experimental ofOpenCV

extend
#1

Hi,
I have some code to test …

ofOpenCV

This new version of the computerVision addon is sort of the result from a previous discussion. The idea was to create a common base class for the image types (ofGrayscaleImage, ofColorImage, ofFloatImage) and put all the overlaping functionality in there.

I also tried to get functionality in that was posted on the forum and things that were obviously missing. I was also working on a multitouch surface and thought it might be nice to have the blob tracker from the surface software as part of this addon. I tried to add this feature without a lot of bloat. Here is an example that uses the blob tracker:

blobTrackingExample

The main purpose of the blob tracker is that blobs have a persistent id and also fire events when they appear, move, or diappear. There is also a way to query the order in which the blobs appeared.

I haven’t entirely tested all the code but it should not be too bad with the bugs (I hope).

This is a complete list of changes:

* image types now share common base class ofCvImage
* image destructor deallocates dynamic memory
* assignment and copy constructor do proper deep copy
* complete set of operator overloads to go between image types
* added image transformations (translate, rotate, scale)
* added image undistortion method
* texture can be toggled with setUseTexture() like in videoGrabber
* use of stl data types and pure c++ where possible
* setFromPixels now only copies to cvImage not to pixels
* ofCvContourFinder slightly adapted
* a blob tracker added
** ofCvBlobTracker
** operates on proximity and best fit
** compensates for ghost frames
** optional callbacks in main app for blob events
** blobOn(…), blobMoved(…), blobOff(…)
** unique ids and order of appearance
* and more

Let me know if anything breaks,
Any feedback appreciated,

#2

hi stefan,

looks really good – I’m gonna fire it up soon.

one question I’d like to pose after looking at the code - could you imagine dropping ofVectorMath from the code? I’d argue that it’s a good idea to try to make the addons less dependent on each other for compiling, so that less things will break as addons are modified etc. some (like ofThread) obviously will become a kind of core for other addons, but I am a bit nervous about creating too many interdependencies. what do you think?

thanks so much for taking this on !!!
zach

#3

or alternatively, maybe we can promote a basic point2 type to OF core?

I had been thinking for some time about a rectangle type. I want avoid promoting too much to of core for obvious reasons - it’s impossible to get it out - but if you think it’s valuable to have we might want to consider…

just a thought.

thanks again, it’s great you took time to hack at this code !

  • z
#4

Yeah I think I would be in favor of some common types like ofPoint3f and ofVec3f - I have too many projects where I am converting the ofOpenCV ofPoint to the ofVectorMath ofVec.

It makes a lot of sense to have these types defined in a smart, consistent way (with appropriate operator overloading between types) in core so that people making addons don’t have to invent their own data types which then need to be converted when used by a project. This is the main point really - which is that addon developers will write their own implementation which will be doing the same thing but will be incompatible with the other addons. If people are having to do that I think it shows that something is lacking in the core :wink:

So yeah I think it is quite essential.

Thoughts?

#5

I feel the same way. A small set of basic data types (ofPoint, ofVec, ofRectangle, … ) in the core would make a lot of sense. The question that arises is how much of the actual functionality should be part of the core. In case of ofPoint2f for example there are a lot class methods. I am not sure whether all of that functionality should go into the core.

In general it probably makes sense to think about core types in conjunction with addon classes that may extend these types. This also makes it necessary to think about a good naming theme because the core type and the one that extends (subclasses) it need to have separate names.

Here are some possible naming themes …

  
  
=== core ========== subclasses ======  
ofPoint2                   ofPoint2f, ofPnt2, ofPnt2d32f, ...  
ofVector2                 ofVec2f, ofVec2, ofVec2d32f, ...  
ofPoint3  
ofVector3  
  
or  
=== core ========== subclasses ======  
ofPoint                      ofPoint2f, ...  
ofPoint3d                   ofPoint3f, ...  
ofVector                    ofVec2f, ....  
ofVector3d                 ofVec3f, ....  
  
  
or the save but unstylish route:  
=== core ========== subclasses ======  
ofPoint2f                   ofVmPoint2f, vmPoint2f, ...  
ofPoint3f                   ofVmPoint3f, vmPoint3f, ...  
ofVec2f                     ofVmVec2f, vmVec2f, ...  
ofVec3f                     ofVmVec3f, vmVec3f, ...
#6

I think my favorite naming them would be the following. I like the idea of ommitting postfixes of the core types whenever they describe exactly what one would expect anyways. For the feature-loaded addon i like the idea of having a very consistent naming theme.

  
  
== core ============= ofVectorMath ===  
ofPoint                  ofPoint2f  
ofPoint3                 ofPoint3f  
ofVector                 ofVector2f  
ofVector3                ofVector3f  
ofRectangle

:slight_smile:

#7

hi stefan

cool !

personally, at first glance, they look way too close for my preferences – I think it’s could be confusing to see ofPoint3 vs. ofPoint3f.

I know it’s safer, but it’s also way more clear to me that this:

ofVmPoint3f
or
vmPoint3f

is part of the vector math library, which ofPoint3 or ofPoint is part of ofCore. I think this is *way* imporant, to be able to look at code and immediately know where it is coming from. I could see myself getting really confused with types as close as that…

an alternative I am thinking is maybe “ofaPoint3f”
(ie, for of addons)

then you can easily see addons, like ofaCvGrayscaleImage vs core OF stuff. this is a big help for example in glu vs gl calls to know which is which…

anyway, I can understand ofPoint as a base class, but do we even need a point3 in ofcore? I’m not entirely convinced. I can see the overlaps w/ the basic point idea (ie, between cv / vector math / physics libraries, etc).

I am always super skeptical to put stuff in the core :slight_smile: it’s a pain to get it out, and I am reluctant to add types that are not entirely useful for… At any rate, my mind isn’t closed at all on this. I think we should play a bit. I’m gonna try the ofa thing and see if it works… I think it’s a matter of trying and feeling it out what looks and feels good.

thanks!!
zach

#8

oh, I like the ‘ofa’ prefix idea! It would be a separate “namespace” for types and classes that are widely used but not absolutely essential. I have to try this out too.

(maybe ofx would look/feel better than ofa … like ofxPoint3f )

#9

yeah ofx is definitely cooler!!

let me know what looks / feels good – I’ll try too…

thanks much !!
zach

#10

The problem I find with the ofVec2f ofVmVec2f approach is that I don’t see a simplified ofVec2f being of much use.

Like what would it offer that a basic ofPoint doesn’t have?
Most of the methods in ofVec2f is what really set it apart as a vector as opposed to a point. ofVec2f without normalize, length, norm_value, etc would be kind of useless it would just be a point (container for x y) which we would sort of think of like a vector.

I would propose maybe something like either a)

  
Core:  
ofPoint   
ofRectangle   
ofQuad (maybe)  
  
ofVectorMath pretty much as is:  
ofVec2f, ofVec3f ofVec4f ( compatible with core ofPoint )  
ofPoint2f, ofPoint3f..... ( compatible with core ofPoint )

This would allow addons etc to return lists of ofPoints without having to define their own types or use ofVectorMath. ofPoint would be treated a simple container for x and y (with assignment and operator overloading) - so we wouldn’t be strict about people using it to return a list of vectors.

ofPoint could either just be a 2D (x,y) which emphasizes the paper/canvas approach of openframeworks. Or it has a hidden z as well which can be used if needed.

  
Like:   
ofPoint(float x, float y, float z = 0);

The other approach b)

Is to put all the types in core - but in a lite way and with less methods.
Then use ofVectorMath as a collection of static functions (Not Types!) that can do the vector math calculations.

  
ofCore:  
ofPoint, ofPoint2f, ofPoint3f, ofVector, ofVector2f, ofVector3f  
  
ofVectorMath.h:  
ofPoint2f ofPointMiddle(ofPoint2f &a, ofPoint2f &b);  
ofPoint2f ofPointAverage(ofPoint2f *a, int numPoints);  
  
etc....

I think both these approaches have their own advantages -
a: doesn’t change things too much and adds a little to core that can be quite useful all round.

b: keeps the data types in core in a simple way and outsources the more complex operations to the ofVectorMath addon. The nice thing about this approach is that other addons could be made in this way using the simple base types - like an ofPhysics addon.

I kind of like the approach of b and I prefer this to have two versions of each type which i think will be quite confusing and also a pain to manage.

What do you guys think?

#11

hmmm – I completely agree with the split, but I personally am super addicted to vector type as a class (but have to admit, I’m not really addicted to the point type as a class), especially because of operator overloading… ie:

ofVec3f a(0,10,0);
ofVec3f b(100,100, 50);
ofVec3f c = a + b;

or even:

float dopenessOfClasses = (a.normalize() + b.normalize()).angle();

whenever I go to java, I cringe having to write stuff like:

a.add(b);
or
vectorAdd(a,b);

anyway, just my two cents - I think considering a simple core type like point (which could be 2d or 3d makes sense), and I wonder if ofVec types couldn’t:

a) extend the point type
b) return themselves as the point type as needed (ie, convert themselved on the fly)

then, things like opencv can use the point type – and users can use of vector math stuff, and just plug it into to opencv without worrying about the type.

ie,

ofVec2f warpPoints[4];
ofCvGrayscaleImage.warpIntoMe(warpPoints…)

since the vector class can convert to ofPoint on the fly, I believe you can use it wherever you expect ofPoint … I *think* this is possible based on return type / operator overloading…

take care,
zach

#12

cool - I would be down for that approach.
I like the idea of ofVec2f… and ofPoint2f… extending from ofPoint.

That would be a really clean solution!

#13

I am really fascinated by all the different possible solution.
This is amazing! Theo’s suggestion of having x,y, *and* z in ofPoint might be just what we needed.

Yet I am still not convinced about meshing vectors with points. Here is why:

When I read through an .h file and see a variable that is a vector I assume a different purpose than for a variable of type point. I even find it misleading. For example in the new blob tracker I have the following fields:

ofVector deltaLoc;
ofPoint predictedPos;

If I had to change this to “ofPoint deltaLoc” I wouldn’t find it as clear anymore. How can a point be a delta?

I understand that from a practical point of view we don’t really need both but conceptually I find it much clearer and more self-documenting.

ofPoint and ofVector in the core would make more sense to me!

=====
Also on a side note. I don’t find the following totally unappealing and the ofVector and ofPoint in the core would allow us to do this as well.

This is sort of a combination of both theo’s two suggestions and a remark from zach:

* ofPoint, ofVector … in the core with x,y,z
* operator overloads in the core
* ofVectorMath as static functions

Zachs example would look like this:
float dopenessOfClasses = angle( (normalize(a) + normalize(b)) );

Since C++ supports function overloading (and the functions are unique by the type of its arguments) the names could be really simple. I am not particular big on all those static function but it is a very flexible solution.

thanks for taking the time to discuss this through,
I find this really interesting,

#14

cool !

I have to admit that I am definitely *not* convinced about putting vector in the core !! I am convinced about point, rectangle, etc. but I strongly feel vector should stay as an addon and should inherit the point / transition seamlessly into being a point. I feel pretty tough about stuff getting in the core, and my current position is yes to points and rects, no to vecs.

I would also argue that by separating out the two classes, you are making things clearer but more difficult… I never had to think as much when it was only vec2f and there were not point2fs… I can completely using a vector to store delta, but in reality, it’s just a “point” like the other points:

float dx = x2 - x1;
float dy = y2 - y1;

obviously, it make sense to have the extra functionality you can call in the vector math (length, angle, etc) but it doesn’t seem to me that a vector ought to be all that different or exclusive from a point and in all honesty, they store the same the exact same information, so it makes sense that your can just bump your point up to a vector if you want to do vector math, and bump it back to a point if you want to pass it to someone who needs a point (like opencv). that fluidity is something that c++ affords and it why I think this would be good:

core
– ofPoint
– ofRectangle

vector math
– ofVec2f, 3f, etc (extends and can become ofPoint)

anyway, we can hack a bit on this at eyebeam next week. I think this conversation is really helpful, and we’re going to solve alot of issues coming around the corner :slight_smile:

thanks!!
zach

#15

hey zach, theo,

I think ofPoint only in the core is an absolutely satisficing solution. It might even be one of the most practical approaches :slight_smile: I am primarily trying to list things to consider before making the decision.

In this tradition, there is one more thing that comes to my mind. Since it
is important that core types are versatile and can be reused in lots of
different places it might make sense to go one step more abstract. The
impression I got is that they will be primarily data storages without much
functionality (methods). Why not call them for what they are. This is sort
of separating the name from the usage case. This would mean a

ofFloat2d in the core

from which advanced concepts like

ofVec2f, ofPoint2f, ofRadialDistortion, ofRotorInclination, etc …

can be derived as needed. All of those can then be used with
methods/functions which require a double float data type. (Needless to say that this would work just as well in 3 and 4 dimensions)

happy pondering,

ofCvColorImage Mirror
#16

sorry to be so disagreeable :slight_smile: just to say that I am super, **super **appreciative of what you are doing… for me it seems strange to call something by what it contains as opposed to what it represents, for example:

struct {
string path;
int nFiles;
} directoryInfo

vs

struct {
char * path;
int nFiles;
} stringInt

I am super interested in adding a point struct to OF – but to call it ofFloat2 vs ofPoint for me doesn’t seem that useful.

I still vote “ofPoint” in OF, ofPoint2f,3f, etc in vector math that extends ofPoint and can become ofPoint on the fly. we can experiment this week on it –

thanks!!
zach

#17

Yay, new files to test …

I have dropped ofOpenCV’s dependency on ofVectorMath. It now uses the newly proposed core type ofPoint. In a related move I made ofVectorMath subclass its types from ofPoint.

ofOpenCV
ofVectorMath

blobTrackingTest
vectorAngleTest

I have also fixed the angle() method in ofVec2f and added one in ofVec3f. The align() methods now tests for relative angles between vectors. If this one is smaller than a certain tollerance the align() methods return true.

Let me know how it goes,
Happy hacking,

#18

… right … I should have posted this earlier. The previous code is based on two new suggested core types. On my compi I put them in ofConstants.h for now …

  
  
// preliminary core types: suggestion for v0.05  
class ofPoint {  
  public:  
    float x;  
    float y;  
    float z;  
      
    ofPoint( float _x=0.0f, float _y=0.0f, float _z=0.0f ) {  
        x = _x;  
        y = _y;  
        z = _z;  
    }  
};  
  
class ofRectangle {  
  public:  
	float x;  
	float y;  
	float width;  
	float height;      
};
problem with opencv
#19

hi Stefanix,
Cool extend feature!
To try this code , it’s necessary include cv.h (ofCvConstants)… and then all libs of opencv/lib?

Thanks.

#20

hey charli_e, I
n your testApp.h there would be a #include “ofCvMain.h”. This is the only include statement that should be necessry. Other then that you will need to add both the openCV lib file and headers to your projects then add the ofOpenCV files to your project and do a little modification to ofConstants.h as described in the previous post.

Upon these steps you should be able to run the blobTracking example or any of your own code.

Maybe you can post the build error so I get a better idea where you are stuck.