ofxCvHaarFinder: OpenCV face/eye/mouth tracking

[kylemcdonald]

Edit: The most recent version is now maintained at http://kyle.googlecode.com/.

ofxCvHaarFinder is for quickly finding features in an image using haar-like features. This is built into OpenCV, but usage is slightly complicated... hence this wrapper.

This wrapper was originally developed by charlie_e, based on an example by stefanix. I rewrote all the code and added some more features.

I spoke with stefanix, and it seems like this would fit with ofxOpenCv but isn't quite ready yet. To avoid creating YAA (yet another addon :) ) I'm going to support/update this code in this thread for now. Download the code and add it to your ofxOpenCv/src.

For Haar cascades that can be used with this addon, I've found two really good compilations:

1 OpenCV, which can be downloaded via SVN from here: http://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/tags/latest_tested_snapshot/opencv/data/haarcascades

haarcascade_eye.xml
haarcascade_eye_tree_eyeglasses.xml
haarcascade_frontalface_alt.xml
haarcascade_frontalface_alt2.xml
haarcascade_frontalface_alt_tree.xml
haarcascade_frontalface_default.xml
haarcascade_fullbody.xml
haarcascade_lowerbody.xml
haarcascade_profileface.xml
haarcascade_upperbody.xml

2 Modesto Fernando Castrillón Santana from University of Las Palmas de Gran Canaria, which can be downloaded here: http://ftp://mozart.dis.ulpgc.es/pub/Software/HaarClassifiers/FaceFeaturesDetectors.zip

ojoD/right eye
nariz nuevo/nose new
nariz/nose
mouth
ojoI/left eye
HS/head and shoulders
parojosG/big eye-pair
parojos/eye-pair

Some links regarding this addon:

Origin: http://forum.openframeworks.cc/index.php/topic,432.0.html
Old bugs: http://forum.openframeworks.cc/index.php/topic,1853.0.html
The OpenCV function itself: http://www.comp.leeds.ac.uk/vision/opencv/opencvref_cv.html#decl_cvHaarDetectObjects

Using this code requires something like this:

Code:

ofxCvHaarFinder finder;
finder.setup("face.xml");
finder.findHaarObjects(img); // an ofxCvGrayscaleImage
for(int i = 0; i < finder.blobs.size(); i++) {
  ofxCvBlob cur = finder.blobs[i];
  printf("found: %i, %i %ix%i\n", 
    cur.x, cur.y, cur.width, cur.height);
}

The attachment is newer than the last one I posted. It implements the copy constructor and correctly deallocates memory.

[vanderlin]

I agree if this were built into ofxOpenCv it would be great. I have a addon that I built ahile back, has function like:

void setRes(w, h);

each haar file when you open them or make them need to be set a a set dimension, or this is what I have read.

I also made a ofxHaarBlob since there is no need for vector pnts and I wanted to add some other stuff, but maybe it could just extend ofxCvBlob to keep clean. I will try to post it up on my google code.

T

[kylemcdonald]

Hey Todd... do you remember where you heard about setting the dimensions of the input when loading the training file?

I know in the xml there is a <size> field that encodes the dimensions of the training target, but that's independent of the files you're loading.

Or maybe you're thinking of the last argument to cvHaarDetectObjects, which specifies the minimum size, in pixels, to search for? (I wrapped this with my code, too).

ofxHaarBlob is nice... but it is kind of ambiguous how the "haar results aren't quite blobs" thing should be handled. Maybe instead of haar blobs extending regular blobs it should be the other way around. You would have ofxCvShape (which could actually just be an ofRectangle, or an ofRectangle plus a few other things), and that would be what haar results are returned as. Blobs from contour tracking would extend that -- as they have a bounding box, length, and center... but vertices, too.

[vanderlin]

I totally agree with you about blobs, that would make so much more sense. I will look for that article, i found it awhile back when working on the haarTraining app ( i still need to post this, sorry, getting better with google code tho). In this function:

Code:

                 /*
		 #define CV_HAAR_DO_CANNY_PRUNING    1
		 #define CV_HAAR_SCALE_IMAGE         2
		 #define CV_HAAR_FIND_BIGGEST_OBJECT 4 
		 #define CV_HAAR_DO_ROUGH_SEARCH     8
		 
		 CVAPI(CvSeq*) cvHaarDetectObjects( const CvArr* image,
		 CvHaarClassifierCascade* cascade,
		 CvMemStorage* storage, 
		 double scale_factor CV_DEFAULT(1.1),
		 int min_neighbors CV_DEFAULT(3), 
		 int flags CV_DEFAULT(0),
		 CvSize min_size CV_DEFAULT(cvSize(0,0)));

		 */

In the XML file

<?xml version="1.0"?>
<!--
    Tree-based 20x20 frontal eye detector with better handling of eyeglasses.
    Created by Shameem Hameed (http://umich.edu/~shameem)

min_size(20, 20) is from what I understand based on the haar file sample size. I just tried leaving that param blank and it still works, maybe cvHaarDetectObjects get that param from the file.. hmmm

[kylemcdonald]

Yeah, you've got it exactly right. If you go further into the xml there is a tag that will say <size>20 20</size> or something similar. If you leave the final argument out of the call to cvHaarDetectObjects, it uses that size. Also, if you set the value as cvSize(0, 0) it uses the default as well.

I left some notes in my code about those #defines btw, and what exactly they do.

[Chris OShea]

In my face finder I just have...

typedef struct {
   float            area;
   ofRectangle      box;
   ofPoint         centre;
} CvHaarResult;

My class is also running in its own thread, but I'm trying to iron out the kinks in it. That way you can have multiple haars running at once, without slowing each other down.

[kylemcdonald]

Does it really take less time to run multiple cvHaarDetectObjects in threads than it does to run them sequentially in a single thread? I don't understand how that works...?

It would be nice to have it, fundamentally, threaded. Then optionally provide a blocking call for people who don't want to deal with that.

[johnavila2]

how can i do for used to another thing???

i just i can do working the example(face) :S, the others xml doesn't work

[kylemcdonald]

how can i do for used to another thing???

i just i can do working the example(face) :S, the others xml doesn't work

Do you mean, "how do i use it with other objects besides faces", or something else?

Also, if you're using Code::Blocks on Windows I could write a simple example for you.

[johnavila2]

Do you mean, "how do i use it with other objects besides faces", or something else?

Also, if you're using Code::Blocks on Windows I could write a simple example for you.

Hi

thx yes i am using code::Blocks on windows

yes i need this i little example for guideme

Sorry for my english i tried to make better this.....

[kylemcdonald]

Your English is good enough for me to understand, so no worries :)

Here's a complete example.

main.cpp:

Code:

#include "testApp.h"
#include "ofAppGlutWindow.h"

int main() {
	ofAppGlutWindow window;
	ofSetupOpenGL(&window, 1024, 680, OF_WINDOW);
	ofRunApp(new testApp());
}

testApp.h:

Code:

#pragma once

#include "ofMain.h"
#include "ofxCvHaarFinder.h"

class testApp : public ofBaseApp{
	public:
		void setup();
		void update();
		void draw();

		ofImage img;
		ofxCvHaarFinder finder;
};

testApp.cpp:

Code:

#include "testApp.h"

void testApp::setup(){
	img.loadImage("test.jpg");
	finder.setup("haarcascade_frontalface_default.xml");
	finder.findHaarObjects(img);
}

void testApp::update(){
}

void testApp::draw(){
	img.draw(0, 0);
	ofNoFill();
	for(int i = 0; i < finder.blobs.size(); i++) {
		ofRectangle cur = finder.blobs[i].boundingRect;
		ofRect(cur.x, cur.y, cur.width, cur.height);
	}
}

Make sure you've downloaded the most recent version and copied the files to the opencv/src folder. I added an inefficient method that uses an ofImage, so you don't have to worry about the ofImage->ofxCvImage conversion. Also, you need to put a file in /data called test.jpg that the test is run on. Here's what it looks like running for me:

[johnavila2]

Hi again i download this

Attachments:  

File comment: r2: May 29, 2009
 ofxCvHaarFinder.zip [2.32 KiB]
Downloaded 3 times
 
File comment: r1: May 24, 2009
 ofxCvHaarFinder.zip [2.26 KiB]
Downloaded 15 times

And download

Attachments:  
File comment: Complete example of using ofxCvHaarFinder
 haarTester.zip [149.26 KiB]
Downloaded 3 times

But appear this error

obj\release\src\testApp.o:testApp.cpp:(.text+0x128)||undefined reference to `ofxCvHaarFinder::findHaarObjects(ofImage&, int, int)'|

i need some else in

finder.findHaarObjects(img);


i put an image test.jpg in data....

thx for your example can you help me

[kylemcdonald]

Make sure that you unzip the code from the ofxCvHaarFinder.zip into of/addons/ofxOpenCv/src, otherwise the project won't be able to find it.

[vanderlin]

poor Damian no face :(

[kylemcdonald]

poor Damian no face :)