openCV - problem finding blob centroid

Gallo · September 5, 2014, 10:52am

Hi all,

I am trying to find the centroid of a blob using openCV.
Well i did it, but centroid doesn’t seem to be where expected !

(see uploaded picture)

here is my code

in update() : solution 1 - find center of blob box area

if(kinect.isFrameNew())
{
	// load grayscale depth image from the kinect source
	grayImage.setFromPixels(kinect.getDepthPixels(), kinect.width, kinect.height);
	    
    grayImage.mirror(false, true);
    flippedDepthImg = grayImage;
    
	// we do two thresholds - one for the far plane and one for the near plane
	// we then do a cvAnd to get the pixels which are a union of the two thresholds
    grayThreshNear = grayImage;
    grayThreshFar = grayImage;
    grayThreshNear.threshold(nearThreshold, true);
    grayThreshFar.threshold(farThreshold);
    cvAnd(grayThreshNear.getCvImage(), grayThreshFar.getCvImage(), grayImage.getCvImage(), NULL);
}

// update the cv images
grayImage.flagImageChanged();

// find contours
contourFinder.findContours(grayImage, 50, (kinect.width*kinect.height)/2, 5, false);

if (contourFinder.nBlobs > 0)
{
    // SOLUTION 1
    ofRectangle bRect = contourFinder.blobs.at(0).boundingRect;
    pos.x = bRect.x + (bRect.width / 2);
    pos.y = bRect.y + (bRect.height / 2);
    // END SOLUTION 1
}

in update() : solution 2 - using centroid() method

if(kinect.isFrameNew())
{
	// load grayscale depth image from the kinect source
	grayImage.setFromPixels(kinect.getDepthPixels(), kinect.width, kinect.height);
	    
    grayImage.mirror(false, true);
    flippedDepthImg = grayImage;
    
	// we do two thresholds - one for the far plane and one for the near plane
	// we then do a cvAnd to get the pixels which are a union of the two thresholds
    grayThreshNear = grayImage;
    grayThreshFar = grayImage;
    grayThreshNear.threshold(nearThreshold, true);
    grayThreshFar.threshold(farThreshold);
    cvAnd(grayThreshNear.getCvImage(), grayThreshFar.getCvImage(), grayImage.getCvImage(), NULL);
}

// update the cv images
grayImage.flagImageChanged();

// find contours
contourFinder.findContours(grayImage, 50, (kinect.width*kinect.height)/2, 5, false);

if (contourFinder.nBlobs > 0)
{
    // SOLUTION 2
    pos = contourFinder.blobs.at(0).centroid;
    // END SOLUTION 2
}

in draw() :

void ofApp::draw()
{
    ofSetColor(255, 255, 255);
    flippedDepthImg.draw(10, 10, 497, 369);
    contourFinder.draw(10, 10, 497, 369);
    ofSetColor(255, 0, 0);
    ofEllipse(pos, 5, 5);
}

Red dot in the picture is the centroid.
In either the first or second solution, centroid is in the same place.

isn’t the centroid supposed to be in the center of the box ?

I am using oF 0.8.3 on mac + xcode 5

Thanks for your help

david_haylock · September 5, 2014, 12:23pm

What is the width and height of the kinect image? Does it comes in at 640x480? If so you are drawing the contour finder at a mismatched resolution. So try drawing it at 0,0,640,480. Also try ofPoint pos = contourFinder.blobs[0].centroid; and draw ofCircle(pos,5); Just incase the ellipse draws at another location. Food for though

david_haylock · September 5, 2014, 12:25pm

Actually I’m fairly certain you need to draw the images the same width and height as the input. So try contourFinder.draw(0,0,kinect.width,kinect.height);

Gallo · September 5, 2014, 2:04pm

Hi,

thanks a lot for the suggestion. This seems ok now.

for the record : same problem with ofCircle and ofEllipse

But problem not solved : i don’t want to draw a 640x480 frame but a 497x369
how can i do that ? or maybe it is just a problem in drawing the dot ? if so i don’t really matter as the dot was only for debugging purpose. If distanceAt(pos) gives me the right value it is ok for me.

By the way, another thing comes to my mind : everything was ok before i did an image flip

grayImage.mirror(false, true);

after i did this, problems appeared.
Any clue ?

david_haylock · September 7, 2014, 8:32pm

No worries.

You could do:

ofPoint newPos(ofMap(pos.x,0,640,0,497),ofMap(pos.y,0,480,0,369));
ofCIrcle(newPos,5);

The mirror problem make sure its called before the background is learnt

Could you post the update section?

Gallo · September 7, 2014, 8:57pm

thank you for your answer.

Here is the update() code :

//--------------------------------------------------------------
void ofApp::update(){
    
    kinect.update();
    
    if(kinect.isFrameNew())
    {
		// load grayscale depth image from the kinect source
		grayImage.setFromPixels(kinect.getDepthPixels(), kinect.width, kinect.height);
		    
        grayImage.mirror(false, true);
        flippedDepthImg = grayImage;
        
		// we do two thresholds - one for the far plane and one for the near plane
		// we then do a cvAnd to get the pixels which are a union of the two thresholds
        grayThreshNear = grayImage;
        grayThreshFar = grayImage;
        grayThreshNear.threshold(nearThreshold, true);
        grayThreshFar.threshold(farThreshold);
        cvAnd(grayThreshNear.getCvImage(), grayThreshFar.getCvImage(), grayImage.getCvImage(), NULL);
    }

    // update the cv images
    grayImage.flagImageChanged();
    
    // find contours which are between the size of 50 pixels and 1/2 the w*h pixels.
    contourFinder.findContours(grayImage, 50, (kinect.width*kinect.height)/2, 5, false);
    
    if (contourFinder.nBlobs > 0)
    {
        pos = contourFinder.blobs.at(0).centroid;
        
        dist = kinect.getDistanceAt(pos);

        // update detected and effect status
        ofxUILabel *sel = (ofxUILabel *)kinectInfos->getWidget("DETECTION");
        sel->setLabel("ETAT : Presence active !");
        sel = (ofxUILabel *)effectsPanel->getWidget("FX STATE");
        sel->setLabel("ETAT : Actif !");
    }
    else
    {
        dist = 0;
        pos = ofPoint(0,0);
        
        // update detected and effect status
        ofxUILabel *sel = (ofxUILabel *)kinectInfos->getWidget("DETECTION");
        sel->setLabel("ETAT : Inactif");
        sel = (ofxUILabel *)effectsPanel->getWidget("FX STATE");
        sel->setLabel("ETAT : Inactif");
    }
    
    // update effect intensity slider value
    ofxUISlider* slider = (ofxUISlider *)effectsPanel->getWidget("INTENSITE");
    slider->setValue(ofMap(dist, 1100, 500, 1, 0));
    
    // update effect variable 1 slider value with kinect position x
    slider = (ofxUISlider *)effectsPanel->getWidget("VAR 1");
    slider->setValue(ofMap(pos.x, 0, canvasWidth, 0, 1));
    
    // update effect variable 2 slider value with kinect position y
    slider = (ofxUISlider *)effectsPanel->getWidget("VAR 2");
    slider->setValue(ofMap(pos.y, 0, canvasHeight, 0, 1));
    
    
    if ( ofGetElapsedTimeMillis() - lastTimeCheck > timeOut)
    {
        if (effectNumber < 6)
        {
            effectNumber++;
        }
        else
        {
            effectNumber = 0;
        }
        ofxUIRadio *sel = (ofxUIRadio *)effectsPanel->getWidget("TYPE INTERFERENCE");
        sel->activateToggle(effects.at(effectNumber));
        lastTimeCheck = ofGetElapsedTimeMillis();
    }
    
    // Send OSC values
    sendOsc("/kinect/distance", ofMap(dist, 500, 1100, 0, 1));
    sendOsc("/kinect/x", ofMap(pos.x, 0, canvasWidth, 0, 1));
    sendOsc("/kinect/y", ofMap(pos.y, 0, canvasHeight, 0, 1));
    sendOsc("/fx/on", contourFinder.nBlobs);
    sendOsc("/fx/active", effectNumber);
    }

//--------------------------------------------------------------

david_haylock · September 8, 2014, 11:21pm

What problems? Like a constant flicker between the mirror?

david_haylock · September 8, 2014, 11:29pm

Ah ok you need to include the grayimage.flag and contourfinder inside the kinect.isFrameNew() bracket.

Gallo · September 10, 2014, 10:20am

Hi,

Sorry for the late answer, didn’t have the email notification…

Ok, i think i didn’t even explain what the problem was. So here we are :

I read the kinect distance value and update a slider using ofxUI addon.
Before mirroring image, distance and slider value updated ok.
Since i use mirroring, distance and slider value seems to go crazy when blob getting close to frame limit.

a bit hard to explain. I will have a look at your suggestions and come back to you with a bunch of code.

thanks a lot.

Gallo · September 10, 2014, 10:42am

Ok, i tried your suggestions but problem persists.

when i flip the image, distance seems ok until i reach left or right frame border…

Maybe you don’t own a kinect sensor but maybe this will help so here you can find the code on github : https://github.com/martialgallorini/ofOscKinect

thank you

david_haylock · September 12, 2014, 6:47pm

Hey fella I’m away from my office for a week. Kinect is unfortunately there, hmm thats strange.
Thanks for posting the link, if memory serves me right blob[0] is the largest blob so as you approach the edge of the frame the CV keeps making a new blob that it thinks it the biggest. What you could try is to blur the image to smooth edges.


openCVImage.blur(value);
openCVImage.blurgaussian(value);