3d object rotation with blob detection

Hello everyone,
I am adding an interactive element to my friends senior thesis in industrial design. He is making a chair/workstation, and he has a good .obj rendering of the image. I am doing some simple blob detection, which would then rotate the 3d image, just like the objLoaderExample does with the mouse, based on the location of one blob. Seems simple enough.

I have detected a single blob from the camera, identified the center of that blob, drawn a circle around that point, but now I just need the 3d object to rotate according to where that center point is.

First, I have
Code:

for (int i = 0; i < contourFinder.nBlobs; i++){
contourFinder.blobs[i].draw(360,540);
//draws a circle around the center of the blob in the color image.
ofCircle(contourFinder.blobs[i].centroid.x, contourFinder.blobs[i].centroid.y, 20);
//now i want to rototate the 3d object
int TheX = contourFinder.blobs[i].centroid.x;
int TheY = contourFinder.blobs[i].centroid.y;
glRotatef(TheY,1,0,0);
glRotatef(TheX,0,1,0);
}

I’m guessing that’s too simple and I can’t get away with doing that. I’ve tried about 10 other things, and nothing seems to work. Either “outside of scope” or the glRotatef doesnt like my int… various errors.

If you’re wondering, in the 3d obj loader example, the object is rotated by grabbing the x and y location of the mouse like this

Code:

glRotatef(mousey,1,0,0);
glRotatef(mousex,0,1,0);

so if i can find out how to grab the x,y of the ofCircle, i can apply those numbers to the glRotate, and have a sweet gesture rotation. Here is a screen shot of what I have thus far:
http://farm4.static.flickr.com/3660/340-…-46de-o.png
notice the red circle in the top left area.

Any help is appreciated, and thanks to everyone who has helped me up to this point and doesnt even know it.
Peace,
Matt

And what are you seeing? No rotation at all?

Are you actually_drawing_the_3d_model after the rotate calls?

It’s easier if you post your code in it’s entirety.

Cheers,

JGL

Hello Matt,

I would keep the glRotatef call outside the for loop. If there are more than one blob found, each call to glRotate will be added to the previous. This is probably not the behaviour you want. To stay simple, you could average all the x centroids and y centroids, and use that for your rotation parameter.

You might also want to consider the case where no blobs are found.

hope this helps

regards
david

Hello Matt,

I would keep the glRotatef call outside the for loop. If there are more than one blob found, each call to glRotate will be added to the previous. This is probably not the behaviour you want. To stay simple, you could average all the x centroids and y centroids, and use that for your rotation parameter.

You might also want to consider the case where no blobs are found.

hope this helps

regards
david

I actually have it set so that only 1 blob is detected at a time. I’ve commented where i do that in the code below. Good call on the no blobs part however, as I will have to use the mouse coordinates as a fallback if number of blobs = 0.

Here is my entire code at the time. I can move the mouse and rotate the object, but not with the blobs yet.

  
  
#include "testApp.h"  
  
  
//--------------------------------------------------------------  
void testApp::setup(){  
  
  
	#ifdef _USE_LIVE_VIDEO  
        vidGrabber.setVerbose(true);  
        vidGrabber.initGrabber(320,240);  
	#else  
        vidPlayer.loadMovie("fingers.mov");  
        vidPlayer.play();  
	#endif  
          
    colorImg.allocate(320,240);  
	grayImage.allocate(320,240);  
	grayBg.allocate(320,240);  
	grayDiff.allocate(320,240);  
	  
	bLearnBakground = true;  
	threshold = 60;  
	  
	//for smooth animation  
	ofSetVerticalSync(true);  
	  
	//turn on alpha blending for colors  
	ofEnableAlphaBlending();  
	  
	//load some obj files from disk  
	crane.loadFile("crane.obj");  
	  
	bMousePressed = false;  
}  
  
//--------------------------------------------------------------  
void testApp::update(){  
	ofBackground(100,100,100);  
      
    bool bNewFrame = false;  
	                                                                         
	#ifdef _USE_LIVE_VIDEO  
       vidGrabber.grabFrame();  
	   bNewFrame = vidGrabber.isFrameNew();  
    #else  
        vidPlayer.idleMovie();  
        bNewFrame = vidPlayer.isFrameNew();  
	#endif  
	  
	if (bNewFrame){  
		  
		#ifdef _USE_LIVE_VIDEO  
            colorImg.setFromPixels(vidGrabber.getPixels(), 320,240);  
	    #else  
            colorImg.setFromPixels(vidPlayer.getPixels(), 320,240);  
        #endif  
		  
        grayImage = colorImg;  
		if (bLearnBakground == true){  
			grayBg = grayImage;		// the = sign copys the pixels from grayImage into grayBg (operator overloading)  
			bLearnBakground = false;  
		}  
		  
		// take the abs value of the difference between background and incoming and then threshold:  
		grayDiff.absDiff(grayBg, grayImage);  
		grayDiff.threshold(threshold);  
  
		// find contours which are between the size of 20 pixels and 1/3 the w*h pixels.  
		// also, find holes is set to true so we will get interior contours as well....  
  
		contourFinder.findContours(grayDiff, 20, (340*240)/3, 1, false, false);	  
		//                                                    ^ only 1 blob detected  
		//int TheX = contourFinder.blobs[i].centroid.x;  
		//int TheY = contourFinder.blobs[i].centroid.y;  
	}  
  
	printf("%f \n", ofGetFrameRate());  
	  
}  
  
//--------------------------------------------------------------  
void testApp::draw(){  
  
	// draw the incoming, the grayscale, the bg and the thresholded difference  
	ofSetColor(0xffffff);  
	colorImg.draw(20,20);	  
	grayImage.draw(360,20);  
	grayBg.draw(20,280);  
	grayDiff.draw(360,280);  
	  
	// then draw the contours:  
  
	ofFill();  
	ofSetColor(0x333333);  
	ofRect(360,540,320,240);  
	ofSetColor(0xffffff);  
      
	// we could draw the whole contour finder  
	//contourFinder.draw(360,540);  
  
	// or, instead we can draw each blob individually,  
	// this is how to get access to them:  
    for (int i = 0; i < contourFinder.nBlobs; i++){  
        contourFinder.blobs[i].draw(360,540);  
		  
		//draws a circle around the center of the blob in the color image.  
		ofCircle(contourFinder.blobs[i].centroid.x, contourFinder.blobs[i].centroid.y, 20);		  
    }  
	  
	// finally, a report:  
  
	ofSetColor(0xffffff);  
	char reportStr[1024];  
	sprintf(reportStr, "bg subtraction and blob detection\npress ' ' to capture bg\nthreshold %i (press: +/-)\nnum blobs found %i", threshold, contourFinder.nBlobs);  
	ofDrawBitmapString(reportStr, 20, 600);  
	  
	ofSetupScreen();  
	  
	//draw in middle of the screen  
	glTranslatef(ofGetWidth()/2,ofGetHeight()/2,0);  
	  
	//tumble according to mouse  
	  
	glRotatef(mouseY,1,0,0);  
	glRotatef(mouseX,0,1,0);  
	  
	//scale large enough to see the model  
	float s = min(ofGetWidth(),ofGetHeight())*0.2; //default is 0.4  
	glScalef(s,s,s);  
	  
	  
	if (bMousePressed == false){  
		  
        //draw the model  
        glColor4f(0.5,1,5,0.4);//lime color  
        crane.fillFaces();//first the faces  
          
        glColor4f(0,0.8,0,1);//outline  
        crane.outlineFaces();//then the edges.  
		  
	} else {  
		  
        //glColor4f(0,0.1,0,0.5);//outline  
        //crane.outlineFaces();//then the edges.  
          
        //wanna see the vertices?  
        glColor4f(1,0,0,0.85);  
        glPointSize(50);  
        crane.pointVertices();  
	}  
	  
}  
  
  
  
  
//--------------------------------------------------------------  
void testApp::keyPressed  (int key){   
	  
	switch (key){  
		case ' ':  
			bLearnBakground = true;  
			break;  
		case '+':  
			threshold ++;  
			if (threshold > 255) threshold = 255;  
			break;  
		case '-':  
			threshold --;  
			if (threshold < 0) threshold = 0;  
			break;  
	}  
}  
  
//--------------------------------------------------------------  
void testApp::mouseMoved(int x, int y ){  
}	  
  
//--------------------------------------------------------------  
void testApp::mouseDragged(int x, int y, int button){  
}  
  
//--------------------------------------------------------------  
void testApp::mousePressed(int x, int y, int button){  
}  
  
//--------------------------------------------------------------  
void testApp::mouseReleased(){  
  
}  
  

hate to do this, but i need to bump this up. Its due Monday, and if anyone has any ideas I would be happy to return the favor.

I quickly hacked your code. It’s far from perfect but should get you started I hope

this would be the new update and draw methods. You should also declare rotationX and rotationY as floats in testApp.h

Finally, I used a teapot instead of the objLoader, but I guess that’s trivial to change.

  
//--------------------------------------------------------------  
void testApp::update(){  
   ofBackground(100,100,100);  
  
    bool bNewFrame = false;  
  
  
       vidGrabber.grabFrame();  
      bNewFrame = vidGrabber.isFrameNew();  
  
   if (bNewFrame){  
  
        colorImg.setFromPixels(vidGrabber.getPixels(), 320,240);  
  
  
        grayImage = colorImg;  
      if (bLearnBakground == true){  
         grayBg = grayImage;      // the = sign copys the pixels from grayImage into grayBg (operator overloading)  
         bLearnBakground = false;  
      }  
  
      // take the abs value of the difference between background and incoming and then threshold:  
      grayDiff.absDiff(grayBg, grayImage);  
      grayDiff.threshold(threshold);  
  
      // find contours which are between the size of 20 pixels and 1/3 the w*h pixels.  
      // also, find holes is set to true so we will get interior contours as well....  
  
      //contourFinder.findContours(grayDiff, 20, (340*240)/3, 1, false, false);  
  
      int totalBlobs = contourFinder.findContours(grayDiff, 50, (340*240)/3, 1, false);	// find holes  
  
      float sumX, sumY;  
  
  
        if (totalBlobs > 0)  
        {  
            rotationY = ofMap((float) contourFinder.blobs[0].centroid.x, 0.0f, 320.0f, -90.0f, 90.0f);  
  
            rotationX = ofMap((float) contourFinder.blobs[0].centroid.y, 0.0f, 240.0f, -90.0f, 90.0f);  
  
             cout<<contourFinder.blobs[0].centroid.y<<endl;  
             cout<<rotationX<<endl;  
        }else  
        {  
            rotationX = rotationY = 0.0f;  
  
        }  
    }  
  
  
}  
  
  
//--------------------------------------------------------------  
void testApp::draw(){  
  
   // draw the incoming, the grayscale, the bg and the thresholded difference  
   ofSetColor(0xffffff);  
   colorImg.draw(20,20);  
   grayImage.draw(360,20);  
   grayBg.draw(20,280);  
   grayDiff.draw(360,280);  
  
   // then draw the contours:  
  
   ofFill();  
   ofSetColor(0x333333);  
   ofRect(360,540,320,240);  
   ofSetColor(0xffffff);  
  
   // we could draw the whole contour finder  
   //contourFinder.draw(360,540);  
  
   // or, instead we can draw each blob individually,  
   // this is how to get access to them:  
    for (int i = 0; i < contourFinder.nBlobs; i++){  
        contourFinder.blobs[i].draw(360,540);  
  
      //draws a circle around the center of the blob in the color image.  
      ofCircle(contourFinder.blobs[i].centroid.x, contourFinder.blobs[i].centroid.y, 20);  
    }  
  
   // finally, a report:  
  
   ofSetColor(0xffffff);  
   char reportStr[1024];  
   sprintf(reportStr, "bg subtraction and blob detection\npress ' ' to capture bg\nthreshold %i (press: +/-)\nnum blobs found %i", threshold, contourFinder.nBlobs);  
   ofDrawBitmapString(reportStr, 20, 600);  
  
   //ofSetupScreen();  
  
  
  
    glPushMatrix();  
       //draw in middle of the screen  
       glTranslatef(ofGetWidth()* 0.5f,ofGetHeight()* 0.5f,0);  
  
       glScalef(1.0f, -1.0f, 1.0f);  
  
       glRotatef(-rotationX,1,0,0);  
       glRotatef(-rotationY,0,1,0);  
  
       glutWireTeapot(200.0);  
    glPopMatrix();  
}  

Hey, matt
Basicly I think it’s the question where you put glRotate() function. While it’s acturally the basic of OpenGL. In your program draw() method there is one of call ofSetupScreen();
this one initlization a lot of Opengl calls which includes glLoadIdentity(); this will load the Identity Matrix means set all transofrmation to null. so if you have your glRotate before that call then you will not see any rotation.

try:

  
  
float TheX = contourFinder.blobs[0].centroid.x;  
float TheY = contourFinder.blobs[0].centroid.y;  
  
then put your rotation call after mouseX call like :  
  
 glRotatef(mouseY,1,0,0);  
 glRotatef(mouseX,0,1,0);  
     
 glRotatef(TheX,1,0,0);  
 glRotatef(TheY,0,1,0);  
  
   //scale large enough to see the model  
   float s = min(ofGetWidth(),ofGetHeight())*0.2; //default is 0.4  
   glScalef(s,s,s);  
  
  
  

THANK YOU TWO SO MUCH!! I can’t thank you enough. I was on the right track but your clues helped a ton. Here is the completed code.

  
  
//--------------------------------------------------------------  
void testApp::update(){  
	ofBackground(100,100,100);  
      
    bool bNewFrame = false;  
	  
#ifdef _USE_LIVE_VIDEO  
	vidGrabber.grabFrame();  
	bNewFrame = vidGrabber.isFrameNew();  
#else  
	vidPlayer.idleMovie();  
	bNewFrame = vidPlayer.isFrameNew();  
#endif  
	  
	if (bNewFrame){  
		  
#ifdef _USE_LIVE_VIDEO  
		colorImg.setFromPixels(vidGrabber.getPixels(), 320,240);  
#else  
		colorImg.setFromPixels(vidPlayer.getPixels(), 320,240);  
#endif  
		  
        grayImage = colorImg;  
		if (bLearnBakground == true){  
			grayBg = grayImage;      // the = sign copys the pixels from grayImage into grayBg (operator overloading)  
			bLearnBakground = false;  
		}  
		  
		// take the abs value of the difference between background and incoming and then threshold:  
		grayDiff.absDiff(grayBg, grayImage);  
		grayDiff.threshold(threshold);  
		  
		// find contours which are between the size of 20 pixels and 1/3 the w*h pixels.  
		// also, find holes is set to true so we will get interior contours as well....  
		  
		contourFinder.findContours(grayDiff, 20, (340*240)/3, 1, false, false);     
		  
	}  
	  
	printf("%f \n", ofGetFrameRate());  
	  
}  
  
  
//--------------------------------------------------------------  
void testApp::draw(){  
	  
	// draw the incoming, the grayscale, the bg and the thresholded difference  
	ofSetColor(0xffffff);  
	colorImg.draw(20,20);     
	grayImage.draw(360,20);  
	grayBg.draw(20,280);  
	grayDiff.draw(360,280);  
	  
	// then draw the contours:  
	  
	ofFill();  
	ofSetColor(0x333333);  
	ofRect(360,540,320,240);  
	ofSetColor(0xffffff);  
      
	  
	// this is how to get access to the blobs:  
     
	for (int i = 0; i < contourFinder.nBlobs; i++){  
        contourFinder.blobs[i].draw(360,540);  
		  
		//draws a circle around the center of the blob in the color image.  
		ofCircle(contourFinder.blobs[i].centroid.x, contourFinder.blobs[i].centroid.y, 20);        
		  
		//declare the x and y angels for rotation  
		float TheX, TheY;  
		  
		//thanks to david.demainlalune for this one  
		TheY = ofMap((float) contourFinder.blobs[0].centroid.x, 0.0f, 320.0f, -90.0f, 90.0f);  
			  
		TheX = ofMap((float) contourFinder.blobs[0].centroid.y, 0.0f, 240.0f, -90.0f, 90.0f);  
				  
		glPushMatrix();  
		//draw in middle of the screen  
		glTranslatef(ofGetWidth()* 0.5f,ofGetHeight()* 0.5f,0);  
		  
		glScalef(1.0f, -1.0f, 1.0f);  
		  
		//here we rotate the object based on the x,y of the center of the blob  
		glRotatef(-TheX,1,0,0);  
		glRotatef(-TheY,0,1,0);  
		  
		glutWireTeapot(200.0);  
		glPopMatrix();  
	}  
	  
	// finally, a report:  
	  
	ofSetColor(0xffffff);  
	char reportStr[1024];  
	sprintf(reportStr, "bg subtraction and blob detection\npress ' ' to capture bg\nthreshold %i (press: +/-)\nnum blobs found %i", threshold, contourFinder.nBlobs);  
	ofDrawBitmapString(reportStr, 20, 600);  
	  
	//ofSetupScreen();  
		  
}  
  
  

I should mention that I needed to add ofMap to the Utils.h and Utils.app that were provided by Vanderlin here: http://forum.openframeworks.cc/t/more-utils/1413/0

Also, you need to add the OfxObjloader library to the libs/addon directory and add “#define OF_ADDON_USING_OFXOBJLOADER” to your .h file, along with delcaring the addition TheX and TheY floats.

not sure if this is common knowledge, but in case a beginner finds themselves here like me.