# 3d object rotation with blob detection

Hello everyone,
I am adding an interactive element to my friends senior thesis in industrial design. He is making a chair/workstation, and he has a good .obj rendering of the image. I am doing some simple blob detection, which would then rotate the 3d image, just like the objLoaderExample does with the mouse, based on the location of one blob. Seems simple enough.

I have detected a single blob from the camera, identified the center of that blob, drawn a circle around that point, but now I just need the 3d object to rotate according to where that center point is.

First, I have
Code:

for (int i = 0; i < contourFinder.nBlobs; i++){
contourFinder.blobs[i].draw(360,540);
//draws a circle around the center of the blob in the color image.
ofCircle(contourFinder.blobs[i].centroid.x, contourFinder.blobs[i].centroid.y, 20);
//now i want to rototate the 3d object
int TheX = contourFinder.blobs[i].centroid.x;
int TheY = contourFinder.blobs[i].centroid.y;
glRotatef(TheY,1,0,0);
glRotatef(TheX,0,1,0);
}

I’m guessing that’s too simple and I can’t get away with doing that. I’ve tried about 10 other things, and nothing seems to work. Either “outside of scope” or the glRotatef doesnt like my int… various errors.

If you’re wondering, in the 3d obj loader example, the object is rotated by grabbing the x and y location of the mouse like this

Code:

glRotatef(mousey,1,0,0);
glRotatef(mousex,0,1,0);

so if i can find out how to grab the x,y of the ofCircle, i can apply those numbers to the glRotate, and have a sweet gesture rotation. Here is a screen shot of what I have thus far:
http://farm4.static.flickr.com/3660/340-…-46de-o.png
notice the red circle in the top left area.

Any help is appreciated, and thanks to everyone who has helped me up to this point and doesnt even know it.
Peace,
Matt

And what are you seeing? No rotation at all?

Are you actually_drawing_the_3d_model after the rotate calls?

It’s easier if you post your code in it’s entirety.

Cheers,

JGL

Hello Matt,

I would keep the glRotatef call outside the for loop. If there are more than one blob found, each call to glRotate will be added to the previous. This is probably not the behaviour you want. To stay simple, you could average all the x centroids and y centroids, and use that for your rotation parameter.

You might also want to consider the case where no blobs are found.

hope this helps

regards
david

Hello Matt,

I would keep the glRotatef call outside the for loop. If there are more than one blob found, each call to glRotate will be added to the previous. This is probably not the behaviour you want. To stay simple, you could average all the x centroids and y centroids, and use that for your rotation parameter.

You might also want to consider the case where no blobs are found.

hope this helps

regards
david

I actually have it set so that only 1 blob is detected at a time. I’ve commented where i do that in the code below. Good call on the no blobs part however, as I will have to use the mouse coordinates as a fallback if number of blobs = 0.

Here is my entire code at the time. I can move the mouse and rotate the object, but not with the blobs yet.

``````

#include "testApp.h"

//--------------------------------------------------------------
void testApp::setup(){

#ifdef _USE_LIVE_VIDEO
vidGrabber.setVerbose(true);
vidGrabber.initGrabber(320,240);
#else
vidPlayer.play();
#endif

colorImg.allocate(320,240);
grayImage.allocate(320,240);
grayBg.allocate(320,240);
grayDiff.allocate(320,240);

bLearnBakground = true;
threshold = 60;

//for smooth animation
ofSetVerticalSync(true);

//turn on alpha blending for colors
ofEnableAlphaBlending();

//load some obj files from disk

bMousePressed = false;
}

//--------------------------------------------------------------
void testApp::update(){
ofBackground(100,100,100);

bool bNewFrame = false;

#ifdef _USE_LIVE_VIDEO
vidGrabber.grabFrame();
bNewFrame = vidGrabber.isFrameNew();
#else
vidPlayer.idleMovie();
bNewFrame = vidPlayer.isFrameNew();
#endif

if (bNewFrame){

#ifdef _USE_LIVE_VIDEO
colorImg.setFromPixels(vidGrabber.getPixels(), 320,240);
#else
colorImg.setFromPixels(vidPlayer.getPixels(), 320,240);
#endif

grayImage = colorImg;
if (bLearnBakground == true){
grayBg = grayImage;		// the = sign copys the pixels from grayImage into grayBg (operator overloading)
bLearnBakground = false;
}

// take the abs value of the difference between background and incoming and then threshold:
grayDiff.absDiff(grayBg, grayImage);
grayDiff.threshold(threshold);

// find contours which are between the size of 20 pixels and 1/3 the w*h pixels.
// also, find holes is set to true so we will get interior contours as well....

contourFinder.findContours(grayDiff, 20, (340*240)/3, 1, false, false);
//                                                    ^ only 1 blob detected
//int TheX = contourFinder.blobs[i].centroid.x;
//int TheY = contourFinder.blobs[i].centroid.y;
}

printf("%f \n", ofGetFrameRate());

}

//--------------------------------------------------------------
void testApp::draw(){

// draw the incoming, the grayscale, the bg and the thresholded difference
ofSetColor(0xffffff);
colorImg.draw(20,20);
grayImage.draw(360,20);
grayBg.draw(20,280);
grayDiff.draw(360,280);

// then draw the contours:

ofFill();
ofSetColor(0x333333);
ofRect(360,540,320,240);
ofSetColor(0xffffff);

// we could draw the whole contour finder
//contourFinder.draw(360,540);

// or, instead we can draw each blob individually,
for (int i = 0; i < contourFinder.nBlobs; i++){
contourFinder.blobs[i].draw(360,540);

//draws a circle around the center of the blob in the color image.
ofCircle(contourFinder.blobs[i].centroid.x, contourFinder.blobs[i].centroid.y, 20);
}

// finally, a report:

ofSetColor(0xffffff);
char reportStr[1024];
sprintf(reportStr, "bg subtraction and blob detection\npress ' ' to capture bg\nthreshold %i (press: +/-)\nnum blobs found %i", threshold, contourFinder.nBlobs);
ofDrawBitmapString(reportStr, 20, 600);

ofSetupScreen();

//draw in middle of the screen
glTranslatef(ofGetWidth()/2,ofGetHeight()/2,0);

//tumble according to mouse

glRotatef(mouseY,1,0,0);
glRotatef(mouseX,0,1,0);

//scale large enough to see the model
float s = min(ofGetWidth(),ofGetHeight())*0.2; //default is 0.4
glScalef(s,s,s);

if (bMousePressed == false){

//draw the model
glColor4f(0.5,1,5,0.4);//lime color
crane.fillFaces();//first the faces

glColor4f(0,0.8,0,1);//outline
crane.outlineFaces();//then the edges.

} else {

//glColor4f(0,0.1,0,0.5);//outline
//crane.outlineFaces();//then the edges.

//wanna see the vertices?
glColor4f(1,0,0,0.85);
glPointSize(50);
crane.pointVertices();
}

}

//--------------------------------------------------------------
void testApp::keyPressed  (int key){

switch (key){
case ' ':
bLearnBakground = true;
break;
case '+':
threshold ++;
if (threshold > 255) threshold = 255;
break;
case '-':
threshold --;
if (threshold < 0) threshold = 0;
break;
}
}

//--------------------------------------------------------------
void testApp::mouseMoved(int x, int y ){
}

//--------------------------------------------------------------
void testApp::mouseDragged(int x, int y, int button){
}

//--------------------------------------------------------------
void testApp::mousePressed(int x, int y, int button){
}

//--------------------------------------------------------------
void testApp::mouseReleased(){

}

``````

hate to do this, but i need to bump this up. Its due Monday, and if anyone has any ideas I would be happy to return the favor.

I quickly hacked your code. It’s far from perfect but should get you started I hope

this would be the new update and draw methods. You should also declare rotationX and rotationY as floats in testApp.h

Finally, I used a teapot instead of the objLoader, but I guess that’s trivial to change.

``````
//--------------------------------------------------------------
void testApp::update(){
ofBackground(100,100,100);

bool bNewFrame = false;

vidGrabber.grabFrame();
bNewFrame = vidGrabber.isFrameNew();

if (bNewFrame){

colorImg.setFromPixels(vidGrabber.getPixels(), 320,240);

grayImage = colorImg;
if (bLearnBakground == true){
grayBg = grayImage;      // the = sign copys the pixels from grayImage into grayBg (operator overloading)
bLearnBakground = false;
}

// take the abs value of the difference between background and incoming and then threshold:
grayDiff.absDiff(grayBg, grayImage);
grayDiff.threshold(threshold);

// find contours which are between the size of 20 pixels and 1/3 the w*h pixels.
// also, find holes is set to true so we will get interior contours as well....

//contourFinder.findContours(grayDiff, 20, (340*240)/3, 1, false, false);

int totalBlobs = contourFinder.findContours(grayDiff, 50, (340*240)/3, 1, false);	// find holes

float sumX, sumY;

if (totalBlobs > 0)
{
rotationY = ofMap((float) contourFinder.blobs[0].centroid.x, 0.0f, 320.0f, -90.0f, 90.0f);

rotationX = ofMap((float) contourFinder.blobs[0].centroid.y, 0.0f, 240.0f, -90.0f, 90.0f);

cout<<contourFinder.blobs[0].centroid.y<<endl;
cout<<rotationX<<endl;
}else
{
rotationX = rotationY = 0.0f;

}
}

}

//--------------------------------------------------------------
void testApp::draw(){

// draw the incoming, the grayscale, the bg and the thresholded difference
ofSetColor(0xffffff);
colorImg.draw(20,20);
grayImage.draw(360,20);
grayBg.draw(20,280);
grayDiff.draw(360,280);

// then draw the contours:

ofFill();
ofSetColor(0x333333);
ofRect(360,540,320,240);
ofSetColor(0xffffff);

// we could draw the whole contour finder
//contourFinder.draw(360,540);

// or, instead we can draw each blob individually,
for (int i = 0; i < contourFinder.nBlobs; i++){
contourFinder.blobs[i].draw(360,540);

//draws a circle around the center of the blob in the color image.
ofCircle(contourFinder.blobs[i].centroid.x, contourFinder.blobs[i].centroid.y, 20);
}

// finally, a report:

ofSetColor(0xffffff);
char reportStr[1024];
sprintf(reportStr, "bg subtraction and blob detection\npress ' ' to capture bg\nthreshold %i (press: +/-)\nnum blobs found %i", threshold, contourFinder.nBlobs);
ofDrawBitmapString(reportStr, 20, 600);

//ofSetupScreen();

glPushMatrix();
//draw in middle of the screen
glTranslatef(ofGetWidth()* 0.5f,ofGetHeight()* 0.5f,0);

glScalef(1.0f, -1.0f, 1.0f);

glRotatef(-rotationX,1,0,0);
glRotatef(-rotationY,0,1,0);

glutWireTeapot(200.0);
glPopMatrix();
}

``````

Hey, matt
Basicly I think it’s the question where you put glRotate() function. While it’s acturally the basic of OpenGL. In your program draw() method there is one of call ofSetupScreen();
this one initlization a lot of Opengl calls which includes glLoadIdentity(); this will load the Identity Matrix means set all transofrmation to null. so if you have your glRotate before that call then you will not see any rotation.

try:

``````

float TheX = contourFinder.blobs[0].centroid.x;
float TheY = contourFinder.blobs[0].centroid.y;

then put your rotation call after mouseX call like :

glRotatef(mouseY,1,0,0);
glRotatef(mouseX,0,1,0);

glRotatef(TheX,1,0,0);
glRotatef(TheY,0,1,0);

//scale large enough to see the model
float s = min(ofGetWidth(),ofGetHeight())*0.2; //default is 0.4
glScalef(s,s,s);

``````

THANK YOU TWO SO MUCH!! I can’t thank you enough. I was on the right track but your clues helped a ton. Here is the completed code.

``````

//--------------------------------------------------------------
void testApp::update(){
ofBackground(100,100,100);

bool bNewFrame = false;

#ifdef _USE_LIVE_VIDEO
vidGrabber.grabFrame();
bNewFrame = vidGrabber.isFrameNew();
#else
vidPlayer.idleMovie();
bNewFrame = vidPlayer.isFrameNew();
#endif

if (bNewFrame){

#ifdef _USE_LIVE_VIDEO
colorImg.setFromPixels(vidGrabber.getPixels(), 320,240);
#else
colorImg.setFromPixels(vidPlayer.getPixels(), 320,240);
#endif

grayImage = colorImg;
if (bLearnBakground == true){
grayBg = grayImage;      // the = sign copys the pixels from grayImage into grayBg (operator overloading)
bLearnBakground = false;
}

// take the abs value of the difference between background and incoming and then threshold:
grayDiff.absDiff(grayBg, grayImage);
grayDiff.threshold(threshold);

// find contours which are between the size of 20 pixels and 1/3 the w*h pixels.
// also, find holes is set to true so we will get interior contours as well....

contourFinder.findContours(grayDiff, 20, (340*240)/3, 1, false, false);

}

printf("%f \n", ofGetFrameRate());

}

//--------------------------------------------------------------
void testApp::draw(){

// draw the incoming, the grayscale, the bg and the thresholded difference
ofSetColor(0xffffff);
colorImg.draw(20,20);
grayImage.draw(360,20);
grayBg.draw(20,280);
grayDiff.draw(360,280);

// then draw the contours:

ofFill();
ofSetColor(0x333333);
ofRect(360,540,320,240);
ofSetColor(0xffffff);

for (int i = 0; i < contourFinder.nBlobs; i++){
contourFinder.blobs[i].draw(360,540);

//draws a circle around the center of the blob in the color image.
ofCircle(contourFinder.blobs[i].centroid.x, contourFinder.blobs[i].centroid.y, 20);

//declare the x and y angels for rotation
float TheX, TheY;

//thanks to david.demainlalune for this one
TheY = ofMap((float) contourFinder.blobs[0].centroid.x, 0.0f, 320.0f, -90.0f, 90.0f);

TheX = ofMap((float) contourFinder.blobs[0].centroid.y, 0.0f, 240.0f, -90.0f, 90.0f);

glPushMatrix();
//draw in middle of the screen
glTranslatef(ofGetWidth()* 0.5f,ofGetHeight()* 0.5f,0);

glScalef(1.0f, -1.0f, 1.0f);

//here we rotate the object based on the x,y of the center of the blob
glRotatef(-TheX,1,0,0);
glRotatef(-TheY,0,1,0);

glutWireTeapot(200.0);
glPopMatrix();
}

// finally, a report:

ofSetColor(0xffffff);
char reportStr[1024];
sprintf(reportStr, "bg subtraction and blob detection\npress ' ' to capture bg\nthreshold %i (press: +/-)\nnum blobs found %i", threshold, contourFinder.nBlobs);
ofDrawBitmapString(reportStr, 20, 600);

//ofSetupScreen();

}

``````

I should mention that I needed to add ofMap to the Utils.h and Utils.app that were provided by Vanderlin here: http://forum.openframeworks.cc/t/more-utils/1413/0