Using a kinect camera to offset Polylines or a single Mesh line?

Hello All

Learning loads, thanks for providing such great tutorials and info on this website.

I have a project that I’m working on atm and I cant get past this basic road block. I have a number of polylines that I need to use as lines that I can then offset using my Kinect.

In the image above I have 3 polylines.

Before, I have been able to use the following code to manipulate a mesh

int w = 640;
int h = 480;

ofMesh mesh;
mesh.setMode(OF_PRIMITIVE_LINES);
int step = 1;
for (int y = 0; y < h; y += step) {
for (int x = 0; x < w; x += step) {
if (kinect.getDistanceAt(x, y) > 0) {
mesh.addColor(kinect.getColorAt(x, y));
//ofDrawCircle(125, 200, 50);
mesh.addVertex(kinect.getWorldCoordinateAt(x, y));
}
}
}

How can I use the getWorldCoordinates to “push out” -for want of a better word" an image from the kinnect camera from the already drawn polylines or mesh line?

Hope I made myself clear enough - any help would be greatly appreciated

Thanks

!

If I’m understanding correctly, theres an example in 3d/meshFromCameraExample that is more or less what you are looking for.

Hi

thank you very much for getting back to me! yes Ive seen that example which is great but I keep running into the same issue. I don’t want to redraw a mesh box that copies the pixels from the camera.

I need each line to be a snapshot of what the camera is doing. I just simply can’t get it to work.

Could you give me any suggestions?

Hi, I dont get what you want to do.
What do you mean by a snapshot of what the camera is doing? can you make a drawing or something.
Do you understand what the getWorldCoordinates function returns?

best

Basically, Im trying to model a shape similar to this

https://scontent-lhr3-1.xx.fbcdn.net/v/t1.0-0/s480x480/21768389_1372015196259080_4325024243582355037_n.jpg?oh=856e1d18ad0badc16a33404539994ee6&oe=5A422492

I know the getWorldCoordinate returns depth values.

I thought If I could someway draw a number of polylines and then use a kinect and the getWorldCoordinates to “push” out the outline from the different thickness polylines

Hope im making myself clearer - and thanks again for the help!

I see. Thats much clearer.
Just use the code you posted, and change the step value to something more than 1 so you get spaced lines.
theres no need to make polylines as you can just use the ofMesh.
You’ll need to set the correct indices for the lines to work correctly.

for (int y = 0; y < h; y += step) {
int currentIndex = 0;
	for (int x = 0; x < w; x += step) {
		if (kinect.getDistanceAt(x, y) > 0) {
			mesh.addColor(kinect.getColorAt(x, y));
			mesh.addVertex(kinect.getWorldCoordinateAt(x, y));
			mesh.addIndex(currentIndex);                
			if(currentIndex>0 && x + step < w && kinect.getDistanceAt(x+step, y) > 0){
				//only add the second index if the vertex is not the first, nor the last
            	mesh.addIndex(currentIndex);
            }
           currentIndex++;
		}
	}
}

I havent tested it but it should work.
cheers

2 Likes

Thank you very much for the advice - i really appreciate it alot!

Hi

just continuing on from what I was saying in the private chat -

I dunno is this too much code to dump in a forum post but maybe it can help with where Im going wrong.

ofBackground(233, 233, 233);
ofSetColor(0, 0, 0);

//	The following code is for creating the vertices for the lines. 
//	From those vertices, I then pick out the ones which work in the style I want 

vector <ofPoint> pointL;
vector <float> xCord;
float yCord;
ofPoint yCordMain;
vector <ofPoint> vecPoint;
vector <ofPoint> pointR;

for (int i = 0; i < height; i++)
{
	pointL.push_back(ofPoint(0, i * 2.5, 1)); // Creating a vector ofPoints the size of the height of the frame
												// at a space of 2.5 apart
}


for (int a = 0; a < pointL.size(); a++)
{
	xCord.push_back(pointL[a].y);
	vecPoint.push_back(ofPoint(width, xCord[a], 1));
	pointR.push_back(ofPoint(vecPoint[a]));
}

yCord = pointL[0].x;
yCordMain = ofPoint(yCord, height, 1);	// Creating a dynamic position for the Y Axis



// The following code could probably be done with a for loop ???? if I was better at c++

ofMesh polyM2; 
polyM2.setMode(OF_PRIMITIVE_LINES);
ofSetLineWidth(4);

polyM2.addVertex(pointL[62]);
polyM2.addVertex(pointR[62]);
polyM2.addVertex(pointL[64]);
polyM2.addVertex(pointR[64]);
polyM2.addVertex(pointL[67]);
polyM2.addVertex(pointR[67]);
polyM2.addVertex(pointL[70]);
polyM2.addVertex(pointR[70]);
polyM2.addVertex(pointL[73]);
polyM2.addVertex(pointR[73]);
polyM2.addVertex(pointL[75]);
polyM2.addVertex(pointR[75]);
polyM2.addVertex(pointL[81]);
polyM2.addVertex(pointR[81]);
polyM2.addVertex(pointL[87]);
polyM2.addVertex(pointR[87]);
polyM2.addVertex(pointL[88]);
polyM2.addVertex(pointR[88]);
polyM2.addVertex(pointL[89]);
polyM2.addVertex(pointR[89]);
polyM2.addVertex(pointL[90]);
polyM2.addVertex(pointR[90]);
polyM2.addVertex(pointL[92]);
polyM2.addVertex(pointR[92]);
polyM2.addVertex(pointL[94]);
polyM2.addVertex(pointR[94]);
polyM2.addVertex(pointL[98]);
polyM2.addVertex(pointR[98]);
polyM2.addVertex(pointL[101]);
polyM2.addVertex(pointR[101]);
polyM2.addVertex(pointL[103]);
polyM2.addVertex(pointR[103]);
polyM2.addVertex(pointL[104]);
polyM2.addVertex(pointR[104]);
polyM2.addVertex(pointL[105]);
polyM2.addVertex(pointR[105]);
polyM2.addVertex(pointL[109]);
polyM2.addVertex(pointR[109]);
polyM2.addVertex(pointL[110]);
polyM2.addVertex(pointR[110]);
polyM2.addVertex(pointL[112]);
polyM2.addVertex(pointR[112]);
polyM2.addVertex(pointL[119]);
polyM2.addVertex(pointR[119]);
polyM2.addVertex(pointL[120]);
polyM2.addVertex(pointR[120]);
polyM2.addVertex(pointL[125]);
polyM2.addVertex(pointR[125]);
polyM2.addVertex(pointL[126]);
polyM2.addVertex(pointR[126]);
polyM2.addVertex(pointL[127]);
polyM2.addVertex(pointR[127]);
polyM2.addVertex(pointL[128]);
polyM2.addVertex(pointR[128]);
polyM2.addVertex(pointL[130]);
polyM2.addVertex(pointR[130]);
polyM2.addVertex(pointL[134]);
polyM2.addVertex(pointR[134]);
polyM2.addVertex(pointL[135]);
polyM2.addVertex(pointR[135]);
polyM2.addVertex(pointL[138]);
polyM2.addVertex(pointR[138]);
polyM2.addVertex(pointL[142]);
polyM2.addVertex(pointR[142]);
polyM2.addVertex(pointL[143]);
polyM2.addVertex(pointR[143]);
polyM2.addVertex(pointL[144]);
polyM2.addVertex(pointR[144]);
polyM2.addVertex(pointL[145]);
polyM2.addVertex(pointR[145]);
polyM2.addVertex(pointL[146]);
polyM2.addVertex(pointR[146]);
polyM2.addVertex(pointL[147]);
polyM2.addVertex(pointR[147]);
polyM2.addVertex(pointL[148]);
polyM2.addVertex(pointR[148]);
polyM2.addVertex(pointL[150]);
polyM2.addVertex(pointR[150]);
polyM2.addVertex(pointL[155]);
polyM2.addVertex(pointR[155]);
polyM2.addVertex(pointL[160]);
polyM2.addVertex(pointR[160]);
polyM2.addVertex(pointL[170]);
polyM2.addVertex(pointR[170]);
polyM2.addVertex(pointL[182]);
polyM2.addVertex(pointR[182]);
polyM2.addVertex(pointL[192]);
polyM2.addVertex(pointR[192]);
polyM2.addVertex(pointL[200]);
polyM2.addVertex(pointR[200]);
polyM2.addVertex(pointL[206]);
polyM2.addVertex(pointR[206]);
polyM2.addVertex(pointL[211]);
polyM2.addVertex(pointR[211]);
polyM2.addVertex(pointL[215]);
polyM2.addVertex(pointR[215]);
polyM2.addVertex(pointL[218]);
polyM2.addVertex(pointR[218]);
polyM2.addVertex(pointL[220]);
polyM2.addVertex(pointR[220]);
polyM2.addVertex(pointL[224]);
polyM2.addVertex(pointR[224]);
polyM2.addVertex(pointL[228]);
polyM2.addVertex(pointR[228]);
polyM2.addVertex(pointL[231]);
polyM2.addVertex(pointR[231]);
polyM2.addVertex(pointL[233]);
polyM2.addVertex(pointR[233]);
polyM2.addVertex(pointL[236]);
polyM2.addVertex(pointR[236]);
polyM2.addVertex(pointL[243]);
polyM2.addVertex(pointR[243]);
polyM2.addVertex(pointL[248]);
polyM2.addVertex(pointR[248]);
polyM2.addVertex(pointL[251]);
polyM2.addVertex(pointR[251]);
polyM2.addVertex(pointL[253]);
polyM2.addVertex(pointR[253]);
polyM2.addVertex(pointL[255]);
polyM2.addVertex(pointR[255]);
polyM2.addVertex(pointL[257]);
polyM2.addVertex(pointR[257]);


//This is the kinect code 

for (int y = 0; y < kinect.getHeight(); y += 1) {
	for (int x = 0; x < kinect.getWidth(); x += 1) {
		if (kinect.getDistanceAt(x, y) > 0) {
			polyM2.addColor(kinect.getColorAt(x, y));
			polyM2.addVertex(kinect.getWorldCoordinateAt(x, y));
		}
	}
}

polyM2.draw();

Any other code left out is just basic initialization stuff

Hopefully you can help - and thank you again for providing assistance for me! I really appreciate it

I’m an idiot - I never added a push and pop matrix + an enabledepth test to store my changes in 3d

this is the code that was missing :

//

ofPushMatrix();
ofScale(1, -1, -1);
ofTranslate(0, 0, -1000);
ofEnableDepthTest();
polyM2.drawVertices();
ofDisableDepthTest();
ofPopMatrix();

easyCam.end();

good to know you were able to solve it. post some screenshots. :slight_smile:

No I actually havent solved it! lol

When I disable the line - polyM2.setMode(OF_PRIMITIVE_LINES);

and just draw the code, I then get this -

But when I comment out this code -

for (int y = 0; y < kinect.getHeight(); y += 1) {
for (int x = 0; x < kinect.getWidth(); x += 1) {
if (kinect.getDistanceAt(x, y) > 0) {
polyM2.addColor(kinect.getColorAt(x, y));
polyM2.addVertex(kinect.getWorldCoordinateAt(x, y));
}
}

I get nothing, which leads me to believe Im not actually drawing the main shapes, but am actually just drawing lines between the x,y values returned from the kinect.getWorldCordinatesAt() function.

This looks to be an indexing issue maybe?

Hope I am making sense?

so, you certainly could add those vertices using a for loop.

When you disable polyM2.setMode(OF_PRIMITIVE_LINES); you simply use the default mesh mode which I think is OF_PRIMITIVE_TRIANGLES.
When set to OF_PRIMITIVE_LINES you should still be able to see something.

It certainly is a problem with indexing.

check the following chapters of ofBook.
http://openframeworks.cc/ofBook/chapters/openGL.html
http://openframeworks.cc/ofBook/chapters/lines.html
http://openframeworks.cc/ofBook/chapters/generativemesh.html

I understand what you want to achieve, but all the code before the following is a bit sense less, I mean its not gonna work how you want it to.

//This is the kinect code 

for (int y = 0; y < kinect.getHeight(); y += 1) {
	for (int x = 0; x < kinect.getWidth(); x += 1) {
		if (kinect.getDistanceAt(x, y) > 0) {
			polyM2.addColor(kinect.getColorAt(x, y));
			polyM2.addVertex(kinect.getWorldCoordinateAt(x, y));
		}
	}
}

The fix is a bit easier than what you might think.
In the above code, in

for (int y = 0; y < kinect.getHeight(); y += 1) {

instead of increasing y by 1 on each loop increase it by a larger number so you can get spaced lines.
you can even set it to update to a random value within a certain range in each loop.

Then, I think that it is a better idea to use OF_PRIMITIVE_TRIANGLE_STRIP as the mesh mode and several meshes to avoid the indexing issue. It is less optimal than doing it with a single mesh and manually doing the indexing but it should work anyways.
do the following

int spacing = 5;
vector<ofMesh> meshes;
float lineWidth = 3;// play with these two values
for (int y = 0; y < kinect.getHeight(); y += spacing ) {
    meshes.push_back(ofMesh());
    ofMesh & m = meshes.back();// this is just a reference, to avoid writing more
    m.setMode(OF_PRIMITIVE_TRIANGLE_STRIP);
	for (int x = 0; x < kinect.getWidth(); x += 1) {
		if (kinect.getDistanceAt(x, y) > 0) {
			auto c = kinect.getColorAt(x, y);
			auto v = kinect.getWorldCoordinateAt(x, y);
			m.addColor(c);
			m.addVertex(v.x, v.y - lineWidth/2, v.z);
                    m.addColor(c);
			m.addVertex(v.x, v.y + lineWidth/2, v.z);
		}
	}
}
 for(int i = 0; i < meshes.size(); i++){
      meshes[i].draw();
}

This should work although I havent tested it as I just coded here in the forum text editor.:stuck_out_tongue:

cheers

Thank you very much man, super detailed explanation all round!

When I go to compile this code - I cant add that code the addVertex() I get an error -

Error (active)	E0415	no suitable constructor exists to convert from "float" to "ofVec3f"

ahh sorry.

 m.addVertex(ofVec3f(v.x, v.y - lineWidth/2, v.z));
 m.addVertex(ofVec3f(v.x, v.y + lineWidth/2, v.z));

Dude perfect, it is in the perfect exact style. It keeps crashing on me though. Could this be related to the reference call & m?

that reference shouldnt be a problem.
if you want to test it change all the m for meshes.back()
and comment out

 ofMesh & m = meshes.back();

if it is crashing, run in debug mode and you should get more insight about it.

Ok, so its finally working! I just want to say thank you so much for the help! Its working exactly how I wanted it to.

The final code ammendment was -

for (int i = 0; i < meshes.size(); i++) {

	easyCam.begin();

	ofPushMatrix();
	ofScale(1, -1, -1);
	ofTranslate(0, 0, -1000);
	ofEnableDepthTest();
	meshes[i].draw();
	ofDisableDepthTest();
	ofPopMatrix();
	easyCam.end();
	
}

This was the final product :slight_smile:

Im really into the kinect and want to learn as much as possible about it and openCV, any suggestions for further reading to learn more on this?

Thanks again!!!

great!
openCv has some really nice tutorials. When using it with openFrameworks I prefer ofxCv instead of ofxOpenCv. look at ofxaddons.con