ofxKinectForWindows2 map color to depth on body index

Hey gang. I am trying to do something that should be relatively simple. I have it working on older OfxKinectV2 code written by rick barazza. But shortly after I got everything working the code was removed from git and replaced with @elliotwoods version – ofxKinectForWindows2. Probably for the best because the old code suffered from a severe FPS throttle when mapping color to depth (max 15fps). But Elliot’s code has been a bit difficult for me to implement so I am looking for any help. I am used to working with pixels and such. But all this ofMesh/openGL stuff has been challenging.

So very simply. How do you map color data to depth data and draw it (with pixels) for the body index? It is key I use body index because I need to remove the ground plane and create a nice greenscreen effect with some openCV contour cleanup :]

I found this code in a previous thread:

mesh = kinect.getDepthSource()->getMesh(true, ofxKinectForWindows2::Source::Depth::PointCloudOptions::TextureCoordinates::ColorCamera);

…but I am not quite sure how to convert this back to pixel data for drawing/exporting.

Also, What is the difference between Kinect body-source and Kinect body-index?

We also have getColorToWorldMap and getDepthToWorldMap. When drawing them I get some really unusual colors. Looks more like a heat-map.


For anyone else wondering the same, there’s an example now in ofxKinectForWindows2 that illustrates this