Porting ofxCLNui app to ofxOpenNI

Hi,

I have an application that used ofxCLNui and I’m currently porting it to ofxOpenNI since it has a lot of cool features, I’m having a problem with a function that I have on CLNui:

// Camera depth frame image data retrieval
GetNUICameraDepthFrameCorrected8(CLNUICamera cam, PBYTE pData, int waitTimeout = 2000);

I couldn’t find something like this on ofxOpenNI… Basically pData is filled with the Depth image information, on CLNui it says that it fills the data with “Corrected8” depth frame data. Do you know how can I get something similar to pData from ofxOpenNI…

Any ideas?

I think you can do it like this:

  
  
ofxOpenNIContext. getIRGenerator().GetIRMap();   
  

This returns XnIRPixel which is just a wrapper for a 16 bit grayscale image.

http://openni.org/docs2/Tutorial/-xn-types-8h.html#a7de7d0c3810f61003294c40709abbb45

Thank you, but unfortunately is not exactly what I’m looking for, with the getIRMap() I get the following image:

and I’m looking something more like this…

Wow, yeah, I don’t know how I got that so wrong, embarrassing. I blame my inability to multi-task. You actually want:

  
ofxOpenNIContext.getXnDepthGenerator().GetDepthMap()  

I’ve never worked with the raw pixels from GetDepthMap() so I’m not sure what you need to do to convert them to friendly values, but I’m guessing they’re just the standard 11 bits in a 16 bit int.

More info on that here: http://openni.org/docs2/Tutorial/classxn-1-1-depth-generator.html

Hey man!

Thanks for your answer, almost there… I got the following:

  
  
class testApp  
{  
	...  
	unsigned char *		pPixels;  
	...  
}  
  
testApp::setup()  
{  
	...  
	//int camWidth = 640, camHeight = 480  
	pPixels = new unsigned char[camWidth * camHeight ];  
	memset(pPixels, 0, camWidth * camHeight * sizeof(unsigned char));  
	...  
}  
  
testApp::update()  
{  
	...  
	xn::DepthMetaData dmd;  
	kDepthGenerator->getXnDepthGenerator().GetMetaData(dmd);  
	const XnUInt8* pImage = (XnUInt8*)kDepthGenerator->getXnDepthGenerator().GetDepthMap();  
	int j = 0;  
	for (int i = 0; i < dmd.XRes() * dmd.YRes()*2; i+=2, j++)   
	{  
		pPixels[j] = pImage[i];  
	}  
	kImageTest.setFromPixels(	pPixels,  
						kDepthGenerator->getWidth(),   
						kDepthGenerator->getHeight(),   
						OF_IMAGE_GRAYSCALE);  
	...  
}  
  

My results are:

it’s almost what I’m looking for… the problem with this image is that whenever I do the binarization process, the result is:

Anyway, thanks for your help

Hmm, looking again I think you can just copy the image bytes directly into your grayscale image, i.e.

  
 kImageTest.setFromPixels(playDepth.getDepthPixels(nearThreshold, farThreshold), playDepth.getWidth(), playDepth.getHeight(), OF_IMAGE_GRAYSCALE);  

check out the example from https://github.com/gameoverhack/ofxOpenNI/blob/master/example/src/testApp.cpp

I’d test but I just shipped my Kinect home, so I’m a little less than fully useful, sorry

I think it’s working now :slight_smile: at the beginning I was trying with:

  
  
playDepth.getDepthPixels(nearThreshold, farThreshold)  
  

But the depthPixels where not correctly formated for my functions… strange… anyways I did the following:

  
  
// Everything stays the same from my last post except the update()  
void testApp::Update()  
{  
	...  
	xn::DepthMetaData dmd;  
	kDepthGenerator->getXnDepthGenerator().GetMetaData(dmd);  
  
	const XnDepthPixel* depth = dmd.Data();  
	  
	static float max	= 255;  
	// This makes the depth brighter... may disappear depending on the application  
	static int depthFIX	= 3;   
	for (XnUInt16 y = dmd.YOffset(); y < dmd.YRes() + dmd.YOffset(); y++)  
	{  
		for (XnUInt16 x = 0; x < dmd.XRes(); x++, depth++)   
		{  
			XnUInt8 a = (XnUInt8)(((*depth) / ( kDepthGenerator->getMaxDepth() / max) * depthFIX));  
			pPixels[y * dmd.XRes() + x] = a;  
		}	  
	}  
	  
	kImageTest.setFromPixels(	pPixels,  
						kDepthGenerator->getWidth(),   
						kDepthGenerator->getHeight(),   
						OF_IMAGE_GRAYSCALE);  
	...  
}  
  

The resulting image looks like this, but is a depth 8 and 1 channel image :slight_smile:

Thanks Joshua :slight_smile:

@alejandro,
the depth image that the computer gets is a 16bit 1 channel (grayscale) image.
There is an already working funtion in ofxOpenNi that converts the 16 bit image to 8 bits. Otherwise you wouldnt be able to see the image in the screen.
unsigned char* ofxDepthGenerator::getDepthPixels(int nearThreshold, int farThreshold).
if you set the near threshold to 0 and farthreshold to 65536 (actually the far threshold max value is 10000)
the code you are trying to implement is already implemented in this function.

Cheers!

@roymacdonald
I try it the first time and it didn’t work as I expected, also the returned array is in binary form and I’m working on a different approach for the binarization that required a grayscale image :), so that was another reason why I didn’t use ofxDepthGenerator::getDepthPixels(int near, int far);

Anyways thank you for your comments

alejandro,
so what do you expect to get from the depth map?
how do you need to get the pixels formated?

Well I’m using ofxOpenNI with OpenCV and some CUDA functions, what I needed was to pass the following arguments

  
  
	IplImage *depth_raw = cvCreateImageHeader(cvSize(640,480),8,1);  
	cvSetData(depth_raw,pPixels/*Kinect depth data in grayscale*/,depth_raw->widthStep);  
  

With CLNUI I was doing the following

  
  
	GetNUICameraDepthFrameCorrected8(cam, pPixels); // CLNui API function  
	  
	IplImage *depth_raw = cvCreateImageHeader(cvSize(640,480),8,1);  
	cvSetData(depth_raw,pPixels/*Kinect depth data in grayscale*/,depth_raw->widthStep);  
  

Maybe I can get it more easily through an ofxOpenNI function but I didn’t find any… any suggestions? For now it’s working well.

yes, quite easy in fact.
Just do a memory copy.
Something like this.

  
  
ofxDepthGenerator depthGen;  
  
IplImage *depth_raw = cvCreateImageHeader(cvSize(640,480),8,1);  
memcpy(depth_raw, depthGen.getDepthPixels(), 640*480);  
  

I’ve used this method lots of times and works well.

good luck

It didn’t work though I’m using gameoverhack/ofxOpenNI I’ll change my ofxOpenNI addon to your version roymacdonald/ofxOpenNI

strange.
what are you expecting to get into the iplimage?

It won’t make much of a change switching to my version of ofxOpenNI as te only diference is that I implemented an automatic skeleton detection in the ofxUserGenerator class.

BTW, check at my github the following:
https://github.com/roymacdonald/ofxCvOpenNIcapture-GUI

it uses opencv to capture directly from the kinect.
I have some other implementations laying around in which I directly extract the image and depth data from openni and pass it to some opencv images to get processed.
I’ll upload those soon.

cheers

@alejandro
i missed a part in the code i posted before.
my apologies.
it should be

  
  
ofxDepthGenerator depthGen;    
        
IplImage *depth_raw = cvCreateImageHeader(cvSize(640,480),8,1);    
memcpy(depth_raw->imageData, depthGen.getDepthPixels(), 640*480);    
  

I hope it now works for you.

cheers!