OpenNI || Kinect GetUserPixels

Hello everybody,

When OpenNI detects a user the ofxOpenNI UserGenerator class draws the scene with the user in it. The silhouette is a perfect match.

I am trying to isolate the pixels for these ‘user blobs’.
I understand that the GetUserPixels method of the UserGenerator has something to do with this.

I think the solution must be somewhere in the drawScene() method of the ofxUserGenerator class. But I can’t seem to figure it out.

Is there somebody who could help me in isolating the users pixels and drawing them onto an ofImage or something similar?

Thank you very much in advance.

Cheers,

1 Like

Got it!

Use SceneMetaData instead of DepthMetaData an then do something like this to create a mask for all User objects.

  
  
  
userGenerator.GetUserPixels(0, sceneMetaData);  
	  
depthPixels = (unsigned short*)sceneMetaData.Data();  
  
for (int i =0 ; i < 640 * 480; i++) {  
		  
		if (depthPixels[i] != 0) {  
                      maskPixels[i] = 0;  
		} else {  
		      maskPixels[i] = 255;  
		}  
		  
  }  
maskImage.setFromPixels(maskPixels, 640, 480, OF_IMAGE_GRAYSCALE);  
  

This gives you perfect User blobs to use in combination with OpenCV.

This is great!

Whos version of ofxOpenNi are you using as I dont get (in roxlu latest) the contours/colours of the people.

thanks

Hi Chris,

I got the latest version from HalfdanJ
https://github.com/HalfdanJ/ofxOpenNI

But I actually I am not using the ofxOpenNI classes from this package.

I work with the OpenNI classes directly, included in the package as a library.
https://github.com/HalfdanJ/ofxOpenNI/t-…-er/include

I instantiate my Context, DepthGenerator and UserGenerator from the XnCppWrapper class.
https://github.com/HalfdanJ/ofxOpenNI/b-…-pWrapper.h

Here you can find the source for my main and testApp:
http://dropbox.littlemissrobot.com/uploads/OpenNI-inOF.zip

Let me know if this helps.

wow - that isolation is amazing!

hm…how can we differentiate between users ? and paint them red, blue, orange… ?

ok, i found it

  
userGenerator.GetUserPixels(0, sceneMetaData);  
	depthRawPixels = (unsigned short*)sceneMetaData.Data();  
	  
	for (int i =0 ; i < 640 * 480; i++) {  
		  
		depthPixels[i*3] = 0;  
		depthPixels[i*3+1] = 0;  
		depthPixels[i*3+2] = 0;  
		  
		if (depthRawPixels[i]>0) {  
		  
		if (depthRawPixels[i]%3 == 0) {  
            depthPixels[i*3] = 255;  
		}  
		if (depthRawPixels[i]%3 == 1) {  
            depthPixels[i*3+1] = 255;  
		}  
		if (depthRawPixels[i]%3 == 2) {  
            depthPixels[i*3+2] = 255;  
		}  
			  
		}	  
		  
    }  
user.setFromPixels(depthPixels, 640, 480, OF_IMAGE_COLOR);  

Just wanted to put up a quick video demoing how fast this is - really is a game changer compared to simple blob detection for people

http://vimeo.com/20110346

an image

Hey, wimvanhenden that method is super-neat!

If you use an image generator stream as well, then something like:

  
  
if (depth_generator.IsCapabilitySupported(XN_CAPABILITY_ALTERNATIVE_VIEW_POINT)) {  
     depth_generator.GetAlternativeViewPointCap().SetViewPoint(image_generator);  
}  

will register the depth map to the image map - and then you can pull/mask the pixels from the image generator aligned to the users silhouette…

Added an example of masks from user data to an ofxOpenNI fork: https://github.com/gameoverhack/ofxOpenNI

More info: http://forum.openframeworks.cc/t/openni-skeleton-tracking/5125/74

Thnkx,
M

sorry if this is super obvious but I couldn’t find it. Is there a way to the the IR image/pixels from OpenNI?

so… another example for masking. this uses imageGenerator, userGenerator and depthGenerator, colors eight different people with depth info plus gives 3d user coords. xCode project for OF 062, have fun! :slight_smile:
http://vimeo.com/20323281

http://www.cucumber.de/discourse/NI-userIsolation-4.zip

Hello,

Just got started with the Kinect and thought I’d give OpenNI a try
after seeing the amazing progress so far.

I had a few compile errors initially(which were discussed in other posts),
but now I am stuck again :frowning:

I am using of_preRelease_v0062_osx_FAT on OSX 10.5.8.

Any hints ?

Ok, I think I understand what’s with the error, but can’t seem to fix it.

I’ve checked the console and this is the output:

  
  
[Session started at 2011-02-24 00:20:55 +0000.]  
dyld: Library not loaded: @executable_path/./../../../data/openni/lib/libnimCodecs.dylib  
  Referenced from: /Users/george/Downloads/FirefoxDownloads/of_preRelease_v0062_osx_FAT/apps/examples/NI_userIsolation_4/bin/emptyExampleDebug.app/Contents/MacOS/emptyExampleDebug  
  Reason: no suitable image found.  Did find:  
	/Users/george/Downloads/FirefoxDownloads/of_preRelease_v0062_osx_FAT/apps/examples/NI_userIsolation_4/bin/emptyExampleDebug.app/Contents/MacOS/./../../../data/openni/lib/libnimCodecs.dylib: unknown required load command 0x80000022  
	/Users/george/Downloads/FirefoxDownloads/of_preRelease_v0062_osx_FAT/apps/examples/NI_userIsolation_4/bin/emptyExampleDebug.app/Contents/MacOS/./../../../data/openni/lib/libnimCodecs.dylib: unknown required load command 0x80000022  
  
The Debugger has exited due to signal 5 (SIGTRAP).The Debugger has exited due to signal 5 (SIGTRAP).  
  

Seems like the app can’t find libnimCodecs.dylib, but it’s in the data folder, and I’ve dragged it onto the xcode project too.

Clueless at the moment :frowning:

Hi ascorbin! Thanks for posting this example. I am getting linker problem like this

  
ld: warning: directory '/Applications/of_osx_062/addons/ofxOpenNI/libs/openni/osx' following -L not found  
ld: warning: directory '/Applications/of_osX_062/apps/neu2/NI_userIsolation_4/src/ofxOpenNI/mac/copy_to_data_openni_path/lib' following -L not found  
ld: warning: directory '/Applications/of_osX_062/apps/neu2/NI_userIsolation_4/src/ofxOpenCv/libs/opencv/lib/osx' following -L not found  
ld: library not found for -lJPEG  
  

Any idea where these directories are hardcoded, and how to get JPEG installed? I am running this against standard 062 download.

Thanks

----b

@boba Sorry to interrupt, but I have a suggestion:

It looks like Xcode Can’t find the libJPEG.dylib file, which you should have in the bin/data/openni/lib folder. Drag the files from the lib folder into your project, but pay attention to the options. You could tick the “Copy items …” option if you like, but the make sure you choose the Relative Path option that suits you from the Reference Type drop down.

I’m guessing the Reference Type somehow got set to Absolute which might explain the ‘hardcoded’ path.

HTH,
george

[quote author=“george.profenza”]@boba Sorry to interrupt, but I have a suggestion:

It looks like Xcode Can’t find the libJPEG.dylib file, which you should have in the bin/data/openni/lib folder. Drag the files from the lib folder into your project, but pay attention to the options. You could tick the “Copy items …” option if you like, but the make sure you choose the Relative Path option that suits you from the Reference Type drop down.

I’m guessing the Reference Type somehow got set to Absolute which might explain the ‘hardcoded’ path.

HTH,
george[/quote]

Thanks george! I used the default settings while adding the whole openni folder and that allowed the build to go to completion. Can’t check it with the device though, being on the bus at the moment.

Hey man,

I checked out your code over at Github. Nice work!

Thanks for sharing.

[quote author=“gameover”]Hey, wimvanhenden that method is super-neat!

If you use an image generator stream as well, then something like:

  
  
if (depth_generator.IsCapabilitySupported(XN_CAPABILITY_ALTERNATIVE_VIEW_POINT)) {  
     depth_generator.GetAlternativeViewPointCap().SetViewPoint(image_generator);  
}  

will register the depth map to the image map - and then you can pull/mask the pixels from the image generator aligned to the users silhouette…

Added an example of masks from user data to an ofxOpenNI fork: https://github.com/gameoverhack/ofxOpenNI

More info: http://forum.openframeworks.cc/t/openni-skeleton-tracking/5125/74

Thnkx,
M[/quote]

@ascorbin

Thanks for sharing how you got the user masks to work correctly -> after looking at your code I realised mine was not working at all :wink:

Have implemented your method in the latest version of my ofxOpenNI (amongst many things)…

More info over here http://forum.openframeworks.cc/t/openni-skeleton-tracking/5125/108

M

:smiley: yey! @gameover great work! i love this forum! open source!

greetings