ofxKinectNui: an addon of Kinect SDK for oF

[sadmb]

Hi there,

I told zach to build an addon of Kinect SDK and here I released ofxKinectNui. : )

https://github.com/sadmb/ofxKinectNui

well, I found there/s already a great addon of ofxMSKinect by simon.
I am happy if some of you might like this also.

– I tried to make the similar APIs as ofxKinect.
– You can grab Video/ Depth/ Label (silhouette)/ Skeleton and CalibratedRGB (adjust video pixels to depth pixels)
– You can capture/ playback them.
– You can change tilt angle of kinect elevation.
– Use addKinectListener if you want callback when kinect is plugged/ unplugged.
– You can use 2 or more kinects (only 1 kinect is allowed to grab Label/ skeleton), note that you have to use the different usb controller on each kinect.
@see http://bit.ly/oG8Hwk
– I implemented near/ far threshold of depth images so that you can use them with OpenCV.
(it may be better to use Label pixels if you want only players’ data for CV.)
– Sorry it/s only for windows + vs2010 'cause Kinect SDK requires them, but I want to wrap OpenNI (+Xtion also) in future for mac/ linux users.

** NOTE **
I know this still contains many bugs, please let me know if you find.

the main bug is that you can’t close app with ‘esc’ key or press ‘x’ button of oF window once after kinect is detected.
I/m trying to fix it now, but please press ‘x’ button of console window to close for now.

thx,

sadam

[sadmb]

Would somebody please help me to solve the problem of closing window?? : (
still I don’t know why impossible to close window with Esc key or close button of main window…

update to ver. 0.02. : )

• changed pixel data to OF style: ofPixels instead of unsigned char*, etc…
• removed ofxKinectNuiCalibration class.
• bug fixed

pic is an example using point cloud. ; )

[sadmb]

Hi all,
I added exampleCV that you can see below:

• Images above are depth images clipped to 700-1700mm(you can change/ save range), thresholded depth images and contours using ofxOpenCV from it.
• Images below are label(users) images, thresholded label images and contours using ofxOpenCV from it.

Differences between them are:

1. Depth images detect not only users but also chairs, cats etc… label images detect users only.
2. When users are overlapped, whole contour will be detected from depth images but you can get separated/ each user’s or whole contour from label images.
3. You have to wait maybe 0.5-1 sec for Kinect to detect users labeled data. A user has to be on proper distance from Kinect to detect him/ her.
4. You can clip distance range only with depth images.

So, if you want to focus on users detection, I recommend you to use label images for cv. And if you don’t need users detection or you need non-user detection, you’d better to use depth images.

thx. : )

[laton]

Looks great!

i will think to move to VS to try your addon

[sadmb]

@laton

Thx for thinking that you will use this.
Let me know if you find any bug or inconvenience.

[jatru]

Grat job! I hope that this library can be used soon with the new sdk!

This addon can run on codeblocks for windows?

Good luck!

[sadmb]

@jatru

Thx!!
Sorry, this time VS only. but I want to make it compliant with code::blocks.
I will try after I implement kinect audio part. Or I’m happy if there’s someone try that. : )

[sadmb]

Here I updated to ver003. : )
https://github.com/sadmb/ofxKinectNui

MIGRATE TO KINECT SDK 1.0 !!

here’s changes

• it’s no longer works on SDK beta2, so please install SDK 1.0 (it works on Kinect for XBOX360 also)
  http://www.microsoft.com/en-us/kinectforwindows/develop/overview.aspx
• near mode is now available (only for Kinect for Windows hardware)
  NOTE: You can’t get skeleton data on near mode. you can get only center position which is now implementing.
• compliant with audio input. ‘exampleAudio’ added.
  Though example is pretty bad, i deside upload this 'cause I don’t want you to wait. : )
  You can record (16kHz/ 16bit/ mono) wav file when press ‘R’.
  This version you should shutdown app to hear the audio file.
  (It seems CloseHandle(HANDLE hFile) make the sound file like crazy… now I researching this problem)
  Anyways I will update this example soon so some of the functions around audio would be changed.
• calibratedVideo (for point cloud) sample is a little bit changed.
  press ‘Q’ on sample app to check

Please let me know if you like it or find any bug. : )

enjoy,

sadam

[antoniomechas]

Hello,

I’m very pleased to find this wonderful addon…

I’m able to compile it and run it, but I’m having a very strange behaviour, when I run the .exe directly, is showing a lot of errors like the screenshot attached, and then crashes… but if I run the application from Visual Studio, then is working fine.

Does anyone found something like this? I’m trying to find what could be the issue with no luck!

Thanks everybody.

Antonio.

[sadmb]

@gizmow

Hello,

thx try it. : )
sorry it’s my mistake.
it seems i forgot initialization of variable somewhere.
i will find and fix it soon. : )

sadam

Hello,

I’m very pleased to find this wonderful addon…

I’m able to compile it and run it, but I’m having a very strange behaviour, when I run the .exe directly, is showing a lot of errors like the screenshot attached, and then crashes… but if I run the application from Visual Studio, then is working fine.

Does anyone found something like this? I’m trying to find what could be the issue with no luck!

Thanks everybody.

Antonio.

[sadmb]

@gizmow

OK, I have fixed the bug. : )
please pull and try it.

sadam

Hello,

I’m very pleased to find this wonderful addon…

I’m able to compile it and run it, but I’m having a very strange behaviour, when I run the .exe directly, is showing a lot of errors like the screenshot attached, and then crashes… but if I run the application from Visual Studio, then is working fine.

Does anyone found something like this? I’m trying to find what could be the issue with no luck!

Thanks everybody.

Antonio.

[antoniomechas]

Hi Sadam,

Many thanks for your quick reply.

I did the pull and now it is working fine.

I changed my program to use your addon instead of ofxKinect to try out and is compiling fine, but works only in debug mode, I don’t know why but it crashes when I compile in release mode (it crashes always in the kinect.update() statement). Anyways it should be something in my app, because both examples from your addon work fine in debug & release.

Regards,

Antonio.

[sadmb]

@gizmow
Hi,
Thx. : )
If you don’t mind share your src, pls let me check.
You can send it via e-mail or share it via git. : )
Take care,
sadam

Hi Sadam,

Many thanks for your quick reply.

I did the pull and now it is working fine.

I changed my program to use your addon instead of ofxKinect to try out and is compiling fine, but works only in debug mode, I don’t know why but it crashes when I compile in release mode (it crashes always in the kinect.update() statement). Anyways it should be something in my app, because both examples from your addon work fine in debug & release.

Regards,

Antonio.

[cyancdesign]

I came across the same issue with the release version crashing immediately but debug working fine.
Check for this line in your setup function:

  
kinect.open(true);  

Try changing it to:

  
kinect.open(false);  

That worked for me, I also tried to leave it empty kinect.open(), but it would still crash with that.
Hope this works for you too.

[sadmb]

Hello cyancdesign,

Great!

that parameter is actually only for Kinect for Windows which enables near field mode.
So it may occur error when using Kinect for XBOX360.

I will change default setting to false.

thx so much,

sadam

I came across the same issue with the release version crashing immediately but debug working fine.
Check for this line in your setup function:

  
kinect.open(true);  

Try changing it to:

  
kinect.open(false);  

That worked for me, I also tried to leave it empty kinect.open(), but it would still crash with that.
Hope this works for you too.

[alexlangberg]

This is very cool!

It seems though, that the skeleton being drawn is off by about 10px. If I change

  
kinectPlayer.drawSkeleton(20, 20, 400, 300);  

to

  
kinectPlayer.drawSkeleton(30, 20, 400, 300);  

it seems to fit better. I’m not that skilled in this yet so I have no idea why but just wanted to mention it.

[jamesalliban]

Thanks for putting this together Sadam. Really great work.

I’m trying to use it for a project but having a nightmare setting it up. This is my first serious attempt to work with the VS2010 OF version. I’m using Visual C++ express.

Trying to compile the application led to a series of errors stating that NuiApi.h could not be found. I added “C:\Program Files\Microsoft SDKs\Kinect\v1.0\inc” to Configuration Properties -> VC++ Directories -> include directories. This fixed the missing NuiApi.h issue but now I’m getting:

fatal error LNK1104: cannot open file ‘Kinect10.lib’

I tried adding “C:\Program Files\Microsoft SDKs\Kinect\v1.0\lib\x86” and “C:\Program Files\Microsoft SDKs\Kinect\v1.0\lib\x86” separately but this didn’t fix it.

Any idea where I might be going wrong here?

I have a couple of ideas but can’t see a way round. Firstly, the Kinect SDK seems to have installed to the 32-bit Program Files folder. Secondly, I learned here:

http://stackoverflow.com/questions/1865069/how-to-compile-a-64-bit-application-using-visual-c-2010-express

that Visual C++ can’t compile a 64 bit app unless the Windows SDK version 7.1 is installed (and a couple of changes to the project settings are made). I tried this but there were errors saying that FOUR_CHAR_CODE (used in OF) wasn’t recognised. I removed the Win SDK and reset the project and this error went away.

PC Spec:
64-bit Windows 7
3.1 GHz Quad Core processor
4 GB RAM

[prisonerjohn]

Hi James,

I’m using the addon right now and it worked pretty much out-of-the-box… I took screenshots of all my paths, maybe one of these isn’t set properly in your project.

Make sure the Kinect environment variables are set to

  
C:\Program Files\Microsoft SDKs\Kinect\v1.0\  

Then make sure the paths are added for compiling and linking.

(BTW, I can’t believe it’s still this complicated to take screenshots on Windows…)

[jamesalliban]

Thanks for the suggestion Elie

I check the environment variables and include directories last night after reading this:

http://stackoverflow.com/questions/7840315/getting-the-kinect-sdk-to-work-with-visual-studio-2010-in-c

They all seemed fine.

However, it seems to work today even though I haven’t changed anything. So I’m experiencing the usual mix of relief and worry - mostly relief though

The frame rate seems to be very low for both sample projects (5fps for skeletal example and 12fps for cv example). I’ve tried opening both on 2 PCs with simlar results. The PC mentioned above has an NVIVIA Quatro 600 graphics card and the second PC has the following spec:

64-bit Windows 7
3.3 GHz Quad Core processor
8 GB RAM
Inter® HD Graphics Family

Any thoughts on why this is so slow? It’s a bit of a show stopper.

[prisonerjohn]

Try running it in RELEASE mode, for some reason DEBUG apps are crazy slow.