Skeleton tracking and depth camera in the post-kinect post-OpenNI world


#21

I’m on linux (Ubuntu 16.04) and have the latest sdk. By PR you mean project?

Running it with realsense d435 on the up board!


#22

Great!
I meant PR against ofxNuitrack which includes your changes. Can be helpful for other Linux users :slight_smile:


#23

Hello Audiophil,

Thanks for reply !!

Ok. I see there are included libraries for mac os. I guess I need to look for something similar for windows in such case.

Any idea if nite libraries are provided with source?


#24

I successfully used ofxNI2 on OSX and Win, using my “vintage” nite libraries; if you did not keep your old libraries, they’re easy to find online (and if you cannot find them, drop me a message and I’ll be happy to share my copy)

Unfortunately nite is not open source, so no source code. Which means ofxNI2 is a temporary solution, which one day will stop working, which is the reason I started this thread :slight_smile:


#25

If you scroll to the bottom of the ofxNI2 github page, you’ll find a link to a page which provides the windows libraries. But I agree with naus3a that using these old NiTE libraries is not a longterm solution.


#26

I managed to get ofxNi2 working with libfreenect2 for kinectV2, it is working for mac and OSX using openCL acceleration. I know it is not futureproof but it works for now and gives skeleton tracking in OF with kinectV2 on windows and mac (linux is an easy addition but I don’t have a machine right now). It should support multiple kinects (again dont have them to test)

Due to the way openNI2 works it should be easy enough to support other depth cameras (as long as someone has built an openni plugin for them)

There is a problem with the example-assimp-model for now. It tracks but does not animate.


#27

Any chance I could get the 2.2.0.11 NITE binaries from someone? I have Linux already, can’t find Mac or Windows. The ones in the fred-dev/ofxNI2 are 2.2.0.12 I think, but I’d rather have 2.2.0.11 so that I can make my version of the addon (pierrep/ofxNI2) completely cross-platform.


#28

New Kinect


#29

You might be able to use the older code with the newer libs. You can find out method descriptions (kind of) on windows by using dumpbin on a lib file

This might help if things have changed. I would just try with the newer libs and the older code first.

At any rate it seems Pierre Proske is still actively developing this addon, I have seen him active on the libfreenect2 repo and his openNi addon has commits over the last week.

For anyone looking I found the openni and nite stuff here:


#30

@fresla I am Pierre Proske :wink:

I’ve decided to use the NITE 2.0.0.11 Linux binaries and the existing binaries in the ofxNI2 repo. I’m actually not 100% sure what the latter are, but the header files point to 2.0.0.12. However there’s no guarantee both Mac and Windows are the same version.

I scoured online and found a Linux 2.0.0.11 NITE version as well as the above link you posted, which are all 2.0.0.5 versions.

I’ve done a lot of clean-up of the ofxNI2 code, mostly to include depth mapped skeleton coordinates and use OpenNI.ini and NiTE.ini files to point to the data and library locations to avoid ~166MB of NITE data for every project created. This has shrunk the ofxNI2 addon to around 198MB (I think it was around 800MB before).

I’ve also added the Orbbec Astra openNI driver, but you can’t run NITE with it and get a skeleton, only depth and IR work.

Other additions include using environment variables to control the Libfreenect2 logging which was driving me nuts, adding an option to toggle tracking out of frame (which I’m not a fan of, it always seems to wig out) and printing out the tracking frame FPS (which is important to know if you’re in a dark environment as the Kinect V2 hardware will drop in framerate - see https://github.com/OpenKinect/libfreenect2/issues/738 )

I still feel the tracking of the Microsoft SDK is probably a tad better - NITE finds the skeleton in 1-2 seconds provided the environment isn’t too cluttered, and it takes around 10 seconds for the tracking to be jitter free. However it’s totally usable…and I’m planning to use it in an upcoming interactive piece.

I haven’t re-tested the Windows and Mac builds yet, but that’s the plan. I have Windows 10 and MacOS 10.12 awaiting me for that task. Feel free to give it a whirl if you like!


#31

Ha, ok well then that explains the activity. I would agree that the Microsoft tracking is better, fast and more stable. The coordinate mapper on the actual sdk is way nicer to use than libfreenect2 (and I had the same problems getting rid of the logger). However the opencl and cuda acceleration give me way better performance than the SDK and can be precompiled into the openni driver (although with no way to select the compute device, something I wanted to attend to at some point).

Realsense also have openni drivers and that is my next target.


#32

PS did you check this out?


#33

I’ve used Realsense cameras before, but mine is an older F200 which is supported in the legacy branch - https://github.com/IntelRealSense/librealsense/tree/development

I currently don’t have one of the new ones unfortunately…

In other news I just finished cleaning up Windows support for ofxNI2. I’ve recreated all projects and I’m also using the .ini files to find drivers. I’ve also added 0.9.8 support in a separate branch. Provided you have the right libusbK driver to control the Kinect2, the project generator will spit out projects that work out of the box. It was a lot of work to sort out all of those .dlls…ugh

Also, I upgraded the NiTE binaries (for Windows) from 2.2.0.5 to 2.2.0.10, not that it makes any difference.


#34

Ok so I’ve got it working now and tested on all 3 platforms. Big thanks to @fresla for the initial leg work! I’ve gone to some effort to make this as easy to install as possible on all platforms as well as both 0.9.8 and 0.10.1. Hope this is useful to someone. The only other thing I might add for the moment is an example with OSC support, so that you can build an app that streams skeleton data more easily.

Current NiTE versions:
OSX: 2.2.0.12
Linux: 2.2.0.11
Windows: 2.2.0.10

If anyone has more recent versions of NiTE for Windows and Linux, I’d love to include them :slight_smile: