Hand Tracking with IISU Middleware

Hi All ,

I know most of everyone on the OF forums using openNI / libfreekinect for kinect hacking. I’ve been playing with SoftKinetic’s iisu middleware and some of the fun things you can do with that. Using the Depthsense camera you have access to a close interaction mode with hand and finger tracking. I made this quick example using the 0071 camera ribbon example.


The code for the ribbons is fairly simple and based off of an old flocking example by roxlu for the more organic movement. The Ribbon source can be found on my github @ git://github.com/benMcChesney/RibbonTrails3D.git

Any feedback would be great! I’m working towards creating an addon to help interface the iisu C++ api.

Really nice work!

Will this work with a Kinect as well? The close interaction mode looks really really interesting. :slight_smile:

Any plans to release an addon for iisu anytime soon?

Thanks for sharing!

Thanks @ lahiru !

The long range interaction should work with a kinect. I’ve used the iisu middleware personally with the ASUS xtion, Panasonic D-Imager, and the Depthsense with almost no problems. Unfortuantely the close range mode will only work with the SoftKinetic Depthsense 311 camera. As for the current state of the addon: right now it’s a split between two different iisu servers for close and long range. In the future I hope to merge them into one coherent class. I’ll upload clean up some code and upload an example for both ranges. The only “downside” is that I’ve only had luck getting iisu to work with visual studio 2010, I never dabbled in their linux library that I believe it uses gcc.

Hi there !

Sorry about the delay. I’ve been away on vacation for the past week or so. I cleaned up my code and made several examples. They are up and available on my company’s github : https://github.com/HeliosInteractive/ofxIisu

I included three examples including : User Representation , Skeleton Tracking, and Close Range Hand Tracking. These examples are written for visual studio 2010 with OF version 0071. Let me know if you have any trouble with the files I tend to have bad luck transferring visual studio projects across computers. I know that SoftKinetic’s iisu is also available via gcc compiler on linux but I’ve never tested it myself. One day hopefully :smiley:

did anyone get this working in osx?

Super nice!
the hand and finger tracking is awesome.

seems that you’ve used a lot of different depth cameras. What are your thoughts about them? most are more expensive than the kinect. Is anyone of this worth the money (compared to the kinect)?

AFAIK the iisu library is not for OSX, at least it’s only specified for windows and linux.


@Stephanshulz the underlying iisu library is Linux and Windows only specific I’m afraid. Windows uses the visual studio compiler and linux uses gcc but I’ve only personally every used the windows side.

@Roymacdonald they each have their specific use. We use the Panasonic in outdoor lighting and it works well but the feed is very noisy and the resolution is quite low. The panasonic is a very robust camera and is tested to run for 5 years straight. The 3106 models works in up to 100,000 lux so we’ve used this camera exclusively for outdoor deployments.

For longer installations ( 3 - 9 months , systems running 18 hours a day ) we have had trouble with the Kinect sensor overheating and then shutting down as a safety precaution to not damage the sensor. We have had better luck with the Asus Xtion, plus it’s USB power only which is quite nice. The softKinetic sensor’s only advantage is that you get the close range tracking with IISU, otherwise an Asus will probably perform better. I have had trouble with Time of Flight cameras having trouble picking up certain clothes such as leather jackets and other highly reflective surfaces, they can get filtered out if you aren’t careful with your calibration.