Multiple hands tracking with Kinect

Hi, I’m working on a hobby project, making a tablet top interface for multiple users, which requires multiple hand tracking.
A Kinect is mounted above the table, capturing images of top view of the table.

I’m making this using Kinect + openFrameworks + OpenNI (ofxOpenNI). I wondered if it’s possible to track hands using OpenNI/NITE’s hand generator, but it seemingly needs body image, not only hand + arm image.

I drew a pic for better understanding what I’m saying here, please look at the attached image.

There are multiple hands moving freely, interacting with the table-top interface.
A hand can be on the top of another, which makes the situation complicated.
When this happens, and if the image of hands (with the background subtracted) is converted into a binary image, it appears as a blob occlusion.
This, occlusion occurrence possibility, is the difficult point for me.
Without occlusions, I think I can do what I want to do. Just subtracting background by depth threshold first, and making the image into a binary image, then each blob represents hand+arm, then just tracking those blobs. That’s it.

I read lots of papers on occlusion handling but they seem difficult and too heavy to implement for me.

Does anybody help me please?

![](http://forum.openframeworks.cc/uploads/default/2759/2013-01-27 14.38.38.jpg)

Hi,
Do you just need the hand position or the fingers?
I can think of several methods but these depend on the previous question.
Glad to help-

Hi, thanks for replying.
I need both.
The hand position is primary, though.
I mean, I’m planning to use a touch display to detect the touches themselves, so the fingers positions are not so accurately needed. What I want to do is to associate touch actions with the one who did the actions. Therefore, hand tracking needs to be robust to occlusion.

Hi, guys, yet the occlusion problem is not settled, my work has finally done:
https://vimeo.com/60233701

Thank you :slight_smile:

![](http://forum.openframeworks.cc/uploads/default/2808/Screen Shot 2013-02-22 at 23.05.56.png)

@takuma7, it looks really nice! Do you pretend to share the code or build and addon? it would be great.
All the best!

@takuma7, I have interest how to do hand tracking, Do you pretend to share the code or addon?