Skeleton tracking and depth camera in the post-kinect post-OpenNI world

#1

Hey everybody,
as the topic’s title suggests, I’m curious to learn how people is tracking their skeletons these days.

Let me go first: most of the time, when I need skeleton tracking, I’m still using the old NiTE library or the Kinect SDK. I imagine this is very common practice.
Sometimes, when the scenario is easy/fitting enough, I also bring back old school, pre-ML, skeletonization techniques.

For most of the real world situations I deal with, I feel OpenPose is still not the most practical solution and again: I think a lot of people feel the same way.
I really think it’s kind of weird how, in the post-kinect world, a lot of new depth cameras hit the market allowing us to re-use most of our depth sensing code without rewriting a single line of code, but there’s a thick silence when it comes to skeleton tracking. There’s some talk about hand tracking, but almost nothing when it comes to robust full figure tracking.

So here I am: anybody working on this? Any paper, or library I should check out?
Any hackish NiTE workaround?

#2

Hi naus3a,

Do you know this? it works with Tvico, Xtion and other devices

Paolo

#3

Thanks Paolo, somehow I did not notice the project: gonna try it!

#4

I use the kinect v2 and the MS sdk on windows. It works really well.
I only use the kinect v1 for non skeleton tracking applications.
Haven’t tried nuitrack. I will give it a try.

#5

Interesting Kinect news from Microsoft

Also, just saw this : JavaScript TensorFlow based skeleton tracking

#6

I use kinect V1 with MS SDK, onLinux I had better results with openNI2 (ofxN2) it gaves more robust results than openNI and it is way easier to use than openNI.
It is also possible to use OpenNI2 on Windows if you install the MS SDK (kinect V1), I didn’t try it out yet since with the MS SDK is enough for me.
I will try the new tools mentioned above.

#7

there is also skeltrack, which i have not used. it could be a good starting point if you want to develop something yourself.


#8

i’ve built this code in 2006. still using it in 2019.
it also has a very simple skeleton tracker, suits my needs just fine, i’ve used it to make projects such as 747.3 (2006), DRACO.WOLFANDDOTCOM.INFO (2015), etc.

i also have a paper glancing through it, human dragons playing in cyberspace https://www.intellectbooks.co.uk/journals/view-issue,id=3470/

by the way, this works on any kind of image, webcams, kinects, jpgs…

#9

Did anyone manage to use the NiTE library for skeleton tracking in of0.10.0 on mac os 10.13?

#10

I did use it with of 0.9.8, still need to test with 10

#11

Hello,
So it is possible to use NITE with ad hoc depth camera to get joints?
Please someone advise how?

#12

I’m using ofxNI2 with oF 0.10 on Mac OS, and skeleton tracking works fine. But I took me some time to set up the project and link all the libraries the right way until it worked. I can’t remember exactly which steps made it work in the end. If you run into some specific problems, let me know, maybe I can help!

#13

Nuitrack is pretty cool although requires a commercial licence! Kinect V2 support was just added very recently.

#14

anyone used openpose?

It looks quite promising

#15

I have not used it, but am really interested in this, there is also vnect:

http://gvv.mpi-inf.mpg.de/projects/VNect/

#16

hmm interesting. I wonder how they differ in practice.

This is a quick test with openpose: https://vimeo.com/301241719

#17

How do I get Nuitrack to openFrameworks? I’ve installed it and got the examples running but I an error when I included <nuitrack/Nuitrack.h> in OF. Might be of topic but still slightly in topic. I guess I have to add something to config.make?

#18

Please try this.

I’m using it with of 0.10.1. on Windows. No support for Linux so far but must be easy.
Also please notice Nuitrack SDK in this addon is little outdated like 2 or 3 months.

#19

Yes! Thanks! Made it!

Had to add to ADDON_CFLAGS -lnuitrack in addon_config.mk. And had to remove all issueCallbacks for some reason. Didn’t compile with them…

#20

Thanks!
Are you on Linux and which SDK version do you use?
May I ask to send PR?