Skeleton Tracking Mac

Hi all,

I’m looking around to see what mac users are using for skeleton tracking in OF these days. I have a kinect v1, but openNI doesn’t seem to be supported anymore. I’m running Catalina and OF 11.

Is it worth upgrading to the new azure kinect? Are there other less expensive options that people are happy with?

Hoping to crowd source this issue and see what other people have been doing.

Kinect v2 is also an option. though support is not as much as other kinect v2 add-ons on macOS, but worth checking out.
you can pick them up pretty cheap on eBay. would save having to spend a lot on the azure, though I would like to get my hands on one :wink:

Thanks for responding so quickly Lewis.

I’ve read a bunch about compatibility issues with the v2 and openNI2 on macOS. Has this been solved or do you just use bootcamp?

honestly could not say right now. been meaning to look a bit further at Kinect v2 to do things like skeleton work, but just not gotten round to it yet. no idea when I will either.
shame that platforms can not work on others, makes things like this that little bit annoying. but im sure there could be a way or workaround

fyi I just got skeleton tracking working with Roy Macdonald’s ofxNI2 on OF 0.11.0 with the kinect 1414:

1 Like

that’s great to know. ill give that a check out myself when I can.
thanks for sharing :kissing_heart:

Hi,

I have used skeleton tracking ofxNI2 on the kinect V2 last year - MACOS 10.14.6 -, and it has relatively close to this one - catalina i believe -.

The hard part is to copy the files right in the build phases, as you can see it here, which comes originally from here.

Hope this helps,

Best,

P

1 Like

really cool. though just trying the setup in Mojave Xcode 10.3 - & having troubles knowing where to put the script stuff. don’t have that popup for the ‘add run script build phase’

I know where the build phase is. but now its where in that. ill play around more & hopefully show something

alright well spotted.

I think you need to add it to the second one :

so if you go for the example called example, the script looks originally like this :

mkdir -p "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/"
# Copy default icon file into App/Resources
rsync -aved "$ICON_FILE" "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/"
# Copy libfmod and change install directory for fmod to run
rsync -aved "$OF_PATH/libs/fmodex/lib/osx/libfmodex.dylib" "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Frameworks/";
install_name_tool -change @executable_path/libfmodex.dylib @executable_path/../Frameworks/libfmodex.dylib "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/$PRODUCT_NAME";

echo "$GCC_PREPROCESSOR_DEFINITIONS";

then it will look something like that :

mkdir -p "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/"
# Copy default icon file into App/Resources
rsync -aved "$ICON_FILE" "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/"
# Copy libfmod and change install directory for fmod to run
rsync -aved "$OF_PATH/libs/fmodex/lib/osx/libfmodex.dylib" "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Frameworks/";
install_name_tool -change @executable_path/libfmodex.dylib @executable_path/../Frameworks/libfmodex.dylib "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/$PRODUCT_NAME";

echo "$GCC_PREPROCESSOR_DEFINITIONS";

cp -R ../../../addons/ofxNI2/libs/OpenNI2/lib/osx/ "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/"; 

cp -R ../../../addons/ofxNI2/libs/NiTE2/lib/osx/libNiTE2.dylib "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/libNiTE2.dylib";

cp -R ../../../addons/ofxNI2/libs/NiTE2/lib/osx/NiTE.ini "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/NiTE.ini";

cp -R ../../../addons/ofxNI2/libs/NITE2-data/NiTE2 "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/";

install_name_tool -change libOpenNI2.dylib @executable_path/../MacOS/libOpenNI2.dylib "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/$PRODUCT_NAME";


cp ../../../addons/ofxNI2/libs/OpenNI2/lib/osx/OpenNI2/Drivers/* "$TARGET_BUILD_DIR/"; 

cp -R ../../../addons/ofxNI2/libs/NiTE2/lib/osx/libNiTE2.dylib "$TARGET_BUILD_DIR/libNiTE2.dylib";


While running it, it should ask for permissions as it’s asking to copy stuff around.

I don’t have a Kinect V2 to test it with, so let me know how it goes for you…

Best,

P

1 Like

coolio - ill give it a look-see :wink:

I bought (foolishly?) a RealSense 435, and was wondering if there was any skeleton tracking available for that device?

Hi, @0a01.wait,

Yes there is.

Free I’m not 100% aware off, but you can try using nuitrack.

You should take a look at @hrs great addon for nuittrack, I’ve tested with an OF D435, it does a good job:

Hope this helps,

Best,

P

thanks for the info - will def check that out

dang just noticed this - I’m on MacBook Pro:

  • Windows 10 or Nuitrack supported OS (Linux, Android)

You might want to check out my fork as well for OpenNI2 skeleton tracking:

Can you do skeleton tracking with the ofxNI2 on intel D435 @grimus?
-i’ve got a D435i, but never tried, i thought it was libFreekinect just for kinect… -

Best,

P

Not you can’t, as the NiTE skeleton tracking library only does Kinect 1 and 2.
You can get the depth map and use OpenNI2 however:

There has been an openNi driver around for a while for the realsense cameras:

Hi, I wanted to test this out for an upcoming project, I am on the current master branch of OF but OSX catalina. I have added the script as described and I see inside the app I have the contents>macos>openNI2>drivers directory with all the requisite dylib files. However when I go to run the user-tracker example I get the following errors:

020-03-05 19:41:56.439724+0100 example-user-tracker[5909:50389] Metal API Validation Enabled
[notice ] LIBFREENECT2_LOGGER_LEVEL = none
[notice ] LIBFREENECT2_LOGGER_LEVEL = info
  2124234 INFO       New log started on 2020-03-05 19:41:58
  2124260 INFO       --- Filter Info --- Minimum Severity: VERBOSE
  2124381 VERBOSE    No override device in configuration file
  2124409 VERBOSE    Configuration has been read from '/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI.ini'
  2124411 VERBOSE    OpenNI 2.2.0 (Build 33)-MacOSX (Nov 12 2013 16:14:14)
  2124413 VERBOSE    Extending the driver path by './OpenNI2/Drivers', as configured in file '/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI.ini'
  2124414 VERBOSE    Using '/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/./OpenNI2/Drivers' as driver path
  2124416 VERBOSE    Looking for drivers in drivers repository '/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/./OpenNI2/Drivers'
  2161310 WARNING    LibraryHandler: Couldn't find function oniDriverStreamSetServices in libPS1080.dylib. Stopping
  2161320 VERBOSE    Couldn't use file 'libPS1080.dylib' as a device driver
  2165606 WARNING    Failed loading lib: dlopen(/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib, 2): Library not loaded: @rpath/libfreenect2.0.2.dylib
  Referenced from: /Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib
  Reason: image not found

  2165616 VERBOSE    Couldn't use file 'libfreenect2-openni2.0.dylib' as a device driver
  2167445 WARNING    Failed loading lib: dlopen(/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib, 2): Library not loaded: @rpath/libfreenect2.0.2.dylib
  Referenced from: /Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib
  Reason: image not found

  2167453 VERBOSE    Couldn't use file 'libfreenect2-openni2.dylib' as a device driver
  2183306 WARNING    LibraryHandler: Couldn't find function oniDriverCreate in libfreenect2.0.2.0.dylib. Stopping
  2183315 VERBOSE    Couldn't use file 'libfreenect2.0.2.0.dylib' as a device driver
  2183852 WARNING    LibraryHandler: Couldn't find function oniDriverCreate in libfreenect2.0.2.dylib. Stopping
  2183859 VERBOSE    Couldn't use file 'libfreenect2.0.2.dylib' as a device driver
  2184235 WARNING    LibraryHandler: Couldn't find function oniDriverCreate in libfreenect2.dylib. Stopping
  2184241 VERBOSE    Couldn't use file 'libfreenect2.dylib' as a device driver
[notice ] OpenNI initialised. Version: 2.2.0.33
[notice ] List devices:
[ fatal ] ofxNI2::Device: invalid device id
[notice ] List devices:

It is strange that is says Reason: image not found

if I ls the Drivers folder I see:

Drivers % ls
PS1080.ini			libfreenect2-openni2.dylib
libOniFile.dylib		libfreenect2.0.2.0.dylib
libPS1080.dylib			libfreenect2.0.2.dylib
libfreenect2-openni2.0.dylib	libfreenect2.dylib

Any clues as to what it would be?

Sorry, spoke to soon, easy to find:

SOLUTION*

As I am on catalina I needed to add these lines to the script:

cp ../../../addons/ofxNI2/libs/OpenNI2/lib/osx/OpenNI2/Drivers/* "$TARGET_BUILD_DIR/"; 

cp -R ../../../addons/ofxNI2/libs/NiTE2/lib/osx/libNiTE2.dylib "$TARGET_BUILD_DIR/libNiTE2.dylib";

@grimus Did you ever find out a way to select the processing device used for depth frame decompression? It automatically selects the first opecCL device, which in a laptop is going to be my integrated card which pulls 15fps…

The output shows:;

[Info] [OpenCLDepthPacketProcessorImpl]  devices:
[Info] [OpenCLDepthPacketProcessorImpl]   0: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz (CPU)[Intel]
[Info] [OpenCLDepthPacketProcessorImpl]   1: Intel(R) UHD Graphics 630 (GPU)[Intel Inc.]
[Info] [OpenCLDepthPacketProcessorImpl]   2: AMD Radeon Pro 5500M Compute Engine (GPU)[AMD]

If i run libfreenect2 with openCL on the radeon card speed is great.