Skeleton Tracking Mac

fyi I just got skeleton tracking working with Roy Macdonald’s ofxNI2 on OF 0.11.0 with the kinect 1414:

1 Like

that’s great to know. ill give that a check out myself when I can.
thanks for sharing :kissing_heart:

Hi,

I have used skeleton tracking ofxNI2 on the kinect V2 last year - MACOS 10.14.6 -, and it has relatively close to this one - catalina i believe -.

The hard part is to copy the files right in the build phases, as you can see it here, which comes originally from here.

Hope this helps,

Best,

P

1 Like

really cool. though just trying the setup in Mojave Xcode 10.3 - & having troubles knowing where to put the script stuff. don’t have that popup for the ‘add run script build phase’

I know where the build phase is. but now its where in that. ill play around more & hopefully show something

alright well spotted.

I think you need to add it to the second one :

so if you go for the example called example, the script looks originally like this :

mkdir -p "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/"
# Copy default icon file into App/Resources
rsync -aved "$ICON_FILE" "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/"
# Copy libfmod and change install directory for fmod to run
rsync -aved "$OF_PATH/libs/fmodex/lib/osx/libfmodex.dylib" "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Frameworks/";
install_name_tool -change @executable_path/libfmodex.dylib @executable_path/../Frameworks/libfmodex.dylib "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/$PRODUCT_NAME";

echo "$GCC_PREPROCESSOR_DEFINITIONS";

then it will look something like that :

mkdir -p "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/"
# Copy default icon file into App/Resources
rsync -aved "$ICON_FILE" "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/"
# Copy libfmod and change install directory for fmod to run
rsync -aved "$OF_PATH/libs/fmodex/lib/osx/libfmodex.dylib" "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Frameworks/";
install_name_tool -change @executable_path/libfmodex.dylib @executable_path/../Frameworks/libfmodex.dylib "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/$PRODUCT_NAME";

echo "$GCC_PREPROCESSOR_DEFINITIONS";

cp -R ../../../addons/ofxNI2/libs/OpenNI2/lib/osx/ "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/"; 

cp -R ../../../addons/ofxNI2/libs/NiTE2/lib/osx/libNiTE2.dylib "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/libNiTE2.dylib";

cp -R ../../../addons/ofxNI2/libs/NiTE2/lib/osx/NiTE.ini "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/NiTE.ini";

cp -R ../../../addons/ofxNI2/libs/NITE2-data/NiTE2 "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/Resources/";

install_name_tool -change libOpenNI2.dylib @executable_path/../MacOS/libOpenNI2.dylib "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Contents/MacOS/$PRODUCT_NAME";


cp ../../../addons/ofxNI2/libs/OpenNI2/lib/osx/OpenNI2/Drivers/* "$TARGET_BUILD_DIR/"; 

cp -R ../../../addons/ofxNI2/libs/NiTE2/lib/osx/libNiTE2.dylib "$TARGET_BUILD_DIR/libNiTE2.dylib";


While running it, it should ask for permissions as it’s asking to copy stuff around.

I don’t have a Kinect V2 to test it with, so let me know how it goes for you…

Best,

P

1 Like

coolio - ill give it a look-see :wink:

I bought (foolishly?) a RealSense 435, and was wondering if there was any skeleton tracking available for that device?

Hi, @0a01.wait,

Yes there is.

Free I’m not 100% aware off, but you can try using nuitrack.

You should take a look at @hrs great addon for nuittrack, I’ve tested with an OF D435, it does a good job:

Hope this helps,

Best,

P

thanks for the info - will def check that out

dang just noticed this - I’m on MacBook Pro:

  • Windows 10 or Nuitrack supported OS (Linux, Android)

You might want to check out my fork as well for OpenNI2 skeleton tracking:

Can you do skeleton tracking with the ofxNI2 on intel D435 @grimus?
-i’ve got a D435i, but never tried, i thought it was libFreekinect just for kinect… -

Best,

P

Not you can’t, as the NiTE skeleton tracking library only does Kinect 1 and 2.
You can get the depth map and use OpenNI2 however:

There has been an openNi driver around for a while for the realsense cameras:

Hi, I wanted to test this out for an upcoming project, I am on the current master branch of OF but OSX catalina. I have added the script as described and I see inside the app I have the contents>macos>openNI2>drivers directory with all the requisite dylib files. However when I go to run the user-tracker example I get the following errors:

020-03-05 19:41:56.439724+0100 example-user-tracker[5909:50389] Metal API Validation Enabled
[notice ] LIBFREENECT2_LOGGER_LEVEL = none
[notice ] LIBFREENECT2_LOGGER_LEVEL = info
  2124234 INFO       New log started on 2020-03-05 19:41:58
  2124260 INFO       --- Filter Info --- Minimum Severity: VERBOSE
  2124381 VERBOSE    No override device in configuration file
  2124409 VERBOSE    Configuration has been read from '/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI.ini'
  2124411 VERBOSE    OpenNI 2.2.0 (Build 33)-MacOSX (Nov 12 2013 16:14:14)
  2124413 VERBOSE    Extending the driver path by './OpenNI2/Drivers', as configured in file '/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI.ini'
  2124414 VERBOSE    Using '/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/./OpenNI2/Drivers' as driver path
  2124416 VERBOSE    Looking for drivers in drivers repository '/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/./OpenNI2/Drivers'
  2161310 WARNING    LibraryHandler: Couldn't find function oniDriverStreamSetServices in libPS1080.dylib. Stopping
  2161320 VERBOSE    Couldn't use file 'libPS1080.dylib' as a device driver
  2165606 WARNING    Failed loading lib: dlopen(/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib, 2): Library not loaded: @rpath/libfreenect2.0.2.dylib
  Referenced from: /Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib
  Reason: image not found

  2165616 VERBOSE    Couldn't use file 'libfreenect2-openni2.0.dylib' as a device driver
  2167445 WARNING    Failed loading lib: dlopen(/Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib, 2): Library not loaded: @rpath/libfreenect2.0.2.dylib
  Referenced from: /Users/fred/openFrameworks/addons/ofxNI2/example-user-tracker/bin/example-user-tracker.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib
  Reason: image not found

  2167453 VERBOSE    Couldn't use file 'libfreenect2-openni2.dylib' as a device driver
  2183306 WARNING    LibraryHandler: Couldn't find function oniDriverCreate in libfreenect2.0.2.0.dylib. Stopping
  2183315 VERBOSE    Couldn't use file 'libfreenect2.0.2.0.dylib' as a device driver
  2183852 WARNING    LibraryHandler: Couldn't find function oniDriverCreate in libfreenect2.0.2.dylib. Stopping
  2183859 VERBOSE    Couldn't use file 'libfreenect2.0.2.dylib' as a device driver
  2184235 WARNING    LibraryHandler: Couldn't find function oniDriverCreate in libfreenect2.dylib. Stopping
  2184241 VERBOSE    Couldn't use file 'libfreenect2.dylib' as a device driver
[notice ] OpenNI initialised. Version: 2.2.0.33
[notice ] List devices:
[ fatal ] ofxNI2::Device: invalid device id
[notice ] List devices:

It is strange that is says Reason: image not found

if I ls the Drivers folder I see:

Drivers % ls
PS1080.ini			libfreenect2-openni2.dylib
libOniFile.dylib		libfreenect2.0.2.0.dylib
libPS1080.dylib			libfreenect2.0.2.dylib
libfreenect2-openni2.0.dylib	libfreenect2.dylib

Any clues as to what it would be?

Sorry, spoke to soon, easy to find:

SOLUTION*

As I am on catalina I needed to add these lines to the script:

cp ../../../addons/ofxNI2/libs/OpenNI2/lib/osx/OpenNI2/Drivers/* "$TARGET_BUILD_DIR/"; 

cp -R ../../../addons/ofxNI2/libs/NiTE2/lib/osx/libNiTE2.dylib "$TARGET_BUILD_DIR/libNiTE2.dylib";

@grimus Did you ever find out a way to select the processing device used for depth frame decompression? It automatically selects the first opecCL device, which in a laptop is going to be my integrated card which pulls 15fps…

The output shows:;

[Info] [OpenCLDepthPacketProcessorImpl]  devices:
[Info] [OpenCLDepthPacketProcessorImpl]   0: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz (CPU)[Intel]
[Info] [OpenCLDepthPacketProcessorImpl]   1: Intel(R) UHD Graphics 630 (GPU)[Intel Inc.]
[Info] [OpenCLDepthPacketProcessorImpl]   2: AMD Radeon Pro 5500M Compute Engine (GPU)[AMD]

If i run libfreenect2 with openCL on the radeon card speed is great.

howdy folks

I’ve got the time finally to look into this & can’t seem to get things working, even with the pointers given.

tried with both kinectv1 & v2 & moved the files into where they were told to go in the bin folder, but its popping up with errors [at the bottom].
I have the run script in as well. so I’m just a bit lost right now.

working in Mojave & Xcode 10.3

thanks

697259 INFO       New log started on 2020-03-06 14:08:47
   697286 INFO       --- Filter Info --- Minimum Severity: VERBOSE
   697361 VERBOSE    No override device in configuration file
   697433 VERBOSE    Configuration has been read from '/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI.ini'
   697441 VERBOSE    OpenNI 2.2.0 (Build 33)-MacOSX (Nov 12 2013 16:14:14)
   697444 VERBOSE    Extending the driver path by './OpenNI2/Drivers', as configured in file '/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI.ini'
   697449 VERBOSE    Using '/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/./OpenNI2/Drivers' as driver path
   697452 VERBOSE    Looking for drivers in drivers repository '/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/./OpenNI2/Drivers'
   703799 WARNING    Failed loading lib: dlopen(/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI2/Drivers/libFreenectDriver.0.6.0.dylib, 2): Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: /Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/libFreenectDriver.0.6.0.dylib
  Reason: image not found

   735377 VERBOSE    Couldn't use file 'libFreenectDriver.0.6.0.dylib' as a device driver
   741537 WARNING    Failed loading lib: dlopen(/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI2/Drivers/libFreenectDriver.0.6.0.dylib, 2): Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: /Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/libFreenectDriver.0.6.0.dylib
  Reason: image not found

   741574 VERBOSE    Couldn't use file 'libFreenectDriver.0.6.dylib' as a device driver
   746994 WARNING    Failed loading lib: dlopen(/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI2/Drivers/libFreenectDriver.0.6.0.dylib, 2): Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: /Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/libFreenectDriver.0.6.0.dylib
  Reason: image not found

   747038 VERBOSE    Couldn't use file 'libFreenectDriver.dylib' as a device driver
   776418 WARNING    LibraryHandler: Couldn't find function oniDriverStreamSetServices in libPS1080.dylib. Stopping
   776448 VERBOSE    Couldn't use file 'libPS1080.dylib' as a device driver
   784467 WARNING    Failed loading lib: dlopen(/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib, 2): Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: /Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/libfreenect2-openni2.0.dylib
  Reason: image not found

   784490 VERBOSE    Couldn't use file 'libfreenect2-openni2.0.dylib' as a device driver
   794858 WARNING    Failed loading lib: dlopen(/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2-openni2.0.dylib, 2): Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: /Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/libfreenect2-openni2.0.dylib
  Reason: image not found

   794878 VERBOSE    Couldn't use file 'libfreenect2-openni2.dylib' as a device driver
   798504 WARNING    Failed loading lib: dlopen(/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2.0.2.0.dylib, 2): Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: /Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/libfreenect2.0.2.0.dylib
  Reason: image not found

   798524 VERBOSE    Couldn't use file 'libfreenect2.0.2.0.dylib' as a device driver
   801924 WARNING    Failed loading lib: dlopen(/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2.0.2.0.dylib, 2): Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: /Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/libfreenect2.0.2.0.dylib
  Reason: image not found

   801947 VERBOSE    Couldn't use file 'libfreenect2.0.2.dylib' as a device driver
   805179 WARNING    Failed loading lib: dlopen(/Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/exampleDebug.app/Contents/MacOS/OpenNI2/Drivers/libfreenect2.0.2.0.dylib, 2): Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: /Users/lepton/Developer/openFrameworks/macOS_0_11_0/examples/ofxNI2/example/bin/libfreenect2.0.2.0.dylib
  Reason: image not found

   805204 VERBOSE    Couldn't use file 'libfreenect2.dylib' as a device driver
[notice ] OpenNI initialised. Version: 2.2.0.33
   805303 VERBOSE    Trying to open device by URI '(NULL)'
   805307 ERROR      Can't open default device - none found
[ error ] ofxNI2: 	DeviceOpen using default: no devices found

Just add these two lines to the script (after the additions you already made)

cp ../../../addons/ofxNI2/libs/OpenNI2/lib/osx/OpenNI2/Drivers/* "$TARGET_BUILD_DIR/"; 
cp -R ../../../addons/ofxNI2/libs/NiTE2/lib/osx/libNiTE2.dylib "$TARGET_BUILD_DIR/libNiTE2.dylib";

I believe you answered that question already here:

On Linux you might be able to blacklist the Intel driver and force the AMD one, but on OSX I have no idea…

Annoying that’s it’s probably easy to implement, they just haven’t done it. You could try that env variable to switch to a different backend, but OpenCL is likely to be faster than all of them I think unfortunately…

Yes, I was wondering if you managed to solve it, I could not. Maybe I will give it another go - lets see if it got smarter.