Blob segmentation / User mask


#1

I need to update an old OF project based on OpenNI. More in details, it uses the user mask feature to avoid blobs merging when users overlap their shape in front of the Kinect.

I would like to keep the project cross-platform and I’m searching for any replacement of such feature that could work with the kinect v1 or v2. I know that the user mask algorithm is based on machine learning and it’s only available in OpenNI or Microsoft Kinect SDK. Of course, I’ve tried to recompile ofxNI2 e ofxOpenNI with OF 0.10 but it crashes. I tried also the solution from this but I didn’t get better results.

I’v read User segmentation/3D blob tracking on multi-kinect data and did some test with ofxKinectBlobTracker but this crashes after a while on OF 0.10.1 OSX 10.11. I have to test it with more than two users, but its instability doesn’t encourage me.

Do you know any other implementation of the “user mask” algorithm?

thanks in advance!!!

Paolo


#2

Honestly, in my average interactive installation projects requiring user mask segmentation, I either:

  • user 0.98 + NiTE if I really, really, really need skeleton tracking
  • perform old school blob segmentation based on size and distance

There are of course alternatives:

  • last year I wrote a "“naif blob skeletonization” (these days it should be implemented in Isadora) module : it was fun and easy to create, good enough for simple scenarios, but not nearly as precise as a NiTE
  • there are Tensor flow/Caffe/etc models that will detect human bodies, but of course it could be tricky to get your 30fps

#3

Thanks very much naus3a,

I don’t need skeleton tracking, just need to keep blobs distinct as viewers overlap, because I have to apply different operations (colors, shape…) to such blobs. See this movie

Do you know some papers or examples about the old school blob segmentation?

Thanks again,

Paolo


#4

got it: will check my links and post here anything that might help :slight_smile:


#5

Great!!! thanks,

Paolo


#6

Hi,
Did you try this addon?
At least it says that it does what you need. I have not tried it, and it is not so outdated.
Let us know how it goes.
cheers


#7

Thanks Roy,

it’s the addon I tryed. More preciesly I tryed ofxKinectBlobTracker that is based on ofxKinectBlobFinder. I have often crashes and I didn’t try with more than two person. I can fix it and I can contact dasaki to get more info. Anyway ofxKinectBlobTracker uses a 3D point cloud to distinguish blobs, while KinectSDK and OpenNI use machine learning, as you can see here. So I’m guessing that the accuracy of ofxKinectBlobTracker cannot be the same… As naus3a says, a machine learning based addon, would not allow 30 fps even if KinectSDK and OpenNI algorithms run fast.

Right now, the only solution is under Microsoft SDK (Kinect V2 and hopefully Kinect v3) or NuiTrack… no open source solution!

Cheers!

Paolo


#8

I have a fork of ofxNI2 here:


It contains the driver for the Kinect V2 and is working cross platform. You may need to change some things to get exactly what you want but it is working well for tracking for me on OSX and windows with OF 0.10


#9

Super,

thanks fresla, this could be the solution

Tomorrow I’ll try it and let you know,

Paolo


#10

Hi fresla,

ofxNI2 runs on OF 0.10 (Windows 10 and OSX 10.11) but I it cannot find any device, so it crashes. I installed OpenNI2 from https://structure.io/openni, remove any Microoft driver to make a test with NiViewer, but still no device found on both Windows and OSX. Last time I used OpenNI, it was on 10.6 a century ago… are there some trick to make OpenNI2 work on today computers?

thanks,

Paolo


#11

Are you using a kinect V2?

If so:

  • Install UsbDk driver
    1. (Windows 7) You must first install Microsoft Security Advisory 3033929 otherwise your USB keyboards and mice will stop working!
    2. Download the latest x64 installer from https://github.com/daynix/UsbDk/releases, install it.
    3. If UsbDk somehow does not work, uninstall UsbDk and follow the libusbK instructions.This doesn’t interfere with the Microsoft SDK. Do not install both UsbDK and libusbK drivers
  • (Alternatively) Install libusbK driverYou don’t need the Kinect for Windows v2 SDK to build and install libfreenect2, though it doesn’t hurt to have it too. You don’t need to uninstall the SDK or the driver before doing this procedure.Install the libusbK backend driver for libusb. Please follow the steps exactly:
    1. Download Zadig from http://zadig.akeo.ie/.
    2. Run Zadig and in options, check “List All Devices” and uncheck “Ignore Hubs or Composite Parents”
    3. Select the “Xbox NUI Sensor (composite parent)” from the drop-down box. (Important: Ignore the “NuiSensor Adaptor” varieties, which are the adapter, NOT the Kinect) The current driver will list usbccgp. USB ID is VID 045E, PID 02C4 or 02D8.
    4. Select libusbK (v3.0.7.0 or newer) from the replacement driver list.
    5. Click the “Replace Driver” button. Click yes on the warning about replacing a system driver. (This is because it is a composite parent.)To uninstall the libusbK driver (and get back the official SDK driver, if installed):
    6. Open “Device Manager”
    7. Under “libusbK USB Devices” tree, right click the “Xbox NUI Sensor (Composite Parent)” device and select uninstall.
    8. Important: Check the “Delete the driver software for this device.” checkbox, then click OK.If you already had the official SDK driver installed and you want to use it:
    9. In Device Manager, in the Action menu, click "Scan for hardware changes."This will enumerate the Kinect sensor again and it will pick up the K4W2 SDK driver, and you should be ready to run KinectService.exe again immediately.