ofxKinectV2 addon

theo · June 27, 2014, 7:59pm

Here is an addon for the new Kinect For Windows sensor.

Notes:
Currently only supports one Kinect V2 at a time.
Kinect needs to have flashed firmware ( this currently needs to be done on Windows 8 )
Requires USB 3 port on machine.
Only tested on OS X though Win / Nix should be possible too with patched libusb ( see: https://github.com/OpenKinect/libfreenect2/blob/master/depends/README.depends.txt )
If you have the ofxKinect ( v1 ) addon in your project remove the ofxKinect libusb lib and use the one that comes with this repo instead.
Depth decoding is not at 30fps as it is decoded on the CPU - GPU decoding is possible, but not stable currently with OF.

bestpaul1985 · June 27, 2014, 10:42pm

Thank you so much Theo, I am going to test it in windows.

bestpaul1985 · June 28, 2014, 12:31pm

Hey Theo,

I patched the libusb in Win8.1.
I use VS2012 and OF 0.8.1.
I got errors.

I am not very sure, if I did right to patch the libusb. I downloaded the patch and put them in ofxKinectV2/libs/libusbx. Is this right?

theo · June 28, 2014, 1:28pm

@bestpaul1985 - check the info here: https://github.com/OpenKinect/libfreenect2/blob/master/depends/README.depends.txt

I think all you’ll need to do is grab the files from the dropbox link:

Download pre-built binaries libusbx-winiso.zip from https://www.dropbox.com/s/madoye1ayaoajet/libusbx-winiso.zip
Unzip into the depends folder such that the resulting folder structure ends up looking like:
./depends/libusbx/
./depends/libusbx/include/
./depends/libusbx/MS64/
Done

You probably could put the libs in a different location - just make sure they are linked / included in the project.

If you don’t get that woking it could also be worth checking out the native ofxKinectCommonBridge which uses the K4W SDK - https://github.com/joshuajnoble/ofxKinectV2 ( sorry @joshuajnoble for duplicating your add-on name, I should off searched githubf first )

jvcleave · June 28, 2014, 5:45pm

I like the name ofxOpenKinect - rock paper scissors for the name rights?

joshuajnoble · June 29, 2014, 9:23am

No worries Theo, I think I’m going to roll it into the ofxKinectCommonBridge library and have that be configurable to be for Kinect V1 or V2 since that’s what it is. I think an addon not leveraging the K4W stuff is more deserving of the more simple name since the library itself is more simplified.

theo · July 2, 2014, 7:18pm

Ahh - okay : )

Sounds good and does make sense that the official Kinect stuff stays with a similar naming convention as the V1 addon and the same for the freenect wrapper.

Excited to give it a whirl!

theo · July 6, 2014, 4:17pm

Actually this might work a bit better for your needs:

And huge apologies to josh as he already had an Addon with the same name.

silverbahamut · July 7, 2014, 7:12pm

@joshuajnoble do you have any visibility in a (public) code release of the KCBv2 ? I’m thinking of switching to your addon but might get stuck if I can’t modify things (as I had to with the KCBv1)

joshuajnoble · July 7, 2014, 8:36pm

I know that MS Open Tech will be opening it up but I do not know when it’s planned. The header and the library for their Kinect Addon should be reference enough to show the functionality provided and you can always just add in a reference to the actual Kinect handle itself and call the MS Kinect SDK methods directly. Does that help at all?

silverbahamut · July 8, 2014, 12:06am

Thanks Josh. It makes sense but I think I’ll probably wait for the dust to settle a bit before doing the switch (as shipping bleeding edge code for production wouldn’t be the best practice ever…).

Jordi · July 28, 2014, 12:07pm

Thank you so much for this addon Theo!

I’m trying to make it work in my mac (OSX 10.9.4) but i can’t manage to open the device.
All I did was to run the SDK in Windows 8 and I did see the depth image there.
Then I runed the project as it is in:

which has everything set up.

Do I have to do anything else to update the firmware?
Kinect has no lights yet when plugged in my mac.

output is:

Using libusbx v1.0.17.10852
Opening device 045E:02C4...
ofProtonect::openKinect  Failed. - handle is NULL

theo · July 28, 2014, 12:32pm

Hmm could be a couple of things.

Does your Mac have USB 3.0?
Also you may have a newer Kinect, could you open the System Profiler app ( I think its in Applications/Utilities ) and see what Product ID and Manufacture ID show up for the Kinect when its plugged in? It may be different from 045E:02C4
Maybe also paste the contents of the device listed in the System Profiler

Thanks!
Theo

Jordi · July 28, 2014, 12:36pm

Thank you very much!

Indeed is the pid, i change it to 02D9 and i went through that one.
Now it complains about a buffer size… i think i recall someone with the same issue…

here is the output:

Opening device 045E:02D9...
libusbx: warning [darwin_open] USBDeviceOpen: another process has device opened for exclusive access
libusbx: error [darwin_reset_device] ResetDevice: device not opened for exclusive access
             speed: 480 Mbit/s (USB HighSpeed)
active configuration: 1, err: 0
Claiming interface 0...
libusbx: error [darwin_claim_interface] IOCreatePlugInInterfaceForService: unknown error
   Failed: -99.

Claiming interface 1...
libusbx: info [darwin_claim_interface] no interface found; setting configuration: 1
libusbx: error [darwin_claim_interface] could not set configuration
   Failed: -4.
running kinect...
Control transfer 1 - set isoch delay
Control transfer error: -9
Control transfer 2 - set sel u1/u2
Control transfer error: -9

Setting interface alt setting...
Setting stream status: Disable
Set stream status error: -5
   Failed: -1.
Control transfer 3 - enable u1
Control transfer error: -9
Control transfer 4 - enable u2
Control transfer error: -9
Control transfer 5 - set feature 768
Setting sensor status: Disable
Set sensor status error: -9
Kinect init done

iso max_packet_size: 4294967291
libc++abi.dylib: terminating with uncaught exception of type std::bad_alloc: std::bad_alloc

Jordi · July 28, 2014, 12:46pm

the device is recognised as Hi-Speed instead of Superspeed. that could be the issue…

Probably i need to plug it through thunderbolt…
I will have a look at that and come back.

Thanks for the help!

Jordi · July 28, 2014, 1:38pm

Well it seems that either my mac mini late 2012 and my macBook pro early 2013 don’t want to take the Kinect2 as a “USB 3.0 SuperSpeed” but as “USB 3.0 Hi-Speed” instead.

It seems that thunderbolt to USB adaptor is not a good idea but i haven’t tried.

Please let me know if anybody is having the same problem.

Thanks!

theo · July 28, 2014, 3:47pm

It might be worth trying a different USB port.
On my iMac I can use some USB ports and not others to get USB 3.0 super speed.

Jordi · July 28, 2014, 3:59pm

I did try all of them, but no luck.

Even if my computers are a bit old they are “USB3 Superspeed” capable so I’m afraid something else is going on… maybe my newer kinect has a patch to prevent non windows machines?

Could you please post the output of your initialization? Does Kinect appear in System Informaion in the Superspeed Bus?

Thanks Theo! let me know if you prefer to keep this as an issue at git.

theo · July 28, 2014, 4:51pm

When did you get the Kinect?

Mine was one of the earliest preview models.
It shows up under USB 3.0 SuperSpeed Bus as: Xbox NUI Sensor

  Product ID:	0x02c4
  Vendor ID:	0x045e  (Microsoft Corporation)
  Version:	 1.00
  Serial Number:	166802534347
  Speed:	Up to 5 Gb/sec
  Manufacturer:	Microsoft
  Location ID:	0x15100000 / 12
  Current Available (mA):	900
  Current Required (mA):	36

Can you paste the same info for yours?
I wonder if the Product ID might be different.

Here is the console output when it starts up for me.

Using libusbx v1.0.17.10852

Opening device 045E:02C4...
             speed: 5000 Mbit/s (USB SuperSpeed)
active configuration: 1, err: 0
Claiming interface 0...
libusbx: info [darwin_claim_interface] no interface found; setting configuration: 1

Claiming interface 1...
running kinect...
Control transfer 1 - set isoch delay
Control transfer 2 - set sel u1/u2

Setting interface alt setting...
Setting stream status: Disable
Control transfer 3 - enable u1
Control transfer 4 - enable u2
Control transfer 5 - set feature 768
Setting sensor status: Disable
Kinect init done

Jordi · July 28, 2014, 7:35pm

I got it this month in Norway.
I tried all usb ports a couple of times, just to make sure.
Didn’t work with my laptop either but here is the mac mini:
And product id it is indeed different! check it out:

My specs:

Mac mini
Late 2012
Processor  2,5 GHz Intel Core i5

result at system info:

USB 3.0 Hi-Speed Bus:
[...]

NuiSensor Adaptor      :
  Product ID:	0x02d9
  Vendor ID:	0x045e  (Microsoft Corporation)
  Version:	 0.70
  Speed:	Up to 480 Mb/sec
  Manufacturer:	Microsoft Corporation  
  Location ID:	0x14100000 / 5
  Current Available (mA):	500
  Current Required (mA):	0
  
Xbox NUI Sensor:
  Product ID:	0x02d8
  Vendor ID:	0x045e  (Microsoft Corporation)
  Version:	 1.00
  Serial Number:	504906442542
  Speed:	Up to 480 Mb/sec
  Manufacturer:	Microsoft
  Location ID:	0x14110000 / 14
  Current Available (mA):	500
  Current Required (mA):	100

output:

[ error ] ofXml: loadFromBuffer(): DOM ERROR
[warning] ofXml: setTo(): empty document
Using libusbx v1.0.17.10852

Opening device 045E:02D9...
libusbx: warning [darwin_open] USBDeviceOpen: another process has device opened for exclusive access
libusbx: error [darwin_reset_device] ResetDevice: device not opened for exclusive access

Reading device descriptor:
            length: 18
      device class: 9
               S/N: 0
           VID:PID: 045E:02D9
         bcdDevice: 0070
   iMan:iProd:iSer: 1:2:0
          nb confs: 1
             speed: 480 Mbit/s (USB HighSpeed)
active configuration: 1, err: 0
Claiming interface 0...
libusbx: error [darwin_claim_interface] IOCreatePlugInInterfaceForService: unknown error
   Failed: -99.

Claiming interface 1...
libusbx: info [darwin_claim_interface] no interface found; setting configuration: 1
libusbx: error [darwin_claim_interface] could not set configuration
   Failed: -4.
running kinect...
Control transfer 1 - set isoch delay
Control transfer error: -9
Control transfer 2 - set sel u1/u2
Control transfer error: -9

Setting interface alt setting...
Setting stream status: Disable
Set stream status error: -5
   Failed: -1.
Control transfer 3 - enable u1
Control transfer error: -9
Control transfer 4 - enable u2
Control transfer error: -9
Control transfer 5 - set feature 768
Setting sensor status: Disable
Set sensor status error: -9
Kinect init done

iso max_packet_size: 4294967291
libc++abi.dylib: terminating with uncaught exception of type std::bad_alloc: std::bad_alloc