How to use data from Asus Xtion Pro in OpenCV

Hello friends! I have an Asus Xtion Pro depth sensor currently and want to use Haar cascades to do some upper body tracking on the Raspberry Pi, which has oF on it. On the OpenCV site, it says that HighGUI ( is the preferred way to get data from the sensors for use by OpenCV. However, based on this post (highgui on the ofxopencv add-on) it seems as if HighGUI is not needed in oF. How then do I get the data? @jvcleave @bakercp @danielJay

Is this for the raspberry pi (Debian Linux)? Mac? Windows?

Had you tested the basic webcam examples?

You’re using a depth camera… Are you asking about how to retrieve depth information? If that’s not an issue, you just need a regular webcam and a 2D skeleton tracker.

This is for the Pi and, yes, I am using a depth sensor. For Haar, don’t you need depth info?

What I basically need is to use the Xtion Pro for upper body tracking but I have never used the RPi, OF, OpenCv, or xtion pro before so I am having a ton of difficulty getting a hang of things. Through research, it seems Haar is the best way to do what I need. However, I do not know how to use OF and ofxOpenCv to do it.

i’d do a read of learning Open CV (im sure people have their preference, this book enlightened me a bit to the lay of the land with openCV (a formalized structure for computer vision basics)

I’d also strongly read an article from Golan Levin :

its a great short read on introducing open CV.

Alright I will go through these. Is there anything specific I should know about OpenCV within OF however? Do the books cover OF? And finally due to a deadline I need to know how to collect data from the sensor as I said before. How do you collect live video data for opencv processing?

So to use the Xtion on the Pi you have to use OpenNI2 as it has a driver that allow you to get the depth info (works ok at 320x240)

The last time I did this was a while ago with this project

Now beware that OpenCv is not GPU optimized so you will be using the much less powerful CPU on the Pi. Using Haar is pretty processing intensive so it will be slow.

If you just need a camera feed you can use a webcamera that doesn’t have a depth sensor on it and make things easier on yourself. Like Jason mentions, Haar tracking can be fairly slow, the smaller you make the image, the faster your application will run. There are Haar cascades for upper body tracking so those seem simpler than trying to use an Xtion.

Okay so with the webcam, I would just need to use ofvideograbber to get the feed and then use ofxOpenCv for Haar? However for my application, I need to not only detect an upper body but also it’s relative distance from the sensor because I am trying to make something that detects an upper body in front of it and as that body moves farther away, motors connected to the pi’s gpio pins will move faster. If a non-human is in front of the camera, the motors won’t move. Would a webcam be able to do that? If anyone could link me to some example Haar analyses with a normal webcam, I would also appreciate it. Additionally, the camera will be static now but I plan on making it mobile later on

Given what you’re describing, I think you probably need to use both the depth and the RGB image then because it’s quite difficult to estimate distance from a camera without either a stereo setup or a depth camera. You can pass an OpenNI image to OpenCV fairly easily, so you’d pass the RGB image to OpenCV and if you find a face or an upper body, then average the depth pixels at the location where the face or body is detected.