Courses to Understand Depth Cameras

What would be the best coursework to understand depth cameras like Kinect and the software written for them? I’ve found some tutorials like this one for Microsoft official Kinect 2, Daniel Shiffman’s “Getting Started with Kinect and Processing,” the great ofBook Computer Vision chapter, and I’ve worked with the Kinect libraries like ofxKinectForWindows2. However, even after completing the courses and using the library, I feel a very shallow understanding of what’s happening. For example, if I’m calibrating the camera… I have no idea what’s actually happening, and if anything goes wrong, I’m completely lost.

Would my best bet be Computer Vision? Photogrammetry? I can’t seem to find any courses that eventually lead to depth cameras. Studying SDK’s in depth? Or am I focusing too much on stuff that can/should be handled by libraries?

I’ve been working with Kinect since 2011 and I still do it. The best book has been my practice and the OF Forum. I just read some stuff on the web and play some video on youtube. Most of the time I spent testing the software in my studio and having experience with the audience in the public events.

You should divide the problem in two parts: working only with the depth frame and blobs, or working with high level features like skeleton tracking and gesture recognition.

In the first case you just need to play with ofxKinect or ofxKinectForWindows2. Once you have the depth frame you can do simple image processing or blob detection with ofxOpenCv for handling 2D stuff. In the second case you need platform specific library and things are a little bit more complicate.

You need calibration only if you want to match the depth frame with the rgb frame, but ofxKinect does it for you.

Regards,

Paolo

1 Like

Thanks so much for replying, Paolo! Your work is very impressive. I have a follow-up question if you have time.

I saw on your site that you have a degree in computer science. I’m learning C++ along with oF, and reviewing Linear Algebra. Do you think any other computer science or math courses would be helpful?

Hi lookingout,

The Italian university has a very theoretical approach to computer science. I learned to program afterwards in a software house. Today I teach Interaction Design in a private university and the approach is totally different, much more practical.

I would do a university course (or any equivalent) in computer science, paying attention to study c++, logic, linear algebra, geometry and algorithm theory.

What is important to learn is not only the subject itself, but the method of study that allows you to learn whatever notion you need to learn in the course of your career. That’s why, imho, the context is very important (unviersity, company, web, friends…). Don’t study alone if you’re having a hard time.

Good luck!

Paolo

1 Like

Paolo - thank you for all of your advice. It’s invaluable to hear from an expert (and professor!)

You are welcome, here there are no professors or students, we are all Openframeworks enthusiast!

1 Like