Kalman filter for kinect depth

Hi all!
I’m trying to stabilize the kinect depth image using kalman filter. What i need is a good shape of people and/or object within some range.
I have seen the example in ofxCv addon about Kalman filter and i used sometimes for stabilize some coordinates. Now i’m trying to understand how to use this filter with pixels, but i’m little confused. Somebody has some example or can explain me what are the steps to follow?
For now i use another app that pass me the depth image using syphon. I draw this image inside an FBO and i take pixels using readToPixels. After that, i’m super confused how to implement Kalman filter. :slight_smile:

Hi Mauro,

I don’t think Kalman filter is appropriate for stabilizing pixel values. Instead, Kalman filter is used for tracking a certain object (for instance, a joint in a skeletal model) as it is based on a physics model, but a pixel may belong to any object. What people usually do is probably to take an average with the neighbors or previous time steps.

you could try this instead:


it fills the holes in the depth image.
If thatis not sufficient, you could do some kind of time smoothing; eg each new depth frame, add part of it it to the smoothed output image. You could do this by iterating over all pixels and doing something like:
char smoothedValue = 0.90 * oldValue + 0.10 * valueFromNewFrame;
(shaders could make this much more effective)

Each new depth frame then only accounts for 10% of the final smoothed image. It works wonders when objects/persons are standing still. For faster moving object, try messing with the percentage values.

Hi @micuat, i found some resources online but for me are really hard to implement. But for example here is one:

http://www.slideshare.net/raviteja1926/kalman-filter-applications-in-image-processing 1

Hi @Kj1, thanks for the link, i have to try.
Anyway, in the past i tried to use shader for smooth my depth image. The final result was not bad, because i could obtain a smoother shape of a siluette. But the problem is that, as you say, it works good when persons are standing still. When they move, i obtain a kind of “delay effects”. If i find my code in the next days, i will put it on github.

@Mauro thanks for sharing the slides. It makes sense to use Kalman filter for such applications, but I think Markov random field (MRF) does a similar thing (or maybe the same) and is widely used compared to Kalman filter particularly for image restoration. A quick search on MRF image restoration library shows these projects:
http://vision.middlebury.edu/MRF/eccv06/
http://cs.brown.edu/~pff/bp/