openCV tracking echo


I’m placing a camera above an LCD monitor, placed horisontally.
I want to track small paper frames (like photo negative frames) placed on
the surface of the screen. I then want to display video loops inside the
frames, so when you move those frames around the video stays inside.

I manage more or less to track the objects. However strangely enough it
works quite well, when “hole” detection is enabled:

contourFinder.findContours(grayDiff, 20, 24000, 5, true);

when last parameter is set to false, detection becomes much more sketchy.

That’s my first question. Why ?

Second issue: Obviously since my camera is looking at the display, everything
I get out there, gets detected and tracked. For now I don’t manage to place my video
exactly below the frame, so it’s often quite off… and it gets picked up by camera and
gets tracked as well! So… the obvious solution is to substract what’s on display from
camera feed. This way I track only physical objects and not what happends on the screen.

My second question: is there a way to access video memory and do this substruction
in an effectient manner ?

Thanks for reading and let me know if you have answers for any of those questions…

Thanks, M

For debugging reasons (for now I don’t manage to get the video placed exactly
below my frames),

I get very unstable tracking results. I don’t quite understand it, cause it
normally should make no difference.

the easier way to get rid of what there is in the screen is to use an infrared or a polarizing filter in the camera any of them will make the camera not ‘see’ the image in the screen.


as arturo said - just use a (linear) polarizing filter in front of your camera. You should get them in common photography stores ether as an lens attachement (out of glass and expensive) or as a foil. If your going to use a infrared bandpass filter you first have to illuminate the scenery with ir light which makes it a lot more complicated then just using a polarisation filter.

The polarisation filter works because the light emitted from your lcd is already polarized because of the inner constructions of the sceen. If you hold the filter in front of your sceen (before the lens of your camera) you have to roted it until the light from the lcd gets blocked. more on the subject of polarisation on wikipedia

excellent advice! thank you very much!

Very nice. I was wondering abour arturo advice, why polarizing filter would do any good.
Now have the answer. Ok - I’ll do that, because as you said illuminating with IR light and modifying
camera seems much more complicated. Thank you much for your explanations.

actually depending on your setup an infrared light can be any plain incandescent bulb there is in the room. And usually an infrared filter is cheaper than a polarized one. About modifying the camera, both solutions require the same: putting a layer of the filter in front of the lens.

:slight_smile: cool, yeah that should work

Well. I guess we didn’t think hard enough…
Polarizing filter works great - camera doesn’t see the screen anymore.
However… the side-effect of this is that camera doesn’t see the object,
since screen doesn’t act as a lightbox anymore!!! :slight_smile: :slight_smile: :slight_smile:

the camera will still see the object, because the light loses its polarization as it passes through or bounces off anything other than the air.

here’s a photo-tutorial for creating a polarized camera