Video input/output latency

Hi all,

I’ve got a project coming up where we have to capture PAL video, process it, and output it again as PAL video. We should get as close to realtime as possible.

The concept works with a webcam as input and VGA as output. I noticed however a lot of delay, which made me search the forums here.

There are a lot of threads on the forum dealing with this, eg http://forum.openframeworks.cc/t/reducing-video-latency/5931/0

So; things I did for reducing the delay:

This results in a latency of ± 120ms; which is unacceptable. It seems that OF’s double buffering has something to do with it; as well as the USB webcam input which buffers in itself, and the video output to LCD screen.

The main question: can this be resolved in the case of pal input/output with external hardware? This would bypass the latency in the USB cam, OF’s buffering and the videocard/LCD buffering.

I thought of the blackmagic intensity shuttle (thunderbolt): http://www.blackmagic-design.com/products/intensity/models/

Can this device (or others) be used simultaneously for input and output? If so, I would proceed as follows:

  • make a threaded grabber/output that first reads the pixels as soon as avialable
  • stores them in the buffer
    – processing thread gets to work
  • reads in another buffer of the previously processed frame (not the one just stored)
  • writing this to the PAL output

Assuming the processing thread always takes less then 40ms/frame (pal = 25fps); this would result in a stable latency of 1 frame. I believe that this would be the theoretical lower limit for the latency.

Any help is very much appreciated!

Thanks,
Kj1

the theoretical lower limit is not just based on your processing time (40 ms) but also on how many framebuffers you have in the way.

if the video capture device has a framebuffer (probably does), that’s 1 frame of latency. if the display has a framebuffer (probably does) that’s another frame of latency. so we’re talking about 2 frames at 25 fps = 80 ms, plus another 40 ms, that’s 120 ms.

in other words, it sounds like you’ve written all your code correctly, and you’re just hitting a limitation of your hardware. i don’t have much experience with devices like that blackmagic one. maybe call them up and ask what they think about getting minimum latency? my guess is that blackmagic is aimed more at editors than broadcasters, though – when you’re editing video, latency isn’t an issue.

one of the big issues is that you’re dealing with PAL. if it was something that ran at 60 fps instead of 25, then you would more than halve your latency.