reducing video latency

if you set the log level to verbose, the videograbber will show all the available modes with their framerates, i think the problem is that you can have a list of modes like:

rgb 30fps
rgb 60fps
yuv 30fsp
yuv 60fps
yuv 85fps
yuv 100fps

but the videograbber is choosing the fastest rgb instead of the fastest of all of them.

OK, that info wasn’t being printed with the gstUtils stuff, I went back to vidgrabber and got this.
Looking at the last few lines it seems it inits with 320x240 at 100fps.

  
  
OF_NOTICE: Probing devices with udev...  
OF_NOTICE: Found device 1415:2000, getting capabilities...  
OF_NOTICE: Detected v4l2 device: USB Camera-B4.04.27.1  
OF_NOTICE: Driver: ov534, version: 134144  
OF_NOTICE: Capabilities: 0x05000001  
  
(<unknown>:4025): GStreamer-WARNING **: pad source:src returned caps which are not a real subset of its template caps  
OF_VERBOSE: Device: USB Camera-B4.04.27.1 (/dev/video0)  
  
  
(<unknown>:4025): GStreamer-WARNING **: pad source:src returned caps which are not a real subset of its template caps  
OF_NOTICE: video/x-raw-rgb 640 x 480 framerates:  
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_NOTICE: 15/1   
OF_NOTICE: video/x-raw-yuv 640 x 480 framerates:  
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_NOTICE: 15/1   
OF_VERBOSE: already added, skipping  
  
OF_NOTICE: video/x-raw-rgb 320 x 240 framerates:  
OF_NOTICE: 100/1   
OF_NOTICE: 75/1   
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_NOTICE: video/x-raw-yuv 320 x 240 framerates:  
OF_NOTICE: 100/1   
OF_NOTICE: 75/1   
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_VERBOSE: already added, skipping  
  
OF_NOTICE: video/x-raw-rgb 640 x 480 framerates:  
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_NOTICE: 15/1   
OF_VERBOSE: already added, skipping  
  
OF_NOTICE: video/x-raw-yuv 640 x 480 framerates:  
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_NOTICE: 15/1   
OF_VERBOSE: already added, skipping  
  
OF_NOTICE: video/x-raw-rgb 320 x 240 framerates:  
OF_NOTICE: 100/1   
OF_NOTICE: 75/1   
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_VERBOSE: already added, skipping  
  
OF_NOTICE: video/x-raw-yuv 320 x 240 framerates:  
OF_NOTICE: 100/1   
OF_NOTICE: 75/1   
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_VERBOSE: already added, skipping  
  
OF_NOTICE: video/x-raw-yuv 640 x 480 framerates:  
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_NOTICE: 15/1   
OF_VERBOSE: already added, skipping  
  
OF_NOTICE: video/x-raw-yuv 320 x 240 framerates:  
OF_NOTICE: 100/1   
OF_NOTICE: 75/1   
OF_NOTICE: 60/1   
OF_NOTICE: 50/1   
OF_NOTICE: 40/1   
OF_NOTICE: 30/1   
OF_VERBOSE: already added, skipping  
  
OF_NOTICE: ofGstUtils: selected format: 320x240 video/x-raw-rgb framerate: 100/1  
OF_NOTICE: gstreamer pipeline: v4l2src name=video_source device=/dev/video0 ! video/x-raw-rgb,width=320,height=240,framerate=100/1 !  ffmpegcolorspace ! appsink name=sink  caps="video/x-raw-rgb, width=320, height=240, bpp=24"  
  
(<unknown>:4025): GStreamer-WARNING **: pad video_source:src returned caps which are not a real subset of its template caps  
OF_VERBOSE: ofVideoGrabber: initied  
  

can you measure the latency with gst-launch using rgb as camera format instead of yuv and see if it’s slower?

I figured out how to reduce the latency when using ofxGStreamer on macOS.

Right after this line:

I added:
g_object_set(G_OBJECT(gstPipeline), "latency", 0, (void*)NULL);

which made the videoPlayer work pretty well.

    player.setPlayer(ofPtr<ofGstVideoPlayer>(new ofGstVideoPlayer));
    player.load("rtsp://root:passw@198.168.0.101/axis-media/media.amp?resolution=800x600&latency=0");
    player.play();
1 Like

Thats great! do you think it should be like this as default?

I am not very familiar with gStreamer and how OF connects with it for other functions. I have a feeling that the default 2 second latency might be used for video file scrubbing.
In my case I am accessing an Axis IP camera with build in microphone. For my use case latency=0 is ideal.

This can be set when setting a custom pipeline with setPipeline but perhaps it would be useful to have a method that would set a latency. Not sure if this would work while the pipeline is already playing so such method might need to be called before setup if that’s not the case

1 Like