i am trying to decode H264 video with OF 0.11.2 on a RPi4 with 32bit bullseye / full KMS drivers (on Pi4, but should be equivalent on Pi3).
The videoplayer exemple compiles and run, but as soon as i play fullscreen FullHD video, it’s very laggy.
gst-launch-1.0 playbin uri=file:///home/pi/media.mp4 video-sink="kmssink"
And it runs smoothly with almost no CPU usage.
So the installed Gstreamer is properly working with v4l2h264dec and properly leverage hardware accel.
I found a different pipeline that create the same laggy result i can see in OF:
gst-launch-1.0 filesrc location=media.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! videoconvert ! "video/x-raw,format=RGB" ! kmssink
So it seems what makes it laggy in OF is the RGB conversion.
The problem can be partially fixed by replacing videoconvert with v4l2convert:
it’s almost smooth, but not as smooth as the dry playbin version.
So my questions are:
Is there a way to dump the actual pipeline selected by gst-launch-1.0 playbin ?
How to enforce v4l2convert instead of videoconvert in OF ?
(i tried tinkering with ofGstVideoPlayer::createPipeline without success…)
Or maybe we can avoid color conversion → but how to work with I420 in OF ?
If i set movie.setPixelFormat(OF_PIXELS_I420); i end up with a green display and this error:
ofGstVideoUtils: buffer_cb(): error on new buffer, buffer size: 3133440!= init size: 3110400