VideoPlayerExample on RaspberryPi

arm
#1

Hi everyone,

I have made a stripped down version of the VideoPlayerExample on Raspberry Pi to experiment with performance of video playback. I’m making a VJ app.

Raspberry Pi Model B2.
Wheezy 3.18.5+.
Current OpenFrameworks.

The code is stripped right down. I’ve removed all the UI stuff. I’m drawing to a 500x500 surface. I’m using my own movie file. I’m running this in single user mode.

in setup()

fingerMovie.loadMovie("movies/film.mp4");
fingerMovie.play();

in update()

fingerMovie.update();

in draw()

fingerMovie.draw(0,0,500,500);

The movie I’m using is the first one, the mp4 file, here:
https://www.dropbox.com/sh/jkd2mffndvsecw4/AAC6kKoCqSiv_7DBFWSCHygja?dl=0
It is mp4, h264, 480x240.

The result is highly erratic.

Sometimes I’m getting a “cannot get pipeline caps” error.
Sometimes I’m getting a repeating error to do with “preroll”.
Sometimes I’m getting just a system freeze, before or after the background draws.
Sometimes it works!!

Short of fully understanding the problem, I’m wondering if anyone can suggest workarounds or some error reporting tricks that would help me build a system that is robust and predictable.

#2

You’ll want to use ofxOMXPlayer for Video Playback on the Pi to get Hardware Decoding.

#3

Thanks Asper, but the absolutely essential part of my project is precise varispeed control. I haven’t tried OMXPlayer out properly yet but I understand that you cannot set speed to an arbitrary float, only increments of 0.5x.

Although the ofVideoPlayer is implemented in an inefficient way, playing a video at a certain speed is all I need to do, and I have seen that it can do this. In my tests, when it works it works very well.

#4

there’s lots of fixes to the linux video player in OF master in github if you are not using that. also in terms of performance the main problem with the current video player is not being hw accelerated, which it is, but that the colorspace conversion is done in the cpu. in master you can also avoid that by doing:

fingerMovie.setPixelFormat(OF_PIXELS_NATIVE);
fingerMovie.loadMovie("movies/film.mp4");
fingerMovie.play();

which will give almost the same performance as omx player.

if you are already using master can you check which version of gstreamer you have using: gst-launch-1.0 --version?

Play video controlling speed and direction
#5

Hi Arturo,

that sounds extremely promising. Thanks. I’ll try this as soon as I can get back to my HDMI monitor.

Do you think that the poor performance is the underlying issue behind all of the erratic behaviour?

#6

no, it shouldn’t. i think i fixed something like what you describe for gstreamer 1.4 that wasn’t happening in 1.2 but not sure wheezy is already on 1.4

#7

N00B C++/OF question, but where is OF_PIXELS_NATIVE?

#8

Also,

if you are already using master can you check which version of gstreamer you have using: gst-launch-1.0 --version?

Do you mean git master branch? I’m cloning now (taking hours to download).
But I don’t have gst-launch-1.0 command.

Compiling for Raspberry Pi 2
#9

Me again.

I have now cloned openFrameworks from git on the RPi but it failed on compile. I found the thread on github discussing the error I have (https://github.com/openframeworks/openFrameworks/issues/3529).

Meanwhile, I wait for my new 6x faster Pi to be delivered :wink:

There is much reason to be hopeful.

#10

@arturo the videoPlayer example runs great on rpi2 with setPixelFormat(OF_PIXELS_NATIVE)- thanks for the tip. Only issue is the video is black and white (tested with multiple videos)- any ideas?

#11

Unless you are developing you are much better off downloading the zip version from github

#12

It was only because I was using the hot new improvements from @arturo.

#13

Those are in the master zip tho - it strips out all the history that makes it so large

#14

OF_PIXELS_NATIVE makes the video pipeline return the native format of the video or camera which is usually some YUV format avoiding the colorspace conversion in the CPU which is usually quite expensive. OF when using the default renderer only shows the first YUV plane which is Y the lunminance so you only see a black and white image. if you change to the programable renderer changing main to:

ofGLESWindowSettings settings;
settings.setGLESVersion(2);
ofCreateWindow(settings);
ofRunApp(new ofApp);

the programmable renderer will internally use a set of shaders to do the conversion from YUV to RGB in the graphics card

1 Like
Video playback problem with RPi4
#15

@arturo thank you. HD video runs smooth (and full colour) at 60fps with GLES2. Also seeking/scrubbing with setPosition() is finally bearable (less than 50ms).

#16

@arturo I just hit another wall with the new gst video player- the app freezes when using setPosition() intermittently (maybe 1 in 10 calls) and I have to force quit. It doesn’t seem to be memory/ram related (i have 384mb split for video memory as suggested by Jason).

I’m syncing video across 6x rpi2’s, and it’s almost there. I could switch back to omx, but the delay when seeking is too large and I can’t get perfect sync. @jvcleave if there is a way around this- i’d love to know.

#17

In trying to implement seeking I found a lot of the limitation is in the codec (mainly h264)

it makes it easier if you compress the video with a keyframe every frame. Apple Compressor was the most reliable in doing this

#18

thanks - ill give it a shot.

#19

I am using this

ofGLESWindowSettings settings;
settings.setGLESVersion(2);
ofCreateWindow(settings);
ofRunApp(new ofApp);

in combination with

videoPlayer.setPixelFormat(OF_PIXELS_NATIVE);

and I am getting a completely blank window. Event native openFrameworks draw functions do not work. I have Gstremer 1.4.4-2. Although gstreamer1.0=omx is at version 1.0.0.1-0+rpi18rpi1g Could that be the problem?

#20

I’m searching for this solution, I’ve been trying ofxOMXPlayer but it doesn’t compile in of_10 anymore.
That code change is applicable to of_v11 in raspbian-buster?