GStreamer 1.4 / OpenGL Pipeline in OF

I’ve recently installed the GStreamer 1.4 Framwork on OSX. The aim is to use a OpenGL Pipeline directly in OF, which I assume, could be very quick and accelerated.

There are some OpenGL Examples on the gStreamer GIT repo:

I had to realize that my knowledge is somehow limited. Maybe somebody likes to join by having a look at the examples? Which one could be a good starting point? There will be threading/OpenGL issues since gStreamer will run in its own thread. That means creating new OpenGL Contexts (maybe like in the Clutter or SDL Example above).
On the other side, there are different approaches: Using either glImageSink directly or fakeSink and getting the Textures from a glfilterapp. But I’m not sure where the differences are.

Could be a cool extension for OF, since gStreamer is easy to install, platform independent and with openGL throughout accelerated.

All the best

i’ve been looking into this but i don’t think it’ll make much difference compared to the current implementation in osx. in linux there’s vaapi elements that’ll do the decoding in the gpu but in osx i think the only difference is that the colorspace conversion will happen in the gpu which the latest master of OF already does if you use the programmable renderer and use OF_PIXELS_NATIVE as pixel format.

anyway, i’ll look into it once ubuntu 14.10 is released in october which will have gstreamer 1.4 as default

Thanks for your reply.

I have the problem that I like to implement a cross platform gapless video player with
network triggering.

I’d like to rely on gStreamer since it’s running well on osx too.

To achieve better performance on linux, I used GTK+ 3 together with OF and gStreamer, running a windowless OF app doing the whole GTK output in a player thread. The appsink approach is slow on old hardware, and with GTK i can use the xoverlay infrastructure of gStreamer.

Now in terms of cross-platform independency I’m stuck somwhere in the middle: GTK on OSX is kind of awkward to port (although it runs, but also crashes). VIdeo Playback on Linux in OF is kind of slow.

What to do? I’m lacking totally the best practice knowledge. Maybe I’ll need to implement an osx and linux version. With avkit on osx and gStreamer openGL on Linux. That’s why I’m looking into the openGL pipeline.

well the opengl pipeline is doing what OF does internally gets a buffer and uploads it to a texture, the only difference is that the colorspace conversion is done in the graphics card but as i said that’s also working now in OF in the master branch. so i doubt there will be any difference, the only way to make it faster would be to use pbos to upload the texture or have decoding in the graphics card which for osx is not possible with gstreamer as far as i know.

also if all you need to do is to play videos you can use the native version for each OS, gstreamer for linux and AVFoundation for osx since OF already hides the implementation details making it multiplatform

using the OF videoplayer is indeed perfectly multiplatform, but my problem is the necessity of gapless and responsive playback. I’m playing clips from a couple of frames up to several minutes.

Maybe I’ll check out ofxThreadedVideo and check how fast it runs on linux with the OF master branch. I worked with a similar approach and was a little bit disappointed by the linux performance. Maybe it’s possible to port that addon to avfoundation as well.