High CPU usage gStreamer OSX

Hi everybody,

I’m running a 640x480 Pixel rtsp stream with gStreamer on OSX, using @arturo’s ofxGStreamer addon. CPU consumption of the stream is pretty high with around 60%. If I drop the appsink and use autovideosink instead (for testing purposes):

GstElement * gstSink = gst_element_factory_make("autovideosink", "app_sink");

CPU usage drops to around 10-12%. Of course, gStreamer opens then its own openGL window then and all video data is not passed to oF.

Is there a possibility to speed up that process on openFrameworks? I saw that openGL is experimental, did anybody try to use #include <gst/gl/cocoa/gstgldisplay_cocoa.h>?

Best
Urs

usually the main difference with an autovideosink pipeline is in the colorspace conversion. OF by default adds a last element to the pipeline to convert to rgb which can be expensive. 60% sounds like a lot for such a small video though

if you set the pipeline using OF_PIXELS_NATIVE for the video format then OF will do the color space conversion in the graphics card which will save a lot of cpu. This will only work with opengl 3 or greater

Thanks!
I tried player.setPixelFormat(OF_PIXELS_NATIVE); which results in 40% less CPU usage, but also b/w only videos (which i could totally live with).

I’m running 6 players simultaneously in a vector, always displaying only one; this works perfect without the pixel format change, with OF_PIXELS_NATIVE the result is odd, showing all movies at the same time flickering around…

As i said this only works in opengl 3+ if you see the image in b&w the you are using opengl 2. try setting the opengl version in main like:

#include "ofMain.h"
#include "ofApp.h"

//========================================================================
int main(){
	  ofGLFWWindowSettings settings;
	  settings.setGLVersion(3,3);
	  ofCreateWindow(settings);
	  ofRunApp(new ofApp);
}

not sure where the flickering is coming from but it might be related

Thanks!

Switching the openGL version indeed restored the colors - but the weird flickering still remains.

    for(int i = 0; i < csv.getNumRows(); i++) {
        ofVideoPlayer p;
        p.setPlayer(ofPtr<ofGstVideoPlayer>(new ofGstVideoPlayer));
        p.setPixelFormat(OF_PIXELS_NATIVE);
        p.load("rtspurl...");
        players.push_back(p);
    }

in the update and draw loop there’s only the update and draw function.

the active player is a pointer to an element in the vector…

player = &players[player_active];  //eg. 0
player->play();

I do some more digging, though…

this is working for me fine using OF from github (or the nightly builds) not sure if the problem might have been fixed since 0.9.8:

#include "ofConstants.h"
#include "ofBaseApp.h"
#include "ofAppGLFWWindow.h"
#include "ofAppRunner.h"
#include "ofVideoPlayer.h"

class ofApp : public ofBaseApp
{

	std::vector<ofVideoPlayer> videos{6};
	size_t current = 0;

public:
	void setup() override {
		for(auto & video: videos){
			video.setPixelFormat(OF_PIXELS_NATIVE);
			video.load("fingers.mov");
			video.play();
		}

	}

	void update() override{
		for(auto & video: videos){
			video.update();
		}

	}

	void draw() override {
		videos[current].draw(0,0);
	}

	void keyPressed(int key) override {
		current += 1;
		current %= videos.size();
	}
private:
};

int main()
{
	ofGLFWWindowSettings settings;
	settings.setGLVersion(3,3);
	ofCreateWindow(settings);
	ofRunApp(new ofApp);
}

I still have the same flickering with the most recent nightly build. I guess it has something to do with the gStreamer implementation - I’m using gStreamer on OSX because of the RTSP implementation.

p.setPlayer(ofPtr<ofGstVideoPlayer>(new ofGstVideoPlayer));

The flickering happens also only for (bad) streaming resources, like webcams with more or less reliable streams. I assume, that whenever an incomplete frame is streamed or some visual errors happen, the texture is corrupted.

(probably I need to switch to linux)

i’m on linux so my code is using gstreamer already without the setPlayer bit but i don’t think there should be any difference with osx apart from that

i have only tried with video files, perhaps if you post a couple of addresses of streams that are being problematic i can look into it

just compile my code on linux (ubuntu) - the flickering does not happen, with the same rtsp sources. so it must be something osx/gstreamer specific. Got them from the shodan search engine.