Gstreamer in openFramework base CCV-Software

Hello hello,

I’m struggling for a couple of days now, so I thought “Why not ask the experts for help”.
I’m trying to get a camera stream from a raspberry pi (sitting inside of a closed sphere, therefore all data has to pass through the wifi) into the Community Core Vision Software to detect touch-events on the surface of the sphere.

So far I have archived a quite delayed mjpeg stream with a low framerate, which is tunnel into the software through a moonware universal driver. But seeing how a gstreamer h.264 pipeline is performing, I can’t stand that solution anymore :wink:

So the question is, is it possible to catch a gstreamer pipeline inside c++? Found a couple of plugins for gstreamer support (arturoc/ofxGStreamer etc) but it seems they’re rather for sending than not for catching a h.264 stream… To make it even more complicated the ccv software is quite outdated and is based on visual studio 2008 and an old version of openFrameworks.

Would be great if anybody can help!
Cheers,
Till

yes you can run any gstreamer pipeline in c++. if you are in linux right away if in other platform using ofxGstreamer. just use ofGstVideoUtils::setPipeline and specify the same pipeline you would specify with gst-launch but without the final element since OF set’s it’s own app sink to receive the pixels and upload them to an opengl texture

Thank your for the fast response arturo. After countless attemps of trying to intgerate the ofGstVideoPlayer in the old Version of oF and in Visual Studio 2008, I’ve tried to get it to work in a blank 2015 project.

So about the generell “app-flow”. used the initialization of your example in the ofApp.cpp

player.setPlayer(ofPtr<ofGstVideoPlayer>(new ofGstVideoPlayer));

player.getPlayer()->load("udpsrc port=5000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264");        (ofApp.cpp)

For the sake of just testing I’ve included the setPipeline in the load-function of the ofGstVideoPlayer.cpp

videoUtils.setPipelineWithSink(pipeSpecs, "rtph264pay0", true);       (ofGstVideoPlayer.cpp)

But somehow (meaning because of a stupid mistake of myself) oF is not able to get the Sink. Is something wrong with the "Gstreamer Specs (although working in the command line)) or is the whole way of constructing the PipelineWithSink in the ofGstUtils false?

I also tried the following line without succes:

videoUtils.setPipeline(pipeSpecs, OF_PIXELS_RGB, true);

Thank you so much in advanced!

you can’t use the player for this but ofGstVideoUtils directly, you need to use setPipeline without the sink cause OF creates it’s own sink and it’s always recommended to add a ! videoconvert at the end of your pipeline so whatever the final format the last element outputs gets converted to rgb which is the default format OF will try to pull from the pipeline

Awesome tipps! Slowly but steadily I get to know oF and the gstreamer. So far I managed to see… pure blackness. But most liked something is utterly wrong with the hole builtup:

void ofApp::setup(){

   videoUtils.allocate(1280, 720, OF_PIXELS_RGB);
   videoUtils.setPipeline("udpsrc port=5000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! videoconvert", OF_PIXELS_RGB, true);

   imgW = 1280;
   imgH = 720;
   img.allocate(imgW, imgH, OF_IMAGE_COLOR);
}

//--------------------------------------------------------------
void ofApp::update(){

   videoUtils.update();
   img.setFromPixels(videoUtils.getPixels());

}

//--------------------------------------------------------------
void ofApp::draw(){

   img.draw(0, 0);
}

that looks correct to me, sometimes gst-launch sets somethings for you that you need to then set manually when using the string pipeline with OF but it’s mostly a trial error process, you can use GST_DEBUG=4 or more to see errors if you run the program in the console like:

GST_DEBUG=4 bin/gstApplication

also you don’t need to use an ofImage for this, better to use an ofTexture instead since ofImage would be copying the pixels too which is not necesary

I ended up using your GstRtp addon and it perfectly does the job. And thanks for the performance tipp with ofTexture.