gstreamer custom pipes

Hi all!
I’ve been working lately in an project using clutter + gstreamer . Since the project needs some features that in clutter aren’t avaliable I’m thinking of porting it to of. The main problem is that I need my own gstreamer pipeline for video streaming (updsink port=5000 ! smokedec ). My idea is editing the gstreamer based ofVideoPlayer.

  
  
gstPipeline = gst_element_factory_make("playbin","player");  
g_object_set(G_OBJECT(gstPipeline), "uri", name.c_str(), NULL);  

is it possible? is there an easier way?

hi martin

in the version that i posted here:

http://forum.openframeworks.cc/t/gstreamer-videograbber/2097/0

there’s a class called ofGstUtils. both the videograbber and the videoplayer use that internally and it allows for custom pipelines, you just need to create an instance of it and then call its setPipeline method like:

  
ofGstUtils gstUtils;  
gstUtils.setPipeline(pipelineStr);  

with this your pipeline is going to be connected to the default appsink so you need to pass a pipeline string without the sink part at the end. with this you will get your video output as normally in videoPlayer or videoGrabber with getPixels, draw or even getTextureReference.

you can also use the play, stop, pause… methods as with the videoplayer to control the pipeline.

also you can setup a custom sink in case you want a different output (like streaming or a different window) with setPipelineWithSink.

arturo

hi Arturo!
hi everyone!

Thank you for your answer, is really really nice that gstreamer can work in of

here is the code I’m using to check things:
http://pastebin.com/d1b4865a4

I’ve been able to work with really simple pipelines with just elements from the base / core like:
gstPipe.setPipeline("videotestsrc ! video/x-raw-rgb, bpp=24, width=320, height=240 ");
gstPipe.setPipeline("videotestsrc ! theoraenc ! theoradec ! ffmpegcolorspace ! video/x-raw-rgb, bpp=24, width=320, height=240 ");

but pipes with with elements from good don’t work like:
gstPipe.setPipeline("videotestsrc ! smokeenc ! smokedec ! ffmpegcolorspace ! video/x-raw-rgb, bpp=24, width=320, height=240 ");
raises a segfault

gstPipe.setPipeline("v4l2src ! video/x-raw-rgb, bpp=24, width=320, height=240 ");
ouputs:
**(:9477): WARNING ++: gstvideo: failed to get caps of pad sink:sink
OF_ERROR:GStreamer: cannot query width and height

I’ll keep doing check and trying my best to make this work. :slight_smile:

hi martin

yes, i’ve noticed some video formats, both files and streams, make the app segfault, have traced the error but it seems to happen inside the gstreamer classes so no idea why it works in totem or gst-launch and not in oF.

about v4l2 take a look at gstUtils.initGrabber or just use that, don’t remember right now but it needs something special to select the device you want to work with. i’m selecting them through hal/dbus.

also if you make any progress with it please post about it, i would like to support as much formats and devices as possible through gstreamer.

ps. didn’t know pastebin… seems great!

arturo

hello!

I’ve been testing lately , yes it might be a bug inside gstreamer. In my case I’m interested in been able to stream video (no sound), the best performance/simplycity/lbanfwidth was with jpegenc or smokenc and udpsink. This 2 codecs are based on jpeg, looking the backtrace It looks the crash comes from jpeglib. A good way to be sure if the bug is in gstreamer it would be to compile a really simple app (kind of gst-launch) using exclusively gstreamer lib and see if it crashes in the same point. Or maybe using another version of gstreamer… but it is strange because the server is coded in python and works perfecto.

Since I have serious time constraints with the deadline, I’m thinking of using another codec (Any suggestion?) or to grab the udp packets with ofxUDP and try to decode the image inside oF. Is it possible?.Any clue on how to decode the jpg from the string received?

thank you!

what you want to do is a stream from the oF app to the network, or receive a network stream in your oF app?

if it’s the first note that you need to use setPipelineWithSink as the other one tries to setup the sink by itself to be able to get the pixels info.

can u post the pipeline u are trying to use so i can give it a try?

What I’m trying is to receive from the network. In the other side of the connection there is an application written in python which behaves like

  
gst-launch-0.10 -v  v4l2src ! video/x-raw-yuv, width=320, height = 240, framerate = 5/1 ! smokeenc ! udpsink port=5000  

I want to grab that stream from OF so the pipe would be something like:

  
udpsrc port=5000 ! smokedec !  

I used smoke because worked fine, but there might be another codec that works fine. As mentioned before the videotestsrc ! theoraenc ! theoradec ! worked inside OF but I’m no able to setup a pipeline that streams though a network correctly.

In the project the connection is not going to be very stable (3G)

thanks

hey martin

the thing is the smoke encoder uses yuv and you need to convert the frames to rgb before being able to use them in oF, don’t think that’s going to solve the problem, but sure you need to do it to be able to use the pipeline. it should be something like this:

udpsrc port=5000 ! smokedec ! ffmpegcolorspace ! video/x-raw-rgb, width=320, height=240

but i cannot make it work even with gst-launch, if you can find what’s wrong

hello arturo!

These pipelines work on my computer:

Sender:

  
gst-launch-0.10 -v  v4l2src ! video/x-raw-yuv, width=320, height = 240, framerate = 5/1 ! smokeenc ! udpsink port=5000  

Reciever:

  
gst-launch-0.10 udpsrc port=5000  ! smokedec !  ffmpegcolorspace ! video/x-raw-rgb, width=320, height=240 ! ximagesink  

nos vemos :slight_smile:

hey martin

is working for me with the ffmpeg mp4 codec, the pipeline in the oF app:

  
udpsrc port=5000 ! video/mpeg, width=320, height=240, framerate=5/1 ! ffdec_mpeg4 ! ffmpegcolorspace  

the sender is:

  
gst-launch-0.10 -v  v4l2src ! video/x-raw-yuv, width=320, height = 240, framerate = 5/1 ! ffenc_mpeg4 ! udpsink port=5000  

hi!
Oh, sounds great! Unfortunately that codec dosen’t come with ubuntu (jaunty) :frowning: . Do you know where can I get it. I’ve been searching but I don’t find the answer.

auto answer: here

sidenote check here to see if there are newer version

thank you. I’m going to check if it works!

oh, didn’t noticed, i have a developer release of gstreamer installed, you can install it by adding this:

deb http://ppa.launchpad.net/gstreamer-deve-…-ppa/ubuntu jaunty main

in System > Administration > Software Sources

also i suppose any other ffmpeg encoder/decoder should work with that pipeline, you can know which you have installed with:

gst-inspect | grep enc

with this code:
http://pastebin.com/f225fc140

plus this line:

gstPipe.setPipeline(“udpsrc port=5000 ! video/mpeg, width=320, height=240, framerate=5/1 ! ffdec_mpeg4 ! ffmpegcolorspace”);
ouputs:
**(:9477): WARNING ++: gstvideo: failed to get caps of pad sink:sink
OF_ERROR:GStreamer: cannot query width and height

could you please show us your example?

thnx

it’s here:

http://pastebin.com/f2dcd3be4

I still have the same error :slight_smile:

EDIT:

this one also works, so it’s not udpsink’s fault :slight_smile: but I want it in the OF Aplication not in another window

  
char pipe[]="udpsrc port=5000 ! video/mpeg, width=320, height=240, framerate=5/1 ! ffdec_mpeg4 ! ffmpegcolorspace ! xvimagesink";  
   bool allocated = gst.setPipelineWithSink(pipe);  

hey martin

i don’t remember modifying ofGstUtils since i last posted but just in case here’s the version i’m using.

i’m getting that error only if the udp sender is not started but with that codec it’s working for me without problem.

perhaps it has something to do with the version of gstreamer i have, try adding this repository:

deb http://ppa.launchpad.net/gstreamer-deve-…-ppa/ubuntu jaunty main

in Software origins or in /etc/apt/sources.list

then your gstreamer version will be updated to the last one.

ofGstUtils.tar.gz

Hi Arturo!

I have tried your new code , with the dev-gstreamer repositories upgraded with no luck :frowning: . I have been able to test it in my computer kubuntu and in another computer (ubuntu jaunty). In both cases I used a fresh install of everything (of+gstreamer). I’ve checked with diff that there are some changes between the gstutils you uploaded to the forum in your last post and the one that comes in the library you uploaded in June. Could be another change in another file what makes that the pipeline is working to you and not to me?

Muchisimas Thanks !

don’t think so, that file doesn’t have dependencies with anything else, only thing i can think of, is the version of gstreamer: i’m not using the one in jaunty but the one from the ppa repositories (10.13.3) i posted before, do you have that installed? also i’m on 64bits but don’t think that matters

Finally!!
well is no a very clean edit but it works!!
GstUtils.cpp line 850

  
if(GstPad* pad = gst_element_get_static_pad(gstSink, "sink")){  
			if(gst_video_get_size(GST_PAD(pad), &width, &height)){  
				pixels=new unsigned char[width*height*3];  
				gstData.pixels=pixels;  
				memset(pixels,0,width*height*3);  
				//allocated = true;  
			}else{  
			    ofLog(OF_LOG_ERROR,"GStreamer: cannot query width and height");  
			    //MARTIN:Terrible edit  
			    width=320;  
                height=240;  
				ofLog(OF_LOG_ERROR,"GStreamer:(you shouldn't be here) Forcing 320x240");  
				pixels=new unsigned char[width*height*3];  
				gstData.pixels=pixels;  
				memset(pixels,0,width*height*3);  
				//MARTIN:Terrible edit  
				//return false;  
			}  
  

mmmh, ok i think the problem is streams doesn’t report the size till they don’t begin to play. there’s a flag for streams that delays the allocation till some data arrives, but it’s not being used in the case of custom pipelines. will take a look

thanks!

arturo