Gstreamer advanced utils or how to stream an OF app

extend
#1

since 0061 all the video in linux is managed using gstreamer, the file that does both the videograbbing and videoplayer is ofGstUtils. although right now is only working in linux, it should work in windows and mac and probably in the iphone just by compiling gstreamer and it’s dependencies.

this is an example of what can be done with it + an addon for recording to video files or streaming through the network.

ofxGstVideoRecorder is an addon that allows to record part of an OF app, the desktop, or any ofBaseImage/ofBaseVideo and save it to a file with different codecs or stream it through the network using udp or tcp. you can even stream it to vlc and use it as an http stream server.

in ofGstUtils there’s a method setPipeline that allows to set a custom gstreamer pipeline. in ofGstExamples there’re some examples of what can be done:

  • receive streams from another OF app using tcp or udp
  • receive the content of the desktop in an OF app (only linux)
  • reading axis cameras jpeg streams

some of this examples need a slight mod to setPipeline in ofGstUtils (wil be in 0062 : )

  
bool ofGstUtils::setPipeline(string pipeline, int bpp, bool isStream, int w, int h){  
	this->bpp = bpp;  
	bHavePixelsChanged 	= false;  
	bIsStream = isStream;  
  
	gstData.loop		= g_main_loop_new (NULL, FALSE);  
  
	if(w!=-1 && h!=-1){  
		width=w;  
		height=h;  
	}  
  
	string caps;  
	if(bpp==1)  
		caps="video/x-raw-gray, depth=8, bpp=8";  
	else if(bpp==3)  
		caps="video/x-raw-rgb, depth=24, bpp=24";  
	else if(bpp==4)  
		caps="video/x-raw-rgb, depth=32, bpp=32";  
  
	gchar* pipeline_string =  
		g_strdup((pipeline + " ! appsink name=sink caps=\"" + caps + "\"").c_str()); // caps=video/x-raw-rgb  
  
	GError * error = NULL;  
	gstPipeline = gst_parse_launch (pipeline_string, &error);  
  
	ofLog(OF_LOG_NOTICE, "gstreamer pipeline: %s", pipeline_string);  
	if(error!=NULL){  
		ofLog(OF_LOG_ERROR,"couldnt create pipeline: " + string(error->message));  
		return false;  
	}  
  
  
	gstSink = gst_bin_get_by_name(GST_BIN(gstPipeline),"sink");  
  
	gst_base_sink_set_sync(GST_BASE_SINK(gstSink), true);  
  
  
	return startPipeline();  
}

advancedGst.zip

howto install gstreamer ???
#2

Hi Arturo,

upon compiling ofxGstExamples project, the definition of file ofGstUtils missed 2 integer parameters width and height.
Should when change line 66 from

  
bool setPipeline(string pipeline, int bpp=3, bool isStream = false);

to

  
bool setPipeline(string pipeline, int bpp=3, bool isStream = false, int=640, int=480);

?

I compiles fine now.

I play a movieExample file, and the terminal prints, while the app runs:
(I uncommented and am using** // getting part of the screen as pixels**)

** (:3371): WARNING **: gstvideo: failed to get caps of pad sink:sink
OF_ERROR: GStreamer: cannot query width and height

And I don know if I’m integrating it tight on the moviePlayer Example… It plays a movie.
Can this source record it?

testApp.h

  
#ifndef _TEST_APP  
#define _TEST_APP  
  
  
#include "ofMain.h"  
  
class testApp : public ofBaseApp{  
  
	public:  
  
		void setup();  
		void update();  
		void draw();  
  
		void keyPressed  (int key);  
		void keyReleased(int key);  
		void mouseMoved(int x, int y );  
		void mouseDragged(int x, int y, int button);  
		void mousePressed(int x, int y, int button);  
		void mouseReleased(int x, int y, int button);  
		void resized(int w, int h);  
  
		ofGstUtils gst;  
		ofTexture tex;  
		ofImage img;  
  
		ofVideoPlayer 		fingerMovie;  
		bool                frameByframe;  
};  
  
#endif

testApp.cpp

  
#include "testApp.h"  
  
//--------------------------------------------------------------  
void testApp::setup(){  
	ofSetVerticalSync(true);  
  
    fingerMovie.loadMovie("fingers.mov");  
	fingerMovie.play();  
  
	// advanced gstreamer examples  
	// ofGstUtils doesn't include a texture so  
	// you'll need to use your own  
  
	// getting part of the screen as pixels  
	gst.setPipeline("ximagesrc do-timestamp=true startx=0 starty=0 endx=640 endy=480 use-damage=false ! video/x-raw-rgb, framerate=30/1, depth=24 ! ffmpegcolorspace ",3,false,640,480);  
  
	gst.play();  
	tex.allocate(gst.getWidth(),gst.getHeight(),GL_RGB);  
  
}  
  
//--------------------------------------------------------------  
void testApp::update(){  
    fingerMovie.idleMovie();  
	gst.update();  
	if(gst.isFrameNew()) tex.loadData(gst.getPixels(),640,480,GL_RGB);  
}  
  
//--------------------------------------------------------------  
void testApp::draw(){  
	ofSetColor(255,255,255);  
	//tex.draw(0,0);  
        fingerMovie.draw(0,0,640,480);  
}

Can I link the moviePlayer to the needed texture?

Thank you!

#3

And, can the ofxGstVideoRecorder project be build upon emptyExample?
I did and got a error. Sorry about bein so clueless about how to set it up.

Wich one, any any of the examples, can record part of its own screen, like a movie playing in it?

Here’s the project:

ofxGstVideoRecorder.tar.gz

#4

hey alexandre

you need to use the testApp in ofxGstVideoRecorder not the ofGstExamples, that are for setting custom pipelines to get pixels info from sources like the desktop, internet streams…

the code is something like:

  
.h:  
ofVideoPlayer player;  
ofGstVideoRecorded recorder;  
  
.cpp:  
player.loadFile(...);  
player.play();  
recorder.setup(player,24,"movie.mpg",ofxGstVideoRecorder::H264);

to record a movieplayer to a movie.avi file using H264 codec

or:

  
.h:  
ofGstVideoRecorder recorder;  
  
.cpp:  
recorder.setupRecordWindow(0,0,640,480,24,"of_output.mpg",ofxGstVideoRecorder::H264,30);

to record the the window of an OF app from 0,0 to 640,480 at 30 fps

#5

I got i clearer now, but compiling ofxGstVideoRecorder based on a empyExample project, I get a few errors.
Here’s the printscreen and the project.

1280×726 178 KB

ofxGstVideoRecorder.tar.gz

#6

there’s a library, missing, thought it was included by default:

go to project > properties

there to the libraries tab, look for gstreamer-app-0.10 and add it to “Libraries used in this project”

#7

OK, Arturo, did that and it compiles OK.
When running the application, I get a few errors on the console and the generated mpg file is made empty with 0 bytes.

PS - The “Cannot query duration” error shows up in other examples too, but the apps run fine.

#8

Hey alexandre

in ofxGstVideoRecorder.cpp, line 43 uncomment:

bIsCamera=true;

and it should work.

have updated the zip in the original post.

#9

trying your new source files as they are, with your uncommented record style:

testApp.cpp…
// records an ofBaseImage or ofBaseVideo
recorder.setup(grabber,24,“movie.avi”,ofxGstVideoRecorder::H264);

generates file movie.avi file with 0 bytes and gives errors:

(:13204): GLib-GObject-WARNING **:invalid cast from ‘GstFileSink’ to GstFileSink
(:13204): CRITICAL: gst_app_sink_set_callbacks: assertion ‘GST_IS_APP_SINK (appsink)’ failed
OF_WARNING: error pushing buffer

Commenting that line (20) and making this one uncommented (26):

// records part of the screen
recorder.setupRecordScreen(0,0,640,480,24,“screenshot.mpg”,ofxGstVideoRecorder::H264);

generates a 0 bytes .mpg file and this error:

(:13155): CRITICAL: gst_app_sink_set_callbacks: assertion ‘GST_IS_APP_SINK (appsink)’ failed

#10

Could these errors mean that my system is missing some lib?

#11

this is a codeblocks project for recording part of the window, it’s working for me with 0061 without problem. if it doesn’t work try with another codec. also for encoding h264, i think you need to install libx264:

sudo apt-get install libx264-dev

gstVideoRecorder.zip

ofxGstVideoRecorder
#12

Installed libx264-dev, got your new project, compiles ok, but still 0 bytes mp4 and these errors:

#13

This system of mine is a 2 weeks old Ubuntu 64 install. All I ever installed on it was:

run of Ubuntu script: sudo ./install_codeblocks.sh
run of Ubuntu script: sudo ./install_dependencies.sh
sudo apt-get install libdrm-dev
sudo apt-get install libx264-dev

This morning I also installed:

gstreamer0.10-alsa
gstreamer0.10-x
gstreamer0.10-plugins-base
libgstreamer-plugins-base0.10-dev

Could that OF_ERROR: GStreamer: cannot query position that shows on other projects mean something?

#14

Just to double check: I changed ofGstUtil.h line 66 to:

bool setPipeline(string pipeline, int bpp=3, bool isStream = false, int w=640, int h = 480);

was that right?

#15

AlexandreRangel good job!
I found this list of dependency OF libraries to install in Ubuntu and save a lot time!!

#16

[quote author=“arturo”]… although right now is only working in linux, it should work in windows and mac and probably in the iphone just by compiling gstreamer and it’s dependencies.
[/quote]

Hey!

I tried to compile on mac osx.
I had to include gstreamer-0.10, glib-2.0 and libxml2 include paths to the openFrameworks project.
Now I cannot continue because I couldn’t find libudev anywhere.

BTW, what’s the current chances of having an official extension to export movies?
That’s the only thing I miss from Processing…

#17

yes, libudev is linux specific. it’s just used to identify the available cameras but gstreamer should work without it. i’ll upload an updated version later, with some ifdefs so libudev is only used in linux

#18

here it is…

ofGstUtils.cpp.zip

#19

continuing…

The source is now compiling!

I had to add the following lins to the project…

/opt/local/lib/libglib-2.0.a
/opt/local/lib/libgstreamer-0.10.a
/opt/local/lib/libgstvideo-0.10.a
/opt/local/lib/libgstapp-0.10.0.dylib
/opt/local/lib/libgstrtp-0.10.0.dylib
/opt/local/lib/libgobject-2.0.0.dylib
/opt/local/lib/libgthread-2.0.0.dylib

But when I run the app it craches and the console give me this…

  
  
  
[Session started at 2010-04-02 20:16:13 +0200.]  
OF_WARNING: No camera settings to load  
  
** (<unknown>:12745): WARNING **: gstvideo: failed to get caps of pad video-sink:sink  
OF_ERROR: GStreamer: cannot query width and height  
  
(<unknown>:12745): GLib-GObject-WARNING **: g_object_set_valist: object class `GstFileSink' has no property named `emit-signals'  
  
(<unknown>:12745): GLib-GObject-WARNING **: invalid cast from `GstFileSink' to `GstAppSink'  
  
** (<unknown>:12745): CRITICAL **: gst_app_sink_set_callbacks: assertion `GST_IS_APP_SINK (appsink)' failed  
  
[Session started at 2010-04-02 20:16:16 +0200.]  
GNU gdb 6.3.50-20050815 (Apple version gdb-967) (Tue Jul 14 02:11:58 UTC 2009)  
Copyright 2004 Free Software Foundation, Inc.  
GDB is free software, covered by the GNU General Public License, and you are  
welcome to change it and/or distribute copies of it under certain conditions.  
Type "show copying" to see the conditions.  
There is absolutely no warranty for GDB.  Type "show warranty" for details.  
This GDB was configured as "i386-apple-darwin".sharedlibrary apply-load-rules all  
Attaching to process 12745.  
warning: Could not find object file "/Users/theo/Documents/CODE/__OPENFRAMEWORKS/SANDBOX/COMPILE_LIBRARIES/buildGlutFramework/libForeground.a(macx_foreground.o)" - no debug information available for "/Users/mcast/Code/GLUT-ToPost/macx_foreground.m".  
  
warning: Could not find object file "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_gstreamer/work/gstreamer-0.10.28/gst/.libs/libgstreamer_0.10_la-gst.o" - no debug information available for "gst.c".  
  
warning: Could not find object file "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_gstreamer/work/gstreamer-0.10.28/gst/.libs/libgstreamer_0.10_la-gstobject.o" - no debug information available for "gstobject.c".  
  
warning: Could not find object file "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_gstreamer/work/gstreamer-0.10.28/gst/.libs/libgstreamer_0.10_la-gstbin.o" - no debug information available for "gstbin.c".  
  
warning: Could not find object file "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_gstreamer/work/gstreamer-0.10.28/gst/.libs/libgstreamer_0.10_la-gstbuffer.o" - no debug information available for "gstbuffer.c".  
  
... and so it goes....
#20

it seems that you’re trying to use a camera? that won’t work since the camera detection relies on libudev and is linux specific, the rest like video player or all the examples here for streaming, recording the screen… should work.

also pierre made gstreamer work on windows some days ago, he needed to add an environment variable to tell gstreamer were the plugins were:

GST_PLUGIN_PATH

since it seems you’re using it from macports it shouldn’t be necessary but just in case