OpenFrameworks HD Video (Windows)

advanced
#1

Hello everyone.
I have been lurking around since a long time and finally thought of creating a topic.

I am trying to achieve a straightforward task. To play HD Video But seems like it is harder than I can imagine.

OS - Win7 x64

Hardware - 2.5Ghz 2nd gen Intel core i5, 4GB RAM,2GB NVidia GEForce GT 630M.

Hardware to deploy the application - Intel Xeon 3.4Ghz , 24GB RAM, 2 GB graphics card (sorry do not remember the model number). I do not have access to this hardware now.

IDE - Visual Studio 2010

Trying to play -
1- 3840*1080 p video
2-1920*540 video (scaled down version of 1)

both are 30fps

What All I have tried ?

ofVideoPlayer - gives me around 10fps on a (1) , and 25fps on (2) but choppy.

gstreamer - I have had so many issues with this.

Started trying with this - http://forum.openframeworks.cc/t/gstreamer-playback-under-windows/3610/35

Got the .mov file playing . But while playing my video , did not see much performance improvement.
Then thought of trying in code::blocks. Went there and followed this post (http://forum.openframeworks.cc/t/gstreamer-playback-under-windows/3610/1).

Here I had to install another gstreamer and now things got messy. I could not get it working in Code::blocks. may be because it was built using OF 0.061, planning to try this tomorrow. Got it working with 0.062 but the exe stops working , even tried in XP compatibility mode as somebody suggested , but no luck. Previously it said it is missing python26.dll , installed Python 2.6 and put the dll in there.

Now when I came back to VS2010 to see if changing video codec improved performance , It has stopped working , No video. I am able to hear the audio. The framerate is around 600 FPS but no image. Here is the screenshot of it running , with the error message.Please see the attached file.

Can anyone hint me how should i proceed now ? Is there any other way I am missing.I am pretty sure this is a big issue and must be very common.

On a side note - Will linux be a better platform for this ?

Thanks for your time.

psc.png1366×768

#2

Hey there,

This is a long standing problem coming from Windows using the old Quicktime C API’s which were created a while before HD video.

I haven’t seen a good solution yet implemented in OF. On the OS X side, the QTKit libraries can handle pretty large videos, as I believe Gstreamer on linux is fairly optimized.

My knowledge on windows video is pretty limited unfortunately. Matthew Gingold (Gameoverhack) I believe has gotten GStreamer on Windows to function so perhaps a ping to him would be a good idea. The other windows/video guru is Elliot Woods.

#3

Hey Peter

Thanks for describing your problem.

I haven’t delved into wrapping video playback from within oF on win32.
I’ve coded against gstreamer in c++ outside of openFrameworks and it’s pretty clean to implement.
The other threads you link demonstrate people with good experience of solving this problem.

Changing the core video player code on win32 seems to be stalling endeavour
There was an initiative at ars electronica which had GStreamer working in win64 *
i feel right now that touching core video code here is a bad use of time, as it’s likely to be stalled / complicated before useful to users
getting a good video player in an addon seems to be the next step.

* i think a sprint to 64bit would be a good place to switch GStreamer in on windows

MS windows video playback has gone VfW->DShow->WMF->something else->something new now with windows 8
DShow still works of course (32 and 64bit), and thankfully they’ve been maintaining backwards compatibility pretty well

Personally I prefer to work towards GStreamer over DShow due to its codec support and consistent API
(and crazy demuxing features which make it compatible with more exotic streams like from an ARDrone).
GStreamer also wraps the other frameworks (e.g. can use DirectShow codecs)

next step seems : get ofxGStreamer as a common functional add-on for win32,win64,osx64
(nicking code from our linux core implementation)

@Peter : In response to your question regarding Linux
Yes, linux would be a more performant (and stable) option for your video playback requirements right now.

Even better would be osx as it’s had a recent overhaul of video playback code, employing GPU decoding and other advanced OS video playback techniques. For a single 4k 30fps video stream, Linux/GStreamer may work well for you.

I’d suggest trying

  
gst-launch-1.0 playbin uri=file:///path/to/your/4k/video/file

on your system to quickly check the video playback of your file using GStreamer without any oF development.

If you want hardware accelerated playback with GStreamer, then check this link:
http://docs.gstreamer.com/display/GstSDK/Playback+tutorial+8%3A+Hardware-accelerated+video+decoding

(note that generally QuickTime isn’t a good codec to use outside of osx, e.g. is generally not hardware accelerated).

Elliot

#4

I have also tried video in OF but capturing from an HD camera and similarly found the performance unacceptably slow. (factor of 5-10 slower than an equivalent c# / WPF version on same computer). My naive guess is therefore that the bottle-neck is not reading the video data from disk via QT or gstreamer but rather the opengl texture upload.

to test where the problem lies you could store a number of HD frames in memory and then play them back as fast as possible and see if this is any faster. this would eliminate any disk or video codec / library issues.

if it turns out disk based video reading is the problem i’d suggest you store you video as a sequence of jpegs (these are very fast and easy to read) and either read them in ahead of time and store them in RAM or read them from disk as they are needed.

#5

to respond a little:

+1 for image sequences as a great way of avoiding difficulties with video playback (when they work fast enough)
but you might have trouble processing your 4k frames

Make sure that you’re compiling with optimisation on when using Visual Studio (e.g. compile as Release) when discussing performance.

@dr.mo :
could you compare persformance of
https://github.com/elliotwoods/videoInput.NET
vs your other c# solution? (make sure that you’re capturing pixels to memory then pushing to video card for a like-for-like comparison).

#6

another easy test: try the file in a gstreamer standalone player (both on windows) and a quicktime standalone player and compare the difference in choppiness

#7

Hey Guys,
Thanks for feedback.

I am pretty sure the disk based reading is not problem. (I have a 5400rpm HDD).

Also as I was pretty language agnostic in developing this application , I started developing using AS3 & stagevideo (http://help.adobe.com/en-US/FlashPlatform/reference/actionscript/3/flash/media/StageVideo.html). It does both hardware based decoding and rendering. And surprisingly I got amazing performance. Video playing at full FPS. Which was pretty surprising because I always considered flash to be slow. (which anyway it is in many other aspects)But for playing video files straight from disk , it seems like to be the easiest option for now.

But if I will try eliotwood’s suggestion to see gstreamer performance. Would love to have this feature as default in Windows and OF :-).

#8

@elliotwoods - I tried to play the file. But I got this as error (which I was getting in VS), I have tried reinstalling ,installing python 2.6 but still without luck. Maybe someone can figure out what the error says exactly -

  
  
  
C:\Program Files (x86)\OSSBuild\GStreamer\v0.10.6\bin>gst-launch.exe playbin uri  
="C:\Dev\FlashDevelop\HDTest\bin\data\video_s2.mp4"  
  
** (gst-launch-0.10:5548): WARNING **: Failed to load plugin 'C:\Program Files (  
x86)\OSSBuild\GStreamer\v0.10.6\lib\gstreamer-0.10\libgstpython-v2.6.dll': `C:\P  
rogram Files (x86)\OSSBuild\GStreamer\v0.10.6\lib\gstreamer-0.10\libgstpython-v2  
.6.dll': %1 is not a valid Win32 application.  
  
** (gst-launch-0.10:5548): WARNING **: Failed to load plugin 'C:\Program Files (  
x86)\OSSBuild\GStreamer\v0.10.6\lib\gstreamer-0.10\libgstpython-v2.6.dll': `C:\P  
rogram Files (x86)\OSSBuild\GStreamer\v0.10.6\lib\gstreamer-0.10\libgstpython-v2  
.6.dll': %1 is not a valid Win32 application.  
Setting pipeline to PAUSED ...  
ERROR: Pipeline doesn't want to pause.  
ERROR: from element /GstPlayBin:playbin0: A C protocol source plugin is required  
 to play this stream, but not installed.  
Additional debug info:  
..\..\..\..\..\Source\gst-plugins-base\gst\playback\gstplaybasebin.c(1691): gen_  
source_element (): /GstPlayBin:playbin0:  
No URI handler for c  
Setting pipeline to NULL ...  
Freeing pipeline ...

Thanks again , would love to see this work with OF.

#9

which download of GStreamer are you using?
one from this page:
https://code.google.com/p/ossbuild/downloads/list ?

#10

GStreamer WinBuilds v0.10.6 GPL (x86)

GStreamer WinBuilds SDK v0.10.6, GPL (x86)

#11

and to double check …

C:\Program Files (x86)\OSSBuild\GStreamer\v0.10.6\lib\gstreamer-0.10\libgstpython-v2.6.dll

exists?

#12

Yes it does. Here’s my complete directory all the files - http://pastebin.com/MmKdiH5P.

I already have reinstalled the package wishing there were something corrupt in last installation, but same issue again :-(.

Tried with CodeBlocks, with OF 0.062 without volume.

Also in between got it to work with OF 0.074 but now after trying CodeBlocks , stopped working directly. Checked my environment variables too , nothing abnormal in there.

I do have Python2.6 installed , it is not in my env variable now , will it help to have it in environment path?

#13

Hi guys, not sure if you found a solution for this in the end but I recently came up against similar issues… In the end I forked and updated the ofxGstStandaloneVideo addon to work with oF 0.8 and VS2012. It requires the OSSBuild version of gstreamer and gstreamer SDK. I’m getting good results with this playing back 1080p video and 4k video too.

My fork includes a list of dependencies and can be found here:
https://github.com/outsidecontext/ofxGstStandaloneVideo

Cheers

#14

Hi,

I managed to compile and build the example program found here:(https://github.com/outsidecontext/ofxGstStandaloneVideo), but unfortunately I am getting some errors when running it.

Behavior: When I run it from Visual Studio, the app starts and then immediately exits with exit code 0 shown in the output. If I run it from the /bin folder, I get the following error on the console, while the actual app screen is freezed and I cannot access it:

I am using OF v.0.8.0, VS 2012, win 7 Home 64bits.
Hardware specs: Intel i3 2.1Ghz, 4GB RAM, Intel HD3000 graphics.

Any tips on how to solve this would be great!

Best,
Andrei

#15

Just use ofxHapPlayer and all the problems will be gone :slight_smile:

#16

the easiest way would be just using this addon https://github.com/arturoc/ofxGStreamer
since OF gst player working very well on windows,and download official gst from here http://gstreamer.freedesktop.org/data/pkg/windows/1.4.5/

hap also does the job,but i had trouble with multi-graphic cards,the content didn’t play somehow.