multiple videos manipulation

hi all,

i am quite new into ofw, but i really love it…however, i am facing a quite big project ahead, and i would like to get some information from you guys…well, i want to play several video clips (and also camera input) and do some manipulations on the clips: RGB values, scale, rotation, speed, etc…

i’ve been exploring ofVideoPlayer and made some tests with it, but i cannot get a high framerate when even when loading short and small (320x240) videos…besides, i would like to make the GPU handle as much video processing as possible, and seems easier to manipulate each frame using its array of pixels than directly on the GPU texture or FBO…

so, my questions:

  • is there a quicker way to play videos? i’m using the quicktime player, is there a way to load all the video directly into memory? i get a huge difference in frame rate when receiving video from a camera (very high framerate) than when loading video from disk…should i use other player?
  • which would be the best architecture to manipulate several videos on real time (usually 5-10)? i would like to reduce CPU processing and also the copies CPU-GPU, and loadData<->getPixels don’t seem very fast, and the ofxShader does not seem to be fully ready (maybe i’m wrong…)

all help is greatly appreciated, as well as suggestions, ideas, comments…or whatever :smiley:

thanks so much!!!


debugging video slow-downs is hard.

try experimenting with different codecs. codecs like h264 do wonders for compressing huge amounts of data into a tiny space, but they do so at the expense of CPU use. at the other end of the scale you’ve got DV or uncompressed formats, which use less CPU but will take up more disk I/O bandwidth. i don’t work with video much but you might want to try Motion-JPEG or Intel Indeo.

basically, you need to find a tradeoff between CPU and disk I/O. one thing you can do to help with the disk I/O problem is to create a RAM disk and copy the movies onto that - but then you’re going to perhaps have size issues, because if you make the RAM disk too big you’ll end up paging virtual memory out to the hard drive, which will kill any performance gained.

yep, gonna try to find the tradeoff between disk I/O and RAM, I’ll post the results as soon as I have something working (a videoplayer extending ofVideoPlayer that is able to handle this in some efficient way…)

btw…any openGL/FBO/Shaders expert around??




Hi, I’ve done a lot of tests with codecs, and i think mjpeg / pjpeg is still the best codec for video performance, balancing quality, filesize (=datarate / streaming data off disk), and decompression cpu load (at least on mac. On PC Pegasus PicVideo mjpeg is still king I think).

The quicktime api is multithreaded (at least on mac), but the bottleneck is mainly going to be sending every frame to the graphics card as a texture. So the bigger the resolution, the slower the performance. In oF, so far video wise I"ve only tried to play 1x 1920x1080p at 30fps, and that works fine on a 2nd gen MBP (pjpeg 80%).

The ofShader does work really well and you can use it to do video processing on the GPU. There are a few threads on that on the forum. The complication comes about because ofTexture defaults to using GL_TEXTURE_RECT, which takes texture coordinates 0…width, 0…height, whereas a lot of GLSL filter code out there uses normalized texture coordinates, so if you copy/pasted such filter code it wouldn’t have worked. You either you need to feed the texture size as uniforms to the shader and mod it accordingly, or hack ofTexture to always use GL_TEXTURE_2D. I believe in 006 this will be addressed.

oh and my 1920x1080 30fps pjpeg80 has a datarate of about 200Mb/s, bit too much for the MBP’s puny little 5400rpm harddrive, it was running off a FW (NOT USB2!!!) external.