Play a really big video

Hi everybody,

I’m building an installation with 6 screens and a total size of 3840 x 3240. I try to play a full screen video in the background.
I used ofVideoPlayer to do that and the drawing time when I put images of the movie in some ofTexture is fine but the decoding time and the quicktime function “MoviesTask” seems really slow with big files.
I tried lots of different format as well (mp4, mov, h264…) and to put the update function of the movie in a different Thread. The framerate of the global app stay between 30 and 60 fps just the movie play really slowly.

I’m on windows so I can’t use QT Kit. Is there a way to use a different plugin than Quicktime ? Do you think I can try to use OpenCV to play the video and that will be better ? Is there a different way to do it I can’t think of ?

Here is the code I did to play the movie

  
  
#include "StdAfx.h"  
#include "ThreadMovie.h"  
  
  
ThreadMovie::ThreadMovie(void)  
{  
	  
  
}  
  
ThreadMovie::ThreadMovie(const ThreadMovie &copy){  
  
}  
  
//Function just call once when the app start  
void ThreadMovie::loadMovie(string url, int w, int h) {  
	width = w;  
	height = h;  
	toggleDraw = false;  
	mov = new ofVideoPlayer();  
	mov->loadMovie(url);  
	mov->setUseTexture(false);  
	mov->play();  
	//mov->setPaused(true);  
  
	text1.allocate(mov->width, mov->height, GL_RGB);  
	text2.allocate(mov->width, mov->height, GL_RGB);  
	  
	start();  
}  
  
//update call in the testapp's update function  
void ThreadMovie::update() {  
  
	if(mov->isFrameNew()) {  
		if(!toggleDraw) {  
			text1.loadData(mov->getPixels(), mov->width, mov->height, GL_RGB);  
			toggleDraw = true;  
		} else {  
			text2.loadData(mov->getPixels(), mov->width, mov->height, GL_RGB);  
			toggleDraw = false;  
		}  
	}  
}  
  
void ThreadMovie::threadedFunction() {  
	  
	while (isThreadRunning() != 0) {  
		if(lock()) {  
			  
			mov->idleMovie();  
			unlock();  
  
		}  
	}  
}  
  
//Function call in the testapp draw function to draw the texture after  
ofTexture ThreadMovie::getTexture() {  
	return (toggleDraw) ? text2 : text1;  
}  
  
ThreadMovie::~ThreadMovie(void)  
{  
	stop();	  
}  
  

Thanks for your answers…

there could be a few bottlenecks here.

1 if you’re somehow using multiple graphics cards, you can have some serious difficulties that are hardware-specific. so i’m going to assume you’re using two matrox triple head 2 gos on a single card.

2 because you’re (probably) using multiple outputs, if you have vertical sync turned on then you might get a constant 30 fps framerate because the graphics card is trying to sync to two separate devices. but it sounds like the playback framerate is your problem, not the app framerate.

3 if the video itself is very large (3840x3240) then you’re going to have problems decoding it quickly. it takes a long time for the CPU to convert a frame of encoded video to a frame of decoded video. some codecs take longer to decode than others. it also takes a long time to upload this frame to the graphics card, but not as long as decoding.

i recommend trying this:

1 find a short, small video. like 640x480 or 320x240, one of the videos that comes with OF is good.
2 play it back on a single monitor.
3 make a big window and play it back on multiple monitors.
4 if it’s not slow yet, repeat the process with a larger video size.

you’ll quickly find the bottleneck this way.

also, i don’t think you need the threaded movie code unless you’re doing something else you didn’t mention.

it should be much faster to use GStreamer instead of the oF default video player.

Hi !

Thanks for your fast answers !

I finally used opencv with the cvCaptureFromAVI function and it’s really faster than quicktime ! I will have 6 screens so I divided my video in 6 movies equal in size (convert in avi 10 bit YUV), decode it and synchronise everything at a given frame rate (I don’t have any sound so I dont really care if I’m not exactly at the video’s frame rate).

I have 3 Nvidia Quadro 600 and I use mosaic (http://www.nvidia.com/object/nvidia-mosaic-technology.html) to join them to do 1 screen so the drawing part seems to stay really fast.

And I’m using thread because I have a really good 4 cores / 8 threads processor so I was thinking that I cant be wrong using more than one thread.

I tried to use GStreamer but I develop with visual c++ and that library seems made to be use with Code Block (I didn’t see any example with vc++, am I wrong ?)

Here is the new code :

  
#include "StdAfx.h"  
#include "ThreadMovie.h"  
  
ThreadMovie::ThreadMovie(void)  
{  
	  
  
}  
  
ThreadMovie::ThreadMovie(const ThreadMovie &copy){  
  
}  
  
void ThreadMovie::loadMovie(string url, int w, int h) {  
	capture = cvCaptureFromAVI(url.c_str());  
	if( !capture ) {  
		printf("wrong format !!");  
		return;  
	}  
	fps = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FPS );  
	  
	bufferSize = 30;  
	width = w;  
	height = h;  
	toggleDraw = false;  
	frame = cvQueryFrame( capture );  
	movWidth = cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH);  
	movHeight = cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT);  
	i = new ofImage();  
	i->allocate(movWidth, movHeight, OF_IMAGE_COLOR);  
	i->setUseTexture(false);  
	text1.allocate(movWidth, movHeight, GL_RGB);  
	text2.allocate(movWidth, movHeight, GL_RGB);  
	currTime = oldTime = percFilm = 0;  
	start();	  
}  
  
void ThreadMovie::update() {  
  
		if(!toggleDraw) {  
			text1.loadData(i->getPixels(), movWidth, movHeight, GL_RGB);  
			toggleDraw = true;  
		} else {  
			text2.loadData(i->getPixels(), movWidth, movHeight, GL_RGB);  
			toggleDraw = false;  
		}  
		currTime = 0;  
}  
  
void ThreadMovie::threadedFunction() {  
	  
	while (isThreadRunning() != 0) {  
		frame = cvQueryFrame( capture );  
		if( !frame ) {  
			cvSetCaptureProperty(capture, CV_CAP_PROP_POS_AVI_RATIO, 0);  
			Sleep(1000.f / 40.f);  
			continue;  
		}  
		  
		cv::cvtColor(frame, rgbFrame, CV_BGR2RGB);  
		  
		i->setFromPixels((unsigned char*)rgbFrame.data, movWidth, movHeight, OF_IMAGE_COLOR);  
		if(isRefvdo) {  
			percFilm = float(cvGetCaptureProperty(capture,CV_CAP_PROP_POS_AVI_RATIO));  
			if(oldFrame == getFrame()) {  
				cvSetCaptureProperty(capture, CV_CAP_PROP_POS_AVI_RATIO, 0);  
			}  
			oldFrame = getFrame();  
		}  
		  
		Sleep(1000.f / 40.f);  
	}  
}  
  
int ThreadMovie::getFrame() {  
	return cvGetCaptureProperty(capture, CV_CAP_PROP_POS_FRAMES);  
}  
  
void ThreadMovie::setFrame(int f) {  
	cvSetCaptureProperty(capture, CV_CAP_PROP_POS_FRAMES, f);  
}  
  
ofTexture ThreadMovie::getTexture() {  
	return (toggleDraw) ? text1 : text2;  
}  
  
ThreadMovie::~ThreadMovie(void)  
{  
	cvReleaseCapture(&capture);  
	stop();	  
}  
  

And in my update function I sync every movies :

  
for(int i = 0; i < playerList.size(); i++) {  
		ThreadMovie *player = playerList[i];  
		if(player->isRefvdo) currFrame = player->getFrame();  
		else player->setFrame(currFrame);  
		player->update();  
	}  

Anyway thanks for your help !