set video position by frame

nay · June 12, 2008, 5:01am

hi all,
i want to sync a vid background across multiple machines and have one computer scrubbing and then sending the position via osc to all machines. is there a way to set the movie position by frame rather than percentage? i need it to be as smooth as possible. thanks in advance. code looks kinda like:

  
  
void testApp::setup() {  
        ofSetFrameRate(25);  
	clip.loadMovie("clip.mov");  
	clip.setPaused(1);  
        vid_pos = 0.0;  
	vid_speed = 1.0 / (clip.getDuration()*25); //vid framerate=25  
}  
void testApp::update(){  
	clip.setPosition(vid_pos);  
	clip.idleMovie();  
	vid_pos += vid_speed;  
	if(vid_pos > 1.0) vid_pos = 0.0;  
}  
void testApp::draw(){  
	clip.draw(0,0);  
}

zach · June 12, 2008, 8:47am

hi !

currently this doesn’t exist in openframeworks, but primarily because quicktime doesn’t give you that functionality off the bat.

it structures movies around a time base, not around frames:

http://tinyurl.com/66l4c3

so it’s quite easy to say go to time n, or got pct.

there is a function called:
**GetMovieNextInterestingTime **
that gets you the next frame, so I guess it would be possible to start at the beginning and count your way through.

I could be way wrong, but I think the problem we might have is that you can use variable frame lengths, so it would be hard to use the timebase toaccurately jump to a specific frame. it would be more acccurate, but less efficient to start at frame 0, and count via GetMovieNextInterestingTime.

Anyway, I am just thinking outloud

is the percentage way off? I’m thinking it should do this quite well…

I am totally happy to test this and see if we can implement this function somehow. one thing that might be helpful for debugging this would be a movie that has the frame number for each frame – ie, frame one says 1, frame two says 2, etc. then it would be easy to test if the functionality is working properly.

take care!
zach

risenparadigm · June 12, 2008, 7:35pm

hi nay/zach,

it’s pretty funny, i was just poring over this issue for the last few days. i have a little code snippet that seems to work fine for static framerate videos. I tested it by setting the playback rate to something really low, then making an infinite loop that would check movie.isFrameNew() and increments a counter if it’s true. for a 3 minute clip, i had about 6000 frames (6055 actually). i also got the duration of the video and when i divided numFrames/duration, i got the standard 29.97 fps. so this was good confirmation that my video is a constant framerate. please test it and let me know!

  
  
Media movieMedia;  
MediaHandler movieMediaHandler;  
  
MovieGetVideoMediaAndMediaHandler(movieClip.moviePtr, &movieMedia, &movieMediaHandler);  
Track videoTrack = GetMovieIndTrackType(movieClip.moviePtr, 1, 'vfrr', movieTrackCharacteristic | movieTrackEnabledOnly );  
  
Media movieMedia = GetTrackMedia(videoTrack);  
MediaHandler movieMediaHandler = GetMediaHandler(media);  
  
double numFrames = GetMediaSampleCount(movieMedia);  
TimeValue64 duration = GetMediaDuration(movieMedia);  
TimeValue64 timeScale = GetMediaTimeScale(movieMedia);  
  
double seconds = (double)duration/(double)timeScale;  
double fps = numFrames/seconds;

I suppose if you wanted to do it for a video with nonstatic framerate, you would have to put it in a loop getting every video track and storing the length of each frame.

if your video is an mpeg, it’s much easier. you would first check:

  
  
boolean isMPEG = false;  
MediaHasCharacteristic(movieMediaHandler, 'mpeg',  &isMPEG);

then if it is an mpeg, all you have to do is:

  
  
MHInfoEncodedFrameRateRecord encodedFrameRate;  
Size encodedFrameRateSize = sizeof(encodedFrameRate);  
MediaGetPublicInfo(movieMediaHandler, kMHInfoEncodedFrameRate, &encodedFrameRate, &encodedFrameRateSize);  
Fixed frameRate = encodedFrameRate.encodedFrameRate;

also, I did try GetMovieNextInterestingTime:

  
  
long frameCount = 0;  
TimeValue curMovieTime;  
curMovieTime = 0;  
while(curMovieTime >= 0)  
{  
   GetMovieNextInterestingTime(theMovie, nextTimeStep, 0, NULL, curMovieTime, 0, &curMovieTime, NULL);  
   frameCount++;  
}  
  
frameCount--; //extra timestep @ eof

however, this gave me a result of 15431, which makes little sense. maybe I’m computing it wrong, so if you guys have an idea on this one, please let me know!

thanks!
ramsin

nay · June 13, 2008, 3:06am

thanks guys.

ramsin, any chance of an example project i could download to try that out? bit baffled by that code at the moment…

first up, i had to paste a function and constant from here to get it to compile:
http://developer.apple.com/qa/qa2001/qa1262.html

  
  
#define   kCharacteristicHasVideoFrameRate  FOUR_CHAR_CODE('vfrr')  
  
MovieGetVideoMediaAndMediaHandler(Movie inMovie, Media *outMedia, MediaHandler *outMediaHandler)  
{ ... }

secondly, even though i have it compiling and using the code for a non mpeg1/2 file, I don’t understand how to actually put it to use?! to specify the frame number and then draw the frame that is.

also, any chance of posting that test clip with the frames numbered? maybe zach could use it to figure out GetMovieNextInterestingTime();

zach, using percentages is close but there is the not-so-occasional hiccup, which will be pretty noticeable when projected large - could just be due to the numbers not dividing evenly, maybe?

the other performance issue i just realised was that i’ve been developing on a mac, but just copied it to the PC it has to run on where QT runs much slower. I’m actually getting the best performance on windows using the animation codec. the clip is 1400x563, 25fps but the card is a 512mb nvidia geforce 8600 gt.

any advice HUGELY appreciated.

nay · June 13, 2008, 5:07am

okay, so i made a 2500 frame / 25fps movie, using quicktime pro in the hope that a more even division would help it increment more smoothly. turned out to be 1:39.98 long instead of 1:40 as i expected.

1/(duration*fps) then gives me .00041 instead .0004 as planned, incrementing by either of these is still not that smooth on mac or PC.

GetMovieNextInterestingTime(); may work as zach suggested though as ramsin’s snippet tells me that the clip is 2499 frames long:

  
  
long frameCount = 0;  
	TimeValue curMovieTime;  
	curMovieTime = 0;  
	while(curMovieTime >= 0)  
	{  
	   GetMovieNextInterestingTime(clip.moviePtr, nextTimeStep, 0, NULL, curMovieTime, 0, &curMovieTime, NULL);  
	   frameCount++;  
	   cout<<frameCount<<endl;  
	}

but as my c++ chops are lacking i’m still unsure about how to use this to set the pos and draw the frame though…

zach · June 13, 2008, 7:20am

hi all,

I would guess that getting 2499 is good – since the from the first frame (0) there are 2499 “interesting times”, ie 2500 frames. the first frame is not an interesting time.

(since the function is called getnextinterestingTime)

ie, have a movie like this.
0 - 1 - 2 - 3 - 4 - 5
there are 5 frames “-”, ie 0-1, 1-2, etc
start at 0, next interesting time is 1, then 2,3,4… so 4 interesting times, but 5 frames for the movie…

I’m currently working on an install, so unable to test stuff, but if someone wants to help the process, they can upload a test movie, it would be awesome. You can likely do it easily by using the ofImage saving routine to generate individual frames (jpegs / pngs, etc) and then premier or qt pro to stitch together a movie. I’m able to give it a try early next week.

take care!!
zach

nay · June 16, 2008, 12:21am

thanks zach, hope you’re install went/goes well

here’s a 10sec, 10fps vid, with frames counting from 0-99:
http://renechristen.net/temp/frame-count.mov

cheers
nay.

mantissa · July 23, 2008, 8:53pm

i’ve been working on this problem … with some help from the developer Q&As for quicktime.

http://developer.apple.com/qa/qa2001/qa1262.html

i use this function for determining the frame count. though i’ve seen others just grab the sample count.

  
  
  
//--------------------------------------------------------------  
  
long ofVideoPlayerExtended::getFrameCount(){  
  
	long        frameCount = 0;  
    TimeValue   curMovieTime;  
	TimeValue   duration;  
	  
	OSType whichMediaType = VIDEO_TYPE;   
	short flags = nextTimeMediaSample + nextTimeEdgeOK;   
  
    curMovieTime = 0;  
    while( curMovieTime >= 0 )  
    {  
	  
		 frameCount++;  
	  
        GetMovieNextInterestingTime(  
                                    moviePtr,  
                                    flags,  
                                    1,   
                                    &whichMediaType,  
                                    curMovieTime,  
									0,  
                                    &curMovieTime,  
                                    &duration );  
									  
		flags = nextTimeMediaSample;  
    }  
  
    frameCount--; // there's an extra time step at the end of the movie  
  
    return frameCount;  
	  
}

then, when i want to jump to a frame, i call setFrame() with the desired frame number;

  
  
  
//--------------------------------------------------------------  
  
void ofVideoPlayerExtended::setFrame(long frame){  
			  
	long timeValuesPerFrame = getTimeValuesPerFrame();  
	  
	// determine time based on frame rate  
	TimeValue t = frame * timeValuesPerFrame;  
	  
	// jump to specific time  
	SetMovieTimeValue(moviePtr, t);  
  
	/* tasking the movie does the actual draw */  
	MoviesTask(moviePtr, 0);  
}

this function requires these other functions … for determining the time values per frame and for finding the video media track.

  
  
  
//--------------------------------------------------------------  
  
int ofVideoPlayerExtended:: getTimeValuesPerFrame(){  
  
	// find the media  
	Media movieMedia= getMovieMedia(moviePtr);  
	  
	// get duration and sample count  
	int duration = GetMovieDuration(moviePtr);  
	int samples = GetMediaSampleCount(movieMedia);  
	  
	// calculate time values per frame  
	int timeValuesPerFrame = duration/samples;  
	  
	return timeValuesPerFrame;  
}  
  
//--------------------------------------------------------------  
  
#define   kCharacteristicHasVideoFrameRate  FOUR_CHAR_CODE('vfrr')  
#define   kCharacteristicIsAnMpegTrack     FOUR_CHAR_CODE('mpeg')  
  
Media ofVideoPlayerExtended::getMovieMedia(Movie inMovie){  
  
	Media movieMedia = NULL;  
	  
	Track videoTrack = GetMovieIndTrackType(inMovie, 1, kCharacteristicHasVideoFrameRate,  
              movieTrackCharacteristic | movieTrackEnabledOnly);  
			    
	if (videoTrack != NULL){  
	  
	  // get media ref. for track's sample data  
      movieMedia = GetTrackMedia(videoTrack);  
	  
	} else {  
	  
		printf("error: failed to get media track");  
	}  
	  
	return movieMedia;  
}

ofxVideoPlayerExtended coming soon!

jeremy

damian · July 23, 2008, 10:50pm

err,

i had this problem some time ago with 0.04, it seemed very straightforward to get frame-accurate seeking, since the setPosition seemed to be converted from a percentage to a frame number internally anyway:

ofVideoPlayerDamians.h:

  
class ofVideoPlayerDamians: public ofVideoPlayer  
{  
public:  
... void setPosition( long frame );  
...  
}

ofVideoPlayerDamians.cpp:

  
void ofVideoPlayerExtended::setPosition( long frame )  
{  
	//--------------------------------------  
#ifdef OF_VIDEO_PLAYER_QUICKTIME  
	//--------------------------------------  
	  
	TimeRecord tr;  
	tr.base 		= GetMovieTimeBase(moviePtr);  
	long total 		= GetMovieDuration(moviePtr );  
	long newPos 	= (long)frame;  
	SetMovieTimeValue(moviePtr, newPos);  
	  
	//--------------------------------------  
#else  
	//--------------------------------------  
	  
	float pct = frame/GetMovieDuration( moviePtr );  
	pct = CLAMP(pct, 0,1);  
	positionPct = pct;  // check between 0 and 1;  
	  
	//--------------------------------------  
#endif  
	//--------------------------------------  
	  
	  
}

(assuming OF_VIDEO_PLAYER_QUICKTIME is #defined somewhere - which it is, I believe)