Record Quicktime video + audio, "succesful" with some issues

As I’m totally new around here, greetings to everyone!

I got commissioned with a project this week regarding the recording of video + audio to different movieclips to basicly make a digital guestbook for a small exhibition. Having no previous experience in C++/Openframeworks (and not being that much of a coder to start with) it turned out to be quite a burden but I managed to pull something off none the less.

After looking around on the forums I ran into the ofxQtVideoSaver which I applied to the MovieGrabberExample (as suggested by James-Patten) After some trial and error, hurray, videorecording!
Now on to the sound part.
I quickly ran accross the ofxQtAudioSaver (thanks Mantissa!) and got the example project running.

Time to combine both! ofxQtVideoSaver+ofxQtAudioSaver=ofxQtMovieSaver (attached)
It’s recording audio from the mono mic and video from the iSight to a Quicktime .mov file.

As I told you earlier, I’m a real beginner here and this thing is full of issues I can’t wrap my head around yet so I really could use some help on completing this.

Issues I’m having atm:
• The ofxQtAudioSaver example project gave me some problem out of the box as it was made for stereo recording and I was recording from my mono iSight mic. Changing NUM_CHANNELS to 1 didn’t seem to fix it, sound was pitched down. I “fixed” (more like hacked) it by hardcoding the ofSoundStreamSetup to 1 channel and keeping the NUM_CHANNELS to 2. Not the cleanest fix, but I haven’t figured out a way to fix this for now.
• The recorded audio runs out of sync with the video quite fast (after about 5secs) and still seems slightly off pitchwise, can’t tell for sure.

Any help is greatly appreciated, logically I’ll put this project up for grabs to anyone needing something similar!

Thanks in advance,
Kevin

QtMovieSaver.zip

Hey there! Recording sync’ed audio and video is quite tricky as a search of the forum will show…

You might find this example based on kritzikratzi’s ffmpeg add-on handy: http://forum.openframeworks.cc/t/export-audio-andor-video-using-ffmpeg/5123/18.](http://forum.openframeworks.cc/index.php/topic,5123.msg28570.html#msg28570)

It was done quickly to test what was possible…sometimes loses sync; sometimes holds sync :wink:

Anyway, see how you go…

M

Thanks mate,

The ffmpeg addon was the first thing I tried, had problems in the early beginning getting the sound to record properly, as was even more clueless at the time, changing the input channels from 2 to 1 fixed it. Now I have both the ffmpeg and the Quicktime versions “working”. Still running into the same issues with both though.

If anyone with more advanced knowledge could have a look at the my QT code, it would be much appreciated as of now, I have no clue how to get it to sync properly.

Thanks alrdy!

Back again!

Tried the ffmpeg approach again, recording vid + sound, both record fine, but get out of sync after a while still. I think this is due to the fact that the audiosamples are being added in the audioReceived() function and the videosamples in the update(). Different threads > running out of sync?

  
  
void testApp::update(){  
	vidGrabber.grabFrame();  
	  
	if( shouldRecord && movieMaker == NULL ){  
		initRec();  
	}  
	  
	if( !shouldRecord && movieMaker != NULL ){  
		movieMaker->finish();   
		movieMaker = NULL;   
	}  
	  
	if( shouldRecord && movieMaker != NULL ){  
		movieMaker->addScreen();  
	}  
}  
  
void testApp::audioReceived 	(float * input, int bufferSize, int nChannels){	  
	for (int i = 0; i < bufferSize * nChannels; i++) audioBuffer[i] = input[i];  
	if(shouldRecord && movieMaker != NULL) {  
		movieMaker->addAudio(audioBuffer, BUFFER_SIZE*NUM_CHANNELS);   
	}  
}  
  

I tried to just run the addAudio part in the update(), but no luck, it only gets me some soundclicks at the start of the vid. Any ideas on how to get the addAudio to run from the update? (also tried running the addScreen in the audioReceived, that didn’t seem to work either)

Thanks again!
Greets,
Kevin

Greetings Kevin,

thanks for the app – it’s really helpful.

I’ve been very suspicious lately about rtAudio and sample rates, and i think we will move to port audio in the future. If you run “audio midi setup” you can see the internal rate of the device, and I have a suspicion there some kind of problem with rtAudio and sample rates.

I’ve gotten good, non out of sync issues with you code doing the following:

a) set the sample rate to whatever audio midi setup tells you, ie:

#define SAMPLE_RATE 96000

b) add some code to calculate the actual time in seconds from frame to frame (instead of just plugging in 1/30.0 as the time per frame).

  
  
if (bRecording == true){  
		float timeNow = ofGetElapsedTimef();  
		float diffTime = timeNow - lastFrameTime;  
		saver.addFrame(vidGrabber.getPixels(), diffTime);   
		  
		lastFrameTime = timeNow; //ofGetElapsedTimef();  
		  
		  
		//  you can also pass in the frameRate:  
		//	saver.addFrame(vidGrabber.getPixels(), 1.0f / 30.0f);  // 30 fps for this frame  
		//	saver.addFrame(vidGrabber.getPixels(), 1.0f / 5.0f);   // 5 fps for this frame, etc....  
	}  
  
.....  
  
(in keyPressed)  
  
else if (key == 'r'){  
		bRecording = !bRecording;  
		if (bRecording) lastFrameTime = ofGetElapsedTimef();  
	}  
  

I hope this helps – I’ve just tested some longer recording and seems to work ok.

take care,
zach

Sorry for the late reply, seems I missed your post as the transition to the new forum seemed to have misplaced the thread’s notification.

Zach you’re an absolute savior :slight_smile:

Seems it was the “sample rate” messing with me all along (had no clue about the “audio midi setup”).
I already implemented a similar thing for the frames like you suggested.
Will post the full working thing one of these days!

Thanks again!

Greets,
Kevin

Hi, I’m new around here and hoping for few pointers…

I’m working on a project where I need to “buffer” live video & audio from a webcam for between 30 secs - 2 mins. I’ve been working with this example, and looking at features of http://www.jamesgeorge.org/ofxslitscan/ (ofxSlitScan) which provides part of the functionality (frame buffer).

Anyone got any ideas or suggestions?

Thank you!

John.

@Kevin & Zach: I’ve been trying Kevin’s code (top post) with the changes indicated by Zach and I cannot get sync’d recording. Seems to drift out of sync consistently -> after 20sec or so it is becoming out of sync (~0.5sec) by 45secs is VERY out of sync (1.5-2secs).

This happens everytime. I’ve tried a number of variations on the theme of setting time interval for the frame (including basing it on the number of audio samples recorded instead of elapsedTime), etc etc but nothing seems to get me sync’d recording.

Could one of you post some code and possibly an example of the video output you are getting? Also could you indicate what a “long” time of recording means? 30 secs? 60 secs? 10 minutes? An hour? (Ok so an hour won’t work with the audio buffering to ram, but you know where I’m going with this :wink:

M

Attached is a quick and dirty test of recording audio and video using QTKit…

No preview image, no setting dimensions, framerates, codecs etc - pure proof of concept…

It’s doing a much better job of sync’ing audio and video than the QT7 SDK or ffmpeg options - of course it’s mac only which is not great…

Anyways…will post if I make this more friendly during the week: ofxQTKitVideoRecorder anyone? Or perhaps just modifying James George’s (obviousjim’s) ofxQTKitVideoGrabber to allow recording is more in order?

Would still like to see a QT7 solution that works for Windoze too…

Mhmm

ofxQTKitVideoRecorderExample.zip

Ok decided to extend ofxQTKitVideoGrabber. More info and code at topic,6390.msg30477.html