I’ve searched the forums but I could not find a working example.
I am trying to record a video with using a webcam and record the sound from the microphone, I’ve read at the forums that the only way seems to capture the video and sound seperatly save them in a folder and combine them with using ffmpeg?
what is the best method of recording videos with sounds?
Hi there,
I am now able to record audio and video together in a quicktime file.
I’ve used libsndfile from http://www.mega-nerd.com/libsndfile/ .
and used ofxvideosaver. I’ve modified some codes from http://forum.openframeworks.cc/t/recording-quicktime-with-sound-in-sync/775/0
How it works is:
#include "testApp.h"
#include "stdio.h"
#include <sndfile.h>
static bool bRecording = false;
//--------------------------------------------------------------
void testApp::setup(){
camWidth = 320; // try to grab at this size.
camHeight = 240;
vidGrabber.setVerbose(true);
vidGrabber.initGrabber(camWidth,camHeight);
saver.listCodecs();
saver.setCodecType(17); //ZACH FIX see recording quicktime with sound in sync on of forum
saver.setCodecQualityLevel(OF_QT_SAVER_CODEC_QUALITY_NORMAL);
saver.setup(320,240,"data/output.mov");
ofSetFrameRate(30);
ofBackground(255,255,255);
// 0 output channels,
// 2 input channels
// 22050 samples per second
// 256 samples per buffer
// 4 num buffers (latency)
ofSoundStreamSetup(0,2,this, 44100, 256, 4);
left = new float[256];
right = new float[256];
bufferCounter = 0;
drawCounter = 0;
sampleRate = 44100;
info.format=SF_FORMAT_WAV | SF_FORMAT_PCM_16;
info.frames = sampleRate*30;
info.samplerate = sampleRate;
info.channels = 2;
outfile = sf_open ("data/mydata.wav", SFM_WRITE, &info) ;
if (!outfile)
{
cerr<<"Error opening ["<<outfilename<<"] : "<<sf_strerror (outfile)<<endl;
}
}
//--------------------------------------------------------------
void testApp::draw(){
vidGrabber.draw(20,400);
// draw the left:
ofSetColor(0x333333);
ofRect(100,100,256,200);
ofSetColor(0xFFFFFF);
for (int i = 0; i < 256; i++){
ofLine(100+i,200,100+i,200+left[i]*100.0f);
}
// draw the right:
ofSetColor(0x333333);
ofRect(600,100,256,200);
ofSetColor(0xFFFFFF);
for (int i = 0; i < 256; i++){
ofLine(600+i,200,600+i,200+right[i]*100.0f);
}
ofSetColor(0x333333);
drawCounter++;
char reportString[255];
sprintf(reportString, "buffers received: %i\ndraw routines called: %i\n", bufferCounter,drawCounter);
ofDrawBitmapString(reportString,80,380);
}
void testApp::update(){
ofBackground(100,100,100);
vidGrabber.grabFrame();
if (bRecording == true){
saver.addFrame(vidGrabber.getPixels(), 1.0f / 30.0f);
// you can also pass in the frameRate:
// saver.addFrame(vidGrabber.getPixels(), 1.0f / 30.0f); // 30 fps for this frame
// saver.addFrame(vidGrabber.getPixels(), 1.0f / 5.0f); // 5 fps for this frame, etc....
}
}
static float adder = 0;
//--------------------------------------------------------------
void testApp::audioReceived (float * input, int bufferSize, int nChannels){
if (bRecording){
// samples are "interleaved"
for (int i = 0; i < bufferSize; i++){
left[i] = input[i*2];
right[i] = input[i*2+1];
}
bufferCounter++;
printf("%i\n",bufferSize);
sf_write_float(outfile, input, bufferSize*2);
}
}
//--------------------------------------------------------------
void testApp::keyPressed (int key){
if (key=='a')
{
bRecording = true;
}
if (key=='s')
{
bRecording = false;
sf_close(outfile);
}
if (key==' ')
{
saver.addAudioTrack("data/mydata.wav");
saver.finishMovie();
}
}
//--------------------------------------------------------------
void testApp::keyReleased(int key){
}
//--------------------------------------------------------------
void testApp::mouseMoved(int x, int y ){
}
//--------------------------------------------------------------
void testApp::mouseDragged(int x, int y, int button){
}
//--------------------------------------------------------------
void testApp::mousePressed(int x, int y, int button){
}
//--------------------------------------------------------------
void testApp::mouseReleased(int x, int y, int button){
}
//--------------------------------------------------------------
void testApp::resized(int w, int h){
}
now my problem is syninc. I have an empty frame at the end of the mov file, and the sound suddenly stops. Any ideas?
I’ve changed the code…
static int myFrameRate = 15;
void testApp::setup(){
camWidth = 640; // try to grab at this size.
camHeight = 480;
vidGrabber.setVerbose(true);
vidGrabber.initGrabber(camWidth,camHeight);
saver.listCodecs();
//saver.setCodecType(17); //ZACH FIX see recording quicktime with sound in sync on of forum
saver.setCodecQualityLevel(OF_QT_SAVER_CODEC_QUALITY_NORMAL);
saver.setup(640,480,"data/output.mov");
ofSetFrameRate(myFrameRate);
ofBackground(255,255,255);
// 0 output channels,
// 2 input channels
// 22050 samples per second
// 256 samples per buffer
// 4 num buffers (latency)
ofSoundStreamSetup(0,2,this, 44100, 256, 4);
left = new float[256];
right = new float[256];
bufferCounter = 0;
drawCounter = 0;
sampleRate = 44100;
info.format=SF_FORMAT_WAV | SF_FORMAT_PCM_16;
info.frames = sampleRate*myFrameRate;
info.samplerate = sampleRate;
info.channels = 2;
outfile = sf_open ("data/mydata.wav", SFM_WRITE, &info) ;
if (!outfile)
{
cerr<<"Error opening ["<<outfilename<<"] : "<<sf_strerror (outfile)<<endl;
}
}
and
void testApp::update(){
ofBackground(100,100,100);
vidGrabber.grabFrame();
if (bRecording == true){
saver.addFrame(vidGrabber.getPixels(), 1.0f / myFrameRate);
// you can also pass in the frameRate:
// saver.addFrame(vidGrabber.getPixels(), 1.0f / 30.0f); // 30 fps for this frame
// saver.addFrame(vidGrabber.getPixels(), 1.0f / 5.0f); // 5 fps for this frame, etc....
}
}
it works better.
i’ll try this week…
Actually this does not makes it a single file, it just adds an audiotrack information to the movie.
hey, i’ve actually made a bit of progress on this front, tho it’s still not completely working yet.
i successfully create a new audio track in QuickTime and fill it with noise encoded in PCM and saved it to disk. BUT only half of the audio (0.5 seconds) is being recorded. 
my code is based on this project: http://xmeeting.sourceforge.net/pages/index.php. here’s some of their relevant source code. http://archive.mildmanneredindustries.c-…-Recorder.m
anybody want to take a look and see if they can spot any errors? i’ve been staring at it all day!
http://mantissa.ca/oF/QuickTimeAudio.zip
thanks,
jeremy
ps - next step: copying sound from quicktime movies.
ok, i’ve got a preliminary version of a quicktime sound-recording addon working. it could easily be added into the functionality for ofxQtVideoSaver.
http://www.mantissa.ca/oF/ofxQtAudioSaver.zip
right now, it’s only recording in 8-bit integer, which doesn’t sound so amazing (i can’t even whistle without distortion).
i’m working on figuring out how to get the raw data from the microphone into the movie in a higher resolution format.
quicktime’s AddMediaSample2 seems to only take arrays of UInt8, so i assume that there’s a way to cast the array to make it UInt8 and adjust the audio stream description to reflect the correct data type.
jeremy
great I will test it soon
Hi, any progress on this? after trawling the forums it seems the best option is to record the audio output from OF in something else. would love to do this in my OF app.
I think the best option is to record audio and video seperatly and bind them using ffmpeg.
thanks. I only need audio but was hoping to record people interacting with a midi + audio app, then at the end of the day run a batch process to convert to mp3 and upload to web.
think if this project goes ahead I may write a separate app to record via mic and patch the first app into this one, and have them communicate via OSC or something
cheers.
okay,
it took a little bit longer than expected, but i was able to debug the code i’d uploaded a few months ago. now you can save a recording of the computer’s stereo audio input (mono and stereo) as a quicktime movie file. this records audio-only files with 32-bit linear-PCM compression.
[attachment=0:1hfkryfj]ofxQtAudioSaver.zip[/attachment:1hfkryfj]
it’ll take a little bit more work to be able to add video frames to the file as well. maybe version 2.0?
jeremy
Thanks heaps for going to the trouble of doing that!
can’t wait to try to try it out…
*edit* example works great
i notice the fps drops quite a bit once recording but if this doesn’t affect overall app performance too much it will be perfect
Does ofxQtAudioSaver work platform independent? Or is it mac only, because of quicktime-files?
ketchupok,
i think that there’s what i believe to be mac-specific code for file loading & creation (fsspecs) – but a little search on google tells me they might work for windows as well. do you wanna give it a try?
the addon code also includes core-audio, but i don’t believe that it requires it.
jeremy
@nay,
you could definitely speed things up by not displaying the waveforms and not using the left and right buffers to displaying the most recent audio samples. otherwise, it’s just copying data over – shouldn’t be an expensive process.
jeremy
hello
where can i download the ofxvideosaver addon?
can someone post a link to it, or upload it to this thread.?
Thanks + regards
Hi,
I was having a lot of trouble with audio and video being out of sync, and I wanted to share some info. One problem was a rounding issue for frame lengths in ofxQTVideoSaver, another was dropped audio frames due to buffer overruns. This was caused by the code writing audio straight to disk every audio callback instead of buffering it. Another problem was the audio and video streams not quite starting at the same time. I dont know what causes this last one. With these issues fixed it works pretty well.
More info and some code here:
http://jamespatten.tumblr.com/post/1471046364/software-recording-audio-and-video-in-sync
Will post more code later but need to clean it up first.
Great work, Jeremy and James!
I’ve been trying to integrate this audio/video recording function into my Quase-Cinema VJ software.
James, I’ll be in NYC next week, doing a Quase-Cinema workshop for a Brooklyn High School and a performance. Any chance I could drop by and see how you got this working?