Managing buffers/recording audio

Hello all, I’m starting a program that involves, in its basic form, taking in audio from the sound card(line in or mic) and then triggering it. I’m using ofSoundStream to do this. Its pretty skeletal at the moment and based off of the AudioInput/Output examples. I can get it to record and spit back audio but it sounds ‘flickery’ and appears to only come out of the left channel and very slightly slower then the original audio.

I can post specific code/sound samples if necessary/helpful (not at my personal computer at the moment) but I’m basically just taking in audio and putting it into a large vector in the for loop for recording. its akin to:

  
  
storage_for_channel[i + (bufferCounter * buffersize)] = input[i];  
  

It seemed relatively straightforward to do something like this but maybe and then basically in reverse for the output…but maybe I am misunderstanding something.

thanks in advance for any and all help!

hi Benjamin,

i haven’t used the of core audio stuff very much (honestly, i find it difficult to wrap my head around writing ordinary code to make sound, so i use Pd instead!). but i imagine that your problem is buffer underflow: the write buffer is not being written to fast enough, so it’s stuttering. the buffer size used in the examples (256 floats) means that at a framerate of 44.1khz the buffer needs to be filled 44100/256 = about 172 times per second, which is probably four or five times faster than your update()/draw() loop is being called.

another thing to take into consideration is that the sound read/sound write may not be happening in the order that you expect. for example, it’s possible that the audio driver is requesting output buffers in advance, and in chunks, so it might fetch 4 buffers in a row and then wait for a bit and then fetch 3 buffers in a row - it all depends on what else is going on in on your computer. against this, the the sound input might be coming in at a steady rate. this the reason that most audio applications will let you set the number of buffers to use internally: so that you can take it as low as possible (for better latency) without dropping any audio from buffer underruns.

to manage this, i think you’ll need at the very least to be double-buffering the audio a little bit. i’m not sure if this is what you mean when you say (bufferCounter*bufferSize)… in any case, when a new buffer becomes available (audioReceived is called) you fetch the data and stick it on the back end of a queue, and then when audioRequested is called, you dump the data on the front of the queue into the buffer. this will need to be implemented as a ring buffer or at least with some thoughtful memory management to avoid allocating memory and copying bytes too many times.

can you post a small example?

with rtAudio, the audio input and audio output are called in sync (ie, input then output) with the same sized buffers, so it should work to do input -> output, but you may need to buffer a bit. If you can post a small example, it would help.

take care,
zach

here’s what I have for the input. The booleans are just flags for mouse buttons.

  
  
void testApp::audioReceived(float * input, int bufferSize, int nChannels)  
{  
    //if you're rec-oh-din  
    if(RMode == true)  
    {  
        for (int i = 0; i < bufferSize; i++)  
        {  
            left[i]     = input[i*nChannels];  
            right[i]    = input[i*nChannels];  
  
            storage_l[i + (bufferCounter*bufferSize)]   = left[i];  
            storage_r[i + (bufferCounter*bufferSize)]   = right[i];  
        }  
  
        bufferCounter++;  
    }  
}  

and output

  
  
void testApp::audioRequested(float * output, int bufferSize, int nChannels)  
{  
    //if you're playin back:  
    if(PMode == true)  
    {  
            for (int i = 0; i < bufferSize; i++)  
            {  
                the_l[i]    = output[i * nChannels]   = storage_l[i + (playbackHead*bufferSize)];  
                the_r[i]    = output[i * nChannels]   = storage_r[i + (playbackHead*bufferSize)];  
            }  
    playbackHead++;  
    }  
}  

this seems pretty straightforward. would I need another intermediary array/vector to make sure everything is delivered in time?

I guess your code sort of makes sense, but it could be much simpler.

So you’d have a buffer, say called buffer, of length LENGTH and a play position counter playPos, and a record counter called recPos

  
  
#define LENGTH 441000 // 10 seconds  
  

in testApp.h definition

  
  
float buffer[LENGTH];  
int recPos;  
int playPos;  
  

then audio received would look like

  
  
void testApp::audioReceived(float * input, int bufferSize, int nChannels)  
{  
    //if you're rec-oh-din  
    if(RMode == true)  
    {  
        // this would be faster with a memcpy()  
        for (int i = 0; i < bufferSize*nChannels; i++)  
        {  
            if(recPos<LENGTH) buffer[recPos++]     = input[i];  
        }  
  
    }  
}  
  

and audioRequested -

  
  
void testApp::audioRequested(float * output, int bufferSize, int nChannels)  
{  
    //if you're playin back:  
    if(PMode == true)  
    {  
            for (int i = 0; i < bufferSize*nChannels; i++)  
            {  
                if(playPos<LENGTH) output[i] = buffer[playPos++];  
            }  
    }  
}  

You’d just have to remember, to start your recording, you’ve got to do:

  
  
RMode = true;  
PMode = false;  
recPos = 0;  
  

and to play

  
  
RMode = false;  
PMode = true;  
playPos = 0;  
  

This will only work if the number of channels in are the same as the number of channels out, but it shouldn’t be too tough to change the code for an arbitary number of channels.

I just tried what I typed, and it appears you must set the number of buffers in ofSoundStreamSetup to 1 to get it sounding ok. Let me know if it works for you!

thaaaaaaAAAAaaaaanks all

yes my code/variable naming is a bit verbose/silly, sometimes on purpose, but I was also basing it on the examples because I wasn’t sure what everything was calling and I was trying to get it to not break.

if I had two samples ready to go (two arrays/vectors or a matrix thereof) how would it work with streaming audio call to the output to get them to play back simultaneously? if you could point me in the right direction (old forum post or even just the term I should be looking for) that would be cool since I guess its very slightly OT

One thing I forgot to mention is that LENGTH is the maximum length a sample can be, so you probably want to save information about how long a sample was recorded for, rather than just stopping playback at LENGTH.

to play two sounds at the same time, you just add their samples. So say there was

  
  
float sound1[LENGTH];  
float sound2[LENGTH];  
  
int sound1Pos; // playback position pointer  
int sound2Pos;  
  
int sound1Length; // length of the sound in samples  
int sound2Length;  
  

you’d do something like this:

  
void testApp::audioRequested(float * output, int bufferSize, int nChannels)  
{  
    //if you're playin back:  
    if(PMode == true)  
    {  
            for (int i = 0; i < bufferSize*nChannels; i++)  
            {  
                // start with zero (silence)  
                output[i] = 0;   
  
                // add sound1 if there's any samples left to play back  
                if(sound1Pos<sound1Length) output[i] += sound1[sound1Pos++];  
  
                // add sound2 if there's any samples left to play back  
                if(sound2Pos<sound2Length) output[i] += sound2[sound2Pos++];  
            }  
    }  
}  
  

Adding 2 sounds might be quite loud, in which case you can write something like

output[i] *= 0.5;

which will half the volume of the sample.

As you can see, it quickly becomes complicated with more and more variables, so it would be a good idea to wrap up your sample playback functionality into a class…

Has anyone started to write a class for recording/playing audio? Something that would work like…

Sample1.length(44100); // 1sec-audio buffer
Sample1.record(start/stop, input, channels);
Sample1.play(start/stop, speed, position);

K.

Hi marek,

I put your suggestions into testApp.h & testApp.cpp - and it works well! Just post it here as a little app to share with others…

Would it be difficult to make a class out of it?

src.zip

Is there anyway to upload that src file again?