How to play sound instantaneously at event (win latency issue)?

advanced
#1

Hello all!

For a project I’m building sort of a video-sequencer in Windows (vs 2012). Each ‘node’ of a sequence consists of one plane in 3d. These planes are textured with videoframes and are animated with FX, translated and scaled when they perform/play video. A node also holds a separate audio file that is played back via multichannel output. To this end I incorporated vcucolo’s ofxMultiSpeakerSoundPlayer in the project. This add-on is based on ofSoundPlayer so it uses FMOD.
Several events can trigger nodes: MIDI, OSC, keypress, … So far so good…

The problem i’m facing is that the sound never really plays instantaneously when an event triggers it. For example, when I load a mono, 16bit, 44.1kHz wave file that really starts at position 0.0 (checked in an DAW) and play back it later through for instance a keypress event, the sound only comes out of the hardware a 150/200 ms later. This is a rough guess, quite difficult to time :wink:
It would be nice if I can get them to play really tight to the event, since a lot of the videos have drumming as content so every delay is very noticeable. All the samples are mono, an last between 0.5 - 3 seconds.

I’m also incorporating Resolume in this project for other parts and I can play back a combo of video/sound directly when triggered via MIDI, but only after setting the driver to ASIO. Both my onboard sound device (asio4all on a 2009 MBpro) and the projects RME fireface 800 produce the sound instantaneously…

So I started digging into FMOD to check if I can set the driver to ASIO (for lower latency):

FMOD_System_SetOutput(sys, FMOD_OUTPUTTYPE_ASIO);

But this always gives an error:

FMOD error! (66) The number of allowed instances of a plugin has been exceeded.

I went ahead and downloaded the FMOD api to check for more possible options. When I set the DSP buffersize like:

FMOD_System_SetDSPBufferSize(sys, 512, 1);

(instead of 1024) it doesn’t change a thing.

I’ve been chewing on this issue for the last two days now but I’m not getting any closer or better performance.
Can somebody give me a hint on how to manage this? Am I right that ASIO is important here? Maybe there are other factors at play here?
(I should still try to test my app on another machine)

Thanks!
Vincent

#2

I don’t know the solution to it in OF, but you can use an external program to play audio for complex setups.
.
For example, Max MSP or PD can receive OSC input and you can let the sound play there. Then you disconnect audio completely from OF and play it back in a program specifically designed for it.

In max’ DSP settings, you should select an asio sound driver.
Also, using the “dac” object, you can output to as many channels you want.

#3

hello @Kj1, thanks for your input!

I was already in the process of splitting things up between max6 and OF, because, as you say, max is specifically designed for these things and surround sound is indeed not really an issue there. Actually max6 performs quite well for video sequencing as well (coupling jit.matrixset with buffer~)…

However, I really wanted to know how to get latency free playback in OF and in the end managed to do this with ofSoundStream. Together with the Maximilian library I’m able to play the wav samples through ASIO with no apparent latency.

(BTW: i can’t seem to post code properly when I copy/paste out of vs. The empty space in the code below shouldn’t be there. I’m using the <pre><code class="cpp"></code></pre> ? Also using the button for preformatted text only indents the first line…)


// 2 output channels,
// 0 input channels
// 22050 samples per second
// 512 samples per buffer
// 4 num buffers (latency)
int bufferSize        = 512;
sampleRate             = 44100;
volume                = 0.8f;
bDrum                = false;
lAudio.assign(bufferSize, 0.0);
rAudio.assign(bufferSize, 0.0);
soundStream.listDevices();
//if you want to set the device id to be different than the default
soundStream.setDeviceID(2);     //note some devices are input only and some are output only 
soundStream.setup(this, 2, 0, sampleRate, bufferSize, 4);
hats.load(ofToDataPath("1xhats.wav", 1));
bass.load(ofToDataPath("1xbass.wav", 1));


//--------------------------------------------------------------
void ofApp::audioOut(float * output, int bufferSize, int nChannels){
    //pan = 0.5f;
    float leftScale = 1 - pan;
    float rightScale = pan;
    if ( bDrum == true){
        for (int i = 0; i < bufferSize; i++){
            float smp = bass.playOnce(1.0) + hats.playOnce(1.0);;
            lAudio[i] = output[i*nChannels    ] = smp * volume * leftScale;
            rAudio[i] = output[i*nChannels + 1] = smp * volume * rightScale;
        }
    } 
}