Generative Sound Synthesis in OF Core

I recently started playing around with the p5 Sound Library which I think does a great job of making the coding of rich audio synthesis quick and easy. I would love to see a similar sound library at the core of OF. What’s interesting to me is how different classes and their corresponding functions work not so much the fact that it’s use is for web audio. I literally just want to take all those classes and make the same thing for OF. I’m still new to c++ but would love any feedback on where I would even begin to maybe start writing such a library. Thanks in advance.

Judging from the classes, p5 is probably wrapping the Web Audio API, which is a pretty handy set of audio primitives and algorithms built into browsers. You could check out that document if you want a decent spec to conform to (assuming you’re mostly interested in the way it all fits together).

There are a few OF addons covering similar ground, handling audio primitives like waveform generation etc. ofxMaxim/Maximillian, ofxTonic and ofxStk are a few good ones. ofxAudioUnit will let you chain in AU plugins if you’re on OSX.

The general concept of chainable bits of audio code has been brought up for the OF core a few times, but the implementations haven’t really been mature enough for inclusion. If you’re interested in writing something like that I’d be excited to see where you go with it :smile:

Ok, so after this initial thought I put together a small addon called ofxGenSound. It’s mainly a starting point for how I envision what an implementation of synthesis methods in OF might look like. The example I included simply plays a tone with an envelope when any key is pressed, mouse x axis controls the frequency and the mouse y axis controls a filter cutoff. Would love to get some people in the community who would be interested in contributing to this one. I think it could be super useful in the future!

I’m currently working in an ambisonic spacialization addon and I feel that openFrameworks lacks some basic audio functionality.
Though the ofSoundPlayer works well and has some very nice functionality, not being able to read it as a buffer limits it’s usage for DSP. I’ve seen that version 0.9.0 includes an ofSoundBuffer class, is that what this is for?
It’d be nice to also have just the 4 basic wave shapes that @michaelpromeo sugested, but his implementation might be a bit too heavy. The most efficient solunt would be to read the values from a table using interpolation (specialy for the sine wave).
I say that because while doing my addon for ambisonics, I had no way of creating examples without using other addons. As I wanted the examples to be read out of the box, I made some oscillators very similar to the ones proposed here. The thing is, without some basic stuff out of the box it becomes hard to implement new audio addons with new functionality. You’re forced to reinvent the wheel everytime.

All that said, I’d love to contribute to this idea and i think we should focus on two things to begin with:

1 - giving easy acces to playing soundbuffers through ofSoundBuffer (@admsyn is this already done? didn’t find either tick or play methods in the class) or reading the buffer of a ofSoundPlayer.

2 - creating an efficient ofOscillator with very simple usage.

The focus should be in design. With that ready, well documented and compatible with the rest of core we can move on to adding more advanced features (that in the mean time may be done with the existing addons).

Hey @michaelpromeo and @cabral1349, thanks for taking up the initiative on this!

@cabral1349:

I’ve seen that version 0.9.0 includes an ofSoundBuffer class, is that what this is for?

Not really, the goal with it is just to get away from passing around a raw float * and sample count as arguments to sound callbacks (C-style). It also has a few utility functions you might want when dealing with sound, but it’s basically just a container.

Though the ofSoundPlayer works well and has some very nice functionality, not being able to read it as a buffer limits it’s usage for DSP.

The issue has more to do with the underlying libraries often being a bit of a black box, and not that anybody hasn’t considered that :slight_smile: The black box issue is tied into a larger issue of finding / wrapping a suitable FMOD replacement, which no-one’s done yet (not to say one doesn’t exist, I’m not aware of it though). The AVFoundation player on iOS has the same issue I believe.

Personally I have my own utils for this sort of thing, but it takes advantage of the fact that it doesn’t also need to support Windows, Linux, Android etc.

  1. giving easy acces to playing soundbuffers through ofSoundBuffer (@admsyn is this already done? didn’t find either tick or play methods in the class) or reading the buffer of a ofSoundPlayer.

These are actually more different than they may seem. ofSoundBuffer doesn’t have a play() because that sort of functionality is out of its scope. It may make sense to have an ofSoundPlayer implementation that just takes an ofSoundBuffer and steps through it as an in-memory sound player. This was considered for 0.9.0 but wasn’t finished.

“Reading the buffer of a ofSoundPlayer” is a different problem and has more to do with convincing FMOD / AVFoundation etc to give you a raw buffer. I doubt this is impossible but it’s at least not trivial. If there’s a straightforward way to do it though, I’d be happy to hear it :slight_smile:

@michaelpromeo & @cabral1349:
For oscillators in general, the issue isn’t so much implementing the raw oscillator itself as it is about all the other stuff that’s involved. Some way of creating chains / graphs of sound primitives, dealing with threading issues, giving people that write addons a nice and sane API to use so it’s actually fun to wrap up your sound thing for other people, etc

Intuitively, I feel that the use case of having just oscillators around doesn’t really cover too many use-cases aside from maybe a computer music 101 situation. If people are going to have to pull in another lib anyway to finish the job it’s not really helping that much, and the theoretical other lib will probably have oscillators that better integrate with their system.

Oscillators inhabit this weird grey area where they’re really so simple it seems negligent to not just drop some in. The issue is that you never really want just an oscillator, though, and then we’re quickly into the territory of having to support another bunch of code that really isn’t so simple anymore :slight_smile: This is just my opinion though, and I’m certainly not the be-all-end-all of things that get into OF. I’d be happy to hear what you think!

It’s partly just a time commitment issue, this could be done but just consider the potential future support issues.

I understand the issues regarding FMOD, I looked at the source code and it’s definetly not trivial to get buffer access.

@admsyn your point about the oscillator is fair, but consider this from a audio-developper perspective:
Say you want to make a distortion plug-in for oF. If you have a very basic oscillator built-in (and that is as simple as adding 1 header file to oF) you can do your work without needing to consider the other audio libraries. I think this is very important to improve the audio part of oF.

ofxMaxim, ofxStk and ofxTonic have a lot of very cool features, but they all achieve pretty much the same goal.

ofxAudioUnit is more interesting in the sense that it allows you to build an AU and then use it with oF, but that’s harder then doing an addon for most people (me included).

My idea is that if you have a per sample oscillator in oF, you can use it to develop audio classes that don’t revolve around a whole new DSP chain.

Of course this may expand in the future, but you can have lots of fun just with some very basic oscillators!

Here’s an oscillator I did (tested for sinewave only) that I think could be added with no compatibilty issues. I expanded on some ideas of @michaelpromeo. “Hoa” is the acronym for the ambisonics addon I’m finishing and wich includes this oscillator :

//
//  ofxHoaOscillator.h
//
//  Created by Claudio Cabral on 21/03/15.
//


#ifndef __OFX_HOA_OSCILLATOR__
#define __OFX_HOA_OSCILLATOR__

#define OSC_BUFFER_SIZE 4096

#include "ofMain.h"

namespace hoa {

    enum oscType { OF_SINE_WAVE, OF_TRIANGLE_WAVE, OF_SAWTOOTH_WAVE, OF_SQUARE_WAVE, OF_NOISE_WAVE  };
    
template <typename T> class ofxHoaOscillator {
    
private:
    
    T _freq;

    T _phase;
    
    int _index, _sampleRate;
    
    
    
public:
    T _buffer[OSC_BUFFER_SIZE];
    ofxHoaOscillator();
    ~ofxHoaOscillator();
    
    void setup(int sampleRate, oscType waveform);
    void setFrequency(T frequency);
    void setPhase(T phase);
    
    T tick();
    
};
}

using namespace hoa;

template <typename T>
ofxHoaOscillator<T>::ofxHoaOscillator(){
    
    _sampleRate = 44100;
    _phase = 0;
    _freq = 440;

}
template <typename T>
ofxHoaOscillator<T>::~ofxHoaOscillator(){

}
//SETUP
template <typename T>
void ofxHoaOscillator<T>::setup(int sampleRate, oscType waveform)
{
    for (int i = 0; i<OSC_BUFFER_SIZE; i++) {
        
        switch (waveform) {
                
            case OF_SINE_WAVE:
                
                _buffer[i] = sin (i * TWO_PI/4096.0);
                
                break;
                
            case OF_TRIANGLE_WAVE:
                
                if (i <= 2048 ) {
                    _buffer[i] = -1.0 + i * (1.0/1024.0) ;
                    
                } else {
                    _buffer[i] =1.0 -  (i-2048) * (1.0/1024.0);
                }
                break;
                
            case OF_SAWTOOTH_WAVE:
                
                _buffer[i] = -1 + i * 1.0/2048.0;
                break;
                
            case OF_SQUARE_WAVE:
                
                if (i<=2048) _buffer[i]=-1;
                
                else  _buffer[i]=1;
                
                break;
                                    
            case OF_NOISE_WAVE:
                                _buffer[i] = ofRandomf();
                
                break;
                
            default:
                _buffer[i] = 0;
                break;
        }
        
    }
    
}
template <typename T>
T ofxHoaOscillator<T>::tick() {
    
    _phase += (4096./(_sampleRate/(_freq)));
    
    if (_phase>=4096.0) _phase-=4096.0;
    
    _index = floor(_phase);
    
    if (_index < 4095) {
        return _buffer[_index] * (1.0 - (_phase - _index)) + _buffer[_index+1] * (_phase - _index);
    }
    else if (_index == 4095) {
        return _buffer[_index] * (1.0 - (_phase - _index)) + _buffer[0] * (_phase - _index);
    }
    
    else { return 0;}
    
}

template <typename T>
void ofxHoaOscillator<T>::setFrequency(T frequency) {
    
    _freq = frequency;
}

template <typename T>
void ofxHoaOscillator<T>::setPhase(T phase) {
    
    _phase = phase * 4096.0;
}

#endif /* defined(__exampleOneSource__ofxHoaOsc__) */

calling it would be as simples as declaring in the header

ofxHoaOscillator<float> osc;

and then

void ofApp::setup(){
osc.setup(44100, OF_SINE_WAVE);
osc.setFrequency(440);
}

void ofApp::output(float * output, int bufferSize, int nChannels){
for (int i = 0; i< bufferSize; i++){
output[i*nChannels] = osc.tick();
}
}

I think it will really help audio developers to have something like that