ofxSoundEngine: a more complete extension of FMOD Ex into OF

extend
#1

I have been thinking of ways in which I could contribute to OF, and seeing that my area of expertise is sound, and my disappointing venture into using ofSoundPlayer to create some sound manipulation programs, I decided I wanted to look into making a more complete version of a FMOD Ex system. I studied a lot of different sources, for some time(being that my background stems from max/MSP), and finally came up with what I thought would work as a good start. Unfortunately for some reason I can’t get a sound to load into the engine, and get a return error from FMOD concerning an invalid parameter in my call to FMOD_System_CreateSound. I am stumped as to what I did wrong. Everything compiles ok, but I know C++ can be funny like that, being such a dynamic and powerful language.

I think it has something to do with the system I set up to create vectors and FMOD_SOUND pointers and properly utilize channels inside FMOD, but I just sorta winged it here, as my understanding of pointers and vectors is still kinda loose.

Anywho, anyone is welcome to take a look/use/improve upon how I did things… Please. I really would like to learn more about this part of things and be able to contribute more on the sound manipulation/DSP part of OF.

not included in the zip is FMOD api library, which my need altering to work properly, altough I think OF includes it somewhere… my be wrong… was planning on altering my include once I could make it stable. Also, the drum.aif is not included, just change that part or add a sound to your data dir.

Thanks in advance for any help/input/advice.

Arick

**PLEASE SEE LINK BELOW FOR CURRENT ZIP**

#2

So after taking a step back for a few days and coming back, I realized that my error was user-related. I forgot to run the initalize. :oops:

Anyhow, I will now be adding in and extending out support for channel groups, DSP, and eventually the 3D stuff, but that is last on my list.

Until then, this code works for basic management of multiple sounds from one system. Please let me know if there is anything I am doing wrong, and expect a big update and better example soon.

#3

Hey Psykel,

Cool endeavor. Something I wanted to do myself was work on making some more bindings to the FMOD API, so I’m glad someone is going to try to tackle it. Good luck, I’ll keep an eye on this when you start posting some more stuff!

Cheers,
Dimitri.

#4

I suspect I will release an update today or tomorrow that will incorporate the following:

*Full DSP functionality(creation, removal, rearrange, and parameter systems for any of FMOD’s built in DSP) per channel, group, or system.

*Function calls for: sound panning, channel group panning, sound levels, channel group sound levels.

*Various Bug Fixes (looping call, etc.)

*A much more complete example

I would also like to get a handle on the feature to create custom DSPs, but that will have to wait till I can get a better handle on that part of FMOD.

More soon!

#5

Hey,
I’ve worked a fair bit with FMOD, and I’ve made custom DSPs before. It’s actually really simple, there’s a good example in the SDK. I haven’t run your code yet, might have some suggestions though so I’ll take a look.

#6

Linked is an early version 0.2 of my ofxSoundEngine. Not everything that I wanted to implement so far for this version is there, but for testing purposes I wanted to supply a better working version/example. PLEASE, let me know what I can improve upon, change, or implement better, any and all help and input is greatly appreciated.

The example comes with a few loops loaded and efx added to them. You can enable/disable the effect on the string sample with ‘s’ and the effect on the drums with ‘d’

Cheers!

http://www.beatsmiths.org/ofxSoundEngine-ver0.2.zip

#7

Also, out of curiosity, how would one go about getting a project at addons.openframeworks.cc?
I have a few other random things I have been working on, and would like to set up an easier way to share code. I have looked at using google, but thought it would be a better use of your efforts to create that section. I registered there, but have failed to find the correct method of requesting a project.

#8

hi psykel,

I’m working on a game which uses ofSoundPlayer as well. It’s working okay but I’ve got like 10 different sounds in my game which need to be played directly and quickly after each other (at the same time as well). I noticed that there is some kind of delay when I want to play 2 or more sounds at the same time.

Do you maybe have an idea what’s going on? (I’ve added my code below).

and while you’re working on ofSoundPlayer; it would be great to get some information about the sound devices which are available. When I run my application on a computer w/o a sound device, ofSoundPlayer stopped it from working.

… about using addons.openframeworks.cc, as far as I know you can create an account at addons.openframeworks.cc and put your code/info over there. I’m using Google Code because I like to use SVN.

Roxlu

  
  
#ifndef SOUNDSH  
#define SOUNDSH  
  
#include "ofMain.h"  
  
class Sounds {  
	public:  
		void addSound(string sName, string sFile);  
		ofSoundPlayer* getSoundPlayer(string sName);  
		void playSound(string sName);  
		void setLoop(string sName, bool bLoop);  
		void setVolume(string sName, float fVolume);  
		void enable();  
		void disable();  
		bool isEnabled();  
	private:  
		map<string, ofSoundPlayer*> sounds;  
		bool enabled;  
  
};  
  
#endif  
  
----------------------  
#include "Sounds.h"  
  
void Sounds::addSound(string sName, string sFile) {  
	sounds[sName] = new ofSoundPlayer();  
	sounds[sName]->loadSound(sFile);  
}  
  
void Sounds::setLoop(string sName, bool bLoop) {  
	if (!isEnabled()) return;  
	ofSoundPlayer* sound = getSoundPlayer(sName);  
	if (sound == NULL) return;  
	sound->setLoop(bLoop);  
}  
  
void Sounds::setVolume(string sName, float fVolume)  {  
	if (!isEnabled()) return;  
  
	ofSoundPlayer* sound = getSoundPlayer(sName);  
	if (sound == NULL) return;  
	sound->setVolume(fVolume);  
}  
  
  
ofSoundPlayer* Sounds::getSoundPlayer(string sName) {  
	if (!isEnabled()) return NULL;  
  
	ofSoundPlayer* sound = sounds[sName];  
	if (sound) return sound;  
	return NULL;  
}  
  
void Sounds::playSound(string sName) {  
	if (!isEnabled()) return;  
  
	ofSoundPlayer* sound = getSoundPlayer(sName);  
	if (sound == NULL) return;  
	sound->setPosition(0.0f);  
	sound->play();  
}  
  
void Sounds::enable() {  
	enabled = true;  
}  
  
void Sounds::disable() {  
	enabled = false;  
}  
  
bool Sounds::isEnabled() {  
	return enabled;  
}
#9

In short I believe it is because of the way ofSoundplayer implements the FMOD_System. Each instance creates a new system, so function calls to play are not tied together. My addon fixes this, as it has no ties to the original ofSoundplayer, and implements proper channels and soon proper soundgroups and channelgroups. I have been able to achieve what you are looking for with my current version, which I will post soon(uber busy this week). So if you give me a bit, and once I get to my computer, I can give you a new version geared to what you want. More later on the device thing, hard to type on my phone.

#10

had a question on preference, if I could get opinions from some of the more advanced folk, maybe even developers and admin. I have been struggling with a good way to create a proper system, and how much I should take care of under the hood VS giving control to others and simplifying the process. Currently my add-on handles all the memory management of multiple sounds and DSPs, and you basically step through everything with function calls in your main application. These functions work off of keys or string names, and will return keys to the main application when necessary. While conceptually this makes things easier in terms of management, the syntax sometimes clunky, and sometimes Im really just writing a management system ontop of FMOD. SO… I branched my project and started working on a object based system in which gives more management control (vectors, arrays, or the such) responsibility over to the programmer. In this version, there are objects for ofxSoundSystem, ofxSound, ofxDSP, ofxSoundGroup, and ofxChannelGroup (currently). You can create and mange these objects as you see fit, and the syntax seems like it will be a tad bit easier once everything is up and going, but a little more on the front end.

The way it works (sorta, typing from memory)

  
  
ofxSoundSystem system;  
ofxSound drums;  
ofxDSP echo;  
  
system.init();  
drums.load("drums.aif");  
echo.init(DSP_ECHO);  
  
system.loadSound(&drums);  
  
drums.addDSP(&echo);

Also, I was considering on this branch to make ofxDSP a base class and create different ofxDSP objects from it, like ofxEcho, ofxFlange, etc. This I can see as making easier to access params, but possibly harder to manage? or should I care? What would everyone else like to see?

#11

@Roxlu - each instance of ofSoundPlayer does not create a new FMOD system each time, as the global bFmodInitialized is effectively static (it should really be labelled so), so I’m not sure what the timing problem was above. The code you posted above looks fine. Send me a more complete example with media files and I will have a look. Are you using mp3s or wav/aiff files? Try using wav/aiff…that way there’s no decoding happening…although it shouldn’t really be noticeable.

@psykel - some suggestions…
For OF stuff in general I think the drive is to make things as simple as possible, without losing any of the important features.

Memory management should if possible be kept far away from the user.
Having vectors of sounds at the user level is of course fine. The user should be able to place the sounds in whatever structures they want, but at the internal level there should be reference counting of sounds and clean allocation/deallocation.

In terms of your proposal, I think the soundgroups and channelgroups should not be accessible to the user…that stuff should be under the hood…why would you need them at this stage?
They should however be accessible to people who want to extend the engines classes at a later stage.

Checkout the FMOD soundmanager here:
http://www.ogre3d.org/wiki/index.php/Fi-…-anager.cpp

You can get some tips from that implementation, for example it would be great to use indices either instead of or in conjunction with filenames or “keys” to reference sound samples. That way a program could step through sounds easily using a for loop and the index of each sound.

A sound engine should be able to play and create both 2D and 3D sounds. The engine should also know whether it is being used for a 2D sound application (i.e. ignore updates related to 3D positioning) or 3D (update the necessary parameters).

DSP stuff initially looks really easy, but actually can get very complex.
Consider being able to chain multiple DSP objects to every sound (DSP in series) and also be able to apply a single DSP (such as a costly reverb for example) to multiple sounds (in parallel). Both “series” and “parallel” solutions would need to make the DSP parameters accessible to the user and be totally transparent (in terms of channelgroups etc). You would also need to give the option to remove any of these DSPs at any time.

oh yeah and ofxDSP should def. be a base class.

You may also want to have the SoundSystem be a singleton, just like in the code above. This would be necessary when access sound system methods across different classes in different files. If a singleton sounds too user unfriendly, then just wrap the calls at a higher level…i.e.

  
  
SoundSystem::playSound(int index)  
{  
     SoundManager::getSingletonPtr()->playSound(index)  
}

Some advanced suggestions:

  • Nonblocking Loading of sounds (do it in an ofxThread and report when they are all loaded)
  • 3D Positional Sound (this is actually super important, and should be compatible with ofx3DUtils
  • Sound Occlusion (importing geometry into a scene to occlude sound (could be compatible with ofx3DModelLoader…although this would probably require a scenegraph at some stage)
#12

It’s funny because most of your suggestions we have actually implemented. (singleton, groups under the hood, and dsp suggestions).

I haven’t uploaded the newest code yet, me and roxlu are trying to make sure everything works well together. We have also fixed gapless playback also (buffering was our main issue so we implemented streaming and stitching features.)

3d stuff is last on the list but I plan on doing this also, just want to solidify 2d first.

Also we have made it optional to use ofxsoundengine management. You can either use the class or the individual objects and manage them on your own.

We have setup a google code for the addon, and as soon as I commit the latest code I will post the link. Good to know we are heading in the right direction.

#13

Sounds great! Looking forward to the code :wink:

Did you ever figure out what the problem was regarding the playback delay that Roxlu had?

Also, the orginal ofSoundPlayer class was writting in a mixture of C and C++. It would be really nice if you just used straight C++ and also used the C++ FMOD api instead of the C one currently found in OF. I guess you’ve probably already done this though!

…and one last thing, it would be great to be able to easily be able to extend ofxDSP to create custom DSP units…

#14

First: there were a few odds and ends in my last post, I was typing from my phone, and hate to do so on boards. So for that, I apologize. All of those suggestion were of major concern for me, and I am really trying to figure out the best way to make something useful.

Now, as to Roxlu’s problem (from what I have gathered anyways, as we only recently started working together, but he says that what we’ve got is working better.)

From what I can tell, it was fixed with the use of FMODs CreateStream instead of CreateSound(which ofSoundPlayer uses), combined with another feature of FMOD. Now I don’t know how is specific program is playing sounds, so I don’t know if he is using this, but there is also a feature in FMOD that I exposed that creates what is called a “Sentence” (composed of what they call Sub-Sounds). Basically you create the “container” stream, and then you can assign any sounds that you have loaded previously (Streams Only) to the container via a flag, tell the container what order to play them in, and then any time you tell it to play, it does it in the order you told it, with out gap, back to back. I need to clean up this and make the creation/management simpler, but it works nicely and I can see it being very useful.

Currently ofxSoundEngine is not a singleton, though the thought is interesting and I am considering that, but my class implementation/handling of FMOD’s System object is singleton, which addresses the issues I was having with ofSoundPlayer. One System object must be shared among all objects/classes created, and can not be copied/duplicated. So this allows the user to have all main objects (ofxSoundEngine, ofxSound, and any of the DSP units) work under one system. You can extend any of the objects without any issues, and all objects will work together. When attempting to extend ofSoundPlayer and create a separate class to embody so DSP processing, it was not working. I don’t claim to understand deeply about static objects, but as soon as I made a part of a singleton design, any class I make just grabs a pointer to the system object and we are good to go.

DSP is hashing itself out currently, with two steps forward and one back, but most of your suggestions are great. SoundGroups/ChannelGroups are what determines the issues of Parallel and Series(for the most part) that you mention, so that is why I am pushing that stuff into both the ofxSound and ofxDSP base object so that it is as simple as possible to deal with chaining.

3D (including occlusion and position) will all be available *eventually*. There are tons of things addressing 3D in FMOD, and I want it to work well with other addons as you say. That being said, I started this project for 2D needs, and want to knock that out of the park first. Plus, 3D being a huge area and one that I am slightly new to; currently my plan is to make my objects so that it will be easy to just expose or possibly extend these later.

I am trying hard to make this a simple and powerful library, and not just a wrap of the API, which is proving fun, yet difficult. This is my first big C++ project, so I am learning TONS. This is what is causing my slow speed, and I apologize.

The code currently follows a lot of the design characteristics of ofSoundPlayer as it started as an extension of it, but turned into it’s own system. Therefore I currently am using the C APIs, but am already working towards changing that also, down the line. :wink:

More to follow soon. I need to create a stable version of the sound portion (DSP is in the middle of a rehaul at the moment) so I can post up code. Source Repository is set up at ofxsoundengine.googlecode.com though currently the source is 4 days old, and may be buggy.

#15

Hmmm, the “sentence” technique you’re describing is typically used for gapless playback of dynamic/interactive audio in computer games. However the FMOD regular sample timing for sounds is pretty good (unless you have a very busy scene and sounds are becoming virtual due to hardware limits), so you shouldn’t notice any timing issues. The stitching stuff is where you might need sample accurate back to back playback of music or voice audio. Having said that, if you want your own sample accurate playback, it’s maybe best to write your own in something like ofSoundStream.

Yeah maybe don’t worry about 3D for the moment…just get something solid working for 2D. If I have time I might jump in and tackle some 3D stuff as I might need it in the near future.
3D occlusion is probably last on the list of todo’s…although it’s a very nice sounding one and I’ve never actually seen it in action yet.

#16

well from what I know, he is creating a game, and needed to do just that, so I guess it worked out for him, and as for me, I intend to make interactive musical installations and instruments, so this seemed to work for that, at least for now. Now I think there may be a simpler way to accomplish this, for example there is a callback for when a sound finishes playing… but Im not to sure on how to use this with of properly… ::shrug:: maybe you could enlighten me.

#17

[quote author=“roxlu”]hi psykel,

I’m working on a game which uses ofSoundPlayer as well. It’s working okay but I’ve got like 10 different sounds in my game which need to be played directly and quickly after each other (at the same time as well). I noticed that there is some kind of delay when I want to play 2 or more sounds at the same time.

Do you maybe have an idea what’s going on? (I’ve added my code below).

[/code]
[/quote]

Hey I think this is to do with FMOD needing FMOD_Update to be called every frame.
Its a bug you might not notice as for the most part you don’t need to call it - but when doing heavy sound stuff / lots of sounds it is the way FMOD figures out how to manage its resources. So you might get issue like sounds not playing or triggering late.

In 0061 we have ofSoundUpdate() which should be called in testApp::update once per frame if you are doing heavy sound stuff.

Maybe this helps with the issue you were having?

Theo