A small demo: Processing local audio with OF, Pure Data, ofxPd, ofxOfelia and Emscripten


1 Like

so, did you put together all the changes you made for it to work?

@roymacdonald thanks. Yes (its all in my audioWorklet branch). I tried to make them as clean as possible (for me). I hope I didnt break any of the other methods, but so far I experienced no issues. I mean, I think about how it would make sense to implement the Java Script filesystem thing in general (I dont always want the possibility to drop mp3 into the website - on the other hand would it be nice to define a drop area in some cases. And it should not conflict with the already implemented filesystem). But it works (and most important, audioWorklet in and out and processing of the input with Pure Data). As ticks per buffer you need to set 2 now (which makes sense, because the audioWorklet processes 128 samples at a time).
Maybe also the possibility to choose between audioWorklet and scriptProcessorNode would make sense, as long as audioWorklet is only part of Emscripten branches (not the master branch). But I guess, audioWorklets will be implemented officially eventually and it works much better for me.

This demo shows how to do the same thing with audio from a video:

For that I just needed to add:

var source = AUDIO.contexts[context_id].createMediaElementSource(video);

to library_html5audio.js and make the video variable from library_html5video.js global (which isnt ideal, but it works well…). The bottleneck in this case seems to be the video with ofxOfelia (it needs a lot of cpu - and switched to another tab the sound is still running and cpu almost 0 - and without audio processing the cpu load is almost the same). Video runs much better directly in OF, like here: https://emscriptenvideoplayerexample.handmadeproductions.de/
But I would have to make a OF class from my ofxOfelia GUI library, to do all the graphics stuff there.