webMidi prototype

Jona · December 9, 2021, 4:54am

I took my webMidi experiments and tried to implement webMidi for Emscripten and OF with a more general approach. Basically, you just need to add the --bind flag to the Emscripten compiler and replace the template.html with the one that is in the bin folder.
For the webMidi functionality I started with this (great) code: https://github.com/cwilso/midi-synth/blob/master/js/midi.js added midi out and made some adjustments…

Here is the code (it is put upon the graphicsExample…): https://github.com/Jonathhhan/Pure-Data-Ofelia-Emscripten/tree/main/webMidiExample

And here you can try it (midi out works with mousePressed and mouseReleased):
https://webmidiexample.handmadeproductions.de/

It could be quite simple to implement…
On the OF side the only changes are in ofApp.cpp, basically two functions (I tried to keep it simple - and maybe some of the midi functionality is missing in my implementation).

For midi in:

void midiIn(int channel, int pitch, int velocity) {
	inChannel = channel;
	inPitch = pitch;
	inVelocity = velocity;
}

EMSCRIPTEN_BINDINGS(Module) {
	emscripten::function("midiIn", &midiIn);
}

and for midi out:

EM_ASM_(window["setMidiOut"] = ([128, 52, 32]));

In html.template I added those 3 lines:

    <b>Input Ports:&nbsp;</b><select id="midiIn"><option>-no MIDI-</option></select><br></p>
    <b>Outputs Ports:&nbsp;</b><select id="midiOut"><option>-no MIDI-</option></select></p>
    <script type="text/javascript" src="midi.js"></script>

Sadly, webMidi does not seem to work with Firefox, but with Chrome, for example, it works very well (latency seems quite small)…
(could also be possible to choose the midi device from the Emscripten app, but didnt figured that out yet - and I made a selector for GM Instruments, but thats only needed in some cases…)

And this way its possible to send midiOut with variable array length:

size_t lengthOfArray = sizeof array / sizeof array[0];
EM_ASM_(
var data = new Uint32Array(HEAPU32.buffer, $0, $1);
window["setMidiOut"] = data, array, lengthOfArray);
}

And this for midiIn with variable length:

void  midiIn(std::string ev) {
unsigned char* uEv = (unsigned char*)ev.c_str();
inChannel= uEv[0];
inPitch= uEv[1];
inVelocity= uEv[2];
}

EMSCRIPTEN_BINDINGS(Module) {
	emscripten::function("midiIn", &midiIn);
}

(Of course it is also possible to send other data between C++ and Java Script with the same methods - and, I am not sure if they are well written. I am still a C++ beginner…)

Jona · December 12, 2021, 6:12am

Sadly I get an error, when I try to use ofxPd and webMIDI together.
I think it is because the emscripten and the pd namespace are conflicting somehow…
WebMidi and ofxPd alone works well.
I can post the whole code if it helps (would be happy about a solution).
This is the error message:

/home/jonathan/Schreibtisch/openFrameworks-AudioWorklet/addons/ofxPd/pdExample/src/ofApp.cpp:19:2: error: unexpected namespace name 'pd': expected expression
        pd.startMessage();
        ^
/home/jonathan/Schreibtisch/openFrameworks-AudioWorklet/addons/ofxPd/pdExample/src/ofApp.cpp:20:2: error: unexpected namespace name 'pd': expected expression
        pd.addSymbol("midiIn");
        ^
/home/jonathan/Schreibtisch/openFrameworks-AudioWorklet/addons/ofxPd/pdExample/src/ofApp.cpp:21:2: error: unexpected namespace name 'pd': expected expression
        pd.addFloat(ev[0]);
        ^
/home/jonathan/Schreibtisch/openFrameworks-AudioWorklet/addons/ofxPd/pdExample/src/ofApp.cpp:22:2: error: unexpected namespace name 'pd': expected expression
        pd.finishList("fromOF");

And this is the conflicting code:

#include "emscripten.h"
#include "emscripten/bind.h"

using namespace emscripten;
	
void midiIn(std::string ev) {
	unsigned char* uEv = (unsigned char*)ev.c_str();
	pd.startMessage();
	pd.addSymbol("midiIn");
	pd.addFloat(uEv[0]);
	pd.addFloat(uEv[1]);
	pd.addFloat(uEv[2]);
	pd.finishList("fromOF");
}

EMSCRIPTEN_BINDINGS(Module) {
	emscripten::function("midiIn", &midiIn);
}

Jona · December 13, 2021, 1:41am

It works with ofEvent, this is the belonging part of the code:

using namespace emscripten;

ofEvent<std::string> midiChangedEvent; 	

void midiIn(std::string ev) {
	midiChangedEvent.notify(ev); 
}

EMSCRIPTEN_BINDINGS(Module) {
	emscripten::function("midiIn", &midiIn);
}

void ofApp::midiChanged(std::string & ev) {
	unsigned char* uEv = (unsigned char*)ev.c_str();
	inChannel = uEv[0];
	inPitch = uEv[1];
	inVelocity = uEv[2];
	ofLog(OF_LOG_NOTICE, "the number is " + ofToString(inPitch));
	pd.startMessage();
	pd.addSymbol("midiIn");
	pd.addFloat(inChannel);
	pd.addFloat(inPitch);
	pd.addFloat(inVelocity);
	pd.finishList("fromOF");
}

//--------------------------------------------------------------
void ofApp::hSlider_1onMousePressed(float & e){
	pd.startMessage();
	pd.addSymbol("sliderTest");
	pd.addFloat(e);
	pd.finishList("fromOF");
}

//--------------------------------------------------------------
void ofApp::setup() {
	ofAddListener(hSlider_1.onMousePressed, this, &ofApp::hSlider_1onMousePressed);
	ofAddListener(midiChangedEvent, this, &ofApp::midiChanged);

Jona · December 15, 2021, 5:12am

I worked on both, the webMidi implementation and the communication between Emscripten and OF.
I would say webMidi works very well now (but not with Firefox and Safari, because they dont support webmidi…).

One drawback is, that midi out only works with audioWorklets together, if there is a user event (otherwise its not in a web enviroment, because audioWorklet puts the code in a worker (as far as I understand it)…). It would be helpful, if someone knows how to send the midi out data to the main thread…

I am quite sure, that it is possible to use it together with audioWorklet, just dont know how to do that.

For now the midi.js file needs to be put into this folder: https://github.com/openframeworks/openFrameworks/tree/master/addons/ofxEmscripten/libs/html5audio/lib/emscripten
Here is an example and the code (also tried my GUI elements). :
https://webmidimarkov.handmadeproductions.de/
https://github.com/Jonathhhan/webMidiMarkov

Jona · February 16, 2022, 3:49pm

It seems Purr Data uses the official webMidi Library (v2.5.2) for Pure Data / ofxPd:

github.com

cuinjune/purr-data/blob/master/libs/webmidi/webmidi.min.js

/*

WebMidi v2.5.2

WebMidi.js helps you tame the Web MIDI API. Send and receive MIDI messages with ease. Control instruments with user-friendly functions (playNote, sendPitchBend, etc.). React to MIDI input with simple event listeners (noteon, pitchbend, controlchange, etc.).
https://github.com/djipco/webmidi


The MIT License (MIT)

Copyright (c) 2015-2019, Jean-Philippe Côté

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
associated documentation files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge, publish, distribute,
sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial
portions of the Software.

This file has been truncated. show original

And here are the official libraries (current version is 3.0.11): https://github.com/djipco/webmidi/tree/master/dist
Maybe I can build a cleaner interface (compared to my current implementation) at some point with that.

Jona · February 18, 2022, 12:50am

I had a look at the webMidi Library, but I think what I use now fits much better, because its in my opinion enough to choose the in and out ports and send / receive raw midi data (it seem more complicated to me to call for example every time a note on message instead of channel, pitch, velocity)…
This is the implementation I adapted for OF (added output functionality in first place) and I still think its great to build on that:

github.com

cwilso/midi-synth/blob/master/js/midi.js

function midiMessageReceived( ev ) {
  var cmd = ev.data[0] >> 4;
  var channel = ev.data[0] & 0xf;
  var noteNumber = ev.data[1];
  var velocity = ev.data[2];

  if (channel == 9)
    return
  if ( cmd==8 || ((cmd==9)&&(velocity==0)) ) { // with MIDI, note on with velocity zero is the same as note off
    // note off
    noteOff( noteNumber );
  } else if (cmd == 9) {
    // note on
    noteOn( noteNumber, velocity/127.0);
  } else if (cmd == 11) {
    controller( noteNumber, velocity/127.0);
  } else if (cmd == 14) {
    // pitch wheel
    pitchWheel( ((velocity * 128.0 + noteNumber)-8192)/8192.0 );
  } else if ( cmd == 10 ) {  // poly aftertouch

This file has been truncated. show original

dimitre · February 18, 2022, 1:39pm

did you try to use your ofxPd object other name than pd?

Jona · February 18, 2022, 10:45pm

@dimitre I tried that, but it does not seem to be the issue (and it works on desktop, so it seems to be an emscripten specific issue). Uploading and opening a new patch already works.