I wanted to share the performance I presented at this year’s Borderline-Biennale: the technical ingredients are skeleton tracking, brainwaves controlled animations and music, projection mapping.
(A little warning: if you are sensible to body piercing and body art in general, don’t watch the video)
Hope you like it 
Congratulation! Very interesting location e awesome thematic!
Bravi!
ed
grazie
the venue is really awesome: everybody passing by Lyon should have at least a short trip there
I can’t stop watching the video! It’s really awesome!
Is it possible to have more technical details (wich sensor you’ve used, how much animation and music were controlled by brain waves etc)?
I find very interesting the brain waves computer interactions and i would like to know more about that!
cheers,
ed
sure, here come the details:
Sensors:
breath: it’s sensed with a wireless mic positioned inside Lukas’ mask. its signal goes directly through a mixer controlled by the audio workstation
heart: it’s sensed with modified stethoscope connected with a wireless mic; signal works just like the breath (we’re not sure, but in the future we may decide to apply some DSP on it)
EEG: we use the cheaper sensor from NeuroSky; it streams brainwaves (already splitted into frequencies) via radio in a serial like protocol; these radio packets arrive to my computer where they’re parsed, conveted into OSC and broadcasted via wifi (we only have 2 computers on stage, but the idea is that if we have an affine hacker soul between the public, he/she can join the jam session )
skeleton tracking: it’s obviously done with ofxOpenNI (as you can see in the video we also stage the infamous “calibration pose”, because we wanted to let people understand as much as possible what was going on)
The audio part maps the brainwave data onto volumes and scales, while the visual part uses spikes (originated i.e. by the piercings and by the winch pulling on the hooks) to trigger events; so, conceptually speaking, the wings are a correct representation of Lukas’s neural response and they really lift him off the ground.
Here’s a couple more words on the piece: http://naufolio.augmentedrealityag.com/installations/danse-neurale/
Very interesting
I live in Lyon and I’ve been to the La Demeure du Chaos / The House of Chaos.
Sorry I missed this, if you return for the next Biennale, please let me know, thanx.
Bonne continuation!
While I was there, I learned there is a vivid media art scene in Lyon: it was a really nice surprise and hope to come back soon (ok, ok: I miss la Demeure 
)
I’ll sure message you when I’m around!
Salut!
There probably is a scene here, but I only know of one, the association called AADN http://www.aadn.org/, (I usually work alone…)
And no problem regarding the name, they’re usually part of the Art Biennale in Lyon.
Ciao!
PS: Awhile back, I worked on an art project in Macchiagodena, Italy, 2 hours outside of Rome, love Italy
Any chance you could let me glance at the code? I’m trying to connect to NeuroSky MindSet and so far - failing.
Thanks.
no problem: at the moment I’m using a Processing sketch translate the Mindset output to OSC and broadcast it to the network. I decided to do it in Processing basically because I wanted to be able to quickly modify it on stage if needed, but it’s really super simple and you should be able to port it in OF without any problem.
here’s the code:
//------------------------
//1. Insert usb radio key
//2. Start TGConnector
//3. Switch headset on
//4. Run OSC sketch
//5. Run performance sw
//------------------------
import processing.net.*;
import oscP5.*;
import netP5.*;
OscP5 osc;
NetAddress ofLocation;
Client tgc;
int meditation, attention;
float[] waves;
int port = 13854;
String localHost = "127.0.0.1";
String localIp;
String localBc;
int updates;
void setup() {
size(200,200);
tgc = new Client(this, localHost, port);
waves = new float[8];
meditation = attention = updates = 0;
localIp = NetInfo.lan();
String[] brkIp = split(localIp,'.');
localBc = brkIp[0]+"."+brkIp[1]+"."+brkIp[2]+".255";
ofLocation = new NetAddress(localBc,12345);
}
void draw() {
background(0);
// Look for available bytes
if(tgc.available() > 0) {
// Grab bytes and save to waves array and att-med
updateMindSet();
//println("Attention is "+attention+" Meditation is "+meditation);
updates++;
//stroke(255,0,0);
//line(updates,height/2,updates,height/2-attention);
//stroke(0,0,255);
//line(updates,height,updates,height-meditation);
//text("Attention: "+attention+" Meditation: "+meditation, 20,20);
sendOsc();
fill(0,255,0);
rect(0,height-20,20,20);
fill(255,255,255);
}else{
//println("not connected");
fill(255,0,0);
rect(0,height-20,20,20);
fill(255,255,255);
}
text("Sending brain activity over network:",0,height-30);
text(localIp+"->"+localBc, 20,height-10);
text("Attention: "+attention+" Meditation: "+meditation, 20,20);
for(int i=0;i<8;i++){
text("Wave: "+waves[i],20,40+15*i);
}
// Draw something based off of wave information
}
void updateMindSet(){
// Typical data string from NeuroSky Device:
// aaaa0200 04390500 812036ec 237b3687 87c73734 edc73870 c7f136f2 af4f3690 bf783513 2d0535bd ab96
// aaaa sync
// 0200 signal (0-200, 0 is good, 200 is off-head)
// 0439 attention (0 is non
// 0500 meditation
// 8120 wave code + length (20 is Hexidecimal for 32, 8 numbers x 4 bytes each (for a float)
// 36ec 237b3687 87c73734 edc73870 c7f136f2 af4f3690 bf783513 2d0535bd ab96
byte[] buffer = new byte[42];
int byteCount = tgc.readBytes(buffer);
if(byteCount == 42) {
meditation = buffer[7];
attention = buffer[5];
for (int i = 0; i<8; i++) {
byte[] floatBuffer = new byte[4];
for(int j=0; j<4; j++) {
floatBuffer[j] = buffer[10 + i*4 + j];
}
waves[i] = byteArrayToFloat(floatBuffer);
}
}
tgc.clear();
}
float byteArrayToFloat(byte test[]) {
int bits = 0;
int i = 0;
for (int shifter = 3; shifter >= 0; shifter--) {
bits |= ((int) test[i] & 0xff) << (shifter * 8);
i++;
}
return Float.intBitsToFloat(bits);
}
void keyPressed(){
switch(key){
case 'c':
//tgc.close();
break;
}
}
void sendOsc(){
OscMessage esenseMsg = new OscMessage("/esense");
esenseMsg.add(attention);//+random(-50,50));
esenseMsg.add(meditation);//+random(-50,50));
OscP5.flush(esenseMsg,ofLocation);
OscMessage wavesMsg = new OscMessage("/waves");
for(int i=0;i<8;i++){
waves[i]*=.001;//random(-50,50);
}
wavesMsg.add(waves);
OscP5.flush(wavesMsg,ofLocation);
}
Thank you so much for the code, I’ve been working on porting it to oF, I think I’m gonna try and pull data as JSON from the localhost socket. But even with that this is extremely helpful!