The EyeHarp: An Eye-Tracking Based musical instrument

Hello everybody!

I am happy to join the openFrameWorks open source community. And I am proud to present the EyeHarp. Inspired by the eyeWriter-project, I thought I should use all this amazing work to create a musical instrument. So as to empower people with paralysis resulting from ALS with more creative opportunities (apart from drawing, playing music as well). This is my first OpenFrameWorks project. Visit the blog

Alternatively, the software can be used using the mouse pointer (MouseHarp version). Then the -camera-mouse-can be used to control the instrument with head movements. Any technology that can take control of the mouse pointer can be used in order to control the instrument. That way the mouseHarp could be an appropriate instrument for many cases of people with physical disabilities. The mouseHarp version is completely independent from the eyeWriter-project. Combining the mouseHarp source with the source of the eyeWriter-project, we get the eyeHarp! A low-cost gaze controlled musical instrument!

The main contribution of this project is the design of an appropriate interface to play music using any eye tracing device. In order to evaluate the interface, a low-cost eye tracking device was constructed using the instructions of the EyeWriter Project. I am grateful to the EyeWriter team. Without them I would never be able to develop the EyeHarp, as I could never afford buying a commercial 10,000 euros eye tracking device in order to have a usability evaluation while developing the project! Instead thanks to the EyeWriter initiative I spent only 50 euros for the eye tracker. The whole program is implemented in openframeworks v0.6.

The EyeHarp project so far is my master in Sound And Music Computing in UPF, Barcelona. My supervisor is Rafael Ramirez.

Read the paper published in SMC Conference 2011, Padova, Italy:

Source code here
(windows, codeblocks 8.02, openframeworks 0.06)

Here is a demo on youtube:

And a screenshot:

Very cool! Hope to see more of this

Im very interested by your project cause i work with handicapped people.
I Tryed to compil your project under codeblocks and openframeworks 0.06 on PC but i have these errors :

…\libs\poco\includes/Poco/FIFOStrategy.h: In member function ‘void Poco::FIFOStrategy<TArgs, TDelegate, TCompare>::add(const TDelegate&)’:
…\libs\poco\includes/Poco/FIFOStrategy.h:109: error: ‘auto_ptr’ is not a member of ‘std’
…\libs\poco\includes/Poco/FIFOStrategy.h:109: error: expected primary-expression before ‘>’ token
…\libs\poco\includes/Poco/FIFOStrategy.h:110: error: ‘pDelegate’ was not declared in this scope

I got the same errors when i try to compile eyewriter project.

An idea?

Could it be due to the version of codeblocks. I use the 10.05 instead your 8.

Great stuff - very inspiring!

And a really excellent demo video of your work :slight_smile: