The ofxGesture addon has been renamed to ofxPatterns due to added functionality. In addition to previous mouse / feature vector gesture classification classes & examples, the new version 0.05 includes:
a class and accompanying example for speech classification, contributed by Dhi Aurrahman and Arif Setiawan (you can train the class to recognize different words or sentences)
a class and accompanying example for on-line mouse gesture recognition, contributed by David Burri (you can use the class to recognize gestures on-line, i.e. without knowing a priori where the gestures begin and end)
There are packages for OS X, Windows (Code::Blocks & MSVC), and Linux. Please note that some examples might not work correctly on some platforms. If you run into problems please let me know.
Future versions of this addon will include general HMM functionality (e.g., a class that allows you to create an arbitrary Markov chain and use it to generate patterns of numbers), and other pattern-related algorithms (e.g., edit distance, dynamic time warping).
The speech recognition add in is great. I’ve played around with Julius a lot but couldn’t ever get a live way of training sorted out and so had to build dictionaries and grammar trees beforehand. Have you figured out a max number of models that the library supports with good (i.e. ~60-75%) accuracy or a max length?
The speech functionality is pretty infant. For one, each model is non-hierarchical (e.g., if you train it to recognize a couple of words, each model does not break down the words into phonemes to help training or recognition). Being that it only does classification at the moment (as opposed to on-line recognition), it is also pretty sensitive to segmentation. If you segment the recordings well (which can be tricky in the provided example, since you have to press/release the mouse button precisely before/after you speak), it works relatively well, otherwise not so much.
In terms of how many models you can reliably recognize, it depends on how distinguishable they are. If you want to distinguish between longer sentences (that are significantly different from each other), I imagine you can distinguish between quite a few of those (say, tens or hundreds). If you’re distinguishing between short words that share some similarity, it will probably start to deteriorate after only a few.
Adding a hierarchical model should help a lot, but that will take some work. The underlying library supports hierarchical models, but getting that functionality into the addon in a way that is easy to use will be a challenge.
If you try this out, I’d love to hear how it works out (or doesn’t work out) for you.
Training 1-10 works more or less like I’d expect, a really rough guess would be ~70% accuracy with a single speaker and invariant environment. I’ll report back when I test some more.
your addon sounds pretty amazing to me but I can’t get it working. I have a Windows XP Laptop and I use Codeblocks so I downloaded the codeblocks version.
I unpacked the file and copied the folders into my OF folders to apps and addons. Then I opened the examples and tried to build them and this is what I got:
||=== patternsExample, release ===|
…\libs\openFrameworks\utils\ofConstants.h|47|glu.h: No such file or directory|
…\libs\openFrameworks\events\ofEventUtils.h|5|Poco/FIFOEvent.h: No such file or directory|
…\libs\openFrameworks\events\ofEventUtils.h|6|Poco/Delegate.h: No such file or directory|
…\libs\openFrameworks\events\ofEventUtils.h|16|error: Poco' has not been declared| ..\..\..\libs\openFrameworks\events\ofEventUtils.h|16|error: expected template-name before '<' token| ..\..\..\libs\openFrameworks\events\ofEventUtils.h|16|error: expected{’ before ‘<’ token|
…\libs\openFrameworks\events\ofEventUtils.h|16|error: expected unqualified-id before ‘<’ token|
…\libs\openFrameworks\events\ofEventUtils.h||In function void ofAddListener(EventType&, ListenerClass\*, void (ListenerClass::\*)(const void\*, ArgumentsType&))':| ..\..\..\libs\openFrameworks\events\ofEventUtils.h|30|error:Poco’ has not been declared|
…\libs\openFrameworks\events\ofEventUtils.h||In function void ofAddListener(EventType&, ListenerClass\*, void (ListenerClass::\*)(ArgumentsType&))':| ..\..\..\libs\openFrameworks\events\ofEventUtils.h|35|error:Poco’ has not been declared|
…\libs\openFrameworks\events\ofEventUtils.h||In function void ofRemoveListener(EventType&, ListenerClass\*, void (ListenerClass::\*)(const void\*, ArgumentsType&))':| ..\..\..\libs\openFrameworks\events\ofEventUtils.h|50|error:Poco’ has not been declared|
…\libs\openFrameworks\events\ofEventUtils.h||In function void ofRemoveListener(EventType&, ListenerClass\*, void (ListenerClass::\*)(ArgumentsType&))':| ..\..\..\libs\openFrameworks\events\ofEventUtils.h|55|error:Poco’ has not been declared|
…\libs\openFrameworks\events\ofEvents.h|48|error: field setup' has incomplete type| ..\..\..\libs\openFrameworks\events\ofEvents.h|49|error: fieldupdate’ has incomplete type|
…\libs\openFrameworks\events\ofEvents.h|50|error: field draw' has incomplete type| ..\..\..\libs\openFrameworks\events\ofEvents.h|51|error: fieldexit’ has incomplete type|
…\libs\openFrameworks\events\ofEvents.h|52|error: field windowResized' has incomplete type| ..\..\..\libs\openFrameworks\events\ofEvents.h|54|error: fieldkeyPressed’ has incomplete type|
…\libs\openFrameworks\events\ofEvents.h|55|error: field keyReleased' has incomplete type| ..\..\..\libs\openFrameworks\events\ofEvents.h|57|error: fieldmouseMoved’ has incomplete type|
…\libs\openFrameworks\events\ofEvents.h|58|error: field mouseDragged' has incomplete type| ..\..\..\libs\openFrameworks\events\ofEvents.h|59|error: fieldmousePressed’ has incomplete type|
…\libs\openFrameworks\events\ofEvents.h|60|error: field mouseReleased' has incomplete type| ..\..\..\libs\openFrameworks\events\ofEvents.h|62|error: fieldaudioReceived’ has incomplete type|
…\libs\openFrameworks\events\ofEvents.h|63|error: field audioRequested' has incomplete type| ..\..\..\libs\openFrameworks\events\ofEvents.h||In member functionvoid ofCoreEvents::disable()’
…\libs\openFrameworks\events\ofEvents.h|66|error: setup' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|67|error:draw’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|68|error: update' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|69|error: request for memberdisable’ in exit', which is of non-class typevoid ()(int)’|
…\libs\openFrameworks\events\ofEvents.h|70|error: keyPressed' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|71|error:keyReleased’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|72|error: mouseDragged' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|73|error:mouseReleased’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|74|error: mousePressed' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|75|error:mouseMoved’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|76|error: audioReceived' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|77|error:audioRequested’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h||In member function void ofCoreEvents::enable()':| ..\..\..\libs\openFrameworks\events\ofEvents.h|81|error:setup’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|82|error: draw' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|83|error:update’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|84|error: request for member enable' inexit’, which is of non-class type void ()(int)'| ..\..\..\libs\openFrameworks\events\ofEvents.h|85|error:keyPressed’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|86|error: keyReleased' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|87|error:mouseDragged’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|88|error: mouseReleased' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|89|error:mousePressed’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|90|error: mouseMoved' was not declared in this scope| ..\..\..\libs\openFrameworks\events\ofEvents.h|91|error:audioReceived’ was not declared in this scope|
…\libs\openFrameworks\events\ofEvents.h|92|error: audioRequested' was not declared in this scope| ..\..\..\libs\openFrameworks\communication\ofArduino.h|277|error: fieldEDigitalPinChanged’ has incomplete type|
…\libs\openFrameworks\communication\ofArduino.h|280|error: field `EAnalogPinChanged’ has incomplete type|
||More errors follow but not being shown.|
||Edit the max errors limit in compiler options…|
||=== Build finished: 50 errors, 0 warnings ===|
are you trying to compile a 0.05 codeblocks project in 0.061? You may need to recreate the project in 0.061 by duplicating an empty project and following the steps to make it. Alternatively, you can download OF 0.05 if you want to try compiling the example provided on the source forge site. The errors you are a getting seem to be from compiling an older project in a newer version of OF.
I’ll take a look – perhaps this addon was built using boost? (which means there might be linking problems… it looks like it’s missing some functions in boost that it needs).
I’m not very familiar with the addon but if you upload this I’ll take a look
pd: Others ways to capture and recognize gestures are by TUIO or PyMT (would need python binding) but they are probably more focused on multitouch finger detection, and I’m more looking for a general patern detection and this one looks pretty good and easy.
After all it would be nice to upload this addon to github and the ofxAddons site.
