Realtime user drawing training and recognition on mobile


I’ve come up over this older project of mine and decided to polish it a bit and get it out of the drawer, see here: Essentially, it is a drawing(or character) training and recognition example that can take user input and that uses a neural network behind the scenes(simple one, not a convolutional net - so just simple drawings, no images). You can draw all kinds of stuff and it will do its best to understand them.
The project is built with OF as a general framework and UI base and I am using openCV for the image pre-processing part and the neural net implementation.
The first version had a horrible UI, so I spent some time to make it prettier. It is still a bit hard to make it look decent without having access to a native interface builder, but this is not the scope of this discussion.
I plan to hold a workshop in Bucharest sometime over the next months with this project as a base - to have a talk about OF, UI in OF, opencv and machine learning. I think it will come up as a neat package. I’ll also put the code on github, but it’ll take some time before I finalise it and make it pretty.


P.S. sorry for the improvised camera setup

Hi Andrei,
This is something similar to what I am looking for. I started with the link below:
I am yet to play with any code on this.

Do you think you can share the code you have/ post some sample code.


Hi Dece,

It might help knowing what you are trying to achieve. I am saying this because if you are not looking for specifically the same example of training custom shapes or optical character recognition, you maybe can find some simpler alternatives. For example, discriminating between basic shapes such as triangles, squares and circles can be done in a cleaner way in opencv.

Anyway, I’ve put the rough code here:, I didn’t have the time to clean it up. If you would like to ignore the UI code, the ‘brains’ of the code is located in the drawingRec class. Here we pass on user input, pre-process it ( crop and resize it ) and then feed it to the neural network ( which is implemented in opencv).

Hi Tazu,
Thanks for the sample code.
I was thinking or being able to draw shapes such as squares, rectangles, arrows, other basic geomoetric shapes. They need not be drawn measured to geometry but approximate recognition is good enough. You can image a drawing interface for kids to play with as an example.
Looking at your vimeo post seems like a good start.

Also a couple of days ago I tried some code from the stonybrook link I posted earlier on shape similarity. I could successfully build that project and run the test program but I am yet to verify the results visibly since the sample code runs without a visual UI. On my todolist as I get some time.

Will checkout your code and thanks again.

Hi Tazu,

I was wondering if your drawRecognition code on github was relocated to somewhere else. I would like to take a look at the sample code for reference purposes but it was nowhere to be found on your site. May I know where I could download the package?