Live drawing following a point in motion?

Hi all,

I’m working on a degree project that’s sort of evolved into a more narrow idea using motion capture.

My idea is that I want to build something that “traces” movement in 2D space. Either with the use of movement a body part like an arm (this would be ideal - but as I understand this would probably require machine learning?) or with the use of a marker - like a brightly colored object that the webcam recognizes as the object to follow. I made a crudely drawn diagram to help explain what I was thinking.

Any ideas on how to start building something like this? I haven’t been able to find many projects similar to this that share their process.

EDIT: I forgot to mention that I’d like to be able to playback the animations in a loop. I’d like to record something like 5 seconds of movement and then the screen would playback the animation for a few cycles and then reset.

1 Like

Without borrowing someone else’s libraries or algorithms, I think you could invent something.

Capturing the video gives you a series of frames of 2D color data, which is a nice easy (in OF) starting point.

If it’s ok to track all 2D movement of anything, then you could devise and tweak an algorithm to do it (no “machine learning” needed, though you might want the algorithm to tune itself a bit for brightness and noise level).

I’d think you’d want to try to skip over as much noise (areas mostly-the-same between frames) as quickly as possible, and when there’s a change, try to find where something like the color in the previous frame can be found where it wasn’t before in the new frame. And then decide how you want to abstract and tune that to show what you want to show.

If you want only the subject’s hand, you might want to have them wear an unusually-colored glove.

I think examples/computer_vision/opencvExample is a good starting point.
In the example contourFinder.blobs[i].boundingRect.getCenter() is used, which you could use as a counterpoint for a circle, that draws into an ofFbo (without clearing the fbo will draw a line instead of a single point) for example.
You can store the position of the blob per frame in an array (while recording), which gives you the data to do a playback.