movement vector from video

i need to do a project something like in this video.
http://www.youtube.com/watch?v=y6izXII54Qc

this is a project they are screening on theaters before the movie, and the
audience participates with the game (simple arkanoid game) via changing
their position or their hands position.

i’ve accomplished this with tracking changing blobs, and if the blobs are on
the right half side of the screen, i would consider a right move, if they
are on the left side of the screen, i would consider it a left move.
however, this would be a cheat and not the best way to do it.

any suggestions? Also, any mailing lists for computer vision, or image
recognition would be very good also.

Best Regards,
Alp

Hey Alp,

I think the best way is to do some sort of frame difference and then create a motion history where old pixels slowly fade to black. You can then look at the gradiant of white -> gray -> black at each point on the image and figure out the overall magnitude of the movement by adding all the small movements together.

I had to do this exact same thing but for movement in a circle. (not frame differencing but similar)
http://blip.tv/file/255536

The nice thing about this approach is it really becomes an average of all the movement in the frame. So if 50 people moved to the left and 50 moved to the right their movement would cancel each other out. This would really force people to work together and all move in the same direction.

Zach and I have actually been working on something very similar the last few days.
We create a vector field from the motion information of a video - to get the overall movement you would just add all the vectors together and divide by the total number.

Let me know what platform you are on (mac / win / linux ) and I could put together a little demo based on where we are at.

I don’t know any great cv resources - but I am sure others on the forums do.

Cheers!
Theo

that’s cool. i’m on windows.

i currently use max/msp with jitter and cv.jit library, there are two methods:

cv.jit.LKflow Estimates the optical flow using the Lucas-Kanade technique.
cv.jit.HSflow Estimates the optical flow using the Horn-Schunk technique.

from here i get the movement vector and send it to director via udp socket. and the game is a 3d car racing game in director that gets left and right commands. acceleration is done automatically. :slight_smile:

but that doesn’t work the way i would like it to.

your technique is the best i could find to date and i’ve looked at some opencv codes in openframeworks and it is not so hard to understand. maybe i won’t use max/msp at all and do all the computer vision stuff in openframeworks.

i’m thinking using a unibrain camera with a wide angle lens (120 degrees) currently i’m using a simple webcam.

http://www.unibrain.com/Products/VisionImg/boardXGA.htm

thanks
alp

Cool - sounds good.

Tomorrow I’ll post a codeblocks project for windows that loads a video file and figures out overall motion. I am sure you can hack it from there to do exactly what you need.

The unibrain cameras are pretty awesome!

The only issue to bare in mind with super wide lenses is that the non-linear nature of the optics will mean magnitude of motion will not be consistent across the image. So movement will be detected as more in the center than at the sides.

To solve that you might want to look at the camera unwarping discussion
http://www.openframeworks.cc/forum/view-…-ight=remap

But it might not be too big of an issue,

Cheers!
Theo

ok, i’ve already installed codeblocks with openframeworks and played with the examples. it compiles and runs fine.

very eager to get my hands on your project.

thanks a lot
alp

Hey Alp!

Here it is - you can set it to use the test video or a camera.

It should be put in the apps/ folder of 005 CB FAT release.
When you unzip it your folder structure will be

apps/
addonExamples/
dev/ <-- this is new from the zip
examples/

http://www.openframeworks.cc/files/moti-…-cb-win.zip

Hope its helpful - I tried to comment through the non-obvious stuff
Let me know if you have any questions,
Theo

wow, i cant believe it, it works. :slight_smile:

thanks so much…

hello…

i checked the forum about the problems with dev c++ and decided to change to code::block…i runned your code and the following warnings appeared. I have noticed that with your code the build target has two options: release and debug…but not the normal default mode.

do you have any suggestions?

-------------- Build: release in allTestExample ---------------

WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxDirList\src\ofxDirList.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxNetwork\src\ofxTCPClient.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxNetwork\src\ofxTCPManager.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxNetwork\src\ofxTCPServer.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxNetwork\src\ofxUDPManager.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxObjLoader\src\ofxObjLoader.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxOpenCv\src\ofxCvColorImage.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxOpenCv\src\ofxCvContourFinder.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxOpenCv\src\ofxCvFloatImage.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxOpenCv\src\ofxCvGrayscaleImage.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxOpenCv\src\ofxCvImage.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxOsc\src\ofxOscMessage.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxOsc\src\ofxOscReceiver.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxOsc\src\ofxOscSender.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxThread\src\ofxThread.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxVectorGraphics\libs\CreEPS.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxVectorGraphics\src\ofxVectorGraphics.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxXmlSettings\libs\tinyxml.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxXmlSettings\libs\tinyxmlerror.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxXmlSettings\libs\tinyxmlparser.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\addons\ofxXmlSettings\src\ofxXmlSettings.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\app\ofAppRunner.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\communication\ofSerial.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\graphics\ofGraphics.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\graphics\ofImage.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\graphics\ofTexture.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\graphics\ofTrueTypeFont.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\sound\ofSoundPlayer.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\sound\ofSoundStream.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\utils\ofMath.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\utils\ofUtils.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\video\ofQtUtils.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\video\ofVideoGrabber.cpp
WARNING: Can’t read file’s timestamp: C:\1.- OpenFrameWorks\0.04_devcpp\apps\libs\openFrameworks\video\ofVideoPlayer.cpp
Linking console executable: bin\allTestExample.exe
mingw32-g++.exe: …/…/…/libs/freeImage/FreeImage.lib: No such file or directory
mingw32-g++.exe: …/…/…/libs/RtAudio/RtAudio.a: No such file or directory
mingw32-g++.exe: …/…/…/libs/quicktime/Libraries/qtmlClient.lib: No such file or directory
mingw32-g++.exe: …/…/…/libs/freetype/lib/libfreetype.a: No such file or directory
mingw32-g++.exe: …/…/…/libs/glut/libglut.a: No such file or directory
mingw32-g++.exe: …/…/…/libs/videoInput/lib/videoInputLib.a: No such file or directory
mingw32-g++.exe: …/…/…/libs/fmodex/lib/libfmodex.a: No such file or directory
mingw32-g++.exe: …/…/…/libs/glee/lib/GLee.lib: No such file or directory
mingw32-g++.exe: …/…/…/libs/glu/glu32.lib: No such file or directory
mingw32-g++.exe: …/…/…/addons/ofxOpenCv/libs/opencv/lib/win32/cv.lib: No such file or directory
mingw32-g++.exe: …/…/…/addons/ofxOpenCv/libs/opencv/lib/win32/cxcore.lib: No such file or directory
mingw32-g++.exe: …\addons\ofxOsc\libs\oscpack\lib\win32\oscpack.a: No such file or directory
Process terminated with status 1 (0 minutes, 0 seconds)
0 errors, 0 warnings

is your app in the right place? it doesn’t look like it based on those errors…

can you download the 0.05 CB
http://www.openframeworks.cc/download
and try the all addons example and see that it compiles?

the apps must be relative to the libs / addons folders in the same level of the heirarchy (ie, two level above libs), ie,

OF/apps/exampleApps/graphicsExample
or
OF/apps/myCoolShinyApps/CSHApp01

if you put the app somewhere else, it wont compile, like
**
OF/apps/CSHApp01 **
or
putting the app not in OF folder, etc, since the paths are all relative, like …/…/libs/…

hope that helps!
best,
zach

Thanks!

it was just a problem of heirarchy.

It runs good. See you soon in Mexico D.F.

thanks again,
Alecs

HI theo,

I am glad that i came across this and was playing something similar to Alp.
Do you mind to post up a Mac version of your little test so I can play around?

Alp,it’s been a while since you posted but if you saw this, I am curious how your project goes as well.

thanks a lot
cath

Although I have some basic C++ experience, I’ve been using processing for quite a while and I just start to play around with oF.

This type of motion detection is exactly what I need.

I cannot compile your code though, is it because of the differences between 005 and 0061? If so, what do I have to change?

thanks