Robust Learning Object Tracking Algorithm (Predator)

Just came across this:
http://www.engadget.com/2011/03/31/zden-…–fly-like/

I think this is ofx-addon worthy and has many applications for HCI.

You can view the video here:
http://www.youtube.com/watch?v=1GhNXHCQ-…-r-embedded

And download the source codes here:
https://github.com/zk00006/OpenTLD

Visit his hompage here:
http://info.ee.surrey.ac.uk/Personal/Z.Kalal/tld.html

Hope this spurs the OF community to build an add-on based on this.

Cheers!

Looks awesome!

It’s even on github: https://github.com/zk00006/OpenTLD
For now, the dependence on matlab may be a deal-breaker for oF inclusion, but they want to get rid of it anyway as per the readme.

I think for our purposes atm MatLab code is pretty useless. I didn’t realize just how much code there was and how hellish MatLab to C++ actually might be. Also, side-note, the first results when searching for “porting Matlab to C++” leads to this charming discussion: http://www.mathworks.com/matlabcentral/-…-ead/282647

well, to be fair, that guy insists on calling it mathlab, so he didn’t exactly have a good start. :slight_smile:
anyways, searching stackoverflow seems to be a bit more fruitful:
http://stackoverflow.com/questions/1329-…-tions-in-c
http://stackoverflow.com/questions/3359-…-ix-library
http://arma.sourceforge.net/

There’s even something from mathworks to convert matlab to C. could be a first step.
http://www.mathworks.com/products/featu-…-dedmatlab/

Fair enough, i didn’t notice that he called it MathLab :slight_smile:

I’m quite curious about their Real-Time Workshop generated code, i.e. how readable and useful it might be. Might give it a shot this evening and see what happens, if it’s not too bad a little commenting/smoothing out might make it almost useful, though in my experience a lot of gen’d code is frightening. I suppose if it’s just compiled into a static lib it doesn’t matter too much tho.

I’ve thought about letting m2html run over it and see if i can get a nice dependency graph - could be useful to get a feeling about the structure of the code. if I can find the time and i’m successful i’ll post it.

OK so I tried that, it was not very successful. C++ files are not parsed, and the GPL license header in every file confuses m2html, so the description of the matlab functions (first comments after function definition) are not picked up correctly. Not very useful.
The command I used was

  
m2html('recursive','on', 'htmldir','OpenTLD/doc', 'source','off','graph', 'on', 'global','on');  

I attached the master dependency graph for interested eyes. the graphs of subdirectories are not so messy, of course.

what is wrong with the forum server for the last month, though? It just took me ~10 attempts to get this message posted??

Update: People are at work on github trying to make it work under octave (open source matlab alternative) and on multiple platform as a first step.

this video i really awesome.
actually world is full of great algos only in matlab (for affecting computing as example)
and something in python that with numpy is a great tool for numerical computing…
had all those in openframeworks should be great

i’ve been reluctant to ask because it feels a bit naive, and i don’t want to sound like a jerk when someone made something cool… but i still don’t really understand why this is receiving so much attention?

i thought this was exactly what’s going on under the hood of tools like PTAM that are using SURF or other fast feature detectors, paired with some basic learning? there’s also sparse optical flow techniques that have been used for at least the last 16 years, i think they could handle maybe 80% of what is shown in this video without any tweaks.

is it just because he gives a good overview of how it’s working, or because it’s focused only on feature tracking and not AR?

Reading through the papers a bit from what I understand I think it’s just the combination of different well-known algorithms in a novel way. Using images obtained during performance for training is pretty cool and afaik isn’t anywhere else. I think the same thing is being done here: http://www.merl.com/projects/ensemble-tracking/ but it’s not open source and not as well documented. Not really revolutionary but cool and very applicable to OF stuff. I think being able to select objects and generate features at runtime could be really helpful in a crowded room installation type scenario.

I, personally, was really impressed by the demo video, but then maybe I’m not well enough informed about the state-of-the-art in object tracking.

Surely there must be some (no marker) based image tracking built into openCV somewhere?
I really hope there is because I need it!

there is already markerless image tracking in opencv, that’s what the sparse optical flow algorithms accomplish. the most popular one is lucas kanade (LK) optical flow, implemented as cvCalcOpticalFlowLK (or cv2 style, cv::calcOpticalFlowPyrLK).

there is an addon floating around that demos the LK flow function. sorry i don’t have a link atm :frowning:

there’s also lots of good stuff related with markerless tracking and feature detection in the newest opencv, module features2d, including ferns http://cvlab.epfl.ch/software/ferns/index.php- and fast http://mi.eng.cam.ac.uk/~er258/work/fast.html

Just as an update: TLD released a C++ port: http://sourceforge.net/projects/qopentld/. Removing the Qt might be difficult, but I can’t imagine that it’s so deeply ingrained into the actual TLD algorithms. Would love to hear how people find it and see what integrating it into oF looks like

===== update

Wow, all the TLD stuff is in a single file, Detector.cpp, and it’s using OpenCV2.2 to do all the imaging+camera, while it handles the tracking in a fairly straightforward fashion. I can’t imagine that it would be too hard to get working in OF actually.

To be more precise, _one guy_ on the google group announced _his_ port to c++, nothing to do with the original developer. there are a couple other guys working on c++ ports as well (one is on github). I tried to encourage them to work together from the beginning, but frankly i’m astonished why they all do their own thing instead of working together on github. well, I did what I could holding together the codebase in my repo (for matlab and octave, at least).

I can’t imagine that it would be difficult to make this independent of Qt, I suspect he used it only to get a GUI. I have my eyes on it regarding an eventual oF version, but we’ll see what develops concerning bugfixing and so on. Too bad I don’t know very much about computer vision.
Also, it seems that the “official” GUI for 007 is really slow to materialize, or the idea has died altogether (nobody says anything), so I’ll be holding back on that end for now, don’t want to commit to the wrong horse.

Hi Christoph.

Did you ever manage to get TLD working within OF ?

It looks super interesting.

s.

No I never got around to it, my focus at the time was the Octave port, and I have abandoned that when it turned out it’s unfeasible. The C++ ports have progressed, though, afaik.
One drawback of OpenTLD that I saw from loosely following discussions in the google group is that tracking is confined to ~one target at a time.