So, I’ve been hitting my head against this program for the better part of 3 weeks without much success. I’ve been going through ofxCv and its pretty impressive and think its the key to solving my issue.
I’m trying to accurately detect when a ball is thrown at a wall and what color that ball is. To make things harder, there is a game projected onto the wall with many changing colors. I am using a kinect to get depth and color data. The kinect is mounted right next to the projector so the balls that the kinect sees are seeing the full image of the projector, unfortunately.
What I’ve tried with decent success depth-wise is doing a simple absdiff on the background, thresholding it with the current kinect depth image and then taking whatever values fit and flattening them to 0 or 255.
With that thresholded depth image I run an ofxCvContourFinder with some simple min/max size constraints. From this point, I could just send the message that the ball hit the wall and depth wise it would work alright. But unfortunately, I need color too!
So if I found a contour using the OF contour finder then take the thresholded depth image and use it as a stencil to cut out the blob from the kinect color camera, leaving everything I don’t want black. I run an ofxCv::contourFinder (The ofxCv one) for the 4 different colors of the balls I’m looking for. If I find a contour (which generally I do as the OF contour finder found one) then I use a ROI to go through all the pixels and find out which color the blob is most similar to.
With this process I would say it works about 80% of the time after a whole bunch of tweaking.
I think the biggest issue is that I’m not using any frame-by-frame tracking, but I think that the kinect depth data isn’t good enough to rely on it solely. If I save the depth image out every frame I see that when things move fast the depth data of the ball is spotty and not uniformly circular. I attempted to use color background differencing but didn’t have much success with that. My other issue is that the tracker in ofxCv looks like it would be super useful but it confuses me in that I don’t know how to tell how many labels were tracked (so I know when to send the message. If I could get it to accurately detect the velocity of the ball, that would be amazing).
I also attempted Hough circle tracking using openCv but it doesn’t seem super accurate and its rather laggy.
Do you have any suggestions on how I should tackle it? Any help would be so so greatly appreciated!