Visual computing geniuses,
I’m doing a project called ShadowFighter, it’s a game installation that let’s you fight with your shadow.
The following video should give you a clue of what it’s about.
The shadows are created by 2 construction lights and we capture the situation using the PS3 Eye camera. Using a beamer we project visuals.
In openFrameworks we do the visual computing, game logic and generate the visuals.
But we are having one big issue, we call it the Suicide issue.
To detect “hit’s” we use the regular openCV blobtracking, we tweak the difference threshold in such a way it only sees the overlapping and thus darker shadows. We validate these hit’s using a few checks and filters. For example, hit’s only count if they are inside a certain area.
But one thing we haven’t been able to filter our, are humans. Because there difference compared to the background is at least as high as the overlapping shadow’s, they are registered as hit’s. This means that when a persons swings his arms far enough, inside the detection zone, it will register as a hit. You can see this in the following image
Of course we have tried a lot of things to filter this out, so here comes the list of what we’ve done, tried and still have to try.
For clarification, normal shadows blobs are the blobs of the normal shadows we also retrieve using a lower threshold
Existing checks and filters
Inside detection area check.
Unique check (ignore hit if there was a prev. hit near it).
Hit’s boundingbox hits a normal shadow blob boundingbox check (this also determines the victim).
Tried / though off
Check if hit blob is inside of detection area. Doesn’t always work because when player’s don’t have sleeves the blobs detected in hands get separated from there body, because skin color is not always detected under the threshold that we use for hit’s.
Check size of hitblobs. Doesn’t really work because when only a piece of someone’s hand is detected as hit blobs, it’s about the same size as a regular overlapping shadow hit.
Detecting shadow’s hit’s speed. Player 1 can only be hit by a movement towards the left side. Issue: a lot of times, you don’t have two prev normal shadows, because they were already combined into one big blob. Blobs get combined into one when players are hitting each other.
Detecting hit speed.
Player 1 can only be hit if the hit blobs moves to the left, and the other way around for player 2. Issue: a lot of times, hit’s go so fast we only detect the hit’s once. Using 60fps, 320x240, and a higher app framerate didn’t seem to help.
Using a script that compares rbg separately, instead of comparing grayscale images. I hoped to differentiate skincolor better from the white background. Sadly that didn’t give a different result at the hit blob detection threshold. Getting more difference by skincolor would connect hitblobs created by hands with the arm’s which would unify a human. This way the inside detection zone check would work better.
Using a wide angle camera. We might be able to place it closer to the shadows, that way creating more distance between shadows and humans. But we don’t think there are lenses wide enough.
Instead of projecting shadows on wall, add a white screen, detect shadows from other side of the screen. But the problem is that the installation already takes up a lot of room, this would increase it almost by a factor 2.
Detect only humans using a 3th threshold, then hittest them with hit blobs, ignoring hit blobs when they do.
Using 2 cam’s. Placing 2 cams off center, each capturing one half of the shadows area. This way human’s won’t overlay shadow. Downside is added complexity and having a less flexible hardware setup.
Doing research into flow detection. Maybe it enables us to detect hit speed, even if the prev blobs are joined together.
Do you guys have any ideas?
We would really appreciate your help.
Let me know if there is something unclear.