Mapping ofxPuppet onto kinect skeleton

I’m trying to figure out the best way of mapping a 2D puppet/character onto a kinect skeleton and am looking for advice.

I had a quick look at cutting up each limb (lower arms, upper arms, body, upper legs, lower legs) and sticking them to each skeleton joint and rotate them accordingly. Trouble is the results are not great with this - and even worse when the limb images have a lot of detail (so when they rotate they do not match up).

Then I found ofxPuppet which looks good. So now, I am creating a rectangle mesh and some control points and mapping the control points to the skeleton joints. So for the body I am mapping:

0,0 - shoulder left
1,0 - shoulder right
1,1 - hip right
0,1 - hip left

My problem is that the mesh does not scale properly and ends up like this:

Is this a problem with the number of control points I am using or am I simply heading down the wrong path here? Has anyone had experience with this? @zach any ideas?

It feels like it’s because ofxPuppet is rigid body and I need some way of scaling it as well as controlling the mesh. I was thinking about using a simple quad mesh distortion but that doesn’t give as much ‘character’ to the output.

I’m looking into doing a similar project. Did you have any luck @csw?

Did you run into any other issues along the way?