offset in homography matrix

Hi all,
I am using a homography matrix to reproject the blob that is identified by kinect depth camera ,
onto a person that is sitting in front of a projector.

The kinect is sitting on top of the projector.

after matching points between kinect and projector coordinate space, I get an offset between myself and the blob that should be drawn across my outline.

when I try to reach for my toes with my fingers, the blob around my hand extends a little below my actual hand.
and when I extend my arm on top the blob is offset higher than my hand.
same for left - right situaltion

is there anyway I can correct this ?
thank you for your help.