Kinect/camera calibration, reverse process?

I’m using opencv to create a projector/kinect calibration system.

I’m using the opencv calibrateCamera then the projectPoint function to calculate the 2d projected point from a 2d camera point. This works well but I want to now do the opposite, calculate the camera point from a 2d projected point. For example I want calculate the corners of where the projected image are in the camera image.

How can I achieve this? Is there a way to reverse the process?

I’ve already tried the undistortPoint function in opencv but it seems to just give me the same output value as input value (or very close).

Did you try https://github.com/genekogan/ofxKinectProjectorToolkit in it you can do what you are saying.

thanks for the reply roy. Yes I did look at that addon but the way it works is:

screen point -> 3d world point -> projector point

I want to do the reverse

projector point -> screen point

hey! I found I had a modified version of ofxKinectProjectorToolkit in which I added a feature where it generates a GLM transform matrix, which allows you to easily do the transform one way or the other.

it has a function called getUnprojectedPoint which should work.
(I was testing and the glm matrix I am building does not invert properly so I am not a 100% sure if the results are correct. Probably these will be a bit off. dont know. please let me know how it works. I dont have the right equipment right now for testing

thanks a lot roy! I’ll test it out later and let you know how it goes.

What do you use for the z input value for the getUnProjectedPoint() function? The results i’m getting are way off no matter if it’s 0 or something else.

I see. for the z value you should use the depth from the kinect, which you need to unproject first. use ofxKinect´s function ofVec3f getWorldCoordinateAt(int cx, int cy) const; and extract the z value from it.
I have not done this in a long time so I dont remember correctly, but it is just a matter of matrix transformations. I think that @genekogan or @elliotwoods might have a clearer idea.

ok thanks for all your help:)