4x4 view projection from 3x3 homography matrix

Hello!

I have already got a 3x3 homography matrix to transform any x,y-coordinate on the zero-z-plane into the correct x,y-coordinate on the screen. Now I need to get a 4x4 view projection matrix, which I can use in OpenGL or Direct3D. That matrix has to also work with vertices, which have a z-component unequal to zero. An easy matrix manipulation by just adding a column and row like in that thread does only work for the zero-z-plane and makes everything else flat.
We already could create a 4x3 matrix to get voluminous projection but unfortunately vertices with z != 0 are still not projected like in the original photo / video. Besides we cannot use a 4x3 matrix because then we have to divide xy by z to get the correct screen coordinates so that z becomes 1, which makes depth buffering impossible.

Finally I do not need a view projection matrix but the extrinsic and intrinsic parameters of the camera. The camera model is simplified to match OpenGL / Direct3D so there is no skew, fish eye distortion and even the optical axis is at the center.
I tried to use OpenCV with some projected points from the plane to the screen but that did not give me the correct camera parameters. Did anybody already implement a working solution by using corresponding points?

I hope that someone could help me.

Thank you in advance!