Hey figured I would throw this out to the collective openframeworks brain.
I am trying to figure out how to warp the entire openGL modelview matrix so that I can easily project onto a surface and match the corners to the four corners of the surface I am projecting onto.
With Laser Tag I did this by drawing into a texture and then warping the four corners of the texture.
But if I want my entire output including OpenGL shapes, textures everything to be able to be warped so that it appears flat on my projection surface I figured I would need to be able to warp my OpenGL Modelview Matrix (or projection matrix ?) .
Does anyone know a good approach to doing this?
It seems that it would be something to do with glMultMatrix:
So I took another approach to this - it seemed like a vertex shader which distorts all the coordinates based on the warped shape could work quite well. Using zach’s ofShader addon and after a little reading up on coordinate warping I got a basic demo working.
I would have recommended multiplying the matrix – I think the best examples of seeing this in action are in the tutorial for planar shadows. but this is a great use of shaders !! – zach
Hmm
I don’t know which tutorial you are referring to.
I have got somewhat of the way there using some warping code from myler and weeks but I don’t think I grasp 100% how to plug in the coefficients into an openGL matrix - maybe someone smarter than me can point me in the right direction.
The code follows bellow:
//lets make a matrix for openGL
GLfloat myMatrix[16];
//we set it to the default - 0 translation
//and 1.0 scale for x y z and w
for(int i = 0; i < 16; i++){
if(i % 5 != 0) myMatrix[i] = 0.0;
else myMatrix[i] = 1.0;
}
//this is the startx and endx of my src quad
// top left corner and bottom right corner
float sx = 0.0;
float sy = 0.0;
float ex = 1.0;
float ey = 1.0;
//this is an array of four points
//that describe the four corners
//of the warped quad -
// currently they are ordered like:
//
// pt0 pt1
//
// pt3 pt2
//
//but I have also tried doing top-left, top-right, bottom-left, bottom-right
float wx[4];
float wy[4];
//corners are in 0.0 - 1.0 range
for(int i = 0; i < 4; i++){
wx[i] = corners[i].x ;
wy[i] = corners[i].y ;
}
//this is code from Myler and Weeks:
//calculates the warp coeffecients that describe
//the transformations between the src and dst quad
float matrix[8];
matrix[0] = (-wx[0] + wx[1]) / (ey-sy); //a
matrix[1] = (-wx[0] + wx[3]) / (ex-sx); //b
matrix[2] = (wx[0] - wx[1] + wx[2] - wx[3]) / ( (ey-sy) * (ex-sx) ); //c
matrix[3] = wx[0]; //d
matrix[4] = (-wy[0] + wy[1]) / (ex-sx); //e
matrix[5] = (-wy[0] + wy[3]) / (ey-sy); //f
matrix[6] = (wy[0] - wy[1] + wy[2] - wy[3]) / ( (ey-sy) * (ex-sx) ); //i
matrix[7] = wy[0]; //j
//mylyer and weeks describes then how to translate a new destination X and Y based
//on these coeffecients
// X = a*x + b*y +c*x*y + d
// Y = e*x + f*y + i*x*y + j;
//I figured I could put those values into a transformation matrix
//and then use glMultMatrix
//I understand how to do a*x and b*y but how would I describe c*x*y ????
//in a openGL matrix.
//nice guide on matrices here:
//[http://www.sacredsoftware.net/tutorials/Matrices/Matrices.xhtml](http://www.sacredsoftware.net/tutorials/Matrices/Matrices.xhtml)
myMatrix[0] = matrix[0]; //a*x
myMatrix[1] = matrix[1]; //b*y
myMatrix[3] = matrix[3]; //d - the translation
myMatrix[4] = matrix[4]; //e*x
myMatrix[5] = matrix[5]; //f*y
myMatrix[7] = matrix[7]; //j - the translation
// myMatrix[0] = 1.0; //this is what normally scales x
// myMatrix[5] = 1.0; //this is what normally scales y
myMatrix[10] = 1.0; //we don't scale z
myMatrix[15] = 1.0; //we don't touch w
glMultMatrixf(myMatrix);
//lets draw a bounding box
ofNoFill();
ofSetColor(0x00FFFF);
ofRect(1, 1, ofGetWidth()-2, ofGetHeight()-2);
I think it might not be possible using the coefficients method from myler and weeks, as I don’t think you can tell a matrix to perform x*y - where x is the input x and y is the input y.
if you try a simple translate matrix or scaling matrix does glMultMatrix() work?
yeah all that stuff works super good.
and the myler and weeks code works for warping an input x and y to and output x and y based on a src and dst quad.
Hi ,
I’m not sure , but I could be useful see 3.3.1 of this RV tutorial http://personales.ya.com/charli-e/tutorial/InteractingWithVirtualsWords.pdf talk about Model View transformation using glMultMAtrix® [multiplying viewpoint with the 4x4 reflection (change this reflection matrix to your personal transformation) matrix]
Any way, I would want to share this tutorial. I Think it’s so useful for OF installations.
I’m not looking at mylar and weeks, and I really don’t have my head around matrix math, but I don’t think the code there gets you a matrix, just how to translate from one x,y position to another given the transform (ie, bilinear or perspective based transform).
to get the matrix you need, you might look at ripping out the code from opencv – I think look at cvGetPerspectiveTransform + cvSolve (which is called in cvGetPerspectiveTransform)…
You might be able to use OpenCV’s cvWarpPerspectiveQMatrix(…). It should give you the right transformation matrix. Well if I think about it you might have to pad it *somehow* to get from a 3x3 to a 4x4 matrix.
This paper has some info about it… It’s also about modifying the projection matrix so it compensates for a projector that is not orthogonal to the wall.
The paper also explains how to go from the 3x3 homography to a 4x4 matrix which can be used in glMatrixMult. Apparently
Yeah I am quite excited about working with the openGL matrix it allows for a lot without having to do expensive image operations. I imagine you could easily combine it with the lens undistortion too! Let the graphics card do the work
I have seen that guys stuff it is totally sick - I spent the last couple of days reading papers on self calibrating projectors.
let me know if the code works for what you are doing -
theo
I am wondering, how can I do the more simple thing, and warp just an ofxCvColorImage ?
I was playing with some quad warping - mapping an FBO onto a quad and then adjusting the quad’s points… but I got poor results, which I’m guessing because I was using just 1 big quad, so it didn’t have enough vertices to accurately transform the texture. But I got even stranger results when I tried to make it a multi-quad surface.
Now I’m wondering if/how I could use this openCV warping approach on just a plane old ofxCvColorImage?
cool… making some progress now on warping my camera feed to calibration points. I’m working with the warpPerspective function that I didn’t even realize existed in the ofxCvColorImage. Just figuring out exactly how it interprets the 4 corner points.
Also had a quick play with the warp class from cj … looks sweet!!! I’m stoked, having that class handy and ready to drop in anywhere is gonna make my life sooo much easier really soon.
Thanks to everyone who worked this problem out! big props to ya all!
