quad warping an entire OpenGL view (solved)

snaps !! that’s nice

I would have recommended multiplying the matrix – I think the best examples of seeing this in action are in the tutorial for planar shadows. but this is a great use of shaders !! – zach

Hmm
I don’t know which tutorial you are referring to.

I have got somewhat of the way there using some warping code from myler and weeks but I don’t think I grasp 100% how to plug in the coefficients into an openGL matrix - maybe someone smarter than me can point me in the right direction.

The code follows bellow:

  
//lets make a matrix for openGL  
GLfloat myMatrix[16];  
  
//we set it to the default - 0 translation  
//and 1.0 scale for x y z and w  
for(int i = 0; i < 16; i++){  
	if(i % 5 != 0) myMatrix[i] = 0.0;  
	else myMatrix[i] = 1.0;  
}  
  
//this is the startx and endx of my src quad   
// top left corner and bottom right corner  
float sx = 0.0;  
float sy = 0.0;  
float ex = 1.0;  
float ey = 1.0;  
  
//this is an array of four points  
//that describe the four corners  
//of the warped quad -   
// currently they are ordered like:   
//  
//  pt0      pt1  
//  
//  pt3      pt2  
//  
//but I have also tried doing top-left, top-right, bottom-left, bottom-right  
  
float wx[4];  
float wy[4];  
  
//corners are in 0.0 - 1.0 range  
for(int i = 0; i < 4; i++){  
	wx[i] = corners[i].x ;  
	wy[i] = corners[i].y ;  
}  
  
//this is code from Myler and Weeks:   
//calculates the warp coeffecients that describe   
//the transformations between the src and dst quad  
float matrix[8];  
  
matrix[0] = (-wx[0] + wx[1]) / (ey-sy); //a  
matrix[1] = (-wx[0] + wx[3]) / (ex-sx); //b  
matrix[2] = (wx[0] - wx[1] + wx[2] - wx[3]) / ( (ey-sy)  * (ex-sx) ); //c  
matrix[3] =  wx[0]; //d  
  
matrix[4] = (-wy[0] + wy[1]) / (ex-sx); //e  
matrix[5] = (-wy[0] + wy[3]) / (ey-sy); //f  
matrix[6] =  (wy[0] - wy[1] + wy[2] - wy[3]) / ( (ey-sy)  * (ex-sx) ); //i  
matrix[7] = wy[0]; //j  
  
//mylyer and weeks describes then how to translate a new destination X and Y based  
//on these coeffecients   
  
// X = a*x + b*y +c*x*y + d  
// Y = e*x + f*y + i*x*y + j;  
  
//I figured I could put those values into a transformation matrix  
//and then use glMultMatrix  
  
//I understand how to do a*x and b*y but how would I describe c*x*y ????  
//in a openGL matrix.  
//nice guide on matrices here:  
//[http://www.sacredsoftware.net/tutorials/Matrices/Matrices.xhtml](http://www.sacredsoftware.net/tutorials/Matrices/Matrices.xhtml)  
  
myMatrix[0] = matrix[0]; //a*x  
myMatrix[1] = matrix[1]; //b*y  
myMatrix[3] = matrix[3]; //d - the translation   
  
myMatrix[4] = matrix[4]; //e*x  
myMatrix[5] = matrix[5]; //f*y  
myMatrix[7] = matrix[7]; //j - the translation  
		  
//	myMatrix[0] = 1.0;   //this is what normally scales x  
//	myMatrix[5] = 1.0;	 //this is what normally scales y  
myMatrix[10] = 1.0;	//we don't scale z  
myMatrix[15] = 1.0;	//we don't touch w   
  
glMultMatrixf(myMatrix);  
  
//lets draw a bounding box  
ofNoFill();  
ofSetColor(0x00FFFF);  
ofRect(1, 1, ofGetWidth()-2, ofGetHeight()-2);  

about tutorials, I guess I meant any of the planar shadow tutorials…

like:
http://www.seebq.com/cs4451/project3/

I had thought there might be a problem w/ column vs row order, but on double checking it looks ok.

if you try a simple translate matrix or scaling matrix does glMultMatrix() work?

  • zach

I think you are right with the column thing - I had a couple switched around the wrong way. I plugged my numbers in wrong.

  
myMatrix[0] = matrix[0]; //a*x   
myMatrix[4] = matrix[1]; //b*y   
myMatrix[3] = matrix[3]; //d - the translation   
  
myMatrix[1] = matrix[4]; //e*x   
myMatrix[5] = matrix[5]; //f*y   
myMatrix[7] = matrix[7]; //j - the translation   

This is sort of how my brain sees it working -

I think it might not be possible using the coefficients method from myler and weeks, as I don’t think you can tell a matrix to perform x*y - where x is the input x and y is the input y.

if you try a simple translate matrix or scaling matrix does glMultMatrix() work?

yeah all that stuff works super good.
and the myler and weeks code works for warping an input x and y to and output x and y based on a src and dst quad.

Hi ,
I’m not sure , but I could be useful see 3.3.1 of this RV tutorial http://personales.ya.com/charli-e/tutorial/InteractingWithVirtualsWords.pdf talk about Model View transformation using glMultMAtrix® [multiplying viewpoint with the 4x4 reflection (change this reflection matrix to your personal transformation) matrix]

Any way, I would want to share this tutorial. I Think it’s so useful for OF installations.

Charli_e

I’m not looking at mylar and weeks, and I really don’t have my head around matrix math, but I don’t think the code there gets you a matrix, just how to translate from one x,y position to another given the transform (ie, bilinear or perspective based transform).

to get the matrix you need, you might look at ripping out the code from opencv – I think look at cvGetPerspectiveTransform + cvSolve (which is called in cvGetPerspectiveTransform)…

or, just use opencv to get the matrix :slight_smile:

hope that helps!
zach

maybe the AGG quad to quad warping routines are helpful to look at?

http://www.antigrain.com/–code/include-…-erspective

  • zach

You might be able to use OpenCV’s cvWarpPerspectiveQMatrix(…). It should give you the right transformation matrix. Well if I think about it you might have to pad it *somehow* to get from a 3x3 to a 4x4 matrix.

This paper has some info about it… It’s also about modifying the projection matrix so it compensates for a projector that is not orthogonal to the wall.

The paper also explains how to go from the 3x3 homography to a 4x4 matrix which can be used in glMatrixMult. Apparently

  
  
a11  a12  a13  
a21  a22  a23  
a31  a32    1  
  

becomes

  
  
a11  a12    0   a13  
a21  a22    0   a23  
0     0     1    0  
a31  a32    0    1  
  

some-more-homography-stuff

haha - yeah that is exactly the approach I took.
uploading the video now : )

  • thanks for that projection link - super useful!

the 3x3 row ordered matrix to a 4x4 column ordered - was what got me - I sort of figured it out be trial and error - arrrrgh.

I am going to try it now with: cvFindHomography - which is meant to be more accurate. see: http://www.nabble.com/Re:-Code-changes-p7216380.html

– thanks for that link zach - that code looks super useful!

Cool - now fixed to work with the more accurate cvFindHomography function.

You can see the results here:
http://www.vimeo.com/805535

and a projection test here
http://www.vimeo.com/806183

download a demo app with all the code here (xcode):
http://openframeworks.cc/files/005-glwarptest.zip

The nice thing is you can contain the warp within a push and pop matrix - so you can do multiple projections with different warps from the same app.

Hope its useful!

Thanks everyone for the help!
Theo

1 Like

Theo,
This is awesome. It looks way smoother than the texture warping Thanks for this.

check this out:
http://www.cs.cmu.edu/~johnny/projects/thesis/

Yeah I am quite excited about working with the openGL matrix it allows for a lot without having to do expensive image operations. I imagine you could easily combine it with the lens undistortion too! Let the graphics card do the work :smiley:

I have seen that guys stuff it is totally sick - I spent the last couple of days reading papers on self calibrating projectors.

let me know if the code works for what you are doing -
theo

nice to see this working :slight_smile:

This was exactly what i needed, thanks a lot!

I’ve put it in a class, if that’s helpful to anyone:
http://carljohanrosen.com/files/glwarp.zip

It’s dependant on my vector class, which could easily be replaced by ofPoint, i suppose.

cheers
cj

theo, thank you so much for that sample source. It works brilliantly!

just tried it out in of_0.06 and it works a treat.
definitely will come in handy when trying to tackle projection mapping.

that is awesome projection mapping code!!

I am wondering, how can I do the more simple thing, and warp just an ofxCvColorImage ?

I was playing with some quad warping - mapping an FBO onto a quad and then adjusting the quad’s points… but I got poor results, which I’m guessing because I was using just 1 big quad, so it didn’t have enough vertices to accurately transform the texture. But I got even stranger results when I tried to make it a multi-quad surface.

Now I’m wondering if/how I could use this openCV warping approach on just a plane old ofxCvColorImage?

Cheers!
-plong0

cool… making some progress now on warping my camera feed to calibration points. I’m working with the warpPerspective function that I didn’t even realize existed in the ofxCvColorImage. Just figuring out exactly how it interprets the 4 corner points.

Also had a quick play with the warp class from cj … looks sweet!!! I’m stoked, having that class handy and ready to drop in anywhere is gonna make my life sooo much easier really soon. :slight_smile:

Thanks to everyone who worked this problem out! big props to ya all!

I have downloaded the source, It can be compiled and linked,but when I run, error occurs! the following is the output! :frowning:

‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\openFrameworksApp.exe’, Symbols loaded.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ntdll.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\kernel32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\glut32.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\user32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\gdi32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\winmm.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\advapi32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\rpcrt4.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\secur32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\glu32.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\opengl32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\msvcrt.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ddraw.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\dciman32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\fmodex.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\msacm32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ole32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\wsock32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ws2_32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ws2help.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\FreeType-6.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\Zlib.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\crtdll.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\FreeImage.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\dsound.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\version.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\shell32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\shlwapi.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\oleaut32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\setupapi.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\WinSxS\x86_Microsoft.VC90.DebugCRT_1fc8b3b9a1e18e3b_9.0.30729.1_x-ww_f863c71f\msvcp90d.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\WinSxS\x86_Microsoft.VC90.DebugCRT_1fc8b3b9a1e18e3b_9.0.30729.1_x-ww_f863c71f\msvcr90d.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\msvcr90.dll’, Symbols loaded.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\shimeng.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\imm32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\lpk.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\usp10.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.5512_x-ww_35d4ce83\comctl32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\comctl32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\uxtheme.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\msctf.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\apphelp.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\msctfime.ime’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\mslbui.dll’
First-chance exception at 0x7c98671e in openFrameworksApp.exe: 0xC0000142: DLL Initialization Failed.
Unhandled exception at 0x7c98671e in openFrameworksApp.exe: 0xC0000142: DLL Initialization Failed.
The program ‘[3808] openFrameworksApp.exe: Native’ has exited with code 0 (0x0).

hmm… it’s kind of hard to tell what’s happening there. do the other examples for OF compile and run?

  • z