quad warping an entire OpenGL view (solved)

cool… making some progress now on warping my camera feed to calibration points. I’m working with the warpPerspective function that I didn’t even realize existed in the ofxCvColorImage. Just figuring out exactly how it interprets the 4 corner points.

Also had a quick play with the warp class from cj … looks sweet!!! I’m stoked, having that class handy and ready to drop in anywhere is gonna make my life sooo much easier really soon. :slight_smile:

Thanks to everyone who worked this problem out! big props to ya all!

I have downloaded the source, It can be compiled and linked,but when I run, error occurs! the following is the output! :frowning:

‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\openFrameworksApp.exe’, Symbols loaded.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ntdll.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\kernel32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\glut32.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\user32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\gdi32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\winmm.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\advapi32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\rpcrt4.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\secur32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\glu32.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\opengl32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\msvcrt.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ddraw.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\dciman32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\fmodex.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\msacm32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ole32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\wsock32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ws2_32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\ws2help.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\FreeType-6.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\Zlib.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\crtdll.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\FreeImage.dll’, Binary was not built with debug information.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\dsound.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\version.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\shell32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\shlwapi.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\oleaut32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\setupapi.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\WinSxS\x86_Microsoft.VC90.DebugCRT_1fc8b3b9a1e18e3b_9.0.30729.1_x-ww_f863c71f\msvcp90d.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\WinSxS\x86_Microsoft.VC90.DebugCRT_1fc8b3b9a1e18e3b_9.0.30729.1_x-ww_f863c71f\msvcr90d.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\Documents and Settings\Jonah\desktop\005-glwarptest\addons\ofOsc\examples\oscReceiveExample\Debug\msvcr90.dll’, Symbols loaded.
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\shimeng.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\imm32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\lpk.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\usp10.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.5512_x-ww_35d4ce83\comctl32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\comctl32.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\uxtheme.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\msctf.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\apphelp.dll’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\msctfime.ime’
‘openFrameworksApp.exe’: Loaded ‘C:\WINDOWS\system32\mslbui.dll’
First-chance exception at 0x7c98671e in openFrameworksApp.exe: 0xC0000142: DLL Initialization Failed.
Unhandled exception at 0x7c98671e in openFrameworksApp.exe: 0xC0000142: DLL Initialization Failed.
The program ‘[3808] openFrameworksApp.exe: Native’ has exited with code 0 (0x0).

hmm… it’s kind of hard to tell what’s happening there. do the other examples for OF compile and run?

  • z

[quote author=“zach”]hmm… it’s kind of hard to tell what’s happening there. do the other examples for OF compile and run?

  • z[/quote]
    The source was compiled and linked by Visual Studio 2008, I have tried several times on different PC and met the same error. but when I download pre release v0.061. examples can be copiled and it can run correctly! :slight_smile:

I have made it , thanks for zach’s reply!

this can also be done without opencv to find the homography by using this:

http://forum.openframeworks.cc/t/quad-warping–homography-without-opencv/3121/5

just call the function findHomography passing the source and destination points and a float[16] matrix to get the result in it, then use that matrix to multiply the opengl matrix by using glMultMatrix as in theo’s original example

can i draw a normal scaled image above all this warped up stuff ?

i tried glLoadIdentity(); but this doesnt seem working. Anyone knows the functions that i have to use?

greetings

ok. solved.

  
// back to normal +++++++++++++++++++++++++++++++++++++++++++++++++  
	float halfFov, theTan, aspect;  
	  
	float eyeX              = (float) ofGetWidth() / 2.0;  
	float eyeY              = (float) ofGetHeight() / 2.0;  
	halfFov                 = PI * 60.0 / 360.0;  
	theTan                  = tanf(halfFov);  
	float dist              = eyeY / theTan;  
	float nearDist  = dist / 10.0;  // near / far clip plane  
	float farDist   = dist * 10.0;  
	aspect                  = (float) ofGetWidth()/(float)ofGetHeight();  
	  
	glMatrixMode(GL_PROJECTION);  
	glLoadIdentity();  
	gluPerspective(60.0, aspect, nearDist, farDist);  
	  
	glMatrixMode(GL_MODELVIEW);  
	glLoadIdentity();  
	gluLookAt(eyeX, eyeY, dist, eyeX, eyeY, 0.0, 0.0, 1.0, 0.0);  
	  
	  
	glScalef(1, -1, 1);           // invert Y axis so increasing Y goes down.  
	glTranslatef(0, -ofGetHeight(), 0);       // shift origin up to upper-left corner.  

wow! strange, but really helps me to ask myself questions in the forum!

I think you can call ofSetupScreen() and it will do all of that for you.

sorry for digging up a old post.
However it would be perfect place for me to start in a project that i am working up.

when i try to compile theo’s program i am getting 4 errors,

ofUDPManager.cpp:329:0 ‘TCP_MAXSEG’ was not declared in this scope

Category: Error: invalid conversion from ‘*’ to ‘*’

/ofAppRunner.cpp:288:0 Invalid conversion from ‘long int*’ to ‘const GLint*’

Category: Error: Other

/ofAppRunner.cpp:288:0 Initializing argument 3 of ‘CGLError CGLSetParameter(_CGLContextObject*, CGLContextParameter, const GLint*)’

C_glWarper/openFrameworksDebug.app/Contents/MacOS/libfmodex.dylib: No such file or directory

the test app works great,
I am on osx 10.6.5,
and i new to this.

thanks so much for the code and openframeworks,
and any help

Ben

Hi, I wrapped Theo’s code into an addon for easier use.
https://github.com/roymacdonald/ofxGLWarper
I also added an activate/deactivate method so the homography matrix is only calculated when the warped corners are being moved.
I hope this might be of any help.
best,
roy

Hola !!

sorry if the thread should be closed (solved), but still i think it’s the best place to answer …

well i’ve tested the ofxGLWarper and works great and really accurate, i was working before with memo’s shader to make quad_warping but it had no perspective correcteness … and this one seems to have . So : beautiful addon !! Thanks Theo & Kyle & all …

now my question is … which will be the process to be able to know the coordinates of a given point (mouse click) made on screen, but getting the point coordinates from the warped quad, but in local (quad) coordinate system ?? …

what i’m trying to ask (i’ll try to explain what i’m trying to ask …jjjj) is how can i get the coordinates of a point clicked over the warped image (eye coordinates) and got the coordinates of that point on the original quad coordinates system ?

what i imagine would be something like multiplying the point by the inverse of the matrix coming from ofxGLWarper … but my difficulty is on understanding what kind of “inversion” of the matrix should i do …

(( i’m sorry if its a confusing question … i’ve tried my best english (i’m not english) to try to explain my doubt … ))

any ideas, suggestions ?

e*

  
  
ofMatrix4x4 inverse = ofMatrix4x4::getInverseOf(matrix);  
  

should give you the inverse of the matrix you get from the warping addon, then multiply it by the vector.

hola arturo !

thanks for your advise … that’s exactly what i was trying to achieve but got not coherent results …
so what i did, i created a new function in ofxGLWarper …
the idea is that i give a point as input and it gives me a point as ouput …

ofVec4f ofxGLWarper::getPoint(float x, float y, float z)
{
ofVec4f mousePoint;
ofVec4f warpedPoint;

// this is the point (in no warped system) i want to know the coordinates in the warped system …
mousePoint.x = x;
mousePoint.y = y;
mousePoint.z = 0.0;
mousePoint.w = 0.0;

// create a ofMatrix4x4 with the ofxGLWarper myMatrix data
ofMatrix4x4 myOFmatrix = ofMatrix4x4(myMatrix[0], myMatrix[1],myMatrix[2],myMatrix[3],
myMatrix[4], myMatrix[5],myMatrix[6], myMatrix[7],
myMatrix[8], myMatrix[9],myMatrix[10],myMatrix[11],
myMatrix[12],myMatrix[13],myMatrix[14],myMatrix[15]);

// invert the matrix
//ofMatrix4x4 invertedMyMatrix = myOFmatrix.getInverse();

// or ?

// don’t invert the matrix
ofMatrix4x4 invertedMyMatrix = myOFmatrix;

// multiply
warpedPoint = invertedMyMatrix * mousePoint ;

return warpedPoint;

}

As you can see i’ve 2 option on this code (one is commented) … so invert or not the matrix …

First approach … if i don’t invert the matrix, i think i should be getting same results as the warped quad, so if i want to calculate the result of this for the middle quad point (1024/2 , 768/2) it should stick to the center of the image while i’m warping … ?? am i guessing right ? and the point doesn’t follow the middle of the image …

I’m starting with this approach because it’s easier to me to think what’s going on …

The second approach is to invert the matrix but then graphically i can’t understand where the pint should go to …

Any idea if i’m doing something wrong ? Thanks …

I’ve been facing almost the same problem, though I’m using this approach to get homography without openCV

http://forum.openframeworks.cc/t/quad-warping–homography-without-opencv/3121/5

After struggling a while to get reverse homography I managed to get it right. What I was doing wrong (and it maybe possibly your case as well) it was forgetting to normalize x,y dividing them by z coordinate (i’m using the 3x3 homography matrix, in case of 4x4 matrix you probably have to normalize dividing by w)

if I do:
x = warped.x/warped.z;
y = warped.y/warped.z;

the homography is perfectly reversed.

see this for reference:
http://tech.groups.yahoo.com/group/OpenCV/message/80121

also, mouse.w must probably be set to 1.0 instead than 0.0

this is my code, I’m afraid It’s orribly ugly and messy, sorry

  
  
ofVec3f mouse;  
ofVec3f warped;  
  
  
ofMatrix4x4 warpMatrix;  
ofMatrix3x3 homographyMatrix3x3;  
float multMatrix[3][3];  
  
mouse.x = (float)x;  
mouse.y = (float)y;  
mouse.z = 1.0;  
  
warpMatrix = findVectorHomography(src, dst);  
   
homographyMatrix3x3 = ofMatrix3x3(warpMatrix(0,0),warpMatrix(0,1), warpMatrix(0,3), warpMatrix(1,0), warpMatrix(1,1), warpMatrix(1,3), warpMatrix(3,0), warpMatrix(3,1), warpMatrix(3,3));  
  
homographyMatrix3x3.invert();  
  
multMatrix = {{homographyMatrix3x3[0], homographyMatrix3x3[1], homographyMatrix3x3[2]}, {homographyMatrix3x3[3], homographyMatrix3x3[4], homographyMatrix3x3[5]}, {homographyMatrix3x3[6], homographyMatrix3x3[7], homographyMatrix3x3[8]}};  
  
// vector-matrix mult   
    for (int i=0; i<3; i++) {  
        for (int j=0; j<3; j++) {  
            warped[j] += (float)multMatrix[j][i] * mouse[i];  
        }  
    }  
  
     
    cout << "warped x = " << warped.x/warped.z <<"\n";  
    cout << "warped y = " << warped.y/warped.z <<"\n";  
    cout << "warped z = " << warped.z <<"\n\n";  
  
  

1 Like

hi francesco !

bravo bravissimo !! thanks a lot for your tip about normalizing the coordinates … without you it could have token me years to find … :wink: yes now it’s working !!

i did exactly what you suggested :

warpedPoint.x = warpedPoint.x / warpedPoint.w;
warpedPoint.y = warpedPoint.y / warpedPoint.w;
warpedPoint.z = warpedPoint.z / warpedPoint.w;

and put

mousePoint.w = 1.0;

!!!

also i’ve changed some minor details in ofxGLWarper … i’ll try to tell the author and maybe they make sense, maybe are no sense . For example : changed the draw() function by begin() and added a end() also to make more clear (in my opinion) how it works, more like a ofFbo or ofShader …

and i’ve added 2 functions :
one to get the Warped space coordinates from a Screen spac coordinates .
and another one to get the Screen space coordinates from a Warped spac coordinates .

i leave it here in case someone needs it …

thanks everyone ;)!

e*

  
  
.h  
        //void draw();  
	void begin();	//changed name from draw to begin  
	void end();		//added to make it easier to use, similar to ofFbo (begin,end)  
  
  
	ofVec4f		fromScreenToWarpCoord(float x,float y,float z);  
	ofVec4f		fromWarpToScreenCoord(float x,float y,float z);  
  
  
  

  
  
.cpp  
  
//--------------------------------------------------------------  
  
//--------------------------------------------------------------  
void ofxGLWarper::begin(){  
	if (active) {  
		processMatrices();  
	}  
	glPushMatrix();  
	glMultMatrixf(myMatrix);  
}  
  
  
//--------------------------------------------------------------  
void ofxGLWarper::end(){  
	glPopMatrix();  
}  
  
  
ofVec4f ofxGLWarper::fromScreenToWarpCoord(float x, float y, float z)  
{  
	ofVec4f mousePoint;  
	ofVec4f warpedPoint;  
	  
	// this is the point on the image which i want to know the coordinates inside the warped system ...   
	mousePoint.x = x;  
	mousePoint.y = y;  
	mousePoint.z = 0.0;  
	mousePoint.w = 1.0;  
	  
	// i create a ofMatrix4x4 with the ofxGLWarper myMatrixData in column order  
	ofMatrix4x4 myOFmatrix = ofMatrix4x4(myMatrix[0], myMatrix[4],myMatrix[8],myMatrix[12],  
										 myMatrix[1], myMatrix[5],myMatrix[9], myMatrix[13],  
										 myMatrix[2], myMatrix[6],myMatrix[10],myMatrix[14],  
										 myMatrix[3],myMatrix[7],myMatrix[11],myMatrix[15]);  
	// do not invert the matrix   
	ofMatrix4x4 invertedMyMatrix = myOFmatrix.getInverse();	  
	//ofMatrix4x4 invertedMyMatrix = myOFmatrix;  
	  
	// multiply  
	warpedPoint = invertedMyMatrix * mousePoint ;  
	  
	warpedPoint.x = warpedPoint.x / warpedPoint.w;  
	warpedPoint.y = warpedPoint.y / warpedPoint.w;  
	warpedPoint.z = warpedPoint.z / warpedPoint.w;  
	  
	return warpedPoint;  
}  
  
//--------------------------------------------------------------  
ofVec4f ofxGLWarper::fromWarpToScreenCoord(float x, float y, float z)  
{  
	ofVec4f mousePoint;  
	ofVec4f warpedPoint;  
	  
	// this is the point on the image which i want to know the coordinates inside the warped system ...   
	mousePoint.x = x;  
	mousePoint.y = y;  
	mousePoint.z = 0.0;  
	mousePoint.w = 1.0;  
	  
	// i create a ofMatrix4x4 with the ofxGLWarper myMatrixData in column order  
	ofMatrix4x4 myOFmatrix = ofMatrix4x4(myMatrix[0], myMatrix[4],myMatrix[8],myMatrix[12],  
										 myMatrix[1], myMatrix[5],myMatrix[9], myMatrix[13],  
										 myMatrix[2], myMatrix[6],myMatrix[10],myMatrix[14],  
										 myMatrix[3],myMatrix[7],myMatrix[11],myMatrix[15]);  
	// invert the matrix   
	//ofMatrix4x4 invertedMyMatrix = myOFmatrix.getInverse();	  
	ofMatrix4x4 invertedMyMatrix = myOFmatrix;  
	  
	// multiply  
	warpedPoint = invertedMyMatrix * mousePoint ;  
	  
	warpedPoint.x = warpedPoint.x / warpedPoint.w;  
	warpedPoint.y = warpedPoint.y / warpedPoint.w;  
	warpedPoint.z = warpedPoint.z / warpedPoint.w;  
	  
	return warpedPoint;  
}  

Hola eloi,
are you using the addon that I published?
Your changes make a lot of sense.
Can you branch the addon and then place a pull request?
thanks for the improvements!

regards!