Stretching, skewing, and stitching cameras (gl not working!)

Hey! I’m trying to skew and crop two images to form two stitched kinect images, and then skew them in order to correct for the fact that they’re pointed downwards. This all looks right according to all the examples I’ve seen, so the fact that it’s not working is driving me a bit nuts.

I feel like I’m missing something very basic, but am running up against deadlines and am still only part of the way through the Orange Book…

Here’s my code (That doesn’t work) for skewing the images, would hugely appreciate feedback!

  
  
void testApp::drawSkew(ofImage * src, ofPoint * tl, ofPoint * tr, ofPoint * br, ofPoint * bl) {  
    // tl,tr,br,bl = normalized 0.-1.   
    ofSetColor(255);  
    src->reloadTexture();  
        //src->draw(0,0);  
        //ofDisableArbTex();  
    src->getTextureReference().bind();  
        //    glEnable(GL_TEXTURE_2D);  
    glMatrixMode(GL_TEXTURE);   
    glPushMatrix();  
    ofScale(src->getWidth(), src->getHeight());    
    glMatrixMode(GL_MODELVIEW);    
    glEnable(GL_DEPTH_TEST);  
    glBegin(GL_QUADS);    
    glNormal3f(0.0f,0.0f,1.0f);  
      
    //Top Left  
    glTexCoord3f(0.0f, 0.0f, 0.0f);   
    glVertex3f( tl->x, tl->y, 0.0f);      
      
    // Top Right  
    glTexCoord3f(1.0f, 0.0f, 0.0f);   
    glVertex3f( tr->x, tr->y, 0.0f);      
  
    // Bottom Right  
    glTexCoord3f(1.0f, 1.0f, 0.0f);   
    glVertex3f( br->x,br->y, 0.0f);      
  
    // Bottom Left  
    glTexCoord3f(0.0f, 1.0f, 0.0f);   
    glVertex3f( bl->x, bl->y, 0.0f);         
    glEnd();    
    glDisable (GL_DEPTH_TEST);  
    glMatrixMode(GL_TEXTURE);    
    glPopMatrix();    
    glMatrixMode(GL_MODELVIEW);   
        //    glDisable(GL_TEXTURE_2D);  
    src->getTextureReference().unbind();  
        //ofEnableArbTex();  
}  
  

and here’s the code for stitching them together.

  
  
ofTexture testApp::stitchKinect(ofImage * _k1, ofImage * _k2) {  
    if(!stitchedImage.isAllocated()) {  
    stitchedImage.allocate(  
                           _k1->width+_k2->width  
                           , _k1->height  
                           , GL_RGBA, 1);  
    }  
      
    ofVec2f k1offset = ofVec2f(  
                           panel.getValueI("k1_x"),  
                           panel.getValueI("k1_y")  
                               );  
  
    ofVec2f k2offset = ofVec2f(  
                               panel.getValueI("k2_x") + _k1->width,  
                               panel.getValueI("k2_y")  
                               );  
      
    float k1crop = panel.getValueI("k1_clear");  
    float k2crop = panel.getValueI("k2_clear");  
    bool flip = panel.getValueB("flipKinect");  
    bool mirror = panel.getValueB("mirrorKinect");  
      
    ofPoint k1tl = (panel.getValueF("k1_tl_x"), panel.getValueF("k1_tl_y"));  
    ofPoint k1tr = (panel.getValueF("k1_tr_x"), panel.getValueF("k1_tr_y"));  
    ofPoint k1br = (panel.getValueF("k1_br_x"), panel.getValueF("k1_br_y"));  
    ofPoint k1bl = (panel.getValueF("k1_bl_x"), panel.getValueF("k1_tr_y"));  
      
    ofPoint k2tl = (panel.getValueF("k2_tl_x"), panel.getValueF("k2_tl_y"));  
    ofPoint k2tr = (panel.getValueF("k2_tr_x"), panel.getValueF("k2_tr_y"));  
    ofPoint k2br = (panel.getValueF("k2_br_x"), panel.getValueF("k2_br_y"));  
    ofPoint k2bl = (panel.getValueF("k2_bl_x"), panel.getValueF("k2_tr_y"));      
      
      
    if(mirror) {  
        _k1->mirror(0,1);  
        _k2->mirror(0,1);  
    }  
          
    stitchedImage.begin();  
        ofClear(255);  
        ofPushMatrix();  
            ofSetColor(255,255,255);   
            ofTranslate(k1offset);  
              
        drawSkew(_k1,&k1tl,&k1tr,&k1br,&k1bl);  
        //            if(flip) _k1->draw(0,0);  
        //            else _k2->draw(0,0);  
  
            if(flip) drawSkew(_k1,&k1tl,&k1tr,&k1br,&k1bl);  
            else     drawSkew(_k2,&k2tl,&k2tr,&k2br,&k2bl);  
  
            ofTranslate(k2offset);  
  
            if(flip) _k2->draw(0,0);  
            else _k1->draw(0,0);  
      
        // glDisable(GL_DEPTH_TEST);  
        ofPopMatrix();  
    stitchedImage.end();  
      
    return stitchedImage.getTextureReference();  
}  
  
  

It currently looks like this (in debug mode).
There’s a bunch of other things going on that work, mostly in the opencv space, but this is the specific part of the chain that is just straight up not working, and I’m fairly certain it’s my just not understanding the horrifying state machine that is openGL well enough.

Here’s a pic of what’s happening in the stitch method.

![](http://forum.openframeworks.cc/uploads/default/2588/Screen Shot 2012-10-15 at 4.17.34 PM.png)

It looks like you’re grabbing the wrong values out of your panel in stitchKinect()

line 27

  
  
ofPoint k1bl = (panel.getValueF("k1_bl_x"), panel.getValueF("k1_tr_y"));   
  

looks like it should be

  
  
ofPoint k1bl = (panel.getValueF("k1_bl_x"), panel.getValueF("k1_bl_y"));   
  

(different y value)

Thanks for picking that out, but still no go. I’m worried that it might be an openGL state somewhere that isn’t caught.

The chain is basically:
Camera or Kinect to ofImage.
OfImages passed to the stitching method, which then outputs to a file that gets passed into an ofImage which is turned into a cv::Mat object.

  
  
            stitchKinect(&depthImg,&depthImg2).readToPixels(stitched);  
  

And then operations are performed on that to generate some meshes and similar.

Those latter parts are mostly working, but I’m trying to get the images stitched together so that the contour finder later down the chain can get a manually stitched/warped image to work with.