Algorithm suggestions for zooming one rectangle into another smaller one

Hi All,

I’m working on an installation that tracks peoples faces, and does an infinite zoom, from face onto face onto face. I want to resolve to the new face when I have zoomed to the pixel level of the past face.

I’ve written some code using Kyles ofxCV example of Face zooming that allows me to zoom in on my own face again and again:

  
  
void testApp::setup() {  
	ofSetVerticalSync(true);  
	ofSetFrameRate(30);  
      
    cameraWidth = 1280;  
    cameraHeight = 720;  
      
	finder.setup("haarcascade_frontalface_alt2.xml");  
	finder.setPreset(ObjectFinder::Fast);  
	cam.initGrabber(cameraWidth, cameraHeight);  
      
    //setting up the image to put the flipped camera image into  
    flippedLiveCameraImage.allocate(cameraWidth, cameraHeight, OF_IMAGE_COLOR);  
      
    //allocating the image to scale into......  
    cropped.allocate(ofGetWidth(), ofGetHeight(), OF_IMAGE_COLOR);  
      
    //setup the correct size for the camera rectangle  
    cameraRectangle = ofRectangle(0, 0, cameraWidth, cameraHeight);  
}  
  
void testApp::update() {  
    t = ofGetElapsedTimef();  
    float tweenValue = fmod(t,1); //tween will go between 0 and 1 every second, stolen from periodicSignalsExample in examples/math  
      
	cam.update();  
	if(cam.isFrameNew()) {  
        flippedLiveCameraImage.setFromPixels(cam.getPixelsRef());  
        flippedLiveCameraImage.mirror(false, true); //vertical, horizontal  
		finder.update(flippedLiveCameraImage);  
          
        if(finder.size() > 0) { //as long as we have at least one face we are safe in this...  
            ofRectangle object = finder.getObject(0);  
            object = matchAspectRatioOfOtherRectangle(object, cameraRectangle);  
            object = tweenBetweenTwoRectangles(cameraRectangle, object, tweenValue);  
              
            cv::Rect roi = toCv(object); //convert to CV co-ordinates  
			Mat camMat = toCv(flippedLiveCameraImage); // make a new CV matrix out of the cam image file  
			Mat croppedCamMat(camMat, roi); //do region of interest on cammat, and put the result into croppedCamMat  
			resize(croppedCamMat, cropped); //resize the croppedcammat to the size of cropped, which is the size of the whole screen  
			cropped.update(); //update the pixels, so when they are displayed in draw, they are the latest pixels  
		}  
	}  
}  
  
void testApp::draw() {  
    ofSetColor(ofColor::white);  
	cropped.draw(0, 0, ofGetWidth(), ofGetHeight());  
}  
  
ofRectangle testApp::tweenBetweenTwoRectangles(ofRectangle start, ofRectangle end, float tweenAmount){  
    ofRectangle result = start; //safety first!  
      
    ofPoint newTopLeft = start.getTopLeft().getInterpolated(end.getTopLeft(), tweenAmount);  
    ofPoint newBottomRight = start.getBottomRight().getInterpolated(end.getBottomRight(), tweenAmount);  
      
    result = ofRectangle(newTopLeft, newBottomRight);  
      
    return result;  
}  
  
ofRectangle testApp::matchAspectRatioOfOtherRectangle(ofRectangle alterMe, ofRectangle matchToMe){  
    ofRectangle result = alterMe; //safety first!  
      
    float matchRatioOfWidthToHeight = matchToMe.getWidth()/matchToMe.getHeight();  
      
    float newWidthOfAltered = alterMe.getHeight() * matchRatioOfWidthToHeight;  
    float halfDifferenceBetweenNewWidthAndOld = (alterMe.getWidth() - newWidthOfAltered)/2.f; //want the halved value, as we are trying to work out the scale factor....  
    float ratioOfNewWidthToOld = newWidthOfAltered / alterMe.getWidth();  
      
    result.scaleFromCenter(ratioOfNewWidthToOld, 1.f); //change the width, not the height...  
      
    return result;  
}  
  
  

This works, but in edge cases where the face is at the edge of the screen, the ROI crashes out the application as my new aspect ratio corrected rectangle goes over the edge of the image - openCV fails when it tries to do an ROI over non existent pixels.

Does anyone have any suggestions of a safe ROI that doesn’t care about non existent pixels?

I suppose the ultimate problem is that I’m resolving to square pixels, but starting with a widedscreen camera image - something has to give!

Do you have to use pixels at all?

How about something like:

  
//--------------------------------------------------------------  
void ofApp::setup(){  
      
    ofSetVerticalSync(true);  
    ofSetFrameRate(60);  
      
    dir.listDir("");  
      
    float maxZoomFactor = 0.01; // stops the zoom happening too fast in this demo but you could ease in the percentageTweened instead  
                                // set it to 1.0 for truly one pixel zoom!!!  
      
    for (int i = 0; i < dir.size(); i++) {  
        string path = dir.getPath(i);  
        ofLogNotice() << "Loading: " << path;  
        ScaledImage scaledImage;  
        scaledImage.image.loadImage(path);  
        ofLogNotice() << "Image size (unscaled): " << scaledImage.image.getWidth() << " x " << scaledImage.image.getHeight();  
        float wRatio = ofGetWidth() / scaledImage.image.getWidth();  
        float hRatio = ofGetHeight() / scaledImage.image.getHeight();  
        float maxRatio = MAX(wRatio, hRatio);  
        if(maxRatio == wRatio){  
            ofLogNotice() << "Scaled for max width: " << wRatio << "(scale) = " << scaledImage.image.getWidth() * wRatio << "(pixels)";  
            scaledImage.minScale = wRatio;  
            scaledImage.maxScale = scaledImage.image.getWidth() * wRatio * maxZoomFactor;  
        }else{  
            ofLogNotice() << "Scaled for max height: " << hRatio << "(scale) = " << scaledImage.image.getHeight() * wRatio << "(pixels)";  
            scaledImage.minScale = hRatio;  
            scaledImage.maxScale = scaledImage.image.getHeight() * hRatio * maxZoomFactor;  
        }  
        scaledImage.percentTweened = 0.0f;  
        images.push_back(scaledImage);  
    }  
    currentImage = 0;  
      
}  
  
//--------------------------------------------------------------  
void ofApp::update(){  
      
}  
  
//--------------------------------------------------------------  
void ofApp::draw(){  
    if(ofGetFrameNum() % 10 == 0){  
        images[currentImage].percentTweened += 0.1;  
        if(images[currentImage].percentTweened >= 1.0f){  
            currentImage++;  
            if(currentImage == images.size()) currentImage = 0;  
            images[currentImage].percentTweened = 0.0f;  
        }  
    }  
    float scale = images[currentImage].minScale + images[currentImage].maxScale * images[currentImage].percentTweened;  
      
    // see: [http://stackoverflow.com/questions/2258910/opengl-scale-then-translate-and-how](http://stackoverflow.com/questions/2258910/opengl-scale-then-translate-and-how)  
    ofTranslate(ofGetWidth() / 2.0f, ofGetHeight() / 2.0f);  
    ofScale(scale, scale);  
    ofTranslate(-images[currentImage].image.getWidth() / 2.0f, -images[currentImage].image.getHeight() / 2.0f);  
    images[currentImage].image.draw(0, 0);  
}  
  

src.zip