Copying a Row of Pixels to another Row, with Sub Pixel Accuracy

advanced
#1

Hi All,

I want to simulate the effect of gravity (for example) on an image. I have made a function that moves each pixel in an image down by a certain amount:

  
  
void testApp::applyVariableGravity(float delta){  
    ofxCvColorImage movedCanvas;  
      
    float gravity = gui.getValueF("GRAVITY_AMOUNT");  
    float pixelsToMoveThisFrame = gravity * delta * 100.f; //useable scale  
      
    movedCanvas.allocate(canvas.getWidth(),canvas.getHeight());  
    cvSet(movedCanvas.getCvImage(), cvScalar(0,0,0)); //all black initially  
      
    int canvasHeight = movedCanvas.getHeight();  
    int canvasWidth = movedCanvas.getWidth();  
      
    for(int j=0; j < canvasHeight; j++){  
        for(int i=0; i < canvasWidth; i++){  
            int positionToCopyToX = i;  
            int positionToCopyToY = (j+((int)pixelsToMoveThisFrame)) % canvasHeight; //wrap around at the last row  
              
            int absolutePositionToCopyTo =  3*(positionToCopyToY*canvasWidth + positionToCopyToX); // 3 * as RGB  
            int absolutePositionToCopyFrom = 3*(j*canvasWidth + i); // RGB again  
              
            movedCanvas.getPixels()[absolutePositionToCopyTo+0] = canvas.getPixels()[absolutePositionToCopyFrom+0]; //Red data  
            movedCanvas.getPixels()[absolutePositionToCopyTo+1] = canvas.getPixels()[absolutePositionToCopyFrom+1]; //Green data  
            movedCanvas.getPixels()[absolutePositionToCopyTo+2] = canvas.getPixels()[absolutePositionToCopyFrom+2]; //Blue data  
        }  
    }  
    canvas = movedCanvas;  
      
    canvas.flagImageChanged();  
}

As you can imagine, as I can only copy to “whole” pixels, this means that even though I am using a time delta value, I don’t get smooth animation, as sometimes, even with constant speed, I might move z pixels one frame and z+1 another.

What I really want is a subpixel accurate method of copying entire rows at a time - I always want to take whole rows at a time, but sometimes I might want to interpolate the destination pixels - “sharing” the colour values between whole pixels in the destination as it were.

http://opencv.willowgarage.com/documentation/geometric-image-transformations.html

Seems to deal with Interpolation of destination pixels:

http://en.wikipedia.org/wiki/Multivariate-interpolation

In the end, I want to use a Perlin noise map to allow individual pixels to move varying amounts downwards on each frame, but this seems to be a good start.

Any thoughts from the forum?

Cheers,

Joel

#2

Hi
use cvRemap(const CvArr* src, CvArr* dst, const CvArr* mapx, const CvArr* mapy, int flags=CV_INTER_LINEAR+CV_WARP_FILL_OUTLIERS, CvScalar fillval=cvScalarAll(0))
it appears in the page that you link.
Check also CV_INTER_LINEAR for the diferent interpolation options.

#3

Thanks for that information, with the help of your advice, Golan Levin
and Kyle Macdonalds Bloggie Unwarp example:

http://www.flong.com/blog/2010/open-source-panoramic-video-bloggie-openframeworks-processing/

And a clear explanation on Stackoverflow:

http://stackoverflow.com/questions/3596385/can-anybody-explain-cvremap-with-a-code-using-cvremap-in-opencv

I’ve managed to make an example that allows me to remap any pixel to any
position in the new image, with interpolation.

.h

  
  
ofVideoGrabber vidGrabber;  
  
   int cameraWidth, cameraHeight;  
  
   ofxCvColorImage unwarpedImageOpenCV;      
   ofxCvColorImage warpedImageOpenCV;  
   ofxCvFloatImage srcxArrayOpenCV;   
   ofxCvFloatImage srcyArrayOpenCV;   
  
   int   interpMethod;   
   CvScalar blackOpenCV;

.cpp

  
  
void testApp::setup(){  
   cameraWidth = 320;  
   cameraHeight = 240;  
  
   vidGrabber.setVerbose(true);  
   vidGrabber.initGrabber(cameraWidth,cameraHeight);  
  
   unwarpedImageOpenCV.allocate(cameraWidth,cameraHeight);     
   warpedImageOpenCV.allocate(cameraWidth,cameraHeight);  
   srcxArrayOpenCV.allocate(cameraWidth,cameraHeight);   
   srcyArrayOpenCV.allocate(cameraWidth,cameraHeight);   
  
   blackOpenCV = cvScalarAll(0);  
  
   /* Interpolation method:   
  
[http://opencv.willowgarage.com/documentation/cpp/geometric-image-transformations.html#cv-resize](http://opencv.willowgarage.com/documentation/cpp/geometric-image-transformations.html#cv-resize)  
  
    CV_INTER_NEAREST nearest-neighbor interpolation  
    CV_INTER_LINEAR bilinear interpolation (used by default)  
    CV_INTER_AREA resampling using pixel area relation. It may be the  
preferred method for image decimation, as it gives moire-free results.  
But when the image is zoomed, it is similar to the INTER_NEAREST method  
    CV_INTER_CUBIC bicubic interpolation over 4x4 pixel neighborhood  
    CV_INTER_LANCZOS4 Lanczos interpolation over 8x8 pixel neighborhood  
  
    */  
  
interpMethod = CV_INTER_CUBIC;   
  
   /* from:  
[http://stackoverflow.com/questions/3596385/can-anybody-explain-cvremap-with-a-code-using-cvremap-in-opencv](http://stackoverflow.com/questions/3596385/can-anybody-explain-cvremap-with-a-code-using-cvremap-in-opencv)  
  
    I use cvRemap to apply distortion correction. The map_x part is in  
image resolution and stores for each pixel the x-offset to be applied,  
while map_y part is the same for the y-offset.  
  
    in case of undistortion  
  
    # create map_x/map_y  
    self.map_x = cvCreateImage(cvGetSize(self.image), IPL_DEPTH_32F, 1)  
    self.map_y = cvCreateImage(cvGetSize(self.image), IPL_DEPTH_32F, 1)  
    # I know the camera intrisic already so create a distortion map out  
    # of it for each image pixel  
    # this defined where each pixel has to go so the image is no longer  
    # distorded  
    cvInitUndistortMap(self.intrinsic, self.distortion, self.map_x,  
self.map_y)  
    # later in the code:  
    # "image_raw" is the distorted image, i want to store the  
undistorted into  
    # "self.image"  
    cvRemap(image_raw, self.image, self.map_x, self.map_y)  
  
    Therefore: map_x/map_y are floating point values and in image  
resolution, like two images in 1024x768. What happens in cvRemap is  
basicly something like  
  
    orig_pixel = input_image[x,y]  
    new_x = map_x[x,y]  
    new_y = map_y[x,y]  
    output_image[new_x,new_y] = orig_pixel  
   */  
  
   /* so lets make a map that just moves all the pixels down 100.4f  
pixels */  
  
   float verticalDistanceToMove = 100.4f;  
  
   float* srcX = srcxArrayOpenCV.getPixelsAsFloats();   
   float* srcY = srcyArrayOpenCV.getPixelsAsFloats();  
  
   for(int j=0; j < cameraHeight; j++){  
       for(int i=0; i < cameraWidth; i++){  
           int positionInArray = j*cameraWidth + i;  
           srcxArrayOpenCV.getPixelsAsFloats()[positionInArray] = (float)i; //x positions do not move  
           srcyArrayOpenCV.getPixelsAsFloats()[positionInArray] = fmodf((float)j+verticalDistanceToMove, (float)cameraHeight); //y positions do move  
       }  
   }  
  
   ofBackground(100,100,100);  
}  
  
//--------------------------------------------------------------  
void testApp::update(){  
  bool bNewFrame = false;  
  
   vidGrabber.grabFrame();  
   bNewFrame = vidGrabber.isFrameNew();  
  
  
if (bNewFrame){  
  
       unwarpedImageOpenCV.setFromPixels(vidGrabber.getPixels(),  
320,240);  
  
       cvRemap(unwarpedImageOpenCV.getCvImage(),  
warpedImageOpenCV.getCvImage(),  
srcxArrayOpenCV.getCvImage(),   
srcyArrayOpenCV.getCvImage(),   
interpMethod | CV_WARP_FILL_OUTLIERS, blackOpenCV );  
  
       unwarpedImageOpenCV.flagImageChanged();  
       warpedImageOpenCV.flagImageChanged();  
}  
  
}  
  
//--------------------------------------------------------------  
void testApp::draw(){  
ofSetColor(255,255,255);  
unwarpedImageOpenCV.draw(20,20);  
  
   warpedImageOpenCV.draw(20,20+unwarpedImageOpenCV.height);  
}

I think this could be a very useful technique for image distortion,
especially as in the new OpenCV 2.2, remap is GPU accelerated:

http://opencv-users.1802565.n2.nabble.com/Demystifying-cvRemap-td5833525.html

Cheers,

JGL

Get sub pixel crop from ofImage
#4

Great! Good work.
I didn’t know that cvRemap is GPU accelerated. Good thing to know.

best!