openCV image/contours don't align with ofImage and/or FBO

beginners
#1

Hello everyone,

I’m trying to use openCV’s contour function to find blobs from clusters of points from a 2D Lidar sensor. My strategy has been to draw the sensor points to an fbo. That fbo is then read to pixels and then to an ofxCvGrayscaleImage. I blur the image and then run the find contours function. While it does kind of work, the ofxCvGrayscaleImage and the corresponding contours are nowhere near the same alignment or scale as the FBO it gets its pixels from (see screenshot below). If I create a regular ofImage from the same pixels, it aligns with the FBO exactly.

My suspicion is that I’m either doing something wrong with creating the ofxCvGrayscaleImage or something wrong with the way I’m translating the image in between Push and Pop.

My apologies if I’m not articulating my problem very well, but thanks in advance for any help or pointers anyone is able to give.

Cheers,
Tom

Here is a screenshot of the mis-aligned/mis-scaled contours and lidar points

45 PM
45 PM1136×1158 93.4 KB

Here is an example of what I am striving for from mayerempl in Berlin.

IMG_FBCECFC95C4F-1
IMG_FBCECFC95C4F-11125×1281 464 KB

//--------------------------------------------------------------
void ofApp::setup(){
//gui setup
gui.setup();
gui.add(label.setup(“label”, “SENSOR PARAMS”));
gui.add(scaleSlider.setup(“scale slider”, 0.1, 0.001, 1));
gui.add(xOffsetSlider.setup(“sensor x offset”, 0.0, -ofGetWidth(), ofGetWidth()));
gui.add(yOffsetSlider.setup(“sensor y offset”, 0.0, -ofGetHeight(), ofGetHeight()))
//sensor setup
auto sensor_list = ofxRPlidar::getDeviceList();
for(auto &sensor_info : sensor_list) {
auto sensor = make_shared();
if(sensor->connect(sensor_info.getDevicePath())) {
sensor->start();
sensors_.push_back(sensor);
}
}

//fbo setup
lidarFbo.allocate(ofGetWidth(), ofGetHeight(), GL_RGBA);
lidarFbo.begin();
ofClear(255);
lidarFbo.end();

pixels.allocate(ofGetWidth(), ofGetHeight(), OF_IMAGE_GRAYSCALE);
lidarImage.allocate(ofGetWidth(), ofGetHeight(), OF_IMAGE_GRAYSCALE);
current.allocate(ofGetWidth(), ofGetHeight());

}

//--------------------------------------------------------------
void ofApp::draw(){
ofBackground(0);
lidarFbo.begin();
ofClear(0);

ofPushMatrix();
ofTranslate(ofVec2f(ofGetWidth() + xOffsetSlider, ofGetHeight()+yOffsetSlider)/2.f);

for(auto &s : sensors_) {
    s->update();
    auto data = s->getResult();
    
    for(auto &d : data) {
        if(d.quality > 0) {
            ofVec2f pos = ofVec2f(d.distance * scaleSlider, 0).getRotated(d.angle);
            ofDrawCircle(pos, 2);
        }
    }
}

ofPopMatrix();
lidarFbo.end();
lidarFbo.readToPixels(pixels);

// lidarFbo.draw(0,0);

//this image looks exactly like the Fbo it gets its pixels from
lidarImage.setFromPixels(pixels);
lidarImage.draw(0,0);

//why does current not allign with lidarImage when they are set from the same pixels?
current.setFromPixels(pixels);
current.blur(15);

// current.draw(0,0);

contour.findContours(current, 1000, 5000, 10, true);
contour.draw(0,0);

gui.draw();

}

#2

Hello,

The scale issue may be relative to color -> grayscale image conversion. Your fbo is RGBA, meaning that you have 4 bytes for each pixels (red, green, blue, alpha). If it is incorrectly converted to a grayscale image (only 1 byte for each pixel), it can lead to an image 4x bigger than the fbo. You can try to comment the current.blur( 15 ) line and to uncomment the current.draw(0,0) line to verify this assumption. If you see a 4x scaled image, this may confirm that you issue is here.

I think that maybe ofFbo.readToPixels() and ofImage.setFromPixels() automatically set the target to a proper format. Meaning that pixels.allocate( ofGetWidth(), ofGetHeight(), OF_IMAGE_GRAYSCALE ) and lidarImage.allocate( ofGetWidth(), ofGetHeight(), OF_IMAGE_GRAYSCALE ) are ignored, and that pixels and lidarImage are in fact color images. This is why lidarImage looks the same than the fbo. This would explain that current.setFromPixels( pixels ) fail to create a correct grayscale CV image.

Possible solutions:

  • Try to work with a grayscale fbo. Use GL_RED instead of GL_RGBA (I’m not sure it will works because I’ve never used a grayscale fbo).
  • Find the proper method to convert the fbo to grayscale. Perhaps read the FBO pixels to a color ofPixels, convert it to ofxCvColorImage, than create the ofxCvGrayscaleImage (hint)

Tip for later posts: format your code with the </> button, and also provide the variables declarations.