What is the best way to set ROI (region of interest) with OF

beginners
#1

I’m doing blob tracking and image processing (with various filters), but would like to have the ability to change the ROI.

I’ve tried setting the ROI once the source image (camera/video) is recieved but that creates mismatched ‘size’ runtime errors/problems when I try to do more down the filter chain.

I’m still new to openCV; what is the best way to set ROI so that I can take an input source (video/camera) and select a portion of the image to filter and then track blobs on? Would I need to reallocate all images everything the ROI is changed since I originally allocate all my images (ofxGrayscaleImages, etc) to full source height/width on loading the application or should I set the first ROI and then getROI/setROI for all other images I use?

Thanks!

#2

There is no reason why this wouldn’t work. Make sure you unset your ROI as well.

Also just capture your video at full size, and make a copy and do ROI on that.

  
  
grayBinaryImage = //your binary blobs image  
cvSetImageROI(grayBinaryImage, cvRect(x, y, w, h));  
contourFinder.. // find contours in grayBinaryImage   
cvResetImageROI(grayBinaryImage);

I’ve done this a number of times. Dont forget that the blobs will be shifted by the ROI offset, so if the ROI starts at x = 100, a blob that normally is in position 150 x is now in position 50, so you need to add the roi x of 100 to a blobs x position to make it normal again. Does that make sense?

#3

There are other ways of masking areas so that you only track blobs in the places you expect them.

  1. set ROI, but only on your calculated images not your video grabber

  2. use a quad to select area and then warp this area
    http://csugrue.com/code/?p=7

  3. use a rectangle region and use cvCopy to copy out just that area into another image

  4. create a polygon mask over an image, useful when your area has funny shaped areas you need to block out. also you can mask one image with another
    http://www.flickr.com/photos/golanlevin-…-593201498/

#4

This might be a helpful reference

  
SetImageROI  
Sets image ROI to given rectangle  
  
void cvSetImageROI( IplImage* image, CvRect rect );  
image  
Image header.  
rect  
ROI rectangle.  
The function cvSetImageROI sets the image ROI to a given rectangle. If ROI is NULL and the value of the parameter rect is not equal to the whole image, ROI is allocated. Unlike COI, most of OpenCV functions do support ROI and treat it in a way as it would be a separate image (for example, all the pixel coordinates are counted from top-left or bottom-left (depending on the image origin) corner of ROI)  
  
ResetImageROI  
Releases image ROI  
  
void cvResetImageROI( IplImage* image );  
image  
Image header.  
The function cvResetImageROI releases image ROI. After that the whole image is considered selected. The similar result can be achieved by  
  
cvSetImageROI( image, cvRect( 0, 0, image->width, image->height ));  
cvSetImageCOI( image, 0 );  
But the latter variant does not deallocate image->roi.  
  
GetImageROI  
Returns image ROI coordinates  
  
CvRect cvGetImageROI( const IplImage* image );  
image  
Image header.  
The function cvGetImageROI returns image ROI coordinates. The rectangle cvRect(0,0,image->width,image->height) is returned if there is no ROI

I would do this and go with option 1 up there. Unless its not a rect then option 4 might be a good way. For option 4 if you dont want to use an image mask you can always set a perimeter point array with the desired shape. Then any blob outside the shape you can chose to ignore using this:
http://forum.openframeworks.cc/t/finding-points-inside/887/0

ding

#5

Hmmm, ok.

A couple things.

  1. I’d like to use ROI before the image is filtered and tracked with contourfinder. This should help with optimizations since then I can cutout pixels from being filtered since ROI will make the image smaller. If I only set ROI right before contourfinder, than I’m doing more calculation than needed beforehand.

  2. I’ve used the quad warp before, but it’s very slow. I tested yesterday and openCV warpPaerspective is 5-14ms which is way to high for my use.

  3. The polygon mask will come in handy for other things, thanks! Is there an OF example of how that is done (like the one in the link)? I’d really like to get rid of pixels though to crop the image smaller from the start, and maybe then apply a polygon mask on that copied (smaller image) that’ll better fit the image.

Right now I have a chain similar to this:

  
grayImage = sourceImage;  
  
grayImage.blur(blurVal);  
  
blurImage = grayImage;  
  
blurImage.highpass(highpassVal);  
  
highpassImage = blurImage;  
  
highpassImage.threshold(threshVal);  
  
thresholdIamge = highpassImg;  
  
contourFinder(thresholdImg, ......);

I would think the most efficient way of setting ROI would be on the first part, either the sourceImage or on the grayImage so that I can cutout pixels that aren’t needed from the start. The problem is, I’ve already allocated space for all those images beforehand. When I try to set ROI on the grayImage, I get mismatched size errors on the next part.

For example doing blurImage = grayImage(with ROI applied); will result in trying to make blurImage (320 x 240) = grayImage (smaller than 320 x 240);

So what would be the proper way to get the rest of the images to use the ROI?

I appreciate the replies so far. Let me know If I’m going about this concept of ROI wrong too.

#6

If I am not mistaken you have to set the ROI on the rest of the IplImage images.

ding

#7

As far as the ROI shifting the blob point, the cvtracker function looks like this

  
int cvFindContours( CvArr* image, CvMemStorage* storage,  
CvSeq** first_contour, int header_size=sizeof(CvContour),  
int mode=CV_RETR_LIST, int method=CV_CHAIN_APPROX_SIMPLE,  
CvPoint offset=cvPoint(0,0) );

if a ROI was natively built in to ofxCvContourFider you can use a cvPoint as an offset

offset
Offset, by which every contour point is shifted. This is useful if the contours are extracted from the image ROI and then they should be analyzed in the whole image context.

ding

#8

I am working on an interactive installation and am also having issues with setting the ROI. To make sure I’m not insane, I tried modifying the opencv addon example that shipped with OF.

in testapp.cpp, line 56ish:

  
  
cvSetImageROI(grayDiff.getCvImage(), cvRect(0,0,200,200));  
contourFinder.findContours(grayDiff, 20, (340*240)/3, 10, true);	// find holes  
cvResetImageROI(grayDiff.getCvImage());

This hangs the whole app up. Any ideas what I am doing wrong here?

Cheers,

-Jer
[/code]

#9

Yeah, I had the same problem.

I’ll be home in a bit and can tell you what the problem may be.

In order to get ROI to fully work how I want, I really had to edit/extent some of the core classes. If your’re just doing ROI on contourfinder I think that was a simpler fix. I’ll let ya know when I get home in a bit.

#10

Thanks!

As you can appreciate, it has been a bit frustrating.

#11

Ok, there’s a couple ways you can handle this problem.

Inside the ofxContourFinder.cpp class, there’s a inputCopy ofxGrayscaleImage. It is allocated based on the height/width of the image. In the case below, the height width of grayDiff is passed in.

  
cvSetImageROI(grayDiff.getCvImage(), cvRect(0,0,200,200));  
contourFinder.findContours(grayDiff, 20, (340*240)/3, 10, true);   // find holes  
cvResetImageROI(grayDiff.getCvImage());

The problem is, the image is using ROI and height width of the grayDiff image doesn’t report the ROI height/width (it reports the full image height/width) which is needed for allocation of the inputCopy image. Therefore, when the following code occurs:

  
inputCopy = input;

You are setting the inputCopy (which is full height/width of the image) = to input (which is only ROI height/width). This will crash since the sizes aren’t the same.

Method 1) In ofxContourFinder you can add the code:

  
  
CvRect rect = cvGetImageROI(input.getCvImage()); //thiss will get the input (graydiff) ROI box  
cvSetImageROI(inputCopy.getCvImage(), rect )//this sets the copy of the input to ROI

That will make them both have the same ROI and therefore the same size. You would need to place that code everywhere before you see

  
inputCopy = input;

in the ofxContourFinder class.

Method 2) You can copy the ROI image into a separate image before you pass it to contourFinder.

For example:

  
cvSetImageROI(grayDiff.getCvImage(), cvRect(0,0,200,200)); //set the ROI  
grayDiffROI.allocate(grayDiff.getCvImage()->roi->width, grayDiff.getCvImage()->roi->height); //set the grayDiffROI to the size of only the ROI  
  
grayDiffROI = grayDiff; //copy the ROI image to the grayDiffROI image  
  
contourFinder.findContours(grayDiffROI, 20, (340*240)/3, 10, true);   // find holes  
cvResetImageROI(grayDiff.getCvImage());

You’d also have to have some if statement where the grayDiffROI.allocate would go so that you only allocate when the ROI size changes. This method makes it so you don’t have to edit any core classes. I haven’t tested this one, but it should work and will probably be what I switch to.

I’m sure others might have other/better ways too.

#12

Wow.

Thanks so much for the quick and thorough reply. It’s a bit too late to try it out tonight, but I will give it a go first thing tomorrow morning. The second option looks like a perfect solution.

Cheers,

-Jer

#13

Hello cerupcat,

It didn´t worked for me :frowning:

There is still the issue of the images having different sizes

  
  
//--------------------------------------------------------------------------------  
void ofxCvGrayscaleImage::operator =	( ofxCvGrayscaleImage& mom ) {  
  
	//------ We are again comparing different heights and widths  
        if( mom.width == width && mom.height == height ) {  
        cvCopy( mom.getCvImage(), cvImage, 0 );  
	} else {  
        cout << "error in =, images are different sizes" << endl;  
	}  
}

And furthermore, cvCopyImage does not consider ROI. See below.

cvCopyImage
Copies entire image to another without
considering ROI.
void cvCopyImage(IplImage* src, IplImage* dst);
src Source image.
dst Destination image.
Discussion
The function cvCopyImage copies the entire image to another without considering
ROI. If the destination image is smaller, the destination image data is reallocated.

Anyway,

I decided to get the pixel data from the ROI rectangle and save it into an image of the corresponding width and height.

  
  
grayDiffROI.setFromPixels( grayDiff.getPixelsRoi(),  
grayDiff.getCvImage()->roi->width,  
grayDiff.getCvImage()->roi->height );  
  


  
// Get Pixel Data from ROI  
//--------------------------------------------------------------------------------  ROI  
unsigned char* ofxCvGrayscaleImage::getPixelsRoi() {  
	int x=cvImage->roi->xOffset;  
	int y=cvImage->roi->yOffset;  
	int w=cvImage->roi->width;  
	int h=cvImage->roi->height;  
  
	for( int i = 0; i < h; i++ ) {  
		memcpy( pixels+(i*w),  
                cvImage->imageData+( (i+y)*cvImage->widthStep), w);  
	}  
	return pixels;  
}  
//--------------------------------------------------------------------------------  ROI

I found the following info about iplCopy which does apply to ROI. Maybe it would make it easier. I have not tested it.

Copy: Copies image data from one image to another.

void iplCopy(IplImage* srcImage, IplImage* dstImage);
srcImage The source image.
dstImage The resultant image.

Discussion
The function iplCopy() copies image data from a source image to a
resultant image. Before calling this function, the source and resultant
headers must be properly constructed and image data for both images must
be allocated; see Example 4-5. The following constraints apply to the
copying:
· The bit depth per channel of the source image should be equal to that
of the resultant image.
· The number of channels of interest in the source image should be
equal to the number of channels of interest in the resultant image; that
is, either the source coi = the resultant coi = 0 or both cois are
nonzero.
· The data ordering (by pixel or by plane) of the source image should be
the same as that of the resultant image.

The origin, align, height, and width field values (see Table 4-2) may
differ in source and resultant images. Copying applies to the areas that
intersect between the source ROI and the destination ROI.

#14

I’ll have to check out what I did again.

BTW, cvCopy and cvCopyImage are different things I believe.

cvCopy does take the ROI. If you were using cvCopyImage, that may be the problem. So you allocate the size of the ROI and then copy the image into that new allocated image.

"The function cvCopy copies selected elements from input array to output array:

B(I)=A(I) if mask(I)!=0.

If any of the passed arrays is of IplImage type, then its ROI and COI fields are used. Both arrays must have the same type, the same number of dimensions and the same size. The function can also copy sparse arrays (mask is not supported in this case)."

#15

Thanks!! :smiley:

The reason i am interested in ROI is because I have a Unibrain Camara with a wide lense and therefore I am only interested In the area of the projection (a 3mx3m wall lighted with IR) and not the whole room.

However I have a huge doubt as to how to calculate the actual position of the person in correspondence to the position of the objects on the projections.

For example is the hand of a person inside a projected rectangle at fullscreen?
The coordinates of the hand of the person are based on a 320x240 image. The coordinates of the projection are based on a 1280 x 1024 Coordinates.

Shall I resize the image form 320x240 to 1280X1024 and then find the contours in order to work with same sizes?

or shall I amplify the contour found in the 320X240 image to meet its correspondent contour in a 1280X1024 image?

Thanks for all the pointers =)

#16

Sorry I missed this ROI conversation –

in japan, theo and I experimented with ROI and I think it will be settable in the next version of ofxOpenCv – there are a few quirks (as you’ve seen) about it, so we’ll need feedback. it’s working in the images, and we need to get it working in the contour finder.

As for what you want to do, you could easily use the quad warping, to pick the 4 points of the image and warp them into 4 points of another image so that the sub image is mapped to the whole of the second image.

there is some code here about that:
http://csugrue.com/code/?p=7

next step, is to normalize the results you get back. For example, if you get a centroid position of a blob, (cx, cy) divide by the width / height and get back a percentage (part / whole), that you can scale to the screen size, such as:

(cx / 320.0f) * 1024.0f;

Shall I resize the image form 320x240 to 1280X1024 and then find the contours in order to work with same sizes?

no ! this will be really slow !!! :slight_smile:

I hope that makes sense and helps !

best,
zach

#17

Thanks zach.

Let us know if you have any ROI code that needs to be tested. Right now, I think im ‘hacking’ around it. It’d be great to have a better approach in the next ofxOpenCv :slight_smile:

@ drogza:

I think what zach said will work great for you. You just want to map the x/y locations to the projector. You don’t need the same resolutions (camera and projector) in order to do that with the technique zach mentioned.

#18

Hello everyone!
I had a question for cerupcat.
I used the last of your examples, method 2, and its all working well except i havent added the if statement for the grayDiffROI.allocate yet.
But I cant use the x and y cordinates of cvRect, only width and height. I would like to, as an example, only do contour finding in the middle of the picture. So how do I change the x and y cordinates? This may be hilariously obvious but Im a total newbie.
Thanks in advance!