OpenCV float color image

extend
#1

The trouble with ofCvFloatImage is that its a grayscale image, and in my case I need to have a color float image.

I saved a copy of ofCvFloatImage and renamed it to ofCvFloatColorImage then changed just a few lines

  
  
void ofCvFloatColorImage::allocate(int _w, int _h){  
....  
 cvImage = cvCreateImage(cvSize(_w,_h), IPL_DEPTH_32F,3);  
 cvImageTemp = cvCreateImage(cvSize(_w,_h), IPL_DEPTH_32F,3);  
 cvImageTemp2 = cvCreateImage(cvSize(_w,_h), IPL_DEPTH_32F,3);  
 cvImage8U = cvCreateImage(cvSize(_w,_h), IPL_DEPTH_8U,3);  
 pixels = new unsigned char[_w*_h*3];  
 floatPixels = new float[_w*_h*3];  
....  
}  
  
void ofCvFloatColorImage::setFromFloatPixels(float * _pixels, int w, int h){  
	for (int i = 0; i < h; i++){  
		memcpy(cvImage->imageData + (i * cvImage->widthStep), _pixels + (i * w * 3), sizeof(float)*w*3);  
	}  
	for (int i = 0; i < h; i++){  
	  memcpy(pixels + (i * w * 3), cvImage->imageData + (i * cvImage->widthStep), sizeof(float)*w*3);  
	}  
}  
  
unsigned char * ofCvFloatColorImage::getPixels(){  
		// copy each line of pixels:   
    cvConvertScale( cvImage, cvImage8U, 1, 0);  
    for (int i = 0; i < height; i++){  
		memcpy(pixels + (i * width * 3), cvImage8U->imageData + (i * cvImage8U->widthStep), width*3);  
	}	  
	return pixels;  
}

Here are those files
[files updated, see posts below]
http://www.chrisoshea.org/storage/ofw/o-…-lorImage.h
http://www.chrisoshea.org/storage/ofw/o-…-rImage.cpp

However it doesn’t work.

Just to confirm, how are the opencv images split, is it rgb or bgr?
are the pixels in the array
rgbrgbrgbrgb
or
rrrrrrr ggggggg bbbbb

if that makes sense? (i forgot which way they worked, havent touched this in a few weeks).

Thanks

#2

opencv is interlaced, but *not* contiguous,

ie, one line of pixels not necessarily connected to the next. there is a variable
“cvImage->widthStep” which is the step in bytes per line of pixels, ie, you could have an image which is 310x235, but it’s actually 320 pixels across in opencv (this is padding, to keep memory block aligned).

http://www.cs.iit.edu/~agam/cs512/lect–…-intro.html

check out :

– Basic OpenCV data structures
– Image data structure

for a breakdown of what’s in an iplImage…

hope that helps!
zach

#3

I think in this example I wouldn’t need to use widthStep. widthStep is used in the ofCvFloatColorImage for settings and getting pixels.

If I want to set the pixels directly of a color image I can do this:

  
  
unsigned char * pix = new unsigned char[width*height*3];  
  
for(int i=0; i<(width*height); i++){  
  
 pix[ i * 3 + 0 ] = 255; // set red colour  
 pix[ i * 3 + 1 ] = 100; // set blue colour  
 pix[ i * 3 + 2 ] = 99; // set green colour  
  
}  
  
colorImage.setFromPixels(pix, width, height);  
  
delete pix;

Now this cycles through the size of the image, setting the r,g,b pixels to a set value, so the image is all one colour.

If I do the same thing with ofCvFloatColorImage :

  
  
float * pix = new float[width*height*3];  
  
for(int i=0; i<(width*height); i++){  
  
 pix[ i * 3 + 0 ] = 255.0; // set red colour  
 pix[ i * 3 + 1 ] = 100.0; // set blue colour  
 pix[ i * 3 + 2 ] = 99.0; // set green colour  
  
}  
  
colorFloatImage.setFromFloatPixels(pix, width, height);  
  
delete pix;

It doesn’t work and I get some distorted white image. The setFloatPixels function probably isn’t working, or getPixels used in Draw. Can you see anything obvious?

It would be great to have a float color image class working and included in future versions of the addon.

Thanks

#4

I didn’t drink coffee yet - so just doing some arm chair debugging…

it looks like this line in set from pixels might be trouble:

  
  
memcpy(cvImage->imageData + (i * cvImage->widthStep), _pixels + (i * w * 3), sizeof(float)*w*3);

for example, check out widthStep, you might be assuming that widthStep is step in pixels, not in bytes… etc.

I’m happy to get it to work,but I have to admit that we haven’t thought too much about float color images - in my experience (which is more with IPP then opencv, but same starting point IPL) float images tend to be very slow… I usually use integer images if I need to accumulate (for example, learning the background over a long time)… I’m with chris (the other chris!) today and I’ll ask her if she can take a look at it, since she was using float images before, too –

take care!
zach

#5

Yeah I am using floatImage for background learning with various functions. Float just enables you to change the background over a much slower rate than an int would, unless I’m wrong.

But of course with floatImage background subtraction is limited to grayscale, and now i’m trying to write shadow removal using hsv colourspace but am stuck without a floating color image.

Thanks for looking into this. I may look at using integer images instead too.

#6

This is the code from setFromPixels for ofCvFloatImage

memcpy(cvImage->imageData + (i * cvImage->widthStep), _pixels + (i * w), w);

This is the same line for ofCvColorImage

memcpy(cvImage->imageData + (i * cvImage->widthStep), _pixels + (i * w * 3), w*3);

for CvFloatImage I made setFromFloatPixels like this, which does work:
memcpy(cvImage->imageData + (i * cvImage->widthStep), _pixels + (i * w), sizeof(float)*w);

So to do setFromFloatPixels for the colorFloatImage I thought it would probably be the same as doing it for cvcolorimage…

memcpy(cvImage->imageData + (i * cvImage->widthStep), _pixels + (i * w * 3), sizeof(float)*w*3);

I’m probably wrong though :frowning:

#7

The core problem I am having with this is getting memcpy working with float values (for set and get pixels).

In a colour image, you can create do this:
unsigned char * pixels = new unsigned char [ width * height * 3 ]

The pixel values are stored r0 g0 b0 r1 g1 b1 r2 g2 b2. Sending this to setFromPixels in a colorImage works fine.

For now, I want to ignore setFromPixels and just make sure the images are being displayed correctly. So as a test, for colorImages I overwrote setFromPixels with this code

  
  
void ofCvColorImage::setFromPixels(){  
  
for(int i = 0; i < h; i++){  
		for(int j=0; j< w; j++){  
		int pixelIndex = w*i+j;  
  
		// red  
		cvImage->imageData[ pixelIndex * 3 + 0 ] = (i/(float)(h))*255;  
  
		// green  
		cvImage->imageData[ pixelIndex * 3 + 1 ] = 0;  
  
		// blue  
		cvImage->imageData[ pixelIndex * 3 + 2 ] = (j/(float)(w))*255;  
		}  
	}  
}

This is to create a red & blue gradient across the image by directly setting the imageData values. This works fine in ofCvColorImage, here is an image of the output…

If I do the same with the floatColorImage class it doesn’t work, due to problems with memcpy I believe.

If getPixels is exactly the same as colorImage, i.e.:

  
  
for (int i = 0; i < height; i++){  
 memcpy(pixels + (i * width * 3), cvImage->imageData + (i * cvImage->widthStep), width*3);  
}

This is the image output…

So it seems like the data isnt filling up the pixels array.

I tried this…(adding sizeof float to the end)…

  
  
memcpy(pixels + (i * width * 3), cvImage->imageData + (i * cvImage->widthStep), width*3*sizeof(float));

And the output was the same as the 2nd image above.

The line that does work now is…

  
  
memcpy(pixels + (i * width * 3 * sizeof(float)), cvImage->imageData + (i * cvImage->widthStep), width*3*sizeof(float));

Does that seem right to you? I’m kind of hacking my way through with this.

In cvFloatImage, for getPixels it does:
cvConvertScale( cvImage, cvImage8U, 1, 0);
converting it to a IPL_DEPTH_8U before memcpy into the pixels array.

Actually it appears that memcpy can cope with copying floats into an unsigned char array fine. Chris & Zach, do you have any thoughts on this?

I am now trying to get setFromFloatPixels working. Here it is so far:

  
  
void ofCvFloatColorImage::setFromFloatPixels(float * _pixels, int w, int h){  
  
 for (int i = 0; i < h; i++){  
 // only updating image data for now  
  memcpy(cvImage->imageData + (i * cvImage->widthStep), _pixels + (i * w), w*3*sizeof(float));  
 }  
}

This makes the output

I also tried
memcpy(cvImage->imageData + (i * cvImage->widthStep), _pixels + (i * w * 3), w*3*sizeof(float));
// added a 3

which makes the output

Do you have any ideas how to modify the memcpy line in setFromFloatPixels to work properly?

Here are the updated files:
http://www.chrisoshea.org/storage/ofw/o-…-lorImage.h
http://www.chrisoshea.org/storage/ofw/o-…-rImage.cpp

Many many thanks. I hope this proves useful in future too.

#8

hi chris,

it’s taking me a bit longer to wrap my head around the problem then I thought, but the main issue is that

->imageData

is *not* a float pointer, even if you made a 32F image. it is a char pointer, so you are wrong I think in treating it like it is a float pointer…

http://www.comp.leeds.ac.uk/vision/opencv/iplimage.html

so for example, this:

  
  
for(int i = 0; i < h; i++){  
      for(int j=0; j< w; j++){  
      int pixelIndex = w*i+j;  
      cvImage->imageData[ pixelIndex * 3 + 0 ] = (i/(float)(h))*255;

is not going to work I believe, since you are adressing imageData as if it is a floating point array: vImage->imageData[ pixelIndex * 3 + 0 ]

imageData[] is stepping in bytes (since it’s a char pointer) not floats, so thus some errors you are getting that look “patterned”.

for example, imageData[4] is the 4th byte, not the 4th float. imageData[4*sizeof(float)] is the 4th float. Also,

float * fp = (float *)imageData;
fp[4]

would be the 4th float…

you will likely have to cast imageData to a float pointer. I tried to jump in there and I couldn’t get it up an running right away, but I am pretty sure this is the main error and fixing that would help other things fall into place… I am running out the door but will look at this as soon as I can…

hope that helps!!
zach

#9

following that logic, this works for me:

  
  
//-------------------------------------------------------------------------------------  
void ofCvFloatColorImage::setFromFloatPixels(float * _pixels, int w, int h){  
	float* fpImgData = (float*)cvImage->imageData;  
	for (int i = 0; i < h; i++){  
	  memcpy(	fpImgData + (i * (cvImage->widthStep/sizeof(float))),   
	  			_pixels + (i * w * 3),   
	  			w*3*sizeof(float));  
	}  
}

it’s tricky not only because imageData is in char pointer, not a float pointer, but also because widthStep is in bytes, so we need to do widthStep/sizeof(float) to find out the number of floats per line. sorry for all the pointer math, I can try to unpack it a bit if necessary.

this is my getPixels()

  
  
//-------------------------------------------------------------------------------------  
unsigned char * ofCvFloatColorImage::getPixels(){  
	cvConvertScale( cvImage, cvImage8U, 1, 0);  
    for (int i = 0; i < height; i++){  
    	  
		memcpy(pixels + (i * width * 3), cvImage8U->imageData + (i * cvImage8U->widthStep), width*3);  
	}	  
	return pixels;  
}

I’m able to at least pass in pixels, I haven’t taken a look at getting them out but the array of pixels in testApp.h is passed in to setFromFloatPixels() and the cvFloatColorImage draw correctly (scaling to unsigned char from 0 to 255.0f in float space) as I would expect…

hope that helps!!
zach