Convert from RGB to BGR w/o OpenCV for ofVideoGrabber frame

beginners
#1

Greetings everyone -

I need to use ofVideoGrabber as opposed to OpenCV to VideoCapture. In previous code I wrote I used Mat::data to pull the actual pixel of the frame as a uchar* … and reading through the ofVideoGrabber::getPixels() call I see that it too returns a unsigned char*.

However, when it comes time to convert the incoming from ofVideoGrabber from RGB to BGR format I get a failure. The code I am using is as follows:

> void ConvertBGR(
> 	const size_t imageWidth,
> 	const size_t imageHeight,
> 	const std::vector<uint8_t*>& srcs,
> 	float* dst,
> 	const std::vector<float>& mean,
> 	const std::vector<float>& scale)
> {
> 	const size_t channels = 3;
> 	const size_t channelSize = imageWidth * imageHeight;
> 	const size_t imageSize = channelSize * channels;
> 
> 	float* rBuffer = &dst[channelSize * 0];
> 	float* gBuffer = &dst[channelSize * 1];
> 	float* bBuffer = &dst[channelSize * 2];
> 
> 	const size_t rOff = 2;
> 	const size_t gOff = 1;
> 	const size_t bOff = 0;
> 
> 	size_t idx = 0;
> 
> 	// swap the bytes
> 	for (size_t mb = 0; mb < srcs.size(); mb++) {
> 		const uint8_t* srcBuffer = srcs[mb];
> 
> 		idx = imageSize * mb;
> 
> 		for (size_t h = 0; h < imageHeight; h++) {
> 			size_t srcIdx = h * imageWidth * channels;
> 
> 			for (size_t w = 0; w < imageWidth; w++, idx++, srcIdx += channels) {
> 				rBuffer[idx] = ((float)srcBuffer[srcIdx + rOff] - mean[0]) / scale[0];
> 				gBuffer[idx] = ((float)srcBuffer[srcIdx + gOff] - mean[1]) / scale[1];
> 				bBuffer[idx] = ((float)srcBuffer[srcIdx + bOff] - mean[2]) / scale[2];
> 			}
> 		}
> 	}
> }

My call to this function using ofVideoGrabber is as follows:

ofVideoGrabber grabber;

< grabber initialization code >
<success!!>

ConvertBGR(inputWidth, inputHeight, { grabber.getPixels() }, inputPtr, mean, scale);

The call to this same function using OpenCV is successful. The call to this function from ofVideoGrabber.getPixels() fails rBuffer, gBuffer and bbuffer for access violations/memory not accessible.

Any and all comments and insights from the community would be greatly appreciated.

Regards,
RudyC

#2

Hi,
the getPixels() method returns a reference to the ofPixels object. It has been this way for quite a while. To get acces to the uchar pointer call grabber.getPixels().getData()

hope this helps.
cherrs.

1 Like
#3

Also,
from this line you posted

ConvertBGR(inputWidth, inputHeight, { grabber.getPixels() }, inputPtr, mean, scale);

Wrapping grabber.getPixels() with the { } is at least unsafe if it actually compiles.
I recommend you to change your functions arguments to have const ofPixels& srcs instead of const std::vector<uint8_t*>& srcs
You’ll need to check what’s going on in the function’s internals which I think have at least one extra for loop, the first one.

#4

Greetings @roymacdonald

Thanks for your prompt reply.

Please bear with my Q’s as I am new to OF.

I rewrote my code to conform to your advice and it is sound. Thanks. However, I am required to RESIZE my image … and as such do not know what is the OF way of doing this. I checked out ofImage and still somewhat lost as to how to effect 1) taking my frame from ofVideoGrabber; 2) resizing it from 640x480 to 224x224; swapping it from RGB to BGR … and doing so without necessarily relying on OpenCV to do this.

Should you or others have any insights on pulling this off within OF I would appreciate hearing from you.

RudyC

#5

Hi,
no problem with your questions. That’s what the forum is for :slight_smile:

So,
First of all a very important thing to understand is the difference between ofImage and ofPixels. Take a look at https://github.com/ofcourseio/SPRING2016/blob/master/week4/pixels.md

As for your question, even when you could do everything in place -destructively modifying the ofVideoGrabber pixels- it is not a good idea, so lets start by copying its pixels.

// ofVideoGrabber grabber; // Declared elsewhere 
//Assuming we have correctly initialized grabber

If you simply want to process the pixels and not display these do the following.

ofPixels pix = grabber.getPixels(); //this will create a copy
pix.resize(224x224);//resizing 
pix.swapRgb(); //fortunately for you, ofPixels has this function that does what you are looking for.

If you want to draw this it might be easier to use an ofImage instance instead.

// ofImage img; //declared elsewhere, probably in the ofApp.h file.
img.setFromPixels(grabber.getPixels());
img.getPixels().resize(224, 224);//or whichever size you might want.
img.getPixels().swapRgb();
img.update();

Thats all. hope this helps. if you want to know whats going on with the pixels just go and read the actual code of each class.


cheers

1 Like
#6

@roymacdonald … thanks for your kind and helpful words. I accomplished what I required to do by switching over to OpenFrameworks from OpenCV highgui.

One final question I have … is there a way to pull an individual plane of pixels from an image? More precisely, given that I have an image with interleaved RGB values, how do I pull the plane of R-pixels - only the R-values from the pixels - is there a construct in OF that enables one to do this?

RudyC

#7

I think you are looking for getChannel(0) in ofPixels. This is the explanation on the ofPixels.h:

/// \brief Get all values of one channel
///
/// For instance, the Red pixel values, from the
/// ofPixels object, this gives you a grayscale representation of the
/// specific channel
///
/// ~~~~{.cpp}
///     // Get red pixels
/// 	ofPixels rpix = pix.getChannel(0);
///     // Get green pixels
/// 	ofPixels gpix = pix.getChannel(1);
///     // Get blue pixels
/// 	ofPixels bpix = pix.getChannel(2);
/// ~~~~
ofPixels_<PixelType> getChannel(size_t channel) const;
1 Like