Faster way to process a video frame pixel by pixel

Hi, I wonder if there’s more efficient way to modify a video frame pixel by pixel than the below example which I copied from videoGrabberExample.

In ofApp.h

    ofVideoPlayer video;
    ofPixels videoInverted;
    ofTexture videoTexture;
    int videoWidth, videoHeight;

In ofApp.cpp

//--------------------------------------------------------------
void ofApp::setup(){
    video.load("fingers.mov");
    video.play();
    videoWidth = video.getWidth();
    videoHeight = video.getHeight();
    videoInverted.allocate(videoWidth, videoHeight, OF_PIXELS_RGB);
    videoTexture.allocate(videoInverted);
}

//--------------------------------------------------------------
void ofApp::update(){
    video.update();
    ofPixels & pixels = video.getPixels();
    for(size_t i = 0; i < pixels.size(); i++){
        //invert the color of the pixel
        videoInverted[i] = 255 - pixels[i];
    }
    //load the inverted pixels
    videoTexture.loadData(videoInverted);
}

//--------------------------------------------------------------
void ofApp::draw(){
    videoTexture.draw(0, 0, videoWidth, videoHeight);
}

Screenshot of the resulting video:


Is there a more efficient method to process a video frame pixel by pixel?

Hi,
Yes, if your goal is to modify the video display. In this case you can let the GPU of your graphics card do the work. For this you have to learn how to program the GPU, which is done with shaders. This is not so straightforward to begin with, but this is really powerfull. Have a look to https://openframeworks.cc/ofBook/chapters/shaders.html
You can also check the examples, see examples\gl\shaderExample in your OF directory.

1 Like

To be a little bit more specific, the shader you will use to invert the color will be what is called a fragment shader, and will be something like that (not tested):

#version 150

uniform sampler2DRect tex0;
in vec2 texCoordVarying;
out vec4 outputColor;
 
void main()
{
    vec4 color = texture(tex0, texCoordVarying);
    outputColor = vec4( vec3(1.0) - color.rgb, color.a);
}

You can use this fragment shader with the 04_simpleTexturing example, in the shader example series mentionned at the beginning of the link I gave you. Let us know if this cause you problems.

1 Like

Thank you for the sample code. I had to add vec4 color; in order to make it work.

#version 150

// this is how we receive the texture
uniform sampler2DRect tex0;

in vec2 texCoordVarying;

vec4 color;

out vec4 outputColor;

void main()
{
    color = texture(tex0, texCoordVarying);
    outputColor = vec4( vec3(1.0) - color.rgb, color.a);
}

The result:

Indeed. Sorry for this mistake.
I’ve corrected the code in my previous post, for future reference. I’m not sure about the best place to declare the variable but I choose to do it within the main function, it seems better for the compilation to me.

1 Like