Alpha Channel in Video

I have checked the previous posts but am stuck.

What I would like to do is load a video, and then get the black pixels (loop through them?) and if the pixel is black, make that pixel transparent.

I have tried OfxHap and it would not complie due to an error (libavulti.56.70.100.dylib can not be opened because the developer can not be verified)

I am sorry to not have any code to show you yet…I am still trying to fiqure out how to get to the alpha channel :slight_smile:

I am on OS 11.3.1

I seem to be going in circles, and would be very grateful if someone could help me out and light the way a bit!

Hey @cirkuit there are a couple of ways to do this. You can use an ofShader and the gpu to set pixels to transparent in an ofFbo, or you can loop thru the colors in an ofPixels with cpu. For video, I would suggest the latter, but both will likely work OK depending on hardware and video resolution.

In either case, if the output is a modified video file, you’ll have to “record it” again somehow (ie save each frame).

Shaders are very fast at this kind of task. With a shader, you could .draw() each frame into an ofFbo and set the transparent pixels all in go. Then you can either .draw() this ofFbo to the screen, or load it into an ofPixels object to save it in a file.

Looping thru the pixels on the cpu is slower, but on a Mac you can thread it with GCD (Grand Central Dispatch), which can help speed things up.

I (and others) can help with some example code if you like, especially if you have a preference for one approach over the other.

You can get/set the value for the alpha in an ofColor like this:

ofColor color = ofColor(255, 255, 255); // an RGB color
color.a = 50; // set the alpha channel to 50
float alpha = color.a; // get the value at .a and store it in alpha

Finally, I don’t know anything about hap video, but the ofVideoPlayer seems to work great for lots of stuff, And if needed, I think there is an addon for a GStreamer player.

1 Like

If you want to do it at the pixel level this is one way:



ofVideoPlayer video;
ofTexture rgbaTexture;

//--------------------------------------------------------------
void ofApp::setup(){
    video.load("black-white-video.mp4");
    video.play();
}

//--------------------------------------------------------------
void ofApp::update(){
    video.update();
    
    bool bThreshold = ofGetMousePressed();
    
    if( video.isFrameNew() ){
        ofPixels pixels = video.getPixels();

        //convert the rgb pixels to rgba
        ofPixels rgbaPixels = pixels;
        rgbaPixels.setNumChannels(4);
        
        //get the red pixel channel ( this will be used to set alpha )
        auto redPixels = pixels.getChannel(0);

        //if we want to make all pixels not completely black 100% alpha
        if( bThreshold ){
            for( auto & pix : redPixels ){
                if( pix > 0 ){ //make non black pixels 100% opaque
                    pix = 255;
                }
            }
        }
        
        //0 is red, 1 is green, 2, is blue and 3 is alpha
        rgbaPixels.setChannel(3, redPixels);
        rgbaTexture.loadData(rgbaPixels);
    }
}

//--------------------------------------------------------------
void ofApp::draw(){
    ofBackground(ofColor::darkBlue);
    
    video.draw(10, 10);
    rgbaTexture.draw(20 + video.getWidth(), 10);
}

2 Likes

Thank you all for helping me get unstuck!

I am thinking along the lines of using shaders.

The things posted are beneficial. I am going to continue to work on this later today. The idea is to remove the black background and have the image on a transparent background to isolate it.

I am layering a few videos so want some of them to have no visible background :slight_smile:

1 Like

Thanks again to both @TimChi and @theo
I have decided to go with shaders as I wish to remove the black background from 8 videos.
I have spent some time learning the basics of shaders and writing code that works.
I think it is not the best code and is rather inelegant.
Is there a better way than repeating the "if / then " statements repeatedly for each texture/video?

1 Like

Thanks for any advice- I am totally new to shaders! The values are not quite black because the response if closer to what I want- it is hard to explain without seeing the full project. The question is about a way to not write a bunch of if/then statements :slight_smile:

Hey looks like a great start! Shaders are a lot of fun and very efficient too. Here are a few thoughts:

  1. A fragment shader should only have one out vec4 variable as its output. This vec4 eventually becomes the color of the pixel when it is rendered. If you want to process more than one texture, you can send each thru the shader independently and render it into an ofFbo. Each texture can have its own ofFbo, or a pair of ofFbo can be used (read from one, write to the other, sometimes referred to as ping-pong).
  2. The RGB values can be used to calculate a float, a gray color that could be used to determine luminosity or lightness like this:
vec4 texel0 = texture(tex0, texCoordVarying);
float gray0 = dot(texel0.rgb, vec3(0.299, 0.587, 0.114)); // humans see green better
if(gray0 < 0.05) {
    texel0.a = 0.0;
} else {
    texel0.a = 1.0;
}
  1. A threshold variable could also come into the shader as a uniform float, so that it can vary if needed according to what is happening in OF (maybe from a gui):
// in OF
float threshold = 0.05; // or ofParameter<float> works great with gui
shader.begin();
shader.setUniform1f("threshold", threshold);
// more shader-related code
shader.end();

// in shader.frag
uniform float threshold;
// threshold can not be used in main();
// copy to a new variable in main() if you need to change its value
}
  1. The shading language (glsl) has a step() function that you could use instead of “if”. step() returns 0.0 below a threshold, and 1.0 above. smoothstep() is a similar function with a graduated transition from 0.0 to 1.0 over a specified range. The Book of Shaders website has a nice section on these functions. smoothstep() is fantastic when you don’t want a “hard edge”.
2 Likes

@TimChi Thanks for the support! I understand now about the shader having only one output. I will keep going- and use FBOs. Thank you also for the advice on finding different functions available in GLSL. It is a learning curve for sure, but I am enjoying it. Thank you again for the advice and help!

1 Like

Hey sure! Glad its helpful and just post back if you run into any issues. Have fun! Shaders are awesome!

1 Like