Using texture as alpha mask to apply transparency

hello,

Ok this seems to work.

I also came up with this solution on my side

for(int y = 0; y < mask.getHeight(); y++) {
    for(int x = 0; x < mask.getWidth(); x++) {
        if (mask.getPixels().getColor(x, y).getBrightness() == 0) {
            mask.getPixels().setColor(x, y, ofColor(0, 0, 0, 0));
        }
    }
}
mask.update();

thanks a lot

yes the mask works as you describe, just allocate the mask as rgb or even luminance and the black parts will be transparent in the final image

sorry, i can’t make it to work like that

imgFront.load("1.jpg");
imgBack.load("2.jpg");

mask.allocate(640, 480, OF_IMAGE_COLOR);
mask.load("mask.jpg");

imgFront.getTexture().setAlphaMask(mask.getTexture());

imgFinal.allocate(640, 480, GL_RGBA);
imgFinal.begin();
imgBack.draw(0, 0);
imgFront.draw(0, 0);
imgFinal.end();

I need to force black transparency in the mask to make it work

mask.load("mask.jpg");
mask.setImageType(OF_IMAGE_COLOR_ALPHA);

for(int y = 0; y < mask.getHeight(); y++) {
    for(int x = 0; x < mask.getWidth(); x++) {
        if (mask.getPixels().getColor(x, y).getBrightness() == 0) {
            mask.getPixels().setColor(x, y, ofColor(0, 0, 0, 0));
        }
    }
}
mask.update();
imgFront.getTexture().setAlphaMask(mask.getTexture());

The comparison with 0 may work, but the mask edges may be rough, because it’s a threshold: either transparent or opaque.

@arturo In my tests black did not become transparent. As in @Gallo 's “what I have” image above, the white part of the mask shows the textures, but the black does not become transparent, but replaces the image texture with black.

tried with latest nightlies just in case. Under macOS Sierra and Debian. Same problem.

oh yeah sorry you are right the mask works for color form rgb and for alpha from alpha. you can remap the channels using:

mask.setSwizzle(GL_TEXTURE_SWIZZLE_A, GL_RED);

which will map the red channel to act as alpha so you will see white as alpha 1 and black as alpha 0. you’ll need a texture allocated with alpha in the first place for this to work

So this is equivalent to the following right ?

mask.getPixels().setChannel(3, mask.getPixels().getChannel(0));

by the way, i did

mask.getTexture().setSwizzle(GL_TEXTURE_SWIZZLE_A, GL_RED);

but didn’t work so far.

I will dig it a bit more tomorrow with a rest brain…

void ofApp::setup() {
    img.load("1.jpg");
    bg. load("2.jpg");
    mask.load("3.jpg");
    mask.setImageType(OF_IMAGE_COLOR_ALPHA);
    mask.getTexture().setSwizzle(GL_TEXTURE_SWIZZLE_A, GL_RED);    
    img.getTexture().setAlphaMask(mask.getTexture());
}
void ofApp::draw() {
    bg.draw(0, 0);
    img.draw(0, 0);
}

I added some blur to the mask to see if it was working right

A comment says swizzle doesn’t work on OpenGL ES. If it doesn’t work for you, is that what you’re using?

Yep, seems we are doing the same thing… but nope

//setup()
    imgFront.load("1.jpg");
    imgBack.load("2.jpg");
    mask.load("mask.jpg");
    mask.setImageType(OF_IMAGE_COLOR_ALPHA);
    mask.getTexture().setSwizzle(GL_TEXTURE_SWIZZLE_A, GL_RED);
    imgFront.getTexture().setAlphaMask(mask.getTexture());

//draw()
    imgBack.draw(0, 0);
    imgFront.draw(0, 0);

I am using oF 0.9.8 under macOS Sierra and XCode

What if you set the OpenGL version in main?

int main( ) {
  ofGLWindowSettings settings;
  settings.setGLVersion(3, 2); // try also 2, 1
  settings.width = 1024;
  settings.height = 768;
  settings.windowMode = OF_WINDOW;
  ofCreateWindow(settings);
  ofRunApp( new ofApp());
}

This one works

mask.getPixels().setChannel(3, mask.getPixels().getChannel(0));

this one doesn’t work

mask.getTexture().setSwizzle(GL_TEXTURE_SWIZZLE_A, GL_RED);

So i ended up using the technique i talked about earlier

mask.load("mask.jpg");
mask.setImageType(OF_IMAGE_COLOR_ALPHA);

for(int y = 0; y < mask.getHeight(); y++) {
    for(int x = 0; x < mask.getWidth(); x++) {
        if (mask.getPixels().getColor(x, y).getBrightness() == 0) {
            mask.getPixels().setColor(x, y, ofColor(0, 0, 0, 0));
        }
    }
}
mask.update();
imgFront.getTexture().setAlphaMask(mask.getTexture());

Thanks a lot to all

I’m happy that you found a solution!

A few things:

  • setSwizzle() takes 5 microseconds on my system, while the two for loops take around 20500 microseconds. Just in case you need to be fast.
  • If your mask is not pure black and white, but has gray values, it might look like this:

Which you can solve like this using your approach:

    for(int y = 0; y < mask.getHeight(); y++) {
        for(int x = 0; x < mask.getWidth(); x++) {
            mask.getPixels().setColor(x, y, ofColor(255, 255, 255, mask.getPixels().getColor(x, y).getBrightness()));            
        }
    }

Or at least I would not compare to 0 (jpg may have introduced artifacts, as you can see on the right side of the pentagon). Using .getBrightness() < 128 would be better, I think.

yes i know setSwizzle() is faster so i would prefer to use it. But unfortunately this doesn’t work for me for a reason i don’t get…

Though, i get my mask from a kinect depth image and i need to threshold it before using it so i am looping in every pixels anyway

Hi all,

is there any news on this topic? I have the exact same problem:

mask.load("masks/circleMask3.jpg");
mask.setImageType(OF_IMAGE_COLOR_ALPHA);
mask.getTexture().setSwizzle(GL_TEXTURE_SWIZZLE_A, GL_RED);
[...]
video.getTexture().setAlphaMask(mask.getTexture());

I expected: white regions of the mask are showing the video, grey regions are partially transparent, black regions are fully transparent.
But it leads to: white regions of the mask are showing the video, some grey regions are in fact partially transparent, but black regions of the mask are non-transparent black.

Same outcome on Ubuntu 16.04 and OS X 10.9.5, both OF 0.9.8

I have in fact the same result when trying the same with a custom shader like:

#version 120
uniform sampler2DRect texture0;
uniform sampler2DRect texture1;
void main(){
vec2 pos = gl_TexCoord[0].xy;
vec4 color0 = texture2DRect( texture0, pos );
vec4 color1 = texture2DRect( texture1, pos );
//Compute resulted color
vec4 color;
color.rgb = color0.rgb;
color.a = color1.r;
gl_FragColor = color;
}

Any help is greatly appreciated.

have a great day!
oe

Hi, did you try the alpha mask example. in examples/gl/alphaMaskingShaderExample
it is pretty much what you are trying to do.
best

you shouldn’t need to set any swizzles or convert the image to color alpha, any rgb or grayscale image should work i think

This Solved the problem.
– set GL version to 3.2. as hamoid suggested –
then the code below does what is expected.

mask.setImageType(OF_IMAGE_COLOR_ALPHA);
mask.getTexture().setSwizzle(GL_TEXTURE_SWIZZLE_A, GL_RED);
imgFront.getTexture().setAlphaMask(mask.getTexture());

The shader example from roymacdonald also has GL set to 3.2, (though there is a test in the example and uses different shaders depending upon GL Version)

good to know.
thanks!

Hi, I’m bumping this thread because I’m trying these methods on a Raspi and none of them work.

When I use the swizzle thing, it can’t find the constants, so I assume it is not on GLES. But the alpha methods don’t work either. Is there any limitation we should be aware of?

Thanks!

I realize now that there is no alphaMaskingShaderExample in the raspi repo. Maybe maybe I should take this as a bad sign :pensive: