Doing masking with ofSetBackgroundAuto(false);

hi all,
i am attempting to do masking of an image in shader.
in ofApp::setup() i have


then I have used the shaders example 04, from “Mastering openFrameworks: Creative Coding Demystified”.
I basically draw my image in an fbo, then I draw the mask in another maskFbo.
finally, I send both to the shader.

The problem with keeping ofSetBackgroundAuto(false) is that the mask does not clear up even though I call

ofSetBackgroundColor( 0, 0, 0 );

after every maskFbo.begin();

how can I fix this (without setting ofSetBackgroundAuto(true) ) ???

thank you!

You could try using ofBackground() instead of ofSetBackgroundColor(). In the documentation both methods have an identical description, but maybe there is a difference… The doumenation of ofSetBackgroundAuto() names ofBackground:

Sets the background clearing function to be auto (default) or not. If
non-auto, then background clearing will not occur per frame (at the
start of draw) but rather, whenever ofBackground is called.

nope :frowning:
unfortunately the problem persists… whether i use ofSetBackgroundAuto(), ofBackground() or both just after maskFbo.begin() …

ofSetBackgroundColor(0, 0, 0) isn’t going to clear the fbo. It is just supposed to change the background color that is drawn when auto background clearing is true. I would use ofClear(0, 0, 0, 0) or ofBackground(0, 0, 0, 0) to clear your fbo.


nope and this is weird.

the code is the following:


ofBackground( 0 );
ofSetColor( 255, 255, 255 );
image.draw( 0, 0 );

ofSetColor( 255, 255, 255 );
ofCircle( posX , posY , openingRadius -1 );


shader.setUniform1f( "time", time );   
shader.setUniform1i( "circleRadius", openingRadius );  
shader.setUniform2f( "mouse", posX , posY );    
shader.setUniformTexture( "texture1", fbo2.getTextureReference(), 1 );
ofSetColor( 255, 255, 255 );
fbo.draw( 0, 0 );

I tried with ofClear() too . Whatever I have drawn on the mask remains… between frames…
could I have done something on the shaders side?
frag shader looks like:

uniform sampler2DRect texture0;
uniform sampler2DRect texture1;    //Second texture , mask, fbo2
uniform float time;
uniform int circleRadius;
uniform vec2 mouse;

float snoise(vec2 v);

void main(){
    vec2 pos = gl_TexCoord[0].xy;
    vec4 color0 = texture2DRect(texture0, pos);
    vec4 color1 = texture2DRect(texture1, pos);

    //computer distance of pixel to current mouse point
    vec2 d = pos - mouse;
    float len =  sqrt(d.x*d.x + d.y*d.y);

    vec4 color;
    if ( len < circleRadius && len > 0 )
        float pct = len / circleRadius ;

        //pulsating hole opening
        vec2 noiseVec = vec2(time*5,time*0.2);
        pct *= pct*3 * (0.8+0.5*snoise(noiseVec)) ;

        color.rgb = color0.rgb;
        color.a = color1.r - pct ;
        color.rgb = color0.rgb;
        color.a = color1.r;
    gl_FragColor = color;

EDIT: Can you share your setup function? Something feels fishy about the background auto clearing turned off (and possibly the fact that you have alpha blending always turned on). Why do you need auto background clearing turned off?

I can take a look at your shader later, but I’m curious as to why you need the shader. Below is some sample code that does a masking operation using fbos, cutting away a circular section of an image at the current mouse position. I enable the blending mode only when I need it (i.e. blending fbo2 into the current frame, which contains fbo). If I were to enable the blending mode a line earlier, or if I were to forget to disable the blending mode when I didn’t need it, then I would end up with a black screen. fbo would be blended into the background color when it is draw.

ofFbo fbo;
ofFbo fbo2;
ofImage image;

void testApp::setup(){
    fbo.allocate(image.width, image.height);
    fbo2.allocate(image.width, image.height);

void testApp::draw(){
    ofSetColor(255, 255, 255);

        ofClear(0, 0, 0, 0);
        image.draw(0, 0);

        ofClear(0, 0, 0, 0);
        ofSetColor(255, 255, 255);
        ofCircle(mouseX , mouseY, 200);

    fbo.draw(0, 0);
    fbo2.draw(0, 0);

Hi @mikewesthad
thanks for the prompt reply.
well, blending was a question on my side too, but the example has the following note next to ofEnableAlphaBlending().

//NOTE: It is important to enable alpha blending for correct shader’s working,
    //because shader performs masking by setting alpha-value of output color

so I did not touch it at all.

also I don’t draw the maskFbo , so I’m not sure where to place the enable-disable of the blending.

Instead, I send it to the shader where a choice is made to draw on the screen, each pixel from either the texture0 (the image) or texture1 (the mask), so i’m not sure if any blending actually takes place. (i’m a noob at shaders so I could be wrong about how i’ve interpreted the shader inner workings though).
I did remove the blending though and the result remains…



and in the constructor of this shader-related scene, has nothing particular :

    shader.load( "sceneCave/shader.vert", "sceneCave/shader.frag" );

    fbo.allocate( ofGetWidth() , ofGetHeight() );

    image.loadImage( "sceneCave/letters1280gray.png" );

Okay I had the time to check out your shader, and I have a variant of your code working on my machine.

So first, why are you using ofSetBackgroundAuto(false)? I can’t seem to figure out a reason why. When the shader is drawn, it’s result is getting blended into the background of the screen. Since you aren’t clearing the screen, and since blending is enabled, each frame is blended into whatever is on the screen. The easy fix would be to set background auto to true or add an ofBackground(0) before starting your shader.

Beyond that, does your shader work as you expected? I had to make a series of tweaks both in testApp.cpp and in the shader to get it working. I’ll list them out in case it might be helpful:

float snoise(vec2 v) This function is not defined in what you posted, so maybe I’m just missing some code. I pulled an implementation from here.

I used GLSL version 150 (the version that corresponds to openGL 3.2), so I had to change texture2DRect(...) to texture(...), gl_TexCoord[0].xy to gl_FragCoord.xy, and define an out vec4 outputColor to use instead of gl_FragColor. (Using this version of GLSL is probably more of a convention choice than anything else.)

texture0 isn’t assigned a value in the code you posted. To do that, testApp.cpp's draw becomes:

        shader.setUniform1f( "time", ofGetElapsedTimef() );
        shader.setUniform1i( "circleRadius", 200 );
        shader.setUniform2f( "mouse", mouseX, ofGetHeight()-mouseY );
        shader.setUniformTexture( "texture1", fbo2.getTextureReference(), 1 );
        ofSetColor( 255, 255, 255 );
        fbo.draw( 0, 0 );

I also had to mirror the image over the x-axis, since textures coordinates in openGL are measured from the bottom left of a texture, whereas openFrameworks coordinates are the upper left.

Other small points:

I’m a little confused as to why you are doing a masking with a circle and also doing a fragment shader that does a masking with a nosily growing/shrinking circle. (Though I haven’t read “Mastering openFrameworks,” so I don’t have the complete context.) You are calculating a noisy circle radius for each fragment. The only thing that the noise depends on is the current time, so each fragment is running the same calculation. It seems like you would want to send a uniform noisyRadius to the shader, letting the CPU calculate the noise.

Also, the calculation using fbo2's circle texture seems redundant for the moment. The fragment can calculate it’s color purely based on distance from the mouse. This would not be true if fbo2 contained another image - like a picture of the sky - to be used as the mask. I assume that’s what you have planned.

Hopefully some chunk of this is useful to you and helps solve your issues getting the code working as you want.

Hi @mikewesthad
lots of info, I have dumb answers to some of your questions (aka “the tutorial said so” ) and I have questions on top of your questions for the rest :smiley:

OK inline comments to your reply follow:

why are you using ofSetBackgroundAuto(false)?

I am implementing a decay effect in some scenes (not the sceneCave that we discuss here). I could be setting ofSetBackgroundAuto() ini the init before running each scene OR I could be setting ofBackground(0). Since both should work, I have to learn why one does not work now (or how will I Learn?). Since I had set ofSetBackgroundAuto(false) randomly for this application , I am just ‘sticking’ with it. I could very well change it I you suggest it is better to do it otherwise.

The easy fix would be to set background auto to true or add an ofBackground(0) before starting your shader.

Well here I seem to miss something. I thought I only need to clear the mask in the fbo. Why do I need to clear the whole screen? The train of thought in my head goes like this (starting from the very first draw of the scene):

  1. fbo is sent to the frag shader, through texture0 (it is somehow sent there directly. At least that is what I have understood from the example04 on shaders , of the book).
  2. maskFbo is sent through setUniform1f(texture1)
  3. since blending is enabled the mask allow texture0 to pass through where the mask does not have a while colour.

on to the next draw operation (2nd pass)
4. screen does not get cleared (because of ofEnableBlending()) , but fbo gets drawn completely on top of the previous screen.
5. this is the tricky part, I assume that the new mask gets drawn on top of the new image, so the old mask is nowhere to be found. This is obviously wrong, but I fail to see where.

So if I set ofBackground(0) before the shader, I can understand that it will solve my problem , but why could clearing only the maskFbo befor every draw() NOT solve it ?


is part of the example code that accompanied the aforementioned book. You can download it from the tab “support” of the book. It is free (registration needed) and the noise shader library comes in a .zip found in the shader tutorials folder.

The book states that OF v080 comes with GLSL versions 120. Did this change recently? I shall switch to 150 and do the changes you suggest.

texture0 isn’t assigned a value

texture0 should get a value, otherwise how can I see the srcImg that i draw in fbo ?
Either way it is not clear for me (little opengl knowledge) what does fbo.bind() do.

I’m a little confused as to why you are doing a masking with a circle
and also doing a fragment shader that does a masking with a nosily
growing/shrinking circle

just untidy code (for now) still learning and I had only recently understood that I was re-implementing this mask. I shall remove the excess and do the noise calculation once in the cpu.

I am doing a simple mask at the moment (just look from the key hole kind of thing). a fading circle is easy to implement in the shader, but more complex shapes (like an actual keywhole), should be easier to draw in a maskFbo, no ?

Thank you for all the help,
it has indeed been gratifying to clear things up in my head. In your spare time please respond to the above as well :smile:


Ahhh, right I should have guessed that. Thanks for clarifying! Go ahead and keep ofSetBackgroundAuto(false).

One clarifying thing about the ofEnableAlphaBlending(). (I’m still learning openGL too, so this was new information to me.) When blending is turned off, the alpha channel is ignored [ref]. Since the shader makes the hole in the image through controlling the alpha (color.a = color1.r - pct and color.a = color1.r), turning off blending will mean that when the pixels of the image are drawn, the transparency is discarded and you will not have a hole. That must be what the book was referencing.

Okay, so your train of thought is mostly spot on. I’ll rephase it and hopefully clarify what is happening:

  1. You send texture0 and texture1 to the shader.
  2. Then the shader calculates the fragments. As per your code, RGB of the fragment is always pulled from texture0, so the alpha channel is the only thing being modified. Outside of the circular mask (texture1), the alpha is set by color.a = color1.r. That really means color.a = 0 since your circular mask is transparent with an opaque black circle in it. Inside of the circular mask, the alpha of the color is set by color.a = color1.r - pct, which creates a gradient from transparent at the edge of the circle to opaque at the center of the circle. I walk through all of this to say that the fragments coming out of your shader are transparent everywhere except for your mask area.
  3. Now the fragments from your shader are blended into the screen buffer. Since you turned off auto background clearing, the screen buffer will be whatever was in the last frame. (On the first draw call of the app, it will usually be set to a grayish value, but you might have been changing that to black in your code.) Let’s assume alpha blending is enabled. Your transparent fragments will be blended with the current colors in the screen buffer. Since the fragments outside of the mask area are completely transparent, that means that will have no effect on the colors in the screen buffer. So yes, the shader is drawing your masked image on top of the last frame, but since most of your masked image is transparent, that isn’t erasing your last frame. (If alpha blending is disabled, then your alpha channels are ignored, and you end up with your original image without a mask.)

Does all that make sense? So clearing your fbos is not going to clear the screen in the way you are expecting, hence the need for background(0) to clear the screen. As way of proof, if your shader looked like this:

if ( len < circleRadius && len > 0 ) {
    color.rgba = color0.rgba;
else {
    color.rgba = vec4(0.0, 0.0, 0.0, 1.0);

Then you wouldn’t be relying on alpha blending to create the hole (so blending could be safely disabled). The RGB channels would be modified directly. This shader would “erase” the last frame. (I realize this isn’t the type of shader you want, just showing it point out a shader that is more in line with what you were expecting in items 4 + 5 of your train of thought.)

Ah thanks!

Here’s the line of code that sets the openGL version for desktop machines. It sets it to openGL 3.2, which corresponds to version 150 [ref]. That line of code is from the github, but it is the same in v0.8.0. Maybe it changed from 120 to 150 within a bug fix to v0.8.0? Anyway, no need to switch from 120 to 150 - use whatever you are comfortable with and whatever works on the machines you are running on.

Since your image was loading, I’m puzzled by this. What must be happening behind the scenes of openFrameworks is that the texture of your first fbo is assigned to texture0 by default. I would have to do some digging in the openFrameworks source code to explain exactly what is happening. The bind operation is a central concept to openGL. It would take a little explaining. Since your shader should work for now, you don’t have to worry about it. If you want, this video is a great overview that goes into all the key concepts. Beware. It is 3+ hours long.

Yup! Though your shader will need to be modified since right now it is creating a circular gradient of transparency.

Hope I got to all of your questions!