Video alpha mask painting outside of shape issue

@arturo any help would be greatly appreciated!

so I’m getting some mixed results with a program I’m trying to write where a shape is used as an alpha mask for incoming video via webcam that then gets painted onto the screen through a second FBO that will ultimately run as a standalone on the raspberry Pi (3b,3b+)

I’m doing my main programming on my macbook and then transferring over the files to run on the RPI

in the main.cpp in wanting to run at SD as I will be outputting via the composite output

 ofSetupOpenGL( 640,480, OF_FULLSCREEN);

ofapp.cpp

#include "ofApp.h"

//--------------------------------------------------------------
void ofApp::setup(){
    
   
    
    
    
    vidGrabber.setDeviceID(0);
    vidGrabber.setDesiredFrameRate(30);
    
    ofSetVerticalSync(true);
    
    
    ofBackground(0,0,0);
    
    vidGrabber.initGrabber(ofGetWidth(), ofGetHeight());
    paint.allocate(ofGetWidth(), ofGetHeight());
    fbo.allocate(ofGetWidth(), ofGetHeight());
    vidGrabber.getTexture().setAlphaMask(fbo.getTexture());
    
    
    paint.begin();
    ofClear(255);
    paint.end();
    
    
    ofSetCircleResolution(100);
    
    
   
}

//--------------------------------------------------------------
void ofApp::update(){
    
    
    vidGrabber.update();
    if(ofGetMousePressed(OF_MOUSE_BUTTON_RIGHT))
    {
        paint.begin();
        ofClear(255);
        paint.end();
    }
    
}

//--------------------------------------------------------------
void ofApp::draw(){
    
    
    fbo.begin();
    ofClear(0,0,0,0);
    ofSetColor(255,255);
    ofDrawCircle(mouseX, mouseY, 55);
    fbo.end();
    
    
    
    paint.begin();
    vidGrabber.draw(0, 0);
    paint.end();
    
    
    paint.draw(0,0);
}


these are declared in .h

    ofFbo fbo;
    ofFbo paint;
    ofVideoGrabber vidGrabber;

so in this picture you can see when I’m running at fullscreen
the bottom right corner is not set all the way to the edge of the screen in both the X and Y directions

however if I change


    vidGrabber.draw(0, 0);

to


    vidGrabber.draw(0, 0,ofGetScreenWidth(),ofGetScreenHeight());


I get this result
the camera image now fills the screen
but there are artifacts at the edge of the shape that show through to the background

in addition the mouse now acts strangely as when in the top left corner the circle is centered on the mouse cursor but in the bottom right corner it is no longer in alignment. I think this has to do with the ofgetW&H being set by the 640,480 in the main.cpp but then getting stretched in the final paint.draw to the screen W&H. I’m less worried about the mouse (although it is a bit annoying)

the first image has the look I want as you don’t see the edges of the shapes but the second allows for actual full screen (all of this has been when using mac so far)

now when I load this onto the raspberry Pi
it gives me the same edge artifacts that I’m getting in the second picture no matter what I change
and when in full screen it only shows black with the mouse cursor unless

vidGrabber.initGrabber(ofGetWidth(), ofGetHeight());

is changed to

vidGrabber.initGrabber(640,480);

I’m testing on a projector right now with the RPI and I believe it is not actually filling the screen too
I’m feeling kind of stuck here
I feel like there is a solution to instead of paint.draw add a third FBO to mix paint and the original image in some way I tried that on the computer with ofblendmode ADD and got feedback that turned the screen white eventually

something like if you comment out the paint.draw(); gives a neat effects but is not what I’m looking for I’m sure I’m just not familiar enough with all this to understand what I’m doing wrong


    bucket.begin();
    
    vidGrabber.draw(0,0,ofGetScreenWidth(),ofGetScreenHeight());
    
    ofEnableBlendMode(OF_BLENDMODE_ADD);
    
    //paint.draw(0, 0,ofGetScreenWidth(),ofGetScreenHeight());
    paint.draw(0, 0,ofGetScreenWidth(),ofGetScreenHeight());
    ofDisableBlendMode();
    bucket.end();
    bucket.draw(0,0);

help on any front would be greatly appreciated again I really just want a full screen version of what is happening in the first picture nice clean edges

so i’ve gotten the screen size issue sorted by using togglefullscreen with a keypress

however the outlining of the shape still persists when running on the raspberry pi it doesn’t seem as bad as before but it is still certainly there

any help would be greatly appreciated

Hi @wednesdayayay,

If your camera support the same resolution than your projector, I think you should use it in the setup:

    vidGrabber.initGrabber( ofGetScreenWidth(), ofGetScreenHeight() );

and size yours fbo at the same size:

    paint.allocate( vidGrabber.getWidth(), vidGrabber.getHeight() );
    fbo.allocate( vidGrabber.getWidth(), vidGrabber.getHeight() );

then in draw():

    fbo.begin();
    ofClear(0,0,0,0);
    ofSetColor(255,255);
    ofDrawCircle( ofGetMouseX(), ofGetMouseY(), 55 );
    fbo.end();
        
    paint.begin();
    vidGrabber.draw(0, 0);
    paint.end();
        
    paint.draw(0,0);

You will not see the entire image in windowed mode but I guess this is not your primary concern.
Or if you want to make it work also in windowed mode:

	float w = ofGetWidth();
	float h = ofGetHeight();
	// Translate the mouse position from window to fbo coordinates
	float mx = ofMap( ofGetMouseX(), 0.f, w, 0.f, fbo.getWidth() );
	float my = ofMap( ofGetMouseY(), 0.f, h, 0.f, fbo.getHeight() );
	
	fbo.begin();
	ofClear(0,0,0,0);
	ofSetColor(255,255);
	ofDrawCircle( mx, my, 55 );
	fbo.end();
		
	paint.begin();
	vidGrabber.draw(0, 0);
	paint.end();
		
	paint.draw( w, h );

Drawback: the image will be distorted if the window has not the same size ratio than the screen. This can be fixed but I will not go into it for now, unless you’re interested.

If your camera doesn’t support the projector resolution, things may be a little more complicated, because:

  • you will need to find how to nicely display it without distortion if its width/height ratio is not the same than the projector. This can be solved with the ofRectangle scaleTo() method, for example
  • translate the mouse coordinates will also need extra work

I can tell you more about that if needed.

For the outline problem, my first try would be to use a target fbo without alpha channel:

paint.allocate( vidGrabber.getWidth(), vidGrabber.getHeight(), GL_RGB); // default is GL_RGBA
paint.begin();
ofClear(0.f,1.f);
paint.end();

Hope this help ! (but untested codes)

1 Like

@lilive so I had gotten the fullscreen stuff working in both cases (mac & raspberry pi) I’ll have to try your solution out at some point too!

my current issue (only on the raspberry pi) is still the outline of shapes being drawn to the screen. I’ll get another screen shot up as it isn’t as bad at this point (at least I don’t think so) after changing the paint FBO to GL_RGB

however whenever I tried to do

ofClear(0.f,1.f); 

I was getting some errors so that may be why the problem persists I suppose. I’m not really sure what the 0.f,1.f are doing either maybe if I understood that I could figure it out further. I understand in the context of ofClear but just not the difference between ofClear(0,0) and ofClear(0.f,1.f)

I’ll give it another go on the RPI as it seems to be working on the mac now that I’m trying it again.

thank you so much for your help
sorry for taking so long to get back to you I’m opening a play at my community theatre this weekend and haven’t had much time for programming