blending multiple FBOs with a shader

kylemcdonald · July 18, 2010, 9:49pm

I’d like to do arbitrary blending of two textures on the GPU. The two textures come from ofxFBOTextures (the version on Damian’s github), and the blending is done with an ofxShader (the version on my Google Code).

It looks like my attempt at setting the sampler2DRect is totally ignored, though.

In the attached example, I draw into two FBOs: a red (.5, 0, 0) circle on the left, a green (0, .5, 0) circle on the right. They are additively blended with glBlend. When you click, it switches to using the shader for blending – except instead you just get a green (0, 1, 0) circle on the right.

I’m going to read through this FBO-tutorial and try it without ofxFBOTexture, but it’d be super helpful if someone knows how to do it with ofxFBOTexture!

  
  
uniform sampler2DRect base, top;  
  
void main(void){  
	vec2 uv = gl_TexCoord[0].xy;  
	gl_FragColor = texture2DRect(top, uv) + texture2DRect(base, uv);  
}

  
  
void testApp::draw() {  
	int r = 200;  
  
	leftFbo.begin();  
	ofBackground(0, 0, 0);  
	glColor4f(.5, 0, 0, 1);  
	ofCircle(w / 3, h / 2, r);  
	leftFbo.end();  
  
	rightFbo.begin();  
	ofBackground(0, 0, 0);  
	glColor4f(0, .5, 0, 1);  
	ofCircle(2 * w / 3, h / 2, r);  
	rightFbo.end();  
  
	glColor4f(1, 1, 1, 1);  
  
	if(isMousePressed) {  
		blendShader.begin();  
  
		leftFbo.bindAsTexture();  
		blendShader.setUniform("left", (int) leftFbo.getTextureData().textureID);  
  
		rightFbo.bindAsTexture();  
		blendShader.setUniform("right", (int) rightFbo.getTextureData().textureID);  
  
		texRect(w, h);  
		blendShader.end();  
	} else {  
		glEnable(GL_BLEND);  
		glBlendFunc(GL_SRC_ALPHA, GL_ONE);  
		leftFbo.draw(0, 0);  
		rightFbo.draw(0, 0);  
		glDisable(GL_BLEND);  
	}  
}

(texRect() just draws a rectangle from the top left corner to the bottom right corner, with unnormalized texCoords.)

MultipleFBOs.zip

arturo · July 18, 2010, 10:51pm

i think the problem is OF uses ARB textures so sampler2DRect won’t work

google first result for “sampler2dRect arb” is another OF thread that seems similar to what you’re trying:

http://forum.openframeworks.cc/t/2-x-fbo-into-sampler2drect/3279/0

kylemcdonald · July 19, 2010, 12:09am

I thought ARB textures = NPOT = sampler2DRect, and non-ARB = POT = sampler2D?

I tried switching the shader to sampler2D and texture2D, and texRect to normalized coordinates, and it just draws black instead.

Could it be that OF mostly uses ARB, but FBOs are non-ARB?

I’ll look through that post, it looks really relevant.

kylemcdonald · July 19, 2010, 12:46am

I got the code in the post to compile, but it has a weird error when it runs.

I tried copying the pattern very directly though, and noticed a few things that are a little different.

For example, ofxFBOTexture::bindAsTexture() is defined as:

  
  
void ofxFBOTexture::bindAsTexture(){  
	glBindTexture(GL_TEXTURE_2D, (GLuint)&texData.textureID);  
}

While a better definition would be:

  
  
void ofxFBOTexture::bindAsTexture(){  
	glBindTexture(texData.textureTarget, (GLuint)&texData.textureID);  
}

Though the code you linked to uses glBindBuffer instead of glBindTexture – I’m not sure why.

I changed the rest of my code accordingly:

  
  
int leftId = leftFbo.getTextureData().textureID;  
glActiveTexture(leftId);  
leftFbo.bindAsTexture();  
leftFbo.draw(0, 0);  
  
int rightId = rightFbo.getTextureData().textureID;  
glActiveTexture(rightId);  
rightFbo.bindAsTexture();  
rightFbo.draw(0, 0);  
  
blendShader.begin();  
  
blendShader.setUniform("base", leftId);  
blendShader.setUniform("top", rightId);  
...

There was also a really obvious bug in the code above, I was setting the wrong uniforms (they should be “top” and “base” rather than “left” and “right”).

I’m not sure why the draw() would be necessary, but I left it in anyway… still not working

pixelnerve · July 19, 2010, 5:39am

before anything, try blending both without using shaders. see if that works.
second, looks like you’re dealing with different texture targets. check your texture targets are the same as fbo’s and which TextureUnit you’re assigning it to, are they valid.

i assume both fbo/textures have same target, if not try to handle each individually.
glEnable( GL_TEXTURE_RECTANGLE );
glBind…
glDisable( GL_TEXTURE_RECTANGLE );

glEnable( GL_TEXTURE_2D );
glBind…

RECT textures = sample2dRect (absolute texcoords)
NPOT & POT = sample2d (normalized texcoords)

hope it works

kylemcdonald · July 19, 2010, 7:16am

Thanks pixelnerve – if you check out the pictures and code on the first post you can see I’ve already tested with glBlend and it works well. And yes, they’re both GL_TEXTURE_RECTANGLE_ARB as is the default in OF.

So I think I get it all now, after reading the tutorial I linked to above. I had to write my own ofxFBO though… sorry!

Instead of creating an FBO with a single texture “inside”, you create an FBO and call setTarget to attach an ofImage to it. This makes working with multiple targets (formerly, multiple FBOs) much cleaner. And, I’ve read, it’s faster too.

The big trick I had to figure out was this: when you’re done setting the uniform values of the shader, and done changing glActiveTexture(), you need to set glActiveTexture back to GL_TEXTURE0. If you don’t do this, calls to draw() an ofImage don’t work. This was really confusing me as I tried to switch back and forth between shader based and glBlend based drawing.

Maybe ofTexture::draw() should have glActiveTexture(GL_TEXTURE0) inside? For now I’ve just added it to the end of ofxShader::setTexture() so you’re always safe.

My minimalist ofxFBO is in the attached code, and pasted below. For the new ofxShader see my Google-Code.

  
  
#pragma once  
  
#include "ofGraphics.h"  
  
class ofxFBO {  
protected:  
	int width, height;  
	GLuint fboId, depthId;  
	ofImage* attached;  
public:  
	ofxFBO() :  
			fboId(0),  
			depthId(0),  
			attached(NULL) {  
	}  
	void setup(int width, int height) {  
		this->width = width;  
		this->height = height;  
		glGenFramebuffersEXT(1, &fboId);  
		glGenRenderbuffersEXT(1, &depthId);  
		glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, depthId);  
		glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT, GL_DEPTH_COMPONENT, width, height);  
		glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT, GL_RENDERBUFFER_EXT, depthId);  
		swapOut();  
	}  
	~ofxFBO() {  
		if(fboId != 0) {  
			glDeleteFramebuffersEXT(1, &fboId);  
			glDeleteRenderbuffersEXT(1, &depthId);  
		}  
	}  
	void setTarget(ofImage& target) {  
		swapIn();  
  
		// detach old texture  
		if(attached != NULL) {  
			ofTextureData texData = attached->getTextureReference().texData;  
			glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, texData.textureTarget, 0, 0);  
		}  
  
		// attach new texture  
		attached = &target;  
		ofTextureData texData = attached->getTextureReference().texData;  
		glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, texData.textureTarget, texData.textureID, 0); // attach  
  
		swapOut();  
	}  
	void swapIn() {  
		glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fboId);  
		glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, depthId);  
	}  
	void swapOut() {  
		glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0);  
		glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, 0);  
	}  
	void begin() {  
		swapIn();  
		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);  
	}  
	void end() {  
		swapOut();  
	}  
};

ofxFBOBlending.zip

pixelnerve · July 19, 2010, 8:40am

ah yes its correct.
drawing image assumes texture unit 0. it uses glTexCoord2f i guess, which goes by default with texture unit 0. if you dont restore that, weird things happen as you have experimented first hand.
Good call on the MRT use.

have fun.