Record ofFbo with MediaCodec

android
#1

Hi all,

i’m trying to record clips of the contents of an fbo, done with OF, with the MediaCodec android object.
I’m looking for some info:

is it in any way possible to “see” the content of a texture from c++ from the java side?
atm i’m passing the GLuint from outFbo.getTextureReference().getTextureData().textureID through jni
to the java side, and trying to use it as input to a shader that writes to the Surface that the
MediaCodec gives me as input for the recording…but i’m getting just noise, as a not initializated texture.

if somebody can give me some information or direction about the possible way to solve this problem
i’ll be very very glad, it’s really like a maze.

#2

it should work, in fact we use textures from jave in the c++ side for the video player… so it should be possible but never tried it. be sure that you are binding the correct target…

1 Like
#3

oh, super nice!!!
thanks for the great direction, ofVideoPlayer is the path.

if i understand well you manually create a ofTextureData, and then pass the texID to java, the exact same thing that i’m trying to do with my fbo.

Can you explain me why :
td.textureTarget = GL_TEXTURE_EXTERNAL_OES; ?

we aren’t on the same render thread of the java side?

and other than the textureTarget there is any other substantial difference beetwen that
texture and the standard texture of an ofFbo?

I’ll try to create a texture, configure it the same as the one of ofVideoPlayer, attach it to an ofFbo, and to use that texID on the java side.

by the way this is what i get on the java side from a standard fbo texture:

Screen Shot 2015-07-19 at 03.06.16.png798×803 662 KB

#4

trying to implement a new texture for the ofFbo i got a really strange problem…

i added the creation of the ofTexture, ofTextureData, and the attach to the fbo in the setup(), and everything seems fine,
but when i launch the app i got a glError on onSurfaceChanged()

code added to setup() :

ofLog() << "HERE SETUP 1";
ofTextureData td;
td.width = vj.outFbo.getWidth();
td.height = vj.outFbo.getHeight();
td.tex_w = td.width;
td.tex_h = td.height;
td.tex_t = 1; // Hack!
td.tex_u = 1;
td.textureTarget = GL_TEXTURE_EXTERNAL_OES;
td.glTypeInternal = GL_RGBA;
td.bFlipTexture = false;
td.useTextureMatrix = true;

// hack to initialize gl resources from outside ofTexture
texture.texData = td;
texture.texData.bAllocated = true;

ofLog() << "HERE SETUP 2 " << texture.texData.textureID;
glGenTextures(1, (GLuint *)&texture.texData.textureID);
    ofLog() << "HERE SETUP 2 " << texture.texData.textureID;

glEnable(texture.texData.textureTarget);

glBindTexture(texture.texData.textureTarget, (GLuint)texture.texData.textureID);

glTexParameterf(texture.texData.textureTarget, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameterf(texture.texData.textureTarget, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameterf(texture.texData.textureTarget, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(texture.texData.textureTarget, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);

glDisable(texture.texData.textureTarget);

	    ofLog() << "HERE SETUP 3";
  vj.outFbo.attachTexture(texture, GL_RGBA, 0);


texID = vj.outFbo.getTextureReference().getTextureData().textureID;

ofLog() << "KKK OF TEXID: " << texID;

Log of the error:

   07-19 04:41:16.992  17306-17503/cc.openframeworks.miumiusic E/KKK1﹕ onSurfaceCreated_E: glGetError: 0x502
  07-19 04:41:17.022  17306-17503/cc.openframeworks.miumiusic W/dalvikvm﹕ threadid=11: thread exiting with uncaught exception (group=0x430f6140)
 07-19 04:41:17.122  17306-17503/cc.openframeworks.miumiusic E/AndroidRuntime﹕ FATAL EXCEPTION: GLThread 3133
Process: cc.openframeworks.miumiusic, PID: 17306
java.lang.RuntimeException: glGetError encountered (see log)
        at cc.openframeworks.GlUtil.checkGlError(GlUtil.java:94)
        at cc.openframeworks.OFAndroidWindow.onSurfaceChanged(OFAndroidWindow.java:427)
        at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1516)
        at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1244)

throwed by:

    public void onSurfaceChanged(GL10 gl, int w, int h) {
	this.w = w;
	this.h = h;
	if(!setup && OFAndroid.unpackingDone){
        try {
            setup();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
	OFGestureListener.swipe_Min_Distance = (int)(Math.max(w, h)*.04);
	OFGestureListener.swipe_Max_Distance = (int)(Math.max(w, h)*.6);
	OFAndroid.resize(w, h);

    GlUtil.checkGlError("onSurfaceCreated_E");
 }

the exact line in setup() that switch on and off the crash in SurfaceChanged() is:

	glGenTextures(1, (GLuint *)&texture.texData.textureID);

possible causes? what i’m missing?

#5

a part the problem on glGenTextures(), for which i opened another thread glGenTextures in ofApp, now i nalied a bit the problem with MediaCodec.

actually i can record a perfect clip with MediaCodec if i pass the textureID of an ofImage, the problem is just and only with ofFbo.

i’m missing something about the inner workings of ofFbo?
why ofImage.getTextureReference().getTextureData().textureID gives me a correct texture on the java side, and ofFbo.getTextureReference().getTextureData().textureID gives me just static glitch (like a non initializated texture) ??

#6

ok, now i see that if i do

outFbo.getTextureReference().draw();

i display the static glitch…

and i see that in ofFbo.cpp :

void ofFbo::updateTexture(int attachmentPoint) {
if(!bIsAllocated) return;
// TODO: flag to see if this is dirty or not
#ifndef TARGET_OPENGLES
if(fbo != fboTextures && dirty) {
	glGetIntegerv( GL_FRAMEBUFFER_BINDING, &savedFramebuffer );

	if (!ofIsGLProgrammableRenderer()){
		// save current drawbuffer
		glPushAttrib(GL_COLOR_BUFFER_BIT);
	}
	// save current readbuffer
	GLint readBuffer;
	glGetIntegerv(GL_READ_BUFFER, &readBuffer);

	glBindFramebuffer(GL_READ_FRAMEBUFFER, fbo);
	glBindFramebuffer(GL_DRAW_FRAMEBUFFER, fboTextures);
	glDrawBuffer(GL_COLOR_ATTACHMENT0 + attachmentPoint);
	glReadBuffer(GL_COLOR_ATTACHMENT0 + attachmentPoint);
	glBlitFramebuffer(0, 0, settings.width, settings.height, 0, 0, settings.width, settings.height, GL_COLOR_BUFFER_BIT, GL_NEAREST);

	glBindFramebuffer(GL_READ_FRAMEBUFFER, savedFramebuffer);
	glBindFramebuffer(GL_DRAW_FRAMEBUFFER, savedFramebuffer);
	glBindFramebuffer( GL_FRAMEBUFFER, savedFramebuffer );

	// restore readbuffer
	glReadBuffer(readBuffer);
	
	if(!ofIsGLProgrammableRenderer()){
	// restore drawbuffer
		glPopAttrib();
	}

	dirty = false;

}
  #endif
}

so probably this is my only problem.

how can i get the texture from an ofFbo with opengles??

#7

Ok, i solved it, the noise like an unitializated texture was, well , an unitializated tex, my fault.

Anyway for anybody that needs to record clips of ofFbo contents on Android, i’ve put my solution here: https://github.com/rcavazza/openFramework-Android-Recording

2 Likes
glGenTextures in ofApp