ofxDOF in Rpi4 Raspberry

Hi! I just managed to run shaders in raspberry 4 by changing the version to gles 2 in the main.cpp. Now I am trying to run a depth of field effect in the Rpi with the ofxDOF addon. Anybody knows how to do it, as the addon will not run in raspberry.

I got this error

/home/pi/openFrameworks/addons/ofxDOF/src/ofxDOF.cpp:25:61: error: ‘GL_TEXTURE_RECTANGLE_ARB’ was not declared in this scope
     dofBuffersSettings.textureTarget = ofGetUsingArbTex() ? GL_TEXTURE_RECTANGLE_ARB : GL_TEXTURE_2D;
 ^~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/openFrameworks/addons/ofxDOF/src/ofxDOF.cpp:25:61: note: suggested alternative: ‘GL_TEXTURE_EXTERNAL_OES’
     dofBuffersSettings.textureTarget = ofGetUsingArbTex() ? GL_TEXTURE_RECTANGLE_ARB : GL_TEXTURE_2D;
 ^~~~~~~~~~~~~~~~~~~~~~~~
 GL_TEXTURE_EXTERNAL_OES

So I went to the ofxDOF.cpp and changed GL_TEXTURE_RECTANGLE_ARB by GL_TEXTURE_EXTERNAL_OES, and it compiled, but it reports several errors on the shader:

[ error ] ofShader: GL_VERTEX_SHADER shader reports:
0:1(10): error: GLSL 1.50 is not supported. Supported versions are: 1.00 ES, 3.00 ES, and 3.10 ES

[ error ] ofShader: GL_VERTEX_SHADER, offending line 10 :
	    8	
	    9	// App uniforms and attributes
	   10	out vec4 vColor;
	   11	out vec2 vTexCoord;
	   12	

[ error ] ofShader: setupShaderFromSource(): GL_FRAGMENT_SHADER shader failed to compile
[ error ] ofShader: GL_FRAGMENT_SHADER shader reports:
0:1(10): error: GLSL 1.50 is not supported. Supported versions are: 1.00 ES, 3.00 ES, and 3.10 ES

[ error ] ofShader: GL_FRAGMENT_SHADER, offending line 10 :
	    8	uniform sampler2DRect range;
	    9	uniform vec2 sampleOffset;
	   10	uniform float focalDistance;
	   11	uniform float focalRange;
	   12	

[ error ] ofShader: checkProgramLinkStatus(): program failed to link
[ error ] ofShader: ofShader: program reports:
error: linking with uncompiled/unspecialized shadererror: linking with uncompiled/unspecialized shader
[ error ] ofShader: setupShaderFromSource(): GL_VERTEX_SHADER shader failed to compile
[ error ] ofShader: GL_VERTEX_SHADER shader reports:
0:1(10): error: GLSL 1.50 is not supported. Supported versions are: 1.00 ES, 3.00 ES, and 3.10 ES

[ error ] ofShader: GL_VERTEX_SHADER, offending line 10 :
	    8	in vec2 texcoord;
	    9	
	   10	// App uniforms and attributes
	   11	out vec4 vColor;
	   12	out vec2 vTexCoord;

[ error ] ofShader: setupShaderFromSource(): GL_FRAGMENT_SHADER shader failed to compile
[ error ] ofShader: GL_FRAGMENT_SHADER shader reports:
0:1(10): error: GLSL 1.50 is not supported. Supported versions are: 1.00 ES, 3.00 ES, and 3.10 ES

[ error ] ofShader: GL_FRAGMENT_SHADER, offending line 10 :
	    8	uniform float focalRange;
	    9	
	   10	in vec4 vColor;
	   11	in vec2 vTexCoord;
	   12	

[ error ] ofShader: checkProgramLinkStatus(): program failed to link
[ error ] ofShader: ofShader: program reports:
error: linking with uncompiled/unspecialized shadererror: linking with uncompiled/unspecialized shader
[ error ] ofFbo: getDepthTexture(): frame buffer object 1 not allocated with depthStencilAsTexture

If anybody knows what should I do to make it work, would be very cool. :slight_smile:

Is the first line of your shader something like:

#version 150

The gles shaders in the shader examples omit that and begin instead with:

precision highp float.

And then if I remember right, you have to use texture2D() in the shader with gles, and not texture() like you would for openGL.

1 Like

Thanks! I tried and changed the version in different ways as you suggested. In the vertex shaders, the precision highp float gave me an error, but not in the fragment shaders. I finally used the version declaration I found in this book (page 11):

#version 300 es
on the vertex shaders, and

#version 300 es
precision highp float;

in the fragment shaders. Some of the errors where corrected, and after that, I changed texture() to texture2D() as you said, and even more errors where corrected. Finally I changed all the uniform sampler2DRect to uniform sampler2D, as suggested by arturo here:

Now the shader is working perfect and I am very happy about that! Looks like I was able to port the shader to GL ES correctly with those suggestions, thank you very much.

Sadly, It still doesn’t work and I can’t see the result, as I still get a white screen, and the following errors:

[ error ] ofAppGLFWWindow: 65544: X11: RandR gamma ramp support seems broken
[warning] ofFbo: allocate(): depthStencilAsTexture is not available for iOS
[ error ] ofFbo: getDepthTexture(): frame buffer object 1 not allocated with depthStencilAsTexture
[ error ] ofTexture: getTextureData(): texture has not been allocated
[ error ] ofFbo: getDepthTexture(): frame buffer object 1 not allocated with depthStencilAsTexture
[ error ] ofTexture: getTextureData(): texture has not been allocated

Any help would be great. I feel I am getting closer to make this DOF work on the Rpi4!

1 Like

Hey you may need one of the RPi folks to chime in here.

In the meantime, what oF window class are you using? Also is the Pi4 loading a non-legacy driver and was oF compiled to use GLFW (so, not legacy mode)? Also, it may be worth a look at this thread if you haven’t already, especially some of the stuff related to window type, openGL vs gles, drivers, etc, and the list of “popular links” at the top.

I’m thinking (but could be wrong) that:
Pi4 + Buster + non-legacy driver + non-legacy oF = GLFW + openGL.
Pii4 + Buster + legacy driver + legacy oF = EGL + GLES.

Hi! Well, about the window class, I am using this on the main file:

#include "ofMain.h"
#include "ofApp.h"

//========================================================================
int main( ){
        ofGLESWindowSettings settings;
        settings.glesVersion = 2;
	settings.setSize(900, 600);
    ofCreateWindow(settings);

    ofRunApp(new ofApp());

}

I am also getting this errors:

➜  DOFtests3 make run
VER ID IS 10
using newer build and GLFW window
[ error ] ofAppGLFWWindow: 65544: X11: RandR gamma ramp support seems broken
[warning] ofFbo: allocate(): depthStencilAsTexture is not available for iOS
[ error ] ofFbo: getDepthTexture(): frame buffer object 1 not allocated with depthStencilAsTexture

The first error, that says ofAppGLFWindow, I get it every time I compile something in the raspberry with openFrameworks.

About the drivers, i am using “Legacy”. I tried to use the "fake KMS GL Driver, but the screen doesn’t work properly, and I usually access my rpi from a anydesk session, which doesn’t work either with those fake KMS GL. Would you recommend to use the fake kms driver? Will that work with the ofxDOF? I am going to run some tests on that too via ssh, so I won’t be needing anydesk to communicate with the Rpi.

That thread you mentioned is very interesting, I am going to read it and reply with more information. Thanks again, TimChi!

Yeah there’s been a lot of discusstion in the forum about the Pi4. Also revisit the setup guide, which has been revised for 0.11.2 (I think at least). That statement above the error “using newer build and GLFW window” is from oF, and means that oF has been compiled to work with the new driver (not the legacy one) and will work with the GLFW window and openGL.

The default makefile has a flag (or even a few) that determines if oF is compiled for use with the legacy driver/EGL or the new driver/GLFW. Scroll down to the " Use EGL instead of GLFW" heading of the setup guide for more info. So, your code and shaders may all work much much better if you recompile oF to use the legacy driver.

Alternatively, you could try using the new driver, your current oF compilation, and a GLFW window and openGL, which is the same setup for a regular linux distribution:

// in main.cpp
    ofGLFWWindowSettings settings;
    settings.setGLVersion(3, 2);
    settings.setSize(1024, 768);
    ofCreateWindow(settings);

    ofRunApp(new ofApp());

Then try setting up the shaders to use openGL with this:

#version 150 // for openGL 3.2

And then don’t forget to change some of the shader code to openGL, like texture() instead of texture2D(), etc.

I have a Pi3 on Buster with oF compiled in legacy mode, and I’ve run shaders with an EGL window and GLES. I tried setting up the Pi3 for GLFW and I think I had some limited success (ran a few examples), but maybe had problems with the shaders too. But, its a Pi3 and not a Pi4, and I didn’t spend too long trying to figure it out. I found the Pi3 ran much slower (in general) with the new driver, and so went back the legacy driver and recompiled oF with the legacy option.

Oh! ok! Now I am beginning to get it. I was reading in the forum, and didn’t really understood what was it all about the type of window related to the driver. So I am trying both of the approaches you suggested:

1-Recompiling openFrameworks to use the legacy driver:
Rasperry os (buster) + desktop + anydesk+legacy driver+ legacy oF = EGL + GLES

2-Preparing a different card to use it on the alternative approach, with
raspberry os headless (buster) + ssh + new driver + non legacy oF =GLFW + openGL

And will try again with the ofxDOF, keeping in mind the changes that I have to make in the shaders depending on if it is GLES or openGL.

This sounds great. This is a lot of homework! Hahaahah thanks again! I will post my results in the next reply.

1 Like

Ok! The first approach did not give as much work as it failed inmediately hahahah. This where the errors:

[notice ] ofAppEGLWindow: display(): eglSwapBuffers failed: ���Ysurface specified
[notice ] ofAppEGLWindow: display(): eglSwapBuffers failed: ���Ysurface specified
[ error ] ofAppEGLWindow: destroySurface(): attempted to destroy uninitialized window

I will post the result of the other approach soon.

Hey I worked on this a bit today too. I got this to work properly:
Pi3 + fake KMS driver + oF 0.11.2 compiled for GLFW (non-legacy) = GLFW (or GLES) + GLES2.

I did a fresh install of Buster on my Pi3, using the fake KMS driver, and compiled oF with the new (non-legacy) option. I can get the first shader example to compile and work properly if I do the following:

// in main.cpp
	ofGLFWWindowSettings settings;
	settings.setGLESVersion(2);
	settings.setSize(1024, 768);
	ofCreateWindow(settings);
    
	ofRunApp(new ofApp());

// in ofApp::setup()
    // load the ES2 shaders
    shader.load("shadersES2/shader");

The shader in shadersGL3 (so, settings.setGLVersion(3,2) and #version 150 in the shader files) will not color the screen, but the app will compile and run.

If I change the window settings to ofGLESWindowSettings, the app also compiles and runs the way it should.

So I’m thinking you just need to switch over to one of the new drivers if oF is compiled in non-legacy mode. The fake KMS one has been around longer I think. Then use either a GLFW or a GLES window, and use GLES2 shaders (like the examples in the /shader folder).

I did this on a Pi3, but hopefully the Pi4 will be the same. And I didn’t test the G3 GL (full KMS) driver yet to see if its any different but would be fun to try that tomrrow.

Ok, the second setup using newer build and GLFW window

using newer build and GLFW window
[ error ] ofAppGLFWWindow: 65544: X11: The DISPLAY environment variable is missing
[ error ] ofAppGLFWWindow: couldn't init GLFW
[ error ] ofAppGLFWWindow: 65537: The GLFW library is not initialized
[ error ] ofAppGLFWWindow: 65537: The GLFW library is not initialized
Segmentation fault

failed the first time with the headless raspberry os. So I had to install X11, blackbox, and lightdm.

After that it gave me the same errors.
I found this post, and did the export DISPLAY=:0

After doing that, I was able to run the 3dPrimitives example. But when I try to run the shader examples, I get back to the beginning: not being able to run them.

VER ID IS 10
using newer build and GLFW window
[ error ] ofShader: sorry, it looks like you can't run 'ARB_shader_objects'
[ error ] ofShader: please check the capabilites of your graphics card: http://www.ozone3d.net/gpu_caps_viewer
[ error ] ofShader: setupShaderFromSource(): failed creating GL_VERTEX_SHADER shader
[ error ] ofShader: sorry, it looks like you can't run 'ARB_shader_objects'
[ error ] ofShader: please check the capabilites of your graphics card: http://www.ozone3d.net/gpu_caps_viewer
[ error ] ofShader: setupShaderFromSource(): failed creating GL_FRAGMENT_SHADER shader
[ error ] ofShader: linkProgram(): trying to link GLSL program, but no shaders created yet
^C^Cmake: *** [/home/pi/openFrameworks/libs/openFrameworksCompiled/project/makefileCommon/compile.project.mk:184: run] Interrupt

the shader example has three folders with different shader versions:

shadersES2  shadersGL2  shadersGL3

So I guess the code should be choosing from those depending on the GL version it finds in the machine. I am not sure which one is choosing, but it is not working.

In this other thread I used the GLES2 version with the shaders, and it ran with no problems, but I can’t make ofxDOF work on it.

I’m glad you found the thread about running headless; I wasn’t sure about that.

I think there are two issues here. One involves the configuration of the Pi4 and oF to run a shader (any shader). The shader examples should compile and run with the changes above (minus the #version 150 in the shader files), provided that the Pi is using one of the KMS drivers and oF 0.11.x has been compiled in non-legacy mode. The shader examples use some ifdefs and etc to pick both the window type and the shader folder, which didnt’ seem to work right with the Pi4 and these other changes. So I omitted the ifdefs and specified the window type and which shader files to load (the ones in /bin/data/shadersES2).

Then there is the issue of ofxDOF. I ran the example-dof from the addon on a linux laptop and (with a few minor changes) it compiled and ran fine. But I get errors (like you did) related to an unallocated fbo and texture (with some similar changes). So, I’m wondering if GLES2 allows for an ofFbo that uses depth and stencil. And if not, then if GLES3 does. And unfortunately this exceeds my knowledge of GLES since I mostly work with linux and openGL. I think if you plug away at it for a while you’ll be able to figure out a workaround. The dof-example shaders also required quite a bit of reworking because they are not GLES, but I eventually did get them to compile.

So for me, getting ofxDOF to work on a Pi4 boils down solving the following error messages, which I think arise from ofxDOF::setup() and the allocation of the ofxDOF::dofTarget (which is an ofFbo):

[warning] ofFbo: allocate(): depthStencilAsTexture is not available for iOS
[ error ] ofFbo: getDepthTexture(): frame buffer object 1 not allocated with depthStencilAsTexture
[ error ] ofTexture: getTextureData(): texture has not been allocated
1 Like

Hey there were some other threads in the forum about depth of field with GLES2: Depth of field effect in GL ES 2.0 and Shaders, fbo and depth texture.

How about using gl_FragCoord.z in the fragment shader, instead of sampling the depth texture?

There may also be a way to bind a depth texture to a conventional ofFbo. There is an android example here: android - Generate a depth texture in OpenGL ES 2.0 or 3.0 - Stack Overflow . And maybe some clues in these forum posts for iOS: ofFBO in iOS - depth buffer and FBO in iOS - stencil buffer. I’m nto sure if the Pi is like iOS though, in that the depth and stencil buffers are combined into 1 buffer.

1 Like

More experiments coming then! I will check out those threads and see what codes I can run on the raspberry to make the depth of field work. I do have an iOs device in which I could run some tests and compare them to the results on the raspberry. Thank you very much, that is very useful information! See you on the next round of tests!