App for Raspberry Pi integrating shaders and ofGraphics objects

#1

Hi,

I am developing an app for Raspberry Pi that uses shaders but also regular ofGraphics objects, such as 2D shapes and text. But I’m noticing that ofGraphics objects are not rendered in screen if the renderer is set to OpenGLES in main.cpp (below is how I currently have main.cpp).

#include "ofMain.h"
#include "ofApp.h"

//========================================================================
int main(){
	ofGLESWindowSettings settings;
	settings.glesVersion = 2;
	ofCreateWindow(settings);
	ofRunApp(new ofApp());
}

In other words, I cannot render any other visual object besides ofShader. Does anyone know why?

I’m using Raspberry Pi 3, Raspian Stretch and the master branch of OF.

#2

can you post a minimal example with code that demos the problem and a screenshot of the output?

#3

Hey @arturo, thanks for your reply. I can reproduce the problem by slightly modifying the “01_simpleColorQuad” example. I have the following code:

main.cpp:

#include "ofMain.h"
#include "ofApp.h"

//========================================================================
int main(){
    ofGLESWindowSettings settings;
    settings.glesVersion = 2;
    ofCreateWindow(settings);
    ofRunApp(new ofApp());
}

ofApp.h:

#pragma once

#include "ofMain.h"

class ofApp : public ofBaseApp{
	public:
		void setup();
		void update();
		void draw();
    
        ofShader shader;
};

ofApp.cpp:

#include "ofApp.h"

//--------------------------------------------------------------
void ofApp::setup(){
	shader.load("shadersES2/shader");
}

//--------------------------------------------------------------
void ofApp::update(){
}

//--------------------------------------------------------------
void ofApp::draw(){
    ofSetColor(255);
    
    shader.begin();
    
    ofRect(0, 0, 1366, 768);
    
    shader.end();
    
    ofDrawBitmapStringHighlight("Shader example", 50, 50);
}

The shader code used here is exactly the same as in the original “01_simpleColorQuad” example.

When I compile this, I can see the shader being rendered, just like in the picture below:

But as you can see, the ofDrawBitmapStringHighlight is not displayed. The same happens with any other ofGraphics object, 2D objects are not displayed. The console output is the following:

HOST_OS=Linux
checking pkg-config libraries:   cairo zlib gstreamer-app-1.0 gstreamer-1.0 gstreamer-video-1.0 gstreamer-base-1.0 libudev freetype2 fontconfig sndfile openal openssl libcurl glfw3 rtaudio gtk+-3.0 libmpg123 
with PKG_CONFIG_LIBDIR=
[notice ] ofAppEGLWindow: setupRPiNativeWindow(): screenRect: 1366x768
[notice ] ofAppEGLWindow: setupRPiNativeWindow(): windowRect: 1024x768
[notice ] ofAppEGLWindow: createSurface(): setting up EGL Display
[notice ] ofAppEGLWindow: createSurface(): EGL Display correctly set 0x1
[notice ] ofAppEGLWindow: createSurface(): no current renderer selected
[notice ] ofAppEGLWindow: createSurface(): GLES2 renderer detected
[notice ] ofAppEGLWindow: createSurface(): surface created correctly
[notice ] ofAppEGLWindow: createSurface(): API bound correctly
[notice ] ofAppEGLWindow: createSurface(): -----EGL-----
[notice ] ofAppEGLWindow: createSurface(): EGL_VERSION_MAJOR = 1
[notice ] ofAppEGLWindow: createSurface(): EGL_VERSION_MINOR = 4
[notice ] ofAppEGLWindow: createSurface(): EGL_CLIENT_APIS = OpenGL_ES OpenVG
[notice ] ofAppEGLWindow: createSurface(): EGL_VENDOR = Broadcom
[notice ] ofAppEGLWindow: createSurface(): EGL_VERSION = 1.4
[notice ] ofAppEGLWindow: createSurface(): EGL_EXTENSIONS = EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap EGL_KHR_vg_parent_image EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_lock_surface 
[notice ] ofAppEGLWindow: createSurface(): GL_RENDERER = VideoCore IV HW
[notice ] ofAppEGLWindow: createSurface(): GL_VERSION  = OpenGL ES 2.0
[notice ] ofAppEGLWindow: createSurface(): GL_VENDOR   = Broadcom
[notice ] ofAppEGLWindow: createSurface(): -------------
[notice ] ofAppEGLWindow: setupPeripherals(): peripheral setup complete
[notice ] ofAppEGLWindow: setupNativeUDev(): created udev object
[notice ] ofAppEGLWindow: setupMouse(): mouse_fd=5 devicePath=/dev/input/event0
[notice ] ofAppEGLWindow: setupMouse(): mouse device name = CHESEN USB Keyboard
[notice ] ofAppEGLWindow: setupKeyboard(): keyboard_fd=6 devicePath=/dev/input/by-path/platform-3f980000.usb-usb-0:1.2:1.0-event-kbd
[notice ] ofAppEGLWindow: setupKeyboard(): keyboard device name = CHESEN USB Keyboard
[notice ] ofAppEGLWindow: setupPeripherals(): native event setup complete
[warning] ofShader: GL_VERTEX_SHADER shader reports:
Compiled
[warning] ofUtils: Couldn't create locale  using default, C.UTF-8
[warning] ofUtils: Couldn't create locale  using default, C.UTF-8
[warning] %extensions%
precision mediump float;
#define IN attribute
#define OUT varying
#define TEXTURE texture2D
#define TARGET_OPENGLES
uniform mat4 modelViewMatrix; uniform mat4 projectionMatrix; uniform mat4 textureMatrix; uniform mat4 modelViewProjectionMatrix; uniform float usingTexture; uniform float usingColors; uniform vec4 globalColor; IN vec4 position; IN vec4 color; IN vec4 normal; IN vec2 texcoord; OUT vec4 colorVarying; OUT vec2 texCoordVarying; void main(){ gl_Position = modelViewProjectionMatrix * position; if(usingTexture>.5) texCoordVarying = (textureMatrix*vec4(texcoord.x,texcoord.y,0,1)).xy; if(usingColors>.5) colorVarying = color; else colorVarying = globalColor; }

[warning] ofShader: GL_FRAGMENT_SHADER shader reports:
Compiled
[warning] ofUtils: Couldn't create locale  using default, C.UTF-8
[warning] ofUtils: Couldn't create locale  using default, C.UTF-8
[warning] %extensions%
precision mediump float;
#define IN varying
#define OUT
#define TEXTURE texture2D
#define FRAG_COLOR gl_FragColor
#define TARGET_OPENGLES
uniform sampler2D src_tex_unit0; uniform float usingTexture; uniform float bitmapText; IN vec4 colorVarying; IN vec2 texCoordVarying; void main(){ vec4 tex; if(usingTexture>.5){ tex = TEXTURE(src_tex_unit0, texCoordVarying); if(bitmapText>.5 && tex.a < 0.5){ discard; }else{ FRAG_COLOR = colorVarying*tex; } }else{ FRAG_COLOR = colorVarying; } }

[ error ] ofShader: checkProgramLinkStatus(): program failed to link
[ error ] ofShader: ofShader: program reports:
ERROR:LEX/PARSE-1 (vertex shader, line 1) Syntax error
glGetError 0x502
[warning] ofShader: GL_VERTEX_SHADER shader reports:
Compiled
[warning] ofUtils: Couldn't create locale  using default, C.UTF-8
[warning] ofUtils: Couldn't create locale  using default, C.UTF-8
[warning] attribute vec4 position;

uniform mat4 modelViewProjectionMatrix;

void main(){
	gl_Position = modelViewProjectionMatrix * position;
}

[warning] ofShader: GL_FRAGMENT_SHADER shader reports:
Compiled
[warning] ofUtils: Couldn't create locale  using default, C.UTF-8
[warning] ofUtils: Couldn't create locale  using default, C.UTF-8
[warning] 
precision highp float;

void main()
{
    // gl_FragCoord contains the window relative coordinate for the fragment.
    // we use gl_FragCoord.x position to control the red color value.
    // we use gl_FragCoord.y position to control the green color value.
    // please note that all r, g, b, a values are between 0 and 1.

    float windowWidth = 1366.0;
    float windowHeight = 768.0;

	float r = gl_FragCoord.x / windowWidth;
	float g = gl_FragCoord.y / windowHeight;
	float b = 1.0;
	float a = 1.0;
	gl_FragColor = vec4(r, g, b, a);
}

These lines refer to an error in the vertex shader:

[ error ] ofShader: checkProgramLinkStatus(): program failed to link
[ error ] ofShader: ofShader: program reports:
ERROR:LEX/PARSE-1 (vertex shader, line 1) Syntax error
glGetError 0x502

But the shader is displayed fine in the screen, so I’m not sure if this is related to the problem.

If I try to modify the same “01_simpleColorQuad” example in macOS by adding an ofGraphics drawing function, the object is displayed correctly, which makes me think that the issue is related to how OF for RPi integrates OpenGLES?

Thank you for the help!

#4

I think there might be some sort of error in the default dynamically generated shaders in the GLES version of OF, because I’ve been seeing the same vertex shader error when I’ve been running my cross platform shader tests. My shader compiles and runs fine (as does the example shader in the post above) which leads me to believe that there’s something wonky going on in either the ofShader class or somewhere else. I was planning to investigate this error and where it’s coming from next week (I’m travelling tomorrow onwards) - and I didn’t run any ofGraphics stuff on top of my cross platform shader implementation so can’t tell if I’m getting the same ofGraphics elements not drawing error - but can look into it next week.

EDIT: looking at ofGLProgrammableRenderer.cpp file here, vertex shader line 1 would have %extensions% - is that causing problems? Also, I had to change the #define OUT varying to just define OUT in my cross platform header and also remove #define TARGET_OPENGLES so those are the first things I’d look at.

1 Like
#5

@ayruos, you nailed it! I commented out the lines you mentioned in ofGLProgrammableRenderer.cpp and now I can see ofGraphics objects rendered with ofShader :slight_smile: Thank you very much!

#6

Huh, wow, that was pretty much a random guess but I’m glad it worked! I must say though, that fully commenting those out may lead to some other issues in the future - I’m gonna take an in depth look at this first week of March + issue #5056 still seems to be open so I can take a look at those too with the main.cpp file I used here if @arturo wants a PR?

PS. Does the current oF master on GitHub have @bakercp’s methods implemented for running oF on Stretch? I don’t want to mess up my currently working nightly installation with the relevant changes implemented to get it up and running if I need to make changes for PR. Maybe I should do a fresh SD card with an install from git if I have to do PRs.

#7

Interesting, it seems like if I have some stuff inside an fbo that uses shaders and then I draw the fbo, it does nothing, too. For example…

mainImage.begin();
shader.begin();
ofDrawRectangle(0, 0, ofGetWidth(), ofGetHeight());
shader.end();
mainImage.end();

in the update, and the draw call being,

mainImage.draw();

outputs nothing.

I’m looking into this now… as well as the ofGraphics not drawing in conjunction with shader stuff.

#8

Did you have any solution to this?

I’m having the same issue. Also been trying to render images from pixel data into textures on RPI
and I get no image( usually texture never allocated error)

thanks

#9

here’s some more details about it…

#10

Thanks - Actually I don’t see this error, but still have issues with fob, shaders, getting pixels into textures on the PI, and many other RPI/OF issues.

Time to go back to a mac mini I think…

#11

Hi!

does anyone have a solution to draw ofFbo between ofShader::begin()/end() on Raspberry Pi ?

it seems issue are still open on github without any news from a while…

And it doesn’t work with of 0.10

#12

finally I got it working, look at https://github.com/openframeworks/openFrameworks/pull/6223