ofGLESWindowSettings | ofShader | OfxGStreamer

#1

hi!

i’m trying to use ofShader with OfxGStreamer but with no success.
later i used shaders with ofGLFWWindowSettings and ofVideoPlayer and worked fine, but the performance was much low with fps alternation.

now, with GStreamer addon, the performance is great, but i get errors when using ofGLFWWindowSettings or ofGLWindowSettings such as:

[ error ] ofShader: setupShaderFromSource(): GL_VERTEX_SHADER shader failed to compile
ERROR: 0:12: ‘’ : #version required and missing.
ERROR: 0:12: ‘varying’ : syntax error: syntax error
[ error ] ofShader: setupShaderFromSource(): GL_FRAGMENT_SHADER shader failed to compile

taking a look at some examples and others code, i tried to use ofGLESWindowSettings and this is the one that i could see positive responses, such as:

[verbose] ofShader: checkAndCreateProgram(): creating GLSL program
[verbose] ofShader: setupShaderFromSource(): GL_VERTEX_SHADER shader compiled
[verbose] ofShader: setupShaderFromSource(): GL_FRAGMENT_SHADER shader compiled
[verbose] ofShader: linkProgram(): attaching GL_FRAGMENT_SHADER shader to program 1
[verbose] ofShader: linkProgram(): attaching GL_VERTEX_SHADER shader to program 1

on this case the video is flickering and there are no returning from shaders commands. and now i really dont know how to fix that
i will try to sum up the code below:

main.cpp


int main( ){
    ofSetLogLevel(OF_LOG_VERBOSE);
    int windowWidth = 1920;
    int windowHeight = 1080;

#ifdef TARGET_OPENGLES
    ofGLESWindowSettings settings;
    settings.width = windowWidth;
    settings.height = windowHeight;
    settings.setGLESVersion(2);
    ofCreateWindow(settings);
#else
    ofSetupOpenGL(windowWidth, windowHeight, OF_WINDOW);
#endif

    ofRunApp( new ofApp());
}

ofApp.h

        // [...]
        ofVideoPlayer player;
        ofShader shader;
        bool doShader;
        ofxToggle doShaderT;
        // [...]

ofApp.cpp

//--------------------------------------------------------------
#include "ofGstVideoPlayer.h"

void ofApp::setup(){
      // [...]
      player.setPlayer(ofPtr<ofGstVideoPlayer>(new ofGstVideoPlayer));
      player.load("videos/test.mov");
      player.play();
      // [...]
      shader.load("shaders/brcosa_GLSL");
      doShader = true;
}
//--------------------------------------------------------------
void ofApp::update(){
    player.update();
    player.setSpeed(speed);
    player.isFrameNew();
    if(doShaderT){
        doShader = !doShader;
    }
    // [...]
}
//--------------------------------------------------------------
void ofApp::draw(){
    fbo.begin();
    ofClear(0,0,0,255);
    
     if( doShader ){
        player.getTexture().bind();
        shader.begin();
        shader.setUniformTexture("tex0", player.getTexture(), 0);
        shader.setUniform1f("saturation", saturacao);
        shader.setUniform1f("contrast", contraste);
        shader.setUniform1f("brightness", brilho);
     }
     player.draw(-player.getWidth()/2, -player.getHeight()/2);
     if( doShader ){
        shader.end();
        player.getTexture().unbind();
     }

     fbo.end();
    
    ofSetColor(255);
    fbo.draw(0,0,1920,1080);
     // [...]
}

OF 10.1
XCode 9.4.1
macOS Mojave

hope someone can help me with some new ideas!
thank you!!

#2

ofGLESWindow is not doing anything there really. This class is only used in linux ARM right now and the way you are setting it up there is not even being used cause #ifdef TARGET_OPENGLES will be false in osx so the part of main that is being run is only the part after the #else, effectively your main file is something like:


int main( ){
    ofSetLogLevel(OF_LOG_VERBOSE);
    int windowWidth = 1920;
    int windowHeight = 1080;

    ofSetupOpenGL(windowWidth, windowHeight, OF_WINDOW);

    ofRunApp( new ofApp());
}

I imagine when using ofGLFWWindowSettings you were setting the opengl version to 3 or more which changes the glsl syntax that’s why your shader weren’t compiling but it has nothing to do with gstreamer

apart from that this in draw:

player.getTexture().bind();

is not needed although i don’t think it’s why the video would flicker.

Does it flicker when not using the shader? or only when using it?

#3

thnx for the reply @arturo

changed again to ofGLFWWindowSettings and reallocated the player.getTexture().bind(); outside if (doShader) as below. it is actually replying as ofGLESWindowSettings.

when NOT setting the GL version or using settings.setGLVersion(2, 1) it is compiling, but with no shader response and flickering. there is no flicker when i turn down the shader, only when i use it.

[verbose] ofShader: checkAndCreateProgram(): creating GLSL program
[verbose] ofShader: setupShaderFromSource(): GL_VERTEX_SHADER shader compiled
[verbose] ofShader: setupShaderFromSource(): GL_FRAGMENT_SHADER shader compiled
[verbose] ofShader: linkProgram(): attaching GL_FRAGMENT_SHADER shader to program 1
[verbose] ofShader: linkProgram(): attaching GL_VERTEX_SHADER shader to program 1
[verbose] ofShader: checkProgramLinkStatus(): program 1 linked
[verbose] ofGstUtils: gstHandleMessage(): unhandled message from player
[verbose] ofShader: unload(): detaching and deleting GL_FRAGMENT_SHADER shader from program 1
[verbose] ofShader: unload(): detaching and deleting GL_VERTEX_SHADER shader from program 1
//[…]

when using settings.setGLVersion(3, 2) it get erros, with no flickering and no shader response. the video responds the same way with or without the shader is on.

[ error ] ofShader: setupShaderFromSource(): GL_VERTEX_SHADER shader failed to compile
[ error ] ofShader: GL_VERTEX_SHADER shader reports:
ERROR: 0:12: ‘’ : #version required and missing.
ERROR: 0:12: ‘varying’ : syntax error: syntax error
//[…]

here is my code now:

main.cpp

#include "ofMain.h"
#include "ofApp.h"
#include "ofAppGLFWWindow.h"

int main(){
    ofGLFWWindowSettings settings;
    settings.setGLVersion(2, 1);
    ofSetLogLevel(OF_LOG_VERBOSE);

    //--------------------------------------------------------------------------------------screen ui
    settings.setSize(1920, 1080);
    settings.setPosition(ofVec2f(0,0));
    settings.resizable = true;
    shared_ptr<ofAppBaseWindow> mainWindow = ofCreateWindow(settings);

    //--------------------------------------------------------------------------------------screen 2
    settings.setSize(RESx, RESy);
    settings.setPosition(ofVec2f(-RESx,0));
    settings.resizable = false;
    settings.shareContextWith = mainWindow;
    shared_ptr<ofAppBaseWindow> secondWindow = ofCreateWindow(settings);

    shared_ptr<ofApp> mainApp(new ofApp);
    mainApp->setupSecondWindow();
    ofAddListener(secondWindow->events().draw, mainApp.get(), &ofApp::drawSecondWindow);

    //--------------------------------------------------------------------------------------run
    ofRunApp(mainWindow, mainApp);
    ofRunMainLoop();
}

ofApp.cpp

//--------------------------------------------------------------
void ofApp::setup(){
    ofBackground(30);
    ofSetFrameRate(30);
    ofSetVerticalSync(true);
    
    player.setPlayer(ofPtr<ofGstVideoPlayer>(new ofGstVideoPlayer));
    dirVideo.allowExt("mov");
    dirVideo.listDir("videos");
    currentVideo = 0;
    player.load(dirVideo.getPath(currentVideo));

    fbo.allocate(1920, 1080, GL_RGBA);
    fbo.begin();
    ofClear(0);
    fbo.end();
    
    shader.load("shaders/brcosa_GLSL");
    doShader = true;

    //[...]
}
//--------------------------------------------------------------
void ofApp::update(){
    player.update();
    player.setSpeed(speed);
    player.isFrameNew();
    if(doShaderT){
        doShader = !doShader;
    }

    //[...]
}
//--------------------------------------------------------------
void ofApp::draw(){
    fbo.begin();
    ofClear(0,0,0,255);
    player.getTexture().bind();
    if( doShader ){
        shader.begin();
        shader.setUniformTexture("tex0", player.getTexture(), 0);
        shader.setUniform1f("saturation", saturacao);
        shader.setUniform1f("contrast", contraste);
        shader.setUniform1f("brightness", brilho);
    }
    player.getTexture().draw(-player.getWidth()/2, -player.getHeight()/2);
    if( doShader ){
        shader.end();
    }
    player.getTexture().unbind();
    fbo.end();
    
    ofSetColor(255);
    fbo.draw(0,0,RESx,RESy);

    //[...]
}

ofApp.h

    // [...]
    ofVideoPlayer player;
    ofShader shader;
    bool doShader;
    ofxToggle doShaderT;
    ofFbo fbo;
    // [...]
    ofDirectory dirVideo;
    int currentVideo;
    // [...]
    void setupSecondWindow();
    void drawSecondWindow(ofEventArgs & args);
    // [...]

any ideas why it is flickering when compiling with OpenGL 2.1, and why it is not changing the parameters from the shader?

thank you very much! :slight_smile:

#4

aren’t this lines in update:

    if(doShaderT){
        doShader = !doShader;
    }

switching the shader every frame?

#5

it can be the flickering problem! indeed!!

now, using the code below into draw(), i have no video and no shader. everything in black:

    player.getTexture().bind();
    shader.begin();
    shader.setUniformTexture("tex0", player.getTexture(), 0);
    shader.setUniform1f("saturation", saturacao);
    shader.setUniform1f("contrast", contraste);
    shader.setUniform1f("brightness", brilho);
    player.draw(-player.getWidth()/2, -player.getHeight()/2);
    shader.end();
    player.getTexture().unbind();
#6

so you don’t need to bind and unbind the texture, setting it as a uniform is enough. in any case i don’t think that’s the problem. it’s probably something going on in your shader. check if it works with an image or something else but probably it won’t

#7

i think the shader is ok, because i am using into another program with ofVideoPlayer. and, also, i tested with other shaders with same problem.

it is working with ofVideoPlayer the way below:

void ofApp::setShaders() {
    ofEnableNormalizedTexCoords();

    //--------------------------------------------------------------------------------------BrCoSa Shader
    brcosaFbo.begin();
    ofClear(0);

    videoTexture.bind();
    brcosaShader.begin();
    brcosaShader.setUniformTexture("tex0", videoTexture, 0);
    brcosaShader.setUniform1f("saturation", saturacao);
    brcosaShader.setUniform1f("contrast", contraste);
    brcosaShader.setUniform1f("brightness", brilho);

    glBegin(GL_QUADS);
    glTexCoord2f(0,0); glVertex3f(0,0,0);
    glTexCoord2f(1,0); glVertex3f(videoTexture.getWidth(),0,0);
    glTexCoord2f(1,1); glVertex3f(videoTexture.getWidth(),videoTexture.getHeight(),0);
    glTexCoord2f(0,1); glVertex3f(0,videoTexture.getHeight(),0);
    glEnd();

    brcosaShader.end();
    videoTexture.unbind();
    brcosaFbo.end();
    ofDisableNormalizedTexCoords();
}

void ofApp::update(){
    videoVector[currentVideo].update();
    videoPixels = videoVector[currentVideo].getPixels();
    videoTexture.loadData(videoPixels);

void ofApp::draw(){
    //--------------------------------------------------------------------------------------fbo draw + warper mapping
    fbo.begin();
    setShaders();
    fbo.end();

    ofSetColor(255);
    fbo.draw(0, 0, videoTexture.getWidth(), videoTexture.getHeight());
#8

if you are using 0…1 texture coordinates then your shader is prepared for non rectangular coordinates but the gstreamer video player uses rectangular ones, try calling ofDisableArbTex() before creating the player

#9

not working with ofDisableArbTex()
the screen continues black

#10

Hi . Just my 2 cent.
I use ofDrawRectangle(0, 0, ofGetWidth(), ofGetHeight()); before shader.end()

#11

@arturo @Dorald
thank you!
but, i really dont know why by yours ways it didnt work.

i found another strange path to make it work, as below. that is to create an ofTexture, iterate it with player.getTexture() and pass throught GL_QUADS inside shader.begin() and shader.end().

if you understand why on this way is working and not on yours or have any more suggestions to make it simpler, please send me!

//--------------------------------------------------------------
void ofApp::setup(){
    //[...]
    player.setPlayer(ofPtr<ofGstVideoPlayer>(new ofGstVideoPlayer));
    dirVideo.allowExt("mov");
    dirVideo.listDir("videos");
    currentVideo = 0;
    player.load(dirVideo.getPath(currentVideo));
    
    tex1.setTextureWrap(GL_REPEAT, GL_REPEAT);
    tex1.allocate(player.getWidth(),player.getHeight(),GL_RGB,GL_RGBA);
    
    brcosaShader.load("shaders/brcosa_GLSL");

    //[...]

//--------------------------------------------------------------
void ofApp::update(){
    player.update();
    player.setSpeed(speed);
    if (player.isFrameNew()) {
        tex1=player.getTexture();
    }

//--------------------------------------------------------------
void ofApp::setShaders(){
    ofEnableNormalizedTexCoords();
    
    tex1.bind();
    brcosaShader.begin();
    brcosaShader.setUniformTexture("tex0", tex1, 0);
    brcosaShader.setUniform1f("contrast", contraste);
    brcosaShader.setUniform1f("brightness", brilho);
    brcosaShader.setUniform1f("saturation", 5*sin(.5));
    brcosaShader.setUniform1f("alpha", saturacao);

    ofPushMatrix();
    ofTranslate(0,0);
    glBegin(GL_QUADS);
    glTexCoord2f(0,0); glVertex3f(0,0,0);
    glTexCoord2f(1,0); glVertex3f(tex1.getWidth(),0,0);
    glTexCoord2f(1,1); glVertex3f(tex1.getWidth(),tex1.getHeight(),0);
    glTexCoord2f(0,1); glVertex3f(0,tex1.getHeight(),0);
    glEnd();
    ofPopMatrix();
    brcosaShader.end();
    tex1.unbind();
    
    ofDisableNormalizedTexCoords();
}

//--------------------------------------------------------------
void ofApp::draw(){
    
    fbo.begin();
    ofClear(0,0,0,255);
    if(doShaderToogle){
        setShaders();
    } else {
        player.draw(0, 0);
    }
    fbo.end();
    
    ofSetColor(255);
    fbo.draw(0,0,RESx,RESy);

Best! :slight_smile: