vidGrabber with shader

beginners
#1

I’m trying to manipulate frames from webcam but seems my input has issues.
I could pass texture from oF to shader, but the input is flipped and a little bit glitchy

49%20PM
Screen Shot 2019-07-20 at 5.47.49 PM.png1024×768 683 KB

Any suggestions are really appreciated.

Also would like to get advice about u_resolution.
I’m writing this, but feel this isn’t a good way.

    float fw =ofMap(w, 0, w, .0, 1.0);
    float fh =ofMap(h, 0, h, .0, 1.0);
    shader.setUniform2f("u_resolution", fw, fh);

My code is the following

ofApp.cpp

//--------------------------------------------------------------
void ofApp::setup(){
    w = ofGetWidth();
    h = ofGetHeight();
    vidGrabber.setDeviceID(0);
    vidGrabber.initGrabber(w, h);
}

//--------------------------------------------------------------
void ofApp::update(){
    vidGrabber.update();
}

//--------------------------------------------------------------
void ofApp::draw(){
    shader.load("", "test.frag");
    shader.begin();
    shader.setUniform1f("u_time", ofGetElapsedTimef());
    float fw =ofMap(w, 0, w, .0, 1.0);
    float fh =ofMap(h, 0, h, .0, 1.0);
    shader.setUniform2f("u_resolution", fw, fh);
    shader.setUniformTexture("tex0",vidGrabber.getTexture(),0);
    ofDrawRectangle(0,0, ofGetWidth(), ofGetHeight());
    shader.end();
}

fragment shader

uniform vec2 u_resolution;
uniform sampler2DRect tex0;

void main(){
    vec2 uv=gl_FragCoord.xy/u_resolution.xy;
    gl_FragColor=texture2DRect(tex0, uv);
}
#2

your first piece of code will always give the same result because w (input) is the same of the third parameter.

I’ve seen you are initializing the camera with your windows dimensions, so maybe it is the case that your window dimensions are outside of the ranges supported by the camera. Can you try it being 640x480?

1 Like
#3

Hi @dimitre,
Thank you for your reply.
Tried 640x480, but still, the output is flipped (y-axis).

#4

In this lines you are actually sending (1.0, 1.0) to the shader.
Maybe you wanted to send different values, as (640, 480)

    float fw =ofMap(w, 0, w, .0, 1.0);
    float fh =ofMap(h, 0, h, .0, 1.0);
    shader.setUniform2f("u_resolution", fw, fh);
#5

After seeing your comment. I changed the code a little bit.
Then it works.

If I send (640, 480), I need uv*=vec2(640.0, 480.0) to normalize, right?

#6

Some comments on your code :

  • In your setup, you assume that your Webcam can provide a video with the resolution of your window.
vidGrabber.initGrabber(w, h);

However your webcam will try to find the resolution that best match the provided resolution. So your vidGrabber texture will surely not be wxh. You can get the dimensions with vidGrabber.getHeight() and vidgrabber.getWidth().

  • You should have your shader.load() in the setup() instead of draw().

  • Instead of using ofDrawRectangle() and setting the vidGrabber texture as a uniform, you could directly use the draw() method of the videoGrabber.

  • If you use some vertex shader, it will take care of the flip.

So I end up with

//--------------------------------------------------------------
void ofApp::setup(){
    w = ofGetWidth();
    h = ofGetHeight();
    vidGrabber.setDeviceID(0);
    vidGrabber.initGrabber(w, h);
    shader.load("test.vert", "test.frag");
    shader.setUniform2f("u_resolution", vidGrabber.getWidth(), vidGrabber.getHeight() );
}

//--------------------------------------------------------------
void ofApp::update(){
    vidGrabber.update();
}

//--------------------------------------------------------------
void ofApp::draw(){
    shader.begin();
    shader.setUniform1f("u_time", ofGetElapsedTimef());
    vidGrabber.draw(0,0,ofGetWidth(),ofGetHeight());
    shader.end();
}

test.vert

#version 120

varying vec2 texCoordVarying;

void main()
{

    texCoordVarying = gl_MultiTexCoord0.xy;
    // send the vertices to the fragment shader
	gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
}

test.frag

#version 120

uniform sampler2DRect tex0;
uniform vec2 u_resolution;

varying vec2 texCoordVarying;

void main()
{
    gl_FragColor = texture2DRect(tex0, texCoordVarying.xy);
    // If use gl_FragCoord, the texture is flipped...
    //gl_FragColor = texture2DRect(tex0, gl_FragCoord.xy);
}
1 Like
#7

I would simply flip vertically the texture back in the fragment shader. Do not use ofMap, it is just a complication more.

In your frag shader, simply:

    vec2 coord = vTexCoord;
    // flip the coordinate system vertically
    coord = vec2(coord.x, 1.0 - coord.y);
    vec4 colCam = texture(tex0, coord);
1 Like