Sampling color from multiple layered videos


#1

Hello,

I’m brand new to OF and am trying to learn by porting a processing sketch. I have multiple videos w/ alpha channels running on top of each other. My goal is to sample color data from multiple points of the overall image while the videos are layered.

Right now, using getColor I can sample color from one of the videos, but instead of sampling from each video individually i’d like to sample whatever shows up on top while the app is running. Could I create some sort of overall class to grab the pixels from instead of the individual videos? Thanks

ofVideoPlayer video, video1, video2;

void ofApp::setup(){
    video.load("black_background.mov");
    video1.load("lightness_Converted.mov");
    video2.load("gray.mov");
}

//--------------------------------------------------------------
void ofApp::update(){
   video.update();
   video1.update();
   video2.update();
    
    ofPixels pix = video1.getPixels();
    ofColor c = pix.getColor(mouseX, mouseY);
    float brightness = c.getBrightness();
    cout << "Brightness: " << brightness << endl;
    
}

#2

“whatever shows up on top while the app is running”. Do the video have the same dimensions? if yes, how do you decide which video to draw last? if no, you could find which picture is under the mouse, getting the coordinates of the image and the dimension.


#3

Yes, the videos are the same dimensions.

When the videos pixels overlap the last video drawn is shown ‘on top’.


So in this example one video is white pixels and one video is gray pixels.


#4

It is probably easier to draw all the videos in an fbo and then get the color from the fbo.
In the ofApp you add a line ofPixels pixels;
In the setup method you allocate the fbo and you clear it. You allocate also the pixels. Pixels and fbo have the same dimensions of the video. In the update method you clear the fbo, in the draw method:

fbo.begin();
video1.draw();
video2.draw();
fbo.end();

fbo.readToPixels(pixels);
auto c = pix.getColor(mouseX, mouseY);

Another solution is to sample the color starting from the last thing drawn on screen, in your case video2 (the white one). If the pixel color is white, you return the color. If it is black, it means that you did not hit any squares in the first video and you have to sample the second video, looking for a color that is not black. If you find it, you return it, if not, it means that your mouse is over a black area.