Workload distribution

Hello! I am trying to use the camera to create generative meshes. At the moment I am using the Lightness values from the camera but eventually it will be based on other parameters. In the test implementation (inspired by Meshes chapter in OF Book) I can see that my cpu (intel 10th gen) can’t really handle the load and it is laggy at times.

I was wondering if there is a way to distribute tasks inside a loop (for/ if) to different threads.

Here is my example code:

 void ofApp::update()
{
    grabber.update();
    if (grabber.isFrameNew()) {
        for (int i = 0; i < width; i++) {
            for (int j = 0; j < height; j++) {
                ofColor c = grabber.getPixels().getColor(i, j);
                float intensity = c.getLightness();
                if (intensity >= intensityThreshold) {
                    glm::vec3 pos(i, j, 0.0);
                    mesh.addVertex(pos);
                    mesh.addColor(c);
                }
            }
        }
    }
}

Maybe I could use two other threads- one to collect pixels and get lightness values, and antother to distribute to the Mesh object.

I understand how to use a mutex to do different tasks on different threads but I do not know how to distribute same task onto different threads.

Threading is just my way of managing cpu workload, if there are more efficient ways of managing, please suggest.

Thank you in advance!

Hey a parallel-for loop (from the TBB library) might be super helpful to parallelize this task. There is a thread about using it here (along with other threading options): Parallel for-loop. With your example above, each thread would take a column and add vertices to its own ofMesh (1 per column), which could be stored in a std::vector. These could then be combined a single ofMesh.

Sometimes using a shader to process a texture (image or data) can be super helpful, but maybe not so much in this case.

Hi, why not, instead of adding vertices and colors to the mesh on every frame, you just create a fixed size mesh and then update its vertices and colors. Also, if you use an ofVboMesh it would be a lot faster. There are several examples in the examples/gl folder.
Then, the getColor function can be slowing you down, as well as the getLightness.
If you check the getLightness() method, you’ll see that it simply averages the rgb values, thus, what you can do is

void ofApp::update()
{

float intensityThreshold3 = intensityThreshold*3;// to avoid this multiplication on every iteration of the loops
    grabber.update();
    if (grabber.isFrameNew()) {

auto * pixels = grabber.getPixels().getData();
            for (int i = 0; i < height* width; i += 3) { // using a single for loop makes it faster. notice that the 3rd parameter, the update, is i+=3 instead of the usual i++ as I am jumping 3 values each time, yet it is below that I read the pixels.
                if (pixels[i]+ pixels[i+1] + pixels[i+2]  >= intensityThreshold3 ) {
                    glm::vec3 pos(i, j, 0.0);
                    mesh.addVertex(pos);
                    mesh.addColor(ofColor(pixels[i]+ pixels[i+1] + pixels[i+2]));
                }
            }
        }
    }
}

hope this helps somehow.

1 Like

Also, I was wondering why this was running slow on a newer processor. A lack of threads may not be the problem. As @roymacdonald points out, the mesh is accruing vertices and colors every frame. So, the vectors in the mesh could quickly become quite large, like really huge. A lot of the vertices would presumably have the same coordinates, but with different colors. Is this what you are wanting to happen?

When vectors outgrow the memory they’ve been allocated, they copy themselves to a another location with a larger allocation of contiguous memory. The bigger they are, the more resources it takes for them to relocate themselves. You could try std::vector::reserve() to reserve enough storage for the vectors in the mesh in ofApp::setup().

A lot depends on how many vertices are added each frame. Perhaps the vertices and colors need to be resized or cleared if they exceed a certain size? Or as Roy suggested, you could fix the size of the mesh and update it with new vertices and colors each frame, either in whole or in part.

You could also draw a much smaller mesh (just the vertices from 1 frame of the grabber) into an ofFbo every frame, without clearing it, as a way to “aggregate” the mesh representations of the grabber over time. This might be a nice, easy, accurate substitute for an ever-growing and eventually super-huge mesh.

Hi, littel late response, sorry about that.

@TimChi, @roymacdonald thank you so much for your advice. I am now using ofVboMesh and I am creating the required vertices+indices in update, destroying them after draw so that they dont accum to millions…
I am also using the getData method as Roy suggested to save more cpu.
I am also toying with the idea of conditionally adding/removing vertices when there is “significant” change in the video frames content.

Thank you so much folks! this is slowly shaping up