Color informations of multiple objects

Hej,
i have a question about the best approach about one of my projects.
I need to get the color information of 900 physical squares that changing their color.
The little squares are arranged in a bigger square so basically it is a “display” with the resolution of 30x30.
I want to place a camera in front of the plane and it should finde the array of squares without manual calibration.
One good point is that i just need to track one at a time but alle at once would also be nice.
So my idea right now is to find the array and just get the bounding box and divide it by 30 to get the center position and then get the average color of the surrounding pixels. But i’m not sure if that is maybe to imprecise.
And i probably have to do some keystone.
Is there anyone how did something like that?
Where should i start?
I’m thankful for every tip!
Best
Florian

use can use https://github.com/julapy/ofxQuadWarp to unwrap your video feed.

i would then cycle through all your pixels

ofPixels & myPixels = bikeIcon.getPixels();

for(int y = 0; y < h; y++) {
    for(int x = 0; x < w; x++) {
        int index = y * w + x;
        
// ofColor pixColor = myPixels.getColor(x, y);
        ofColor pixColor = myPixels.getColor(index);

        ofSetColor(pixColor);
        ofDrawCircle(x * diameter,y * diameter, diameter / 2);
    }
}