Tracking colours in frame


I’m working with an openframeworks/opencv colour tracking example that uses the mouse position to determine which hue to track. Can anyone give me an idea of how to track many different colour that appear in the frame without having to click on a position?

so instead of

void testApp::mousePressed(int x, int y, int button) {  
    //calculate local mouse x,y in image  
    int mx = x % w;  
    int my = y % h;  
    //get hue value on mouse position  
    findHue = hue.getPixels()[my*w+mx]  

I would think it would just be



but maybe not so easy?