Rpi4 + ofx frame rate sluggish

Like said in the note, I’m having FR issues on my pi4, I’ve not added a patch before and would like to know where / what file I should modify.

This thread had this solution, I tried it, but no performance change.
Ofx slow on rpi4
Just modify main.cpp with this?


 ofGLFWWindowSettings msettings;
    msettings.setSize( 1024, 768 );
    msettings.numSamples = 0; 
    auto mainWin = ofCreateWindow( msettings );

    auto app = make_shared<ofApp>();
    ofRunApp( mainWin, app );
    ofRunMainLoop();

Documentation here

  • Currently the 0.11.0 release has 4x Anti Aliasing as default for the GLFW window. This can cause quite a big hit to framerate. 0.11.1 Patch release will fix this, but you can also set the window settings manually in main.cpp to set numSamples to 0.

FYI - The patch/fix should be enabled in the 0.11.1 or 0.11.2 release.
So changing the main.cpp for releases later than 0.11.0 shouldn’t make any difference.

This is the fr im getting running the basic polygon example full screen.

Taken from the Rpi documentation
I guess I downloaded 11.0, is there a way to update the ofx on rpi?

cd
wget https://openframeworks.cc/versions/v0.11.0/of_v0.11.0_linuxarmv6l_release.tar.gz
mkdir openFrameworks
tar vxfz of_v0.11.0_linuxarmv6l_release.tar.gz -C openFrameworks --strip-components 1

I added the patch and got desired frame rate for the example polygon sketch.

Unfortunetly for my computer vision installation my fr is down at 5fps, if I run the basic ofx background differencing cv I get 30fps.

Any advice on speeding it up?

Also, how could I set full screen mode from the patch?

For CV stuff I would look at making the images you are working with as small as possible.
ie: if you have a 1024x768 image / video try resizing it down before you do any CV stuff.

Also ofxOpenCv images each have a texture. If you are not drawing the CV image you can save some processing with ofxCvGrayscaleImage::setUseTexture(false);

In main.cpp change OF_WINDOW to OF_FULLSCREEN

1 Like

I decimated my grey image by 0.25 but it hasnt affected the fps, I think its the Haar face tracker causing the issue

made this image smaller with this code

void VideoPlayer::update(){
    
   
        vidPlayer.update();
   
    
    //Do computing only if the new frame was obtained
    if ( vidPlayer.isFrameNew() ) {
        //Getting a new frame
        image.setFromPixels( vidPlayer.getPixels() );
        
        //Convert to ofxCv images
        ofPixels & pixels = vidPlayer.getPixels();
        image.setFromPixels( pixels );
        
        float decimate = 0.25;              //Decimate images to 25% (makes calculations faster + works like a blurr too)
        ofxCvColorImage imageDecimated1;
        imageDecimated1.allocate( image.width * decimate, image.height * decimate );
        imageDecimated1.scaleIntoMe( image, CV_INTER_AREA );             //High-quality resize
        grayImage = imageDecimated1;

  
        

can anything be reduced here?

void FaceDetector::setup(){
   
	finder.setup("haarcascade_frontalface_default.xml");

    videoGrabber.setDeviceID(0);
    videoGrabber.setDesiredFrameRate(30);
    videoGrabber.initGrabber(320, 240);
    bool faceDet = false;
}

//--------------------------------------------------------------
void FaceDetector::update(){
    nFacesDetected = finder.blobs.size();
    finder.setScaleHaar(1.1);
    videoGrabber.update();

    if(videoGrabber.isFrameNew()){
        finder.blobs.clear();
       
        finder.findHaarObjects(img);
        img.setFromPixels(videoGrabber.getPixels());
    }
}

Here is the sketch running on my Mac

1 Like

maybe ?:

videoGrabber.initGrabber(160, 120);
videoGrabber.setUseTexture(false);  // then you won't see the video 

Also your update order is a little strange.
This would make more sense:


    videoGrabber.update();

    if(videoGrabber.isFrameNew()){     
        img.setFromPixels(videoGrabber.getPixels());
        finder.findHaarObjects(img);
        nFacesDetected = finder.blobs.size();
    }

And move:

    finder.setScaleHaar(1.1);

to after finder.setup();

But not sure if you will get much performance from any of the above.
You might want to check you are building in Release and not Debug.

You could run the haar finder in a separate thread at a slower rate, but that would be a more complex solution.

It might be good to comment out the finder code and see how fast the video grabbing is running before trying any threading.

Hope that helps!
Theo

1 Like

Thanks for the advice, I will tidy up the code like you suggested.
When I run the ofx open cv examples like Haar and frame differencing im getting 35-50fps.

I will start from the bottom up and slowly build sketch to find the issue.
:slight_smile:

I tidied up the code and have found its the ps3 eye and Haar face detection thats slowing it down. Do you have any tricks to help?

Currently im running from the rpi video drivers for the ps3eye. Would the addon help or perhaps a different camera?

Its not possible to run the resolution at 160 X 120 either.

#include "FaceDetection.hpp"


//--------------------------------------------------------------
FaceDetection::FaceDetection(){

    finder.setup("haarcascade_frontalface_default.xml");
    videoGrabber.setDeviceID(0);
    videoGrabber.setDesiredFrameRate(30);
    videoGrabber.initGrabber(320, 240);
    videoGrabber.setUseTexture(false);  // then you won't see the video
    finder.setScaleHaar(1.1);
    bool faceDet = false;
}

//--------------------------------------------------------------
void FaceDetection::update(){
    
    videoGrabber.update();
    
    if(videoGrabber.isFrameNew()){
        img.setFromPixels(videoGrabber.getPixels());
        finder.findHaarObjects(img);
        nFacesDetected = finder.blobs.size();
    }
    

}

//--------------------------------------------------------------
void FaceDetection::draw(){
    ofSetColor(255);
    ofNoFill();
    img.draw(0, 0);
    
    ofSetLineWidth(3);
    for(int i = 0; i < finder.blobs.size(); i++) { // loop over all the found faces/blobs
        cur = finder.blobs[i].boundingRect;     // and put a rectangle around the face
        proximity = ofMap (finder.blobs[i].boundingRect.width, 0, 200, 0,10); // map closeness to installation
        cout <<proximity<<endl;
        ofDrawRectangle(cur);
    }
    
    
    if(finder.blobs.size() > 0 ) {
        
        faceDet = true;
        millis = ofGetElapsedTimeMillis();
    }
    
    if (ofGetElapsedTimeMillis() > (millis + 2000))
    {
        faceDet = false;
    }
    
    
    
   // cout<<faceDet<<endl;
    


}

//--------------------------------------------------------------
void FaceDetection::installation(){
    ofSetColor(255);
    ofNoFill();
    
    // blobs, bounding box and mapping proximity
    ofSetLineWidth(3);
    for(int i = 0; i < finder.blobs.size(); i++) { // loop over all the found faces/blobs
        cur = finder.blobs[i].boundingRect;     // and put a rectangle around the face
        proximity = ofMap (finder.blobs[i].boundingRect.width, 0, 200, 0,10); // map closeness to installation
        
    }
    
    // is there a face present boolean
    if(finder.blobs.size() > 0 ) {
        
        faceDet = true;
        millis = ofGetElapsedTimeMillis();
    }
    
    if (ofGetElapsedTimeMillis() > (millis + 2000))
    {
        faceDet = false;
    }
    
    
    
    cout<<faceDet<<endl;
    
}

//--------------------------------------------------------------
void FaceDetection::facesDetected(){
    
    if (counter%60==0){
        nFacesDetected = finder.blobs.size();
    }
    counter++;
}
//--------------------------------------------------------------

cheers

Hmmm.
You could try lowering the videoGrabber frameRate to 10 or 15 and see if that does anything.

Are you running in Release?

You could add this line and see if it helps too:

if(videoGrabber.isFrameNew()){
    img.setFromPixels(videoGrabber.getPixels());
    img.resize(img.getWidth()/2, img.getHeight()/2);  //<--  resize ofPixels by 50%
    finder.findHaarObjects(img);
    nFacesDetected = finder.blobs.size();
}

Also some useful info here:

1 Like

Yh im running in release. I will try what you suggested, thanks!

Regards
Chris Courage

I used the ofxCvGrayscaleImage::warpIntoMe()

 if(videoGrabber.isFrameNew()){
        colorImg.setFromPixels(videoGrabber.getPixels());
        init = colorImg;

        ofPoint src[4];
        src[0] = ofPoint(0, 0);
        src[1] = ofPoint(320, 0);
        src[2] = ofPoint(320, 240);
        src[3] = ofPoint(0, 240);

        ofPoint dst[4];
        dst[0] = ofPoint(0, 0);
        dst[1] = ofPoint(160, 0);
        dst[2] = ofPoint(160, 120);
        dst[3] = ofPoint(0, 120);

        reSize.warpIntoMe(init, src, dst);



        img.setFromPixels(reSize.getPixels());
        finder.findHaarObjects(img);
        nFacesDetected = finder.blobs.size();
    }

FR upto 20-25fps

@theo Thanks for the advice, finally have a working installation :slight_smile:

Final thing, I need the sketch to run full screen with no window surround as it ruins the magic. Is it possible to chnge the colour or open in a different type of window?

Im sure it opened up and filled the screen when I didnt use the patch

installtionTest

Glad it is working faster!!!
btw this should be faster than warpIntoMe

//assumes reSize is allocated to be a smaller size than init 
reSize.scaleIntoMe(init); 

For the window issue.
Do you have OF_FULLSCREEN in your main.cpp?