Memory increases at each rerun, how to free it

I am new to Openframework and GStreamer, and I am trying to build a robust pipeline that can handle two RTSP streams where it stops and replays when needed.

It works fine getting it to stop and replay (I simply run close() on a ofGstVideoUtils object and setup() again), however, when I monitor the memory I can see that it increases each time I rerun the stream.

In the implementation of close(), in ofGstUtils, it seems that the pipeline is closed down and freed correctly, so I’m wondering why the memory increases each time I rerun the stream.

void ofApp::keyPressed(int key){
    switch(key)
    {
        case 'p':
        case 'P':
            setup();
            printf("REPLAY\n");
            break;
        case 's':
        case 'S':
            vutilA->stop();
            vutilB->stop();
            printf("STOP\n");
            break;
        case 'q':
        case 'Q':
            vutilA->close();
            vutilB->close();
            printf("CLOSE\n");
            break;
        default:
            break;
    }
}

void ofApp::setup(){

#ifdef TARGET_OPENGLES
...
#endif

    // ofGstVideoPlayer*
    playerA = new ofGstVideoPlayer();
    playerB = new ofGstVideoPlayer();
    
     // ofVideoPlayer
    movieA.setPlayer(ofPtr<ofGstVideoPlayer>(playerA));
    movieB.setPlayer(ofPtr<ofGstVideoPlayer>(playerB));
    
    // ofGstVideoUtils*
    vutilA = playerA->getGstVideoUtils();
    vutilB = playerB->getGstVideoUtils();

    vutilA->setPipeline("rtspsrc location=rtsp_url1protocols=GST_RTSP_LOWER_TRANS_TCP latency=0 ! rtph265depay ! h265parse ! avdec_h265 ! autovideoconvert", OF_PIXELS_RGB, true);
    vutilB->setPipeline("rtspsrc location=rtsp_url2 protocols=GST_RTSP_LOWER_TRANS_TCP latency=0 ! rtph265depay ! h265parse ! avdec_h265 ! autovideoconvert", OF_PIXELS_RGB, true);
    
    vutilA->startPipeline();
    vutilB->startPipeline();

    vutilA->play();
    vutilB->play();

    ...

In the picture below you can see the program being rerun 4 times where the memory keeps increasing.

Hi @sarah,

I’m thinking that calling .close() may not be freeing the memory associated with it, which should happen when delete is called. So, try calling delete someplace that makes sense, maybe after .close(). Another option is to try using a smart pointer with std::make_shared or maybe std::make_unique; smart pointers manage their own memory:

// in ofApp.h
    std::shared_ptr<ofGstVideoPlayer> playerA;

// in ofApp::setup()
    //    ofGstVideoPlayer playerA = new ofGstVideoPlayer(); // doesn't compile for me at least
    //    ofVideoPlayer playerA = new ofGstVideoPlayer(); // same, doesn't compile

    // the following compiled
    playerA = std::make_shared<ofGstVideoPlayer>();

    ofVideoPlayer movieA;
    movieA.setPlayer(playerA);

    auto vUtilsA = playerA->getGstVideoUtils();

Edit: I edited my code a bit, and put the shared_ptr into ofApp.h

1 Like

Hi @TimChi,

Thank you for your help!

Unfortunately, in both suggestions I get double free or corruption (out) w.r.t. the ofGstVideoUtils object (vutil). Both whenever I try to delete or free it, and if I use the smart pointers as you suggest.

I can delete the player object, and deleting the movie object gives a Segmentation fault.

I have also tried to use vutil->reallocateOnNextFrame();, but I’m not sure if it reuses the memory allocation since the memory still increases - but that could be due to the movie object.

Do you have an idea of the cause of this?

Here’s another sparse example of what I’m doing:


//--------------------------------------------------------------
void ofApp::update(){

    if(!movie)
    {
        player = new ofGstVideoPlayer();
        movie  = new ofVideoPlayer();
        movie->setPlayer(ofPtr<ofGstVideoPlayer>(player));
        vutil = player->getGstVideoUtils();
        vutil->setPipeline("rtspsrc location=rtsp_url protocols=GST_RTSP_LOWER_TRANS_TCP \
                            latency=0 ! rtph265depay ! h265parse ! avdec_h265 ! \
                            autovideoconvert", OF_PIXELS_RGB, true);
        vutil->startPipeline();
        vutil->play();
    }
        
    movie->update();
    vutil->update();
    
    return;
}


//--------------------------------------------------------------
void ofApp::draw(){
    
    ofClear(0, 0, 0,255);
    if(movie)
    {
        movie->draw(0,0,1920,1080);
    }
}


//--------------------------------------------------------------
void ofApp::keyPressed(int key){

    switch(key)
    {
        case 'n':
        case 'N':
            delete player;
            delete movie;      // Segmentation fault
            delete vutil;      // double free or corruption (out)
           
            player = nullptr;
            vutil  = nullptr;
            movie  = nullptr;
            break;
        default:
            break;
    };
}

1 Like

I can’t test or look it up myself right now, but in this case i think ‘player’ is a part of ‘movie’ and ‘vutil’ is a part of ‘player’, so its (probably) enough just to delete ‘movie’.
Take this with a grain of salt though, as i’m unable to check it myself right now. But maybe you can give it a try and see what happens.

Another guess is, that your first approach should be fine with some adjustments as well. Instead of calling setup() again after your call to .close(), something like this could work:

void ofApp::setup(){
    // ofGstVideoPlayer*
    playerA = new ofGstVideoPlayer();
    playerB = new ofGstVideoPlayer();
    
     // ofVideoPlayer
    movieA.setPlayer(ofPtr<ofGstVideoPlayer>(playerA));
    movieB.setPlayer(ofPtr<ofGstVideoPlayer>(playerB));
    
    // ofGstVideoUtils*
    vutilA = playerA->getGstVideoUtils();
    vutilB = playerB->getGstVideoUtils();
  
    play();
}

void ofApp::play() 
{
    vutilA->setPipeline("rtspsrc location=rtsp_url1protocols=GST_RTSP_LOWER_TRANS_TCP latency=0 ! rtph265depay ! h265parse ! avdec_h265 ! autovideoconvert", OF_PIXELS_RGB, true);
    vutilB->setPipeline("rtspsrc location=rtsp_url2 protocols=GST_RTSP_LOWER_TRANS_TCP latency=0 ! rtph265depay ! h265parse ! avdec_h265 ! autovideoconvert", OF_PIXELS_RGB, true);
    
    vutilA->startPipeline();
    vutilB->startPipeline();

    vutilA->play();
    vutilB->play();
}

void ofApp::keyPressed(int key){
    switch(key)
    {
        case 'p':
        case 'P':
            play();
            printf("REPLAY\n");
            break;
        case 's':
        case 'S':
            vutilA->stop();
            vutilB->stop();
            printf("STOP\n");
            break;
        case 'q':
        case 'Q':
            vutilA->close();
            vutilB->close();
            printf("CLOSE\n");
            break;
        default:
            break;
    }
}

Again, this is just a guess. I’m unable to check properly right now.

1 Like

Hi @marco_v,

Thank you so much! You are right about the movie object and it doesn’t cause a segfault when I delete that and not the player, however, the memory still increases at each run (although slightly less at a time).

And it’s a nice code suggestion you have, but the memory also keeps increasing with it.

It is very strange that the memory keeps increasing like this.

Well. I don’t have a lot of experience with GStreamer and basically no experience with custom pipelines. Although it looks like you don’t need an ofVideoPlayer instance at all? See here.

Maybe this helps. If not, i’m sure someone else, who is more experienced with Gst, will help you out sooner or later here.

2 Likes

Hey I like the solution marco_v found in the link, as the ofGstVideoPlayer is an object instead of a pointer, so there’s nothing delete when it does out of scope. But I’m pretty inexperienced with GStreamer stuff too.

It seems like new players are getting created and the old ones are hanging around as they have not been deleted or otherwise destructed. Using a player (or even a few) as an object for the lifetime of the application should work. If it needs to be on the heap instead of the stack, I’d vote to use a smart pointer to manage it.

1 Like

maybe related:

1 Like

Thank you for all your input @marco_v , @TimChi and @dimitre!

I have now tried the recommended solution from the link @marco_v shared, and while it’s a shorter and nicer solution, it still causes the memory to increase. The same goes for the solution using smart pointers, which is also a nice suggestion.

I have tested the GStreamer pipeline outside OP and it doesn’t create any memory leaks, so I’m thinking that it must be the way OP handles GStreamer pipelines.

Do any of you know if you can read in a stream of bytes in OP? Perhaps I can manually create the pipeline and process it as a stream of bytes, rather than a GStreamer object/pipeline. What do you think? Do you have other suggestions?

Sorry, im out of ideas then. Maybe you can look through the oF GStreamer code to see if you find the leak yourself.

I usually handle everything video related with FFmpeg. In your case, I’d probably spawn a FFmpeg process to receive the frames and pipe the outgoing frames to my oF app. Not trivial, and i’ve never needed/tried it, but it will probably work.

1 Like

Hi again,

I have solved the problem!

I went away from using OP’s built in handlers for GStreamer (GstVideoUtils etc.) and implemented the whole GStreamer execution manually, and then I transfer the stream using appsink and load it into a texture using GstBuffer, GstMapInfo and ofTexture.loadData (note that this should be done in ofApp::update).

Between each rerun, I close down and remove all resources.

In addition I use a thread to run this process such that the GStreamer pipeline doesn’t block anything, and it all runs smoothly without any memory leaks.

Just wanted to share in case anyone runs into the same issues in the future :slight_smile:

2 Likes

Beautiful! Glad you solve it.
if you share the code in the future I’ll be using it too. :slight_smile:

Hey Sarah I’m glad you figured out something that works! If you’re up for it, you could post your code here, or turn it into a addon that others could use, or maybe even review the Gst headers in oF to see if you can figure out if the leak is coming from there even propose a fix for it. oF has lots of development support from the people who use it and find ways to improve it over time.