Gstreamer in paused mode - frame-by-frame

Hello.

I ran into problems when using ofGstVideoPlayer in paused mode to do a frame-by-frame process with certain video-files.
The video-track in those files is shorter than the file itself. In the attached file: testmovie.mov the 2 audio-tracks are a couple of frames longer than the video-track.
testmovie.zip (60.8 KB)

The problem occurs, when jumping after the last video-frame but before the end of the file. gstreamer waits for a sample which never appears.

try this with attached videofile:

gplayer = new ofGstVideoPlayer;
gplayer->setFrameByFrame(true);

player.setPlayer(ofPtr<ofGstVideoPlayer>(gplayer));
player.loadMovie("testmovie.mov");
player.setFrame(42); // last videoframe is frame 41, the file is 46 frames long

/* setFrame is an async call, we need to wait until async is done -> more about that in another forum-entry*/
sleep(1);

/*hangs*/
player.update();

Looking into ofGstUtils update() function the problem appears to be that gst_app_sink_pull_preroll does never return when pulling a sample form an invalid position.

Surrounding gst_app_sink_pull_preroll with a gst_element_query_position fixes the problem:

GstSample * sample = NULL;

if(isPaused()){

    /*test for valid position*/
    if (gst_element_query_position (GST_ELEMENT (getSink()),GST_FORMAT_TIME,NULL)) {
        sample = gst_app_sink_pull_preroll (GST_APP_SINK (getSink()));
    } else {
        ofLogError("ofGstVideoUtils") << "update(): invalid position";
    }
}else{
    sample = gst_app_sink_pull_sample (GST_APP_SINK (getSink()));
}

/*check for sample*/
if (sample != NULL) {
     // get_buffer, map buffer etc...
}

When using this fix a sample is not pulled from an invalid position, but ofGstUtils::getPosition() function would still return frames, even they are not valid in the video-track. E.g.: Using same file as in the example above, when using player.getCurrentFrame() on frame 42 it would return 42, eventhough frame 42 is in fact not existing in the video-track.
That is because ofGstUtils::getPosition() queries the position of the gstreamer pipeline and not the appsink.
I am not sure what would be a good solution for this situation. If we would distinct here between running and paused state then player.getCurrentFrame() would reflect what happens when trying to call update().

What i mean is this:

float ofGstUtils::getPosition(){
    // omitted code here
    GstElement* element;

    if(isPaused()){
        element = GST_ELEMENT(getSink());
    }else{
        element = GST_ELEMENT(gstPipeline);
    }
    if(!gst_element_query_position(element,GST_FORMAT_TIME,&pos)){
        ofLogVerbose("ofGstUtils") << "getPosition(): couldn't query position";
        return -1;
    }
}

This would distinguish between paused and play-mode, and query the appsink when paused. in above example, this would return -1 when querying the position at frame 42.

Not quite sure what side-effect this has, but it solves the issues when coming across video-files like the one attached.

happy to hear any thoughts about above suggestions.

EDIT:
What i don’t know about gstreamer is how to get the real video-frames in the file.
querying the duration of either the pipeline or the appsink gives back exactly 2 seconds for attached file.

cheers
inx

@arturo have you seen this?