Problem playing URL in GStreamer ofVideoPlayer

linux
#1

I’m playing back direct links to youtube clips, and while 90-95% of them work fine, the occasional one gives the following error when about to play:

[ error ] ofGstVideoUtils: buffer_cb(): error on new buffer, buffer size: 218880!= init size: 218160

The video loads fine otherwise.

string movie = "https://r3---sn-ppoxu-hxal.googlevideo.com/videoplayback?mv=m&source=youtube&itag=18&ms=au&mt=1396240717&requiressl=yes&sparams=id%2Cip%2Cipbits%2Citag%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&ip=203.166.237.73&upn=4eDNnj1DgNc&sver=3&expire=1396262225&ratebypass=yes&fexp=927606%2C945011%2C934804%2C927872%2C916625%2C902545%2C937417%2C913434%2C936910%2C936913%2C934022&ipbits=0&signature=7789A411F8885C5DD1C1BB64109B15814024A4A3.83708F7B62E4D524A3B43E3F77C0AA90E10EC042&key=yt5&id=dc848efa12ad4bb8";
        
video.loadMovie(movie);

Seems like the initially advertised buffer size has changed. I’m not sure if there’s anything that can be done to improve this. It’s hard to ignore them either as they don’t return any errors when loading.

The other thing is that these play fine when pasted into a browser, and Totem plays them fine as well.

#2

yes this is because we calculate the size before starting the video or in the first buffer in the case of streams and never check again except to see if the size is different in which case it fails. we should probably reallocate in that case and change the reported size of the player

#3

I looked into this and it’s a little more tricky. Basically as an example, the video was advertised as being 202x360x3 pixels = 218160, however the actual buffer size is 720 pixels greater, which is not a multiple of three. I tried putting a manual offset in there to see if it would fix it but that didn’t seem to work. I also played around with gst_buffer_resize() but that didn’t work on the readonly buffer coming in.

That link above expires after a day or so it seems so it won’t work anymore. I could get you some example code if you wanted to look into it further, at the moment I’ll just have to make some work around.

#4

i’m super busy right now so won’t be able to take a look but the idea is that you can get a GstVideoFrame from the GstBuffer with: gst_video_frame_map

http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-gstvideo.html#gst-video-frame-map, the videoframe among other things has a GstVideoInfo which gives you all sorts of info on the buffer

http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-gstvideo.html#GstVideoInfo

and would allow to reallocate the internal ofPixels, it would also tell you if there’s any sort of padding, cropping… how many planes it has if any (it shouldn’t have more than 1 being RGB) and the offset and stride of each plane

#5

Info about offsets, strides and padding would be super useful here. I’ll have another look. I’ve hacked around it for the moment but it would be good to fix. Thanks.

#6

Ok I tried to fix this but still no love.

I got the GstVideoInfo this way:

GstPad* pad = gst_element_get_static_pad(getSink(), "sink");
GstCaps *caps = gst_pad_get_current_caps (GST_PAD (pad));
GstVideoInfo vinfo;
gst_video_info_init (&vinfo);
gst_video_info_from_caps (&vinfo, caps);

GstVideoFrame f;
gst_video_frame_map(&f,&vinfo,_buffer,GST_MAP_READ);
cout << "INFO ----------------------------------------------------" << endl;
cout << "Width=" << f.info.width  << " Height=" << f.info.height  << endl;
cout << "interlace=" << f.info.interlace_mode << " size="<< f.info.size << endl;
cout << "stride=" << f.info.stride[0] << " offset="<< f.info.offset[0] << endl;
cout << "flags=" << f.info.flags << " views="<< f.info.views << endl;
cout << "pixel stride =" << f.info.finfo->pixel_stride[0]  << " format flags = " << f.info.finfo->flags << endl;
cout << "video format=" << f.info.finfo->format << " n_components=" << f.info.finfo->n_components << endl;
cout << "shift=" << f.info.finfo->shift[0] << " depth" << f.info.finfo->depth[0] << " plane=" << f.info.finfo->plane[0] << " poffset=" << f.info.finfo->poffset[0] << endl;
cout << "w_sub=" << f.info.finfo->w_sub[0] << " h_sub=" << f.info.finfo->h_sub[0] << " pack_lines=" <<  f.info.finfo->pack_lines <<  " unpack format=" << f.info.finfo->unpack_format << endl;

Basically GStreamer rounds up the stride of a buffer to be divisible by 4. So in the case of a 270x360 image, the stride (i.e.each row of pixels in the buffer) will be 812, not 270x3 = 810. So there are 2 extra bytes used for padding it seems.

However I can’t fix this, I’ve tried to push the pixels in all sorts of ways but still the image is broken (I can see the video but it’s distorted).

For example I’ve tried this (assuming that there is a byte either side of each line of pixels from the image:

int h = f.info.height;
int w = f.info.width;
int stride = f.info.stride[0];

ofPixels    newpixels;
newpixels.allocate(w,h,3);

for(int i=0; i < h;i++) {
    memcpy(mapinfo.data+1+i*stride,newpixels.getPixels()+i*w*3,w*3);
}

Instead of the memcopy, I found this function and tried to use it to replace the memcopy but it didn’t work either:

f.info.finfo->unpack_func(f.info.finfo,GST_VIDEO_PACK_FLAG_NONE,newpixels.getPixels(), f.data ,f.info.stride,0,i,w);

Here’s an example of the output I get if I don’t return prematurely from the ofGstVideoUtils::buffer_cb() and display the image instead:

#7

yeah we need something like this for hw accelerated playback on some arm boards too, i think the videoinfo or the caps packs some crop information, we should pass that to the texture which total width would be the full size of the frame including the paddding then when it draws it creates a quad of the size of the video without the padding, it can also be done with a texture matrix but i think using the dimensions would be much easier.

only problem with that approach is that the pixels will contain the padding but i think it’s ok, if you want to do some computer vision you can set a roi to avoid the padding in opencv

#8

In the meantime, how to get rid of the padding? I just can’t seem to do it.

By the way, running these videos through the RPi OMXPlayer works fine,

#9

ofPixels has a setFromAlignedPixels that should work if you pass the right parameters

#10

Weird, that’s almost exactly what I’m doing in the above code. I’ve tried with a +0, +1 and +2 pixel offset but neither gives any results. I’ll have another look to see if I’m doing something wrong.

#11

Uggh. I had the parameters for memcpy swapped. How embarrassing. Anyhow, I also tried with setFromAlignedPixels() and that works well and is a lot cleaner.

This is a problem for all GStreamer ofVideoPlayer videos that have widths that when multiplied by the number of channels doesn’t equal a multiple of four, so it’s a general problem. I’ll submit my fix with a PR. Thanks!

#12

Hey, is this supposed to be fixed on master ?

I am getting the same error ( for some videos ) if I set the internal pixel format to OF_PIXELS_NATIVE although if I force it to continue by removing the return I get what seems to be the expected result. i.e the video renders with the default I420 format properly.

Thanks for any insights !

Petros

P.S I am working with d216b948 from github.

#13

that might be a bug, can you submit it to the issue tracker?

#14

Will do…

I am actually facing another issue also, related to YUV-RGB conversion when setting OF_PIXELS_NATIVE on the video player .

I will submit both them separately though since I am not sure if they are related .

Thanks.