ofxAVFVideoPlayer

announcements
#1

Hi all,
I’ve been working on an addon that uses the most recent video playback library on the Mac, AV Foundation. For a project I’m working on at Sosolimited we ran into lots of performance and threading issues with the QTKit video player, so I looked around and found that AV Foundation seems to be Apple’s preferred framework moving forward. And it seems to work great.

Check out the addon here: https://github.com/kronick/ofxAVFVideoPlayer

It’s a different implementation than what’s used in ofxiPhone-- the main difference is pixels are never copied from the video card to the CPU unless you call getPixels(). Instead it natively renders a CALayer into an FBO using a technique I randomly found in the QT project (the windowing framework, not QuickTime) source-code. This should be much faster and allows an implicit “texture only” rendering mode without having to declare it during initialization as you do with ofQTKitVideoPlayer. I’m getting acceptable performance playing back 10 H.264 full HD videos on a recent iMac, which is better than anything else I’ve tried.

I don’t think I’ve implemented the full interface yet of ofVideoPlayer (really just focused on what I needed first) but maybe you can fork and add in the bits that it’s missing. Hope it’s useful!

Sam

3 Likes
Video playback stability - OSX 10.9 vs 10.8 (or earlier) QTKit vs QT 7
#2

thank you a lot kronick!
i used your AVF-Player a lot and can confirm a very good performance!

greetings ascorbin

#3

Hey,

I’m having trouble trying to get the ofxAVVideoPlayer drawing all of my video - it only appears to draw the upper right quarter, I’ve played around with the x,y,h,w values but don’t seem to be able to get it drawing as intended.

I wondered if you knew what ay be causing this?

Thanks,

A

#4

@kronick do you have a working example of your extension playing a video with alpha channel? I used your extension to display a regular video and it works very well, but when I try to load one with alpha all I get is a white rectangle. Of course I set pixelFormat to RGBA before loading the movie.

I tried all the movie formats I know about that support alpha but no one worked :frowning:

Any idea?

#5

OK I’ll answer this myself. It seems that the only video codec supporting alpha channel on Mavericks is Apple ProRes 4444. It is a high quality format, and produces large files (about 2 - 4 times the original .mov in my tests) but works like a charm and lets you do alpha blending with existing background :smiley:

All other formats (like animated png and so) are not supported anymore by Apple. Just a word to future readers, be careful about the tool you use to convert existing files, many 3d party utilities will misbehave with alpha channel or happily ignore it. I got successful results using apple’s “compressor”.

#6

Hello @agibson,
I think I’m having the same problem as you.
Only part (a top corner) of the loaded video is being rendered / displayed.
I’m playing a rather large video (2470x2048 h264).
The important information is that the same project worked perfectly in OSX 10.6.8.
Have any of you guys experienced the same problem and have any solution in mind?
Thanks,
Rodrigo
ps.: I’ve tried to make the [change][1] suggested by Kunsberg but it didn’t solved the problem.
[1]: https://github.com/kronick/ofxAVFVideoPlayer/issues/2

#7

Hmm, ofxAVFVideoPlayer doesn’t support OS X 10.6 (AV Foundation didn’t land on OS X until 10.7), so I’m guessing you used a different player previously, no?

Could you post your drawing code?

#8

Hey,

I think what I ended up doing was duplicating the example project and just tearing out all the code and starting fresh.
Certainly not ideal, as I never discovered what was causing the problem, but at least I have all of the video being drawn!

Sorry I can’t be of more help.

#9

Hmm, ofxAVFVideoPlayer doesn’t support OS X 10.6 (AV Foundation didn’t land on OS X until 10.7), so I’m guessing you used a different player previously, no?

Oops, sorry, in my previous post I gave you the wrong version.
The OS X version of the project that’s working is 10.7.5.

#10

Hi guys, firstly thanks for such an awesome video addon. I am getting really fantastic results using this. I have a slight issue though. When I am playing a video and then call the code below to replace the video with a new one I get a frame or 2 of a white rectangle before the video starts to play. I cant seem to work out how to get rid of this. Is anyone else experiencing this?

video.close();
video.setPixelFormat(OF_PIXELS_RGBA);
video.loadMovie(paths.at(_fileID));
video.play(); 
video.setLoopState(OF_LOOP_NORMAL);
#11

Just a guess but could it be a delay due to the time needed to load the movie? AFAIK this player does not stream from disk but loads all the movie to ram before playing it.

#12

hi!
I made some modifications to ofxAVFVideoPlayer (as ‘kunzberg’ on github), positioning should work from 10.7 to 10.9, and some sound implementation are in.
greetings

Archiv.zip (6.0 KB)

1 Like
#13

Hi ascorbin,

Great additions to kronick’s work! After replacing the files, cleaning my project (took some time before I realized this was necessary :p) and rebuilding, the image is now positioned properly. I see you also implemented more functionality like the setSpeed method.

Shouldn’t your additions be added to kronick’s repository, perhaps you could make a pull request?

One suggestion, in analogy with the ofQTKitPlayer functionality, perhaps you could change the play() function to:

void ofxAVFVideoPlayer::play() {
    [moviePlayer setMyRate:speed];
    [[moviePlayer player] setRate:speed];
    [moviePlayer play];
}

Thanks for your work guys!

#14

Glad to see that ofxAVFVideoPlayer is being integrated into OF’s core! I have a small code suggestion that will make looping video play more smoothly without noticable hiccups. It really has proven itself in my app, hopefully someone could check it and add it to the OF sourcetree. (I do not master the powers of git, sorry)

in /libs/openFrameworks/video/ofAVFMovieRenderer.m, add in the init function (line 48):

self.player.actionAtItemEnd = AVPlayerActionAtItemEndNone;

and at line 244, this function:

- (void)playerItemDidReachEnd:(NSNotification *)notification
{
    _bMovieDone = YES;

    if (self.bLoops) {
        // Start over.
        _bMovieDone = NO;
        [self stop];
        [self play];
    }
}

can be replaced by:

- (void)playerItemDidReachEnd:(NSNotification *)notification
{
    _bMovieDone = YES;
    
    if (self.bLoops) {
        //start over
        _bMovieDone = NO;
        AVPlayerItem *p = [notification object];
        [p seekToTime:kCMTimeZero];
    }
}

This should do the trick, thanks all!

#15

Another thing I did was commenting out the close functioning loadMovie but initializing the player only if it is not initialized already. Like this, sequential loading of new clips drop down from 20ms to 1ms on my MBP, which makes a threaded solution not necessary anymore.

//--------------------------------------------------------------
bool ofAVFoundationPlayer::loadMovie(string path)
{
  - if (bInitialized) {
  -      close();
  -  }
    
    @autoreleasepool {
       + if (!bInitialized) {
            moviePlayer = [[AVFMovieRenderer alloc] init];
            [moviePlayer setUseAlpha:(pixelFormat == OF_PIXELS_RGBA)];
            [moviePlayer setUseTexture:YES];
        +}


        if (Poco::icompare(path.substr(0, 7), "http://")  == 0 ||
            Poco::icompare(path.substr(0, 8), "https://") == 0 ||
            Poco::icompare(path.substr(0, 7), "rtsp://")  == 0) {
            [moviePlayer loadURLPath:[NSString stringWithUTF8String:path.c_str()]];
        }
        else {
            path = ofToDataPath(path, false);
            [moviePlayer loadFilePath:[NSString stringWithUTF8String:path.c_str()]];
        }
    }
    
    bShouldPlay = false;
    return true;
}

I’m currently testing this in a loop, it seems to run ok with master. Don’t know the long term consequences, if any…

best,
Urs

#16

Hey i’m trying to get a prores 444 video with an alpha channel to play but it doesn’t seem to display the alpha channel. This is what i’m using:

// in setup
    proResVid = new ofxAVFVideoPlayer();
    proResVid->setPixelFormat(OF_PIXELS_RGBA);
    proResVid->loadMovie("movies/prores.mov");

// in update
 proResVid->update();

      if(proResVid->isLoaded()) 
                proResVid->play();
    

// in draw
    ofSetColor(0,255,0);
    ofRect(0,0,1280,720);
    ofEnableAlphaBlending();
	ofSetColor(255,255,255);
    proResVid->draw(0,0,1280,720);

But I can’t see the green square behind my image?

Any ideas?

#17

Hey, I’m having the same issue.
Trying to play a prores 4444 video in a OS X 10.9.4
I’ll work on a simplified version of my code (it’s in the middle of a fairly complex project…), but I think it’ll look the same as yours.

Hope someone helps us, hehe

Claudio

#18

A found a solution for my problem today, it was actually a bit dumb.
I didn’t have the decoder for ProRes installed.
You can download it here:
http://support.apple.com/kb/dl1396

Now everything works perfectly =)

#19

Does it work on iOS?