Multiple ofWMFoundationPlayer in single oF instance, juggling 4K videos/memory release causes crash - options?

(this question is replicated from an email thread between myself and Mr. James George of the CLOUDS project. I’m replicating it here so he can respond to me and the community at once)

I’m employing the feature-videobuffer branch of CLOUDS-Interactive-Documentary/openFrameworks/tree/feature-videobuffer for hardware-accelerated 4K h264 decoding on Windows, as this branch has an amazing 64-bit Direct3D ofWMFoundationPlayer (which I’m sure is planned for eventual release in oF x64 stable).

However, I’ve run into a nontrivial roadblock.

I’m trying to get an app that has 3 portrait instances of ofWMFoundationPlayer to load/switch/mutex other 4K videos on-the-fly. The issue that must be familiar to some here: the non-threadsafe nature of the GPU and oF’s video.draw() calls cause a bad memory access crash when the video player attempts to do a player->lockSharedTexture() after said texture has already been released.

With a very limited timeframe in mind, what’s the best way I can try to handle this? Some ideas that come to mind:

  • Option 1. A dedicated ofWMFoundationPlayer allocated for every video file I try to play back. Create a dynamic vector of shared_ptr<ofWMFoundationPlayer>. I swap between only calling update()/draw() on the “active” ofWMFoundationPlayer. Unwanted update()/draw() calls due to C++'s threading (hopefully) do not cause a crash because I never actually release the shared texture / GPU memory. I know this is inefficient / inelegant, but my machines are quite beefy and I think this is the best option I have given the time frame.
  • Option 2. Attempt to write a class / drop in an ofx class that juggles ofWMFoundationPlayer’s to be thread-safe. Perform a lock() on video rendering threads whenever attempting to load or unload a video so that no erroneous video.draw() calls are committed when a video texture is being released or loaded. I honestly doubt I could pull off this option, given my C++ experience and the timeframe.
  • Option 3. Anything y’all could think of? :wink:

Thanks again for any knowledge that can be shed on this.


Update: I’m currently running with Option 1 and it hasn’t caused me any troubles yet. I pretty much just stop the clip and halt asking for video->update()'s besides the active 3 videos in question. I’ll come back here to let everyone know how it goes, my install is in 1 week (ಠ_ಠ)

Hi,

I’ve used and misused this poor video player in many ways. Could you describe exactly what’s going on ? Do you load and draw the videos in multiple threads ? Issuing drawing call outside of the main openGL thread is generally speaking a bad idea (for anything openGL related, not just the video player)
Ideally my setup would be having 4-6 instances of ofxwmfvideoplayer all used in the same thread so you can preload the next file you have to play and swap the players as seamless-ly as possible. I believe the version on the github should be able to cope with reloading files without exploding. If not, when you have time I’d be glad you share some code sample that cause the crash so I can try to reproduce the bug and fix the player.

Good luck with your install, and please share pictures or videos :smiley:

When you say “main OpenGL thread”, does that mean the ofApp::draw() method? Where exactly is it thread safe to issue texture lock / release / etc commands?

Thanks!

Yup that’s what I meant. Basically if you call the update and the draw in oF update and draw that should be fine. Do you manually release and lock the texture ?
I noticed something that may or may not explain your bug. lockSharedTexture() test for gl_handle == 0 but if you release the texture and then try to draw it, it will cause some unexpected behavior as when the texture is release the gl_handle isn’t set to 0 ( and actually the variable is not set to 0 when the EVR is created which is bad) .

So a potential fix would be to add gl_handle = 0 in releaseSharedTexture in PresentEngine.cpp (like at the end of the function). This will ensure that if the shared texture is release, the player wont try to lock it again

another ‘quick fix’ might be to use ofxHapPlayer player and a very fast SSD drive for loading the files. Its a gpu based codec; so on a good system with dedicated graphics card you should be able to have 4 4K video’s playing simultaniously in the main thread @ 60fps.

Hey Drew, Thanks for posting this on the forum, nice to have some input from others.

We deal with a similar problem to this in CLOUDS and we go with basically what you describe in option2.

The WMF player is asynchronously loading, so that you can call load() on the main thread to ‘prep’ a video player without causing a glitch in playback on any other videos currently displayed. Then when that video is loaded, you can swap it in. So if you’re playing 4 videos, you’ll want 8 video players so you can have a ‘front’ and 'back player for each of the four. It’s not unlike FBO pingpong-ing

here’s what it would look like for 1 player, and you can do the leap to 4 (also typing this without testing, may be some obvious bugs)

shared_ptr<ofWMFPlayer> frontPlayer;
shared_ptr<ofWMFPlayer> backPlayer;
bool playerSwitching;
bool playerShouldSwitch;

void update(){    
   if(playerShouldSwitch){
      backPlayer->load("newMoviePath.mov");
      playerShouldSwitch = false;
      playerSwitching = true;
   }
   if(backPlayer->isLoaded() && playerSwitching){
      playerSwitching = false;
      backPlayer->play();
      std::swap(frontPlayer,backPlayer)
   }
}

void draw(){
  frontPlayer->draw(0,0);
}
1 Like