Announcing ofxHPVPlayer: cross-platform high-res/high-fps video playback


#1

Dear all,

It is with pleasure that I announce the release of the ofxHPVPlayer addon.

Description

This addon aims at providing a cross-platform solution for playing high-res and high-fps video content without being tied to system specific media-frameworks such as Quicktime, AVFoundation, GStreamer, …

You can find the addon, the Qt GUI encoder and example projects on the ofxHPVPlayer Github.
Pre-compressed HPV files accompanying the examples can be found here.

The HPV library was originally created by the Belgian University of Hasselt/EDM / iMinds in the context of the European funded FP7 program Dreamspace. Their aim was to construct a toolset for the Unity game engine allowing to create immersive multi-projector installations. HPV files can be played both through this addon and by using the provided Unity tools. For more information, visit the ofxHPVPlayer Github.

###Specifications in a nutshell:

  • Play FullHD/4K/8K video files at high framerates

    • Max achievable framerate is limited by the performance of your computer (HDD read speed, CPU speed, throughput speed of PCI-Express bus)
  • Optimized for playing multiple videofiles at the same time.

  • Supports blitting (direct CPU texture to GPU texture) and double buffered playback (on OpenGL, using Pixel Buffer Objects)

  • Self-contained custom HPV file format with no dependencies to platform specific media frameworks.

  • Frames are compressed using texture compression methods (DXT). Open source GUI HPV encoder is provided for Windows & Mac

    • Supported compression types are:
      • DXT1 (no alpha)
        • reasonable image quality
      • DXT5 (alpha)
        • reasonable image quality + alpha
      • SCALED DXT5 (CoCg_Y)
        • good image quality, no alpha
        • bigger filesize
    • Allows for future extensions: eg. BPTC, ASTC, …
    • The encoder expects an image sequence where each frame is a separate image with a incremental number in the filename.
      Supported filetypes are: png, jpeg, jpg, tga, gif, bmp, psd, gif, hdr, pic, ppm, pgm
  • Texture compressed frames are then further compressed via LZ4 HQ to get even smaller file sizes.

  • Each videoplayer generates playback state events that can be captured in the openFrameworks application.

  • Render backend agnostic, can be attached to OpenGL or DirectX context

  • Extensible format that can contain multiple texture compression formats. Succesful tests have been made with BPTC and ASTC which will be available in a future update.

  • Built-in logging system, able to log to file.

  • Built-in timed statistics for HDD read time, LZ4 de-compress time and GPU upload time, to debug playback issues.

Compatibility

  • Made with OF 0.9.8
  • Tested on Windows 7, 8 and 10.
  • Tested on Max OSX 10.12.4 (should work from 10.9 up)
  • Tested on Linux Ubuntu Desktop 16.04

The HPV system was tried and tested in several installations and VR showcases and was able to provide stable FullHD, 4K and 8K playback at high frame rates (given a powerful machine with SSD and good CPU).

Hopefully this addon will serve the community!
Please try it out and let me know any comments or issues!

Note

It is important that the HPV library doesn’t aim to replace the fantastic HAP Codec by Vidvox nor has it the specifications to. The context in which the HPV system was realised required complete independence from platform specific frameworks. It furthermore required an open container that allows for supporting newer types of texture compression (BPTC, ASTC, …) in the future.


#2

Fixed some issues, now working on Linux, tested on Ubuntu Desktop 16.04 with OF 0.9.8

Check the GitHub.


#3

Hi,

Does it work in Raspberry Pi?


#4

@ nununo: no. See in readme on https://github.com/vjacob/ofxHPVPlayer:

OpenGL: needs at least an OpenGL 4.1 context [...]


#5

btw, the HPVPlayer in general works great, I can play and scrub the example videos very smoothly at maximum framerate.

I could not succeed in running the HPVCreator though in either OSX or Windows due to some issues with the qt framework (I assume)… is there another way of creating hpv files?
I opened issues in the github repository of the tool, where I also posted the error messages I got. Any help or pointing in the right direction is highly appreciated!

Apart from that I had issues with setting the play speed above 1, and palindrome looping didn’t work. I tested in OSX, Linux & Windows.
Both things I could easily fix with an ugly hack in my application code.
I just pause the video and then seekToFrame() in update whenever necessary and flip direction at beginning and end… I could show my code, but I wouldn’t recommend using it to anybody :wink:

If I would be able to play my own videos somehow, then i’d be already happy!


#6

Hello @nununo,

In short: no.

The GPU on a Raspberry Pi (Broadcom VideoCore IV) doesn’t support hardware texture de-compression of the types that are currently used for HPV files (DXT). This is not likely to change in the future.

However, I’m experimenting with texture compression types that are supported by the Pi (ETC1, RGB only) but don’t expect support for this soon. Another restraint will be the slower disk read speeds achievable on embedded devices such as the PI, which might be too slow.
I’m also experimenting with newer types of compression, such as ASTC, which has much more flexible control over the compression, and thus file size. ASTC is supported by a lot of mobile/embedded devices through OpenGLES 3.0, so I’m curious to test there and see what the results are.


#7

Hi @atanyimebutnow,

thanks for the feedback! I’m pretty sure we can easily fix your Qt issues. I will took a look at it tonight or tomorrow!
EDIT: new binaries uploaded to the HPVCreator git, should be fixed!

I never really had issues with the palindrome looping and settings speed higher than 1. Do the other loop modes work and do speeds below 0 zero work (video should play backwards)?


#8

Hello @Vjacobs,

Understood. Thanks for the detailed explanation. And good luck with your innovative experiments! I’ll try to follow your work.

Cheers,
Nuno


#9

hello,

I’ve updated the HPVCreator binaries on the Github releases page to support:

  • Mac OSX Sierra (10.12). Binary should work until Mac OS 10.9
  • Linux. Binary provided as AppImage.

You can find them here.

Can anybody with Sierra and/or another Linux distro than Ubuntu try these on their machine?

Thanks!


#10

amazing! thanks, the HPV creator works now out of the box.
I tested it in Linux (Ubuntu 16.04)
I do Sierra tomorrow and will report if there are issues.

Is there a possibility to use it as command line interface? I love making shell scripts to batch convert many video files, this would be super handy.

What I might add as a remark, you could let people know that they have to select a folder with single frame images, and not try to convert an existing video, because that crashed the HPVCreator for me.

btw, weirdly enough with my new video file from HPVCreator the palindrome loop + other loopstates + speed above one and negative values actually work.
Setting speed dynamically still gives me jitterish movement though.


#11

Thanks for testing, first time I use the AppImage approach and happy to hear it works out of the box! :slight_smile:

A command line encoder doesn’t exist for the moment. If there’s interest from the community in this tool I can invest time in this. It’s actually not so hard to implement.

Good remark about the frame selection hint, I’ll see what I can do about that!

With ‘Setting speed dynamically’ you mean programmatically from code, or using the UI slider in the controls example?


#12

I’m trying this and although it works great for basic cases it has some issues:

  • The aux thread is constantly allocating buffers for the compressed data which might be slowing down things a lot in certain cases
  • Playing back in “normal” mode, ie: with no manual seek, is just sleeping the aux thread for an arbitrary amount of time and hoping it’ll wake up on time to read the next frame but there’s no sync primitives between the main and aux thread and there’s only one buffer so if a frame took longer to load or the app took longer to render you’ll surely get tearing.
  • Seeking means that the main thread locks until the aux thread awakes reads the next frame and returns it which virtually is almost the same as doing it in the main thread (except if you play several videos where the uncompression would happen in parallel)

I’ve done some quick and really hacky changes in this branch: https://github.com/Entropy/ofxHPVPlayer/tree/buffering using some prebuffering and a pool of frames so instead of sleeping the aux thread it’ll automatically block until the main thread sends it a new frame to read. that way there’s almost no locking but also when seeking sequentially most of the time the next frame is already loaded so there’s no blocking of the main thread

with this changes i’m getting about 3.5x speed up in the multivideo player example

As i said that branch is really hacky and wouldn’t recomend to merge it, it also removes the ability to playback in normal mode, without manually seeking since that’s all we need but perhaps it’s useful to checkout how to do prebuffering

Some other stuff that can be optimized:

  • I’ve added an extra thread on the manager to send all disk reads there so there’s only 1 simultaneous read which seems to make a difference with some drives
  • You could share a pointer to a pbo’s memory and then uncompress directly instead of making one extra copy
  • If you allow for no compression at all then you can also use unbuffered reads in windows or memory mapped files in osx/linux directly to a gpu buffer which can be faster then uncompressing in some cases

#13

Hi @arturo,

thanks a lot for your comments and feedback! I’m aware of some of your points (thread waiting and superfluous frame copy) but haven’t found the time to re-implement things. Always eager to learn and make this a better add-on.

I’m on holidays now but will check your branch in one week. Thanks again.


#14

Hi @Vjacobs,
Very interested in the HPVplayer.
We are planning a project where we would have to play 12 fullHD videos in sync. If we can build a machine fast enough, with powerfull enough GPU’s and lightning fast HDD’s would it then be possible to do this with the HPVPlayer?
-Per


#15

Hi @squaresquid,

There are some improvements to be made under the hood that specifically address multiple player performance (cf. Arturo’s remarks) but yes, playing 12 FullHD video’s should absolutely be possible. Check the sync multiple video’s example, where video’s are pauzed after loading and then all receive seek messages to advance to the desired frame. Make sure not to miss this pauze, otherwise one players internal clock will constantly ‘fight’ against the external seek messages. This is a bit ugly and will be cleaner in a next update.

I don’t think multiple GPU’s will give any benefit. Multiple SSD disks might, but I can’t confirm your specific use case. Count let’s say that one second of 25fps 1920x1080 HPV video results, using DXT1, in ~20Mb/sec. This times 12 results in roughly 240Mb/sec read throughput. It’s possible this is too much random seeking for one SSD (while in theory recent SSDs offer sequential read rates much higher than that). So you’ll have to try this out yourself!

Best,
Vincent


#16

@Vjacobs did you get a chance to implement arturos changes? also can you comment on how this is different from the Hap codec and ofxHapPlayer which seems to be very similar. We’re you just not aware of it or did you have some specific design decisions you made differently?


#17

hi @jrs,

sorry for the late reply.

I made this addon mainly because I was annoyed that most videoplayers are dependent on platform specific media framework which tend to make them monolithic and un-transparent. For this reason I wanted to build something that was more open, totally cross-platform and could be easily used in video projects, at the same time providing very robust playback and more control about how resources are dealt with. After using the core in several long term art installations, without issues, I decided to release it as an Openframeworks addon. I am very familiar with the HAP codec (and the Resolume DXV codec) but at the time, HAP was still quite tied to either Quicktime or Directshow. If you’re comfortable using one of those, by all means do so!

That being said, there are certainly improvements possible. I already worked on implementing a PBO pool for a more streamlined upload process. I would be beneficial to move all disk reading operations to one thread and streamline synchronisation primitives, both CPU and GPU. Last but not least it would be better to have the frame queuing/preloading more closely linked to the actual timing of the display thread, provisioning new frames when older ones have been displayed.

I plan to work on this in February.


#18

Hi @Vjacobs and others! Thanks for this superb addon. I’m extremely impressed by the performance even on very modest hardware (an NUC with core i3).

I’m using the orginal project (not @arturo 's) fork, which I presume is not quite ready for production use although it clearly contains some important optimisations.

Any news on the roadmap or the level of interest in continuing work on this? I’m not particularly experienced with the kind of programming this requires but just thought I’d mention how useful I have found this addon to be.


#19

@anselan, the code in my branch could be cleaner but i’ve been using it in a 40 minute show playing, loading and unloading several videos without any problem


#20

Wow, this sounds good! Should the buffering branch be merged into master, then?