Standalone video player?

[FrayxRulez]

While this library is quite cool MediaPlayer(Element) definitely isn't.
Have you considered building your own?

[brabebhin]

That would be pretty huge undertaking that neither me nor @lukasf have the time for. Technically it is possible to create our own wrapper around the MediaFoundation interfaces that MediaPlayer wraps, but even that is quite complicated.

What exactly do you not like about MediaPlayer? It is mostly quite OK, except for the bugs in MediaPlaybackList that can be usually worked around with some effort.

[lukasf]

You say you used the custom solution with WriteableBitmap because its low resource usage. I do not agree with that. Both video files I tested did not play properly in the feed. CPU usage was very high. One of the video was stuttering and not fluid. The other video was even worse, it only rendered a frame every few seconds. Sometimes I was not even sure if it is playing. This happened on my laptop which was pretty high-end 5 years ago, probably a standard PC these days. Both videos play flawless and 100% fluid using FFmpegMediaSource with HW decoding at very low resource usage. Even when using SW decoding, both files are playing at the same time, nice and fluid using FFmpegMediaSource. CPU usage is similar to your solution, but the playback is smooth and nice. If your users complain that the app cannot play videos smooth, then I bet this is the reason. Your solution might be fast to start and low on RAM (which most PCs have more than enough), but you burn a lot of other resources instead (CPU and battery - GPU decoding is much more efficient!), and the result is not even fluid on a lot of systems. It might work well for low-res stuff. But these days, FullHD is pretty standard and lots of people start recording 4K. Codecs are more and more demanding. Good luck SW decoding multiple 4K files. Many PCs won't even be able to SW decode a single 4K stream properly.

Of course, you need to limit the number of videos you have prepared at the same time. And you need to properly cleanup the ones not needed anymore. I would not play more than 2 or 3 files at the same time. And maybe keep 2-3 more prepared. You could also have a pool of MediaPlayers, so you re-use those instead of creating new ones all the time.

The MediaPlayerElement is actually pretty lightweight, if you do not display the transport controls. And you do not need to connect a MediaPlayer to it until the point where you want to start playback. Rendering is fully hardware accelerated and should not have any real impact on scrolling or animations. The only thing that is slow is resizing a MediaPlayerElement (true resize, not transform) while it is playing, because it will allocate new textures on each size change. But video sizes in your feed are constant, so this won't be an issue for you.

Frankly, I never tried using MediaPlayerElement in a way like in your feed. But I am sure you will continue to get complaints about performance, if you do not use HW decoding in the feed. I also do not understand why your software decoding is so much slower than our SW decoder. Maybe you should try using our FrameGrabber, calling ExtractNextVideoFrameAsync in a loop. It uses the same underlying decoders. It even allows to grab frames into an existing IBuffer, so you could pass the WriteableBitmap's buffer right into it (that is exactly the scenario why that overload was added). FFmpeg will use that buffer as decoder render target, so there is zero copying involved (copying large SW frames in RAM is slow!). Still, I'd rather recommend going for HW decoding, since that is the only way to get truly smooth playback across a vast amount of target devices.

[brabebhin]

It is also worth mentioning syncing between audio and video streams, which is also very important to proper playback.

I'd go for the preload solution with multiple MediaPlayers. Limit your preload to 5-10 videos and recycle the MediaPlayer objects. It should only cost you a few dozen MB at most. Having multiple MediaPlayers is actually a supported scenario and I've used it successfully in the past.

Remember, RAM is cheap and windows is pretty good at RAM management. When optimising, it is almost always better to optimize CPU and GPU usage at the cost of higher RAM usage.

[FrayxRulez]

I only receive complains about the full screen gallery so far. Videos rarely play automatically in the feed, it’s mostly GIFs and stickers that are quite low resolution MP4/WEBM files. Of course you won’t use more than 2-3 MediaPlayerElements at the same time, but a single message can contain a hundred custom emojis and even ignoring these you can have dozens of GIFs visible at the same time on screen by just opening the menu to send GIFs.

Worth mentioning that we did receive a lot of complains about performances when using MediaPlayerElement in the feed.

[brabebhin]

I honestly don't know what else to advise you. I definitely think MF is fast enough for your goal (after all, the vast majority of Windows applications, messaging apps included use MF at some point or another).

As for standalone players, you don't really have much choice: libVLC does provide its own MediaPlayerElement for Xamarin Forms. Technically FFmpeg can also render video to any HWND. Pretty sure various game engines will also provide basic video playback capabilities. But all of these come with both advantages and disadvantages. Creating a standalone player is not quite the scope of this library, although I did give it some thought at some point. Too much work.

[lukasf]

For the gallery, I have already listed how things can be improved. Especially the load times when switching files within the gallery can be reduced to effectively zero by preloading files into ready-to-play MediaPlayer instances. This should be pretty simple. As opposed to the feed, you do not need to load hundreds of files super fast. You only need to keep 3 files opened, and files cannot change faster than the user can click or swipe. No worries abour resources here.

For the load time from clicking on an item in the feed to the player playing: When I open the video in our sample app and press Ctrl+Return to re-open it, the time till playback starts is shorter than the duration of the opening animation you use. So with this solution, you should be able to get a video ready to play when the animation ended, without any visible delay. Please note that this is when re-using one (or more) existing MediaPlayers. When I create a new MediaPlayer, the total delay is definitely longer! So you really want to use a pool of MediaPlayers where you fetch one for the next job and put unused ones back after resetting their source.

And I verified with a modified sample app: If I open a file in one MediaPlayer without even playing it, and then open the file again in a new MediaPlayer and render it, I do not see the long "first video playback after app startup" load delay. So opening a hidden dummy file at app startup should indeed help you get rid of the first time playback delay. And then you could even put that player into the pool, since it is already initialized and will be able to start playback faster than a new instance.

As I see it, all the complain points in the gallery should be solveable, to make it feel like there is more or less zero delay.

Creating a custom player is definitely out of scope here. It is way too much work to get right for our small team, including frame synced video playback, sync audio playback and subtitle rendering...

[FrayxRulez]

In the end I opted for forking libVLC and libVLCSharp to dynamically link to ffmpeg and remove any other unneeded modules. This way it's quite compact in size and performance (even with a single player, no preloading, etc) is pretty much up to the expectations.

So if you're not planning anything like this, I'll just close this.