Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Add hidden "Dummy media API" experimental feature and add integration tests linked to DRM #1478

Open
wants to merge 9 commits into
base: misc/no-eme-type
Choose a base branch
from

Conversation

peaBerberian
Copy link
Collaborator

@peaBerberian peaBerberian commented Jul 9, 2024

Based on #1477

Reason

Today, we can only test the RxPlayer's DRM-related logic either through:

  • unit tests. Writing those are very time-consuming considering that EME-related code changes often, and thus often lead to unit test rewrites.

    To lower the lost time, we added the concept of "global unit tests", which instead of performing tests at the lower function-level, it performed them at the module-level, which has its own API much less subject to change.

    Even then, we would prefer more stable and representative tests.

  • rx-tester: Our closed-source repository that tests real production contents through one of our main application at canal+.

    This is nice but we have to go through another application that do not use the API at its full extent, so it's not an exhaustive test for us.
    It's also closed-source and relying on specific real-life contents where we would prefer the idea of directly running in the RxPlayer CI tests on contents with all kinds of quirks.

  • manually, which is also time-consuming.

The idea is here to actually run DRM-related integration tests by completely mocking in them:

  • EME API
  • MSE API
  • HTML5 video API

To be able to run those tests through npm run test:integration either locally or in our CI.

Implementation

I decided to add an [incomplete] in-JS HTML5 media element implementation as an "experimental tool" (in src/experimental/tools/DummyMediaElement).

That fake HTMLMediaElement actually implements the IMediaElement type defined in #1397 and so should be compatible with what the RxPlayer expects of a media element.

It has also those supplementary properties:

When those are present, the RxPlayer will rely on them instead of the usual MSE and EME implementations.

I also added:

  • In the demo page: I added a Dummy Media API switch allowing to rely on our MSE+EME+HTML5 implementation instead of the browser's. This is mainly to be able to debug the RxPlayer logic on contents that cannot normally be played on the demo page, usually because of DRM.

    dummy

    With this on, those contents will load - even if with a black screen (though subtitles are displayed) - and the RxPlayer will believe that the content can play.

  • In integration tests: I added a fake encrypted content with several keys and some tests checking for errors thrown, fallbacks, license request configuration etc.. More tests should be quick to add.

    On that matter, I already spotted a potential issue when the getLicense respond much faster than a SourceBuffer.prototype.appendBuffer call. I removed that test as I try to fix the issue.

What's missing

Not the full extent of the MSE, EME nor HTML5 media API is implemented here.

Most notably, I did not yet implement:

  • actual content playback, as in the loaded content will always stay in pause.

    The fake implementation will block actual playback by relying on the NotAllowedError exception on each play call, so the RxPlayer still understands the issue.

    I did not implement playback yet because I did not want to take time implementing time-related complexities (stalling, ending etc.). Though I plan to.

  • persistent licenses. This does not seem hard to do, but I did not take the time yet. My implementation will just gives the standard answer indicating that the feature is not supported

  • MSE "sequence" mode: this can become complex and we do not need it at all for now in the RxPlayer

  • Only "mp4" files are handled for now, as we have to do some container parsing here

  • Multiple other minor features

@peaBerberian peaBerberian added tests Relative to the RxPlayer's tests tools Relative to the RxPlayer's tools Demo Relative to the RxPlayer's demo page labels Jul 9, 2024
@peaBerberian peaBerberian force-pushed the misc/mse-mock branch 3 times, most recently from bb18117 to 7743e95 Compare July 9, 2024 15:59
@peaBerberian peaBerberian added this to the 4.2.0 milestone Jul 9, 2024
@peaBerberian peaBerberian added the proposal This Pull Request or Issue is only a proposal for a change with the expectation of a debate on it label Jul 9, 2024
@peaBerberian peaBerberian force-pushed the misc/mse-mock branch 5 times, most recently from ae80343 to 9527be7 Compare July 9, 2024 21:11
@peaBerberian peaBerberian force-pushed the misc/no-eme-type branch 2 times, most recently from c4aae81 to a7cc348 Compare July 24, 2024 16:49
@peaBerberian peaBerberian added the Priority: 3 (Low) This issue or PR has a low priority. label Jul 26, 2024
@peaBerberian peaBerberian force-pushed the misc/mse-mock branch 2 times, most recently from b5c4a45 to 418170b Compare August 1, 2024 12:24
@peaBerberian peaBerberian force-pushed the misc/no-eme-type branch 3 times, most recently from 6429671 to ce097d5 Compare August 1, 2024 13:56
peaBerberian added a commit that referenced this pull request Aug 13, 2024
While working on new DRM integration test in #1478, to add tests for the
not-yet-released features proposed by #1484, I noticed that some
contents anounced HEVC by beginning the codec string with `hev` and
others with `hvc`.

When comparing family of codecs, we thus had false negative which may
mean we would have ended up playing encrypted hevc on devices where it
wasn't supported, if it was anounced in the MPD as `hev1...`.

I now treat both the same way.
peaBerberian added a commit that referenced this pull request Dec 23, 2024
Thanks to the PR #1478, that basically allows integration tests on our
DRM logic without a real encrypted or decodable contents nor MSE/EME
available, I noticed that I very recently brought a minor regression in
the rewrite of the `MediaKeySystemAccess` cache reusage rules.

We relied on the `keySystems[].type` API to see if the cached one was
compatible to the new wanted one, we probably wanted to compare the
former with the `type` actually provided as argument to the
`navigator.requestMediaKeySystemAccess` API instead (e.g.
`keySystems[].type` could be set just to `widevine`, in which case the
RxPlayer will probably ask for `com.widevine.alpha` instead).

Thankfully, EME defines a
[`MediaKeySystemAccess.prototype.keySystem`](https://www.w3.org/TR/encrypted-media-2/#dom-navigator-requestmediakeysystemaccess)
property that seems to (according to that recommendation) always be equal
to the asked `keySystem`.
peaBerberian added a commit that referenced this pull request Dec 23, 2024
Thanks to the PR #1478, that basically allows integration tests on our
DRM logic without a real encrypted or decodable contents nor MSE/EME
available, I noticed that I very recently brought a minor regression in
the rewrite of the `MediaKeySystemAccess` cache reusage rules.

We relied on the `keySystems[].type` API to see if the cached one was
compatible to the new wanted one, we probably wanted to compare the
former with the `type` actually provided as argument to the
`navigator.requestMediaKeySystemAccess` API instead (e.g.
`keySystems[].type` could be set just to `widevine`, in which case the
RxPlayer will probably ask for `com.widevine.alpha` instead).

Thankfully, EME defines a
[`MediaKeySystemAccess.prototype.keySystem`](https://www.w3.org/TR/encrypted-media-2/#dom-navigator-requestmediakeysystemaccess)
property that seems to (according to that recommendation) always be equal
to the asked `keySystem`.
peaBerberian added a commit that referenced this pull request Dec 24, 2024
This is based on #1478.

On #1478, I was mocking MSE+EME + a part of the [HTML5 media
spec](https://html.spec.whatwg.org/#media-elements) to allow advanced
integration tests without having to create encrypted contents nor
having to physically test on all encountered environments.

The subset of the MSE and EME API that we use isn't very complex, the
harder part was to mock an `HTMLMediaElement`: especially the evolution
of the current position, rebuffering, ending, readyState, autoplay etc.

A shortcut I consequently took in #1478 was to rely on a potential
browser behavior: [the allowed to play](https://html.spec.whatwg.org/#allowed-to-play)
concept.
By forbidding playback through this mechanism, I could avoid all those
complex features while still being spec-compliant.

However forbidding playback prevent us from making a lot of useful
tests: e.g. on freezing, rebuffering, ending, auto-play etc.

So this commit is an attempt to implement that. I relied on the WHATWG
HTML spec but I took many shortcuts, especially by removing parts that
aren't relied on at all by the RxPlayer.

It's possible that I missed some quirks or that it still has some bugs,
but it right now seems to following the behavior of a real
`HTMLMediaElement` well enough.

I also did not implement "directfile" playback for now. For all those
missing features, we may ideally throw or indicate in some way that this
is not yet implemented.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Demo Relative to the RxPlayer's demo page Priority: 3 (Low) This issue or PR has a low priority. proposal This Pull Request or Issue is only a proposal for a change with the expectation of a debate on it tests Relative to the RxPlayer's tests tools Relative to the RxPlayer's tools
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant