Allow seeking through seekTo before the HTMLMediaElement is ready

As #1600 signalled, there's currently no practical way to update the
initial position (configured through the `startAt` `loadVideo` option)
until `startAt` has actually been applied.

More generally, it's not possible to perform a `seekTo` call until the
initial seek has been performed on the content (the `seekTo` call
will just be ignored).

This commit proposes the following code updates to improve on that
situation:

  1. Tangentially related, we now schedule a JavaScript micro-task right
     at `ContentTimeBoundariesObserver`'s instantiation so the caller
     can catch `events` that were previously synchronously sent (I'm
     thinking here of the warnings for the `MEDIA_TIME_BEFORE_MANIFEST`
     and the `MEDIA_TIME_AFTER_MANIFEST` codes).

     Without this, we would wait for the next playback observation,
     which could happen a whole second later.

  2. I noticed that the position indicated through the `startAt`
     `loadVideo` option was bounded so it's inside the
     `[minimumPosition, maximumPosition]` range (as obtained from the
     Manifest). As [stated
     here](#1600 (comment))
     I personally think this is suboptimal.

     In my opinion, we should let the initial position go outside the
     range of the Manifest and let the application do its thing based on
     `MEDIA_TIME_BEFORE_MANIFEST` / `MEDIA_TIME_AFTER_MANIFEST` events.

     As to not totally change the behavior, I've only done so for
     dynamic contents (contents for which segments are added or removed,
     like live and contents that are being downloaded locally).

     VoD contents continue to have the previous behavior for now.

  3. I added a "seek-blocking" mechanism to the
     `MediaElementPlaybackObserver` and made all seek operations,
     including the one from `seekTo` go through it.

     The idea is that when it is blocked (as it is initially), we'll delay
     any seek operation (by marking it as a "pending seek") and only seek
     to the last one of those once the `unblockSeeking` method is called
     (when the RxPlayer considers that the initial seek should be done).

     I also had to add `getPendingSeekInformation` method, sadly. I feel
     that we could do without it if we're smarter about things, but I
     wanted to avoid changing too much code here.

     I also thought about reworking the initial seek so it is completely
     handled by the `MediaElementPlaybackObserver` - as it could make
     everything simpler - but I chose for now to implement that less
     disruptive "seek-blocking" mechanism for now.

     Note that technically, we could still have an application directly
     updating the `HTMLMediaElement.property.currentTime` property, or
     even a web feature such as picture-in-picture doing that.
     I ensured that this eventuality did not break anything. Still, there
     will be a preference for pending seeks performed through the
     `MediaElementPlaybackObserver` over such "HTML5 seeks" performed
     during that time (if there is a "pending seek", we will apply it
     regardless of if an "HTML5 seek" happened since then).

I'm now unsure if the `getPosition` or `getCurrentBufferGap` API should
now return that planned position. I did nothing for those yet.

doc/api/Basic_Methods/seekTo.md

@@ -8,8 +8,8 @@ The argument can be an object with a single `Number` property, either:
 
 - `relative`: seek relatively to the current position
 
-- `position`: seek to the given absolute position (equivalent to
-  `player.getVideoElement().currentTime = newPosition`)
+- `position`: seek to the given absolute position (equivalent to what you would give to
+  `player.getVideoElement().currentTime)
 
 - `wallClockTime`: seek to the given wallClock position, as returned by
   `getWallClockTime`.

src/core/main/common/content_time_boundaries_observer.ts

@@ -29,6 +29,7 @@ import type { IReadOnlyPlaybackObserver } from "../../../playback_observer";
 import type { IPlayerError } from "../../../public_types";
 import EventEmitter from "../../../utils/event_emitter";
 import isNullOrUndefined from "../../../utils/is_null_or_undefined";
+import queueMicrotask from "../../../utils/queue_microtask";
 import SortedList from "../../../utils/sorted_list";
 import TaskCanceller from "../../../utils/task_canceller";
 
@@ -94,29 +95,35 @@ export default class ContentTimeBoundariesObserver extends EventEmitter<IContent
     this._maximumPositionCalculator = maximumPositionCalculator;
 
     const cancelSignal = this._canceller.signal;
-    playbackObserver.listen(
-      ({ position }) => {
-        const wantedPosition = position.getWanted();
-        if (wantedPosition < manifest.getMinimumSafePosition()) {
-          const warning = new MediaError(
-            "MEDIA_TIME_BEFORE_MANIFEST",
-            "The current position is behind the " +
-              "earliest time announced in the Manifest.",
-          );
-          this.trigger("warning", warning);
-        } else if (
-          wantedPosition > maximumPositionCalculator.getMaximumAvailablePosition()
-        ) {
-          const warning = new MediaError(
-            "MEDIA_TIME_AFTER_MANIFEST",
-            "The current position is after the latest " +
-              "time announced in the Manifest.",
-          );
-          this.trigger("warning", warning);
-        }
-      },
-      { includeLastObservation: true, clearSignal: cancelSignal },
-    );
+
+    // As the following code may send events synchronously, which would not be
+    // catchable as a caller could not have called `addEventListener` yet,
+    // we schedule it in a micro-task
+    queueMicrotask(() => {
+      playbackObserver.listen(
+        ({ position }) => {
+          const wantedPosition = position.getWanted();
+          if (wantedPosition < manifest.getMinimumSafePosition()) {
+            const warning = new MediaError(
+              "MEDIA_TIME_BEFORE_MANIFEST",
+              "The current position is behind the " +
+                "earliest time announced in the Manifest.",
+            );
+            this.trigger("warning", warning);
+          } else if (
+            wantedPosition > maximumPositionCalculator.getMaximumAvailablePosition()
+          ) {
+            const warning = new MediaError(
+              "MEDIA_TIME_AFTER_MANIFEST",
+              "The current position is after the latest " +
+                "time announced in the Manifest.",
+            );
+            this.trigger("warning", warning);
+          }
+        },
+        { includeLastObservation: true, clearSignal: cancelSignal },
+      );
+    });
 
     manifest.addEventListener(
       "manifestUpdate",

src/main_thread/api/public_api.ts

@@ -1014,10 +1014,27 @@ class Player extends EventEmitter<IPublicAPIEvent> {
       });
     }
 
+    /** Global "playback observer" which will emit playback conditions */
+    const playbackObserver = new MediaElementPlaybackObserver(videoElement, {
+      withMediaSource: !isDirectFile,
+      lowLatencyMode,
+    });
+
+    /*
+     * We want to block seeking operations until we know the media element is
+     * ready for it.
+     */
+    playbackObserver.blockSeeking();
+
+    currentContentCanceller.signal.register(() => {
+      playbackObserver.stop();
+    });
+
     /** Future `this._priv_contentInfos` related to this content. */
     const contentInfos: IPublicApiContentInfos = {
       contentId: generateContentId(),
       originalUrl: url,
+      playbackObserver,
       currentContentCanceller,
       defaultAudioTrackSwitchingMode,
       initializer,
@@ -1108,16 +1125,6 @@ class Player extends EventEmitter<IPublicAPIEvent> {
     //     content.
     this.stop();
 
-    /** Global "playback observer" which will emit playback conditions */
-    const playbackObserver = new MediaElementPlaybackObserver(videoElement, {
-      withMediaSource: !isDirectFile,
-      lowLatencyMode,
-    });
-
-    currentContentCanceller.signal.register(() => {
-      playbackObserver.stop();
-    });
-
     // Update the RxPlayer's state at the right events
     const playerStateRef = constructPlayerStateReference(
       initializer,
@@ -1722,7 +1729,7 @@ class Player extends EventEmitter<IPublicAPIEvent> {
       throw new Error("invalid time given");
     }
     log.info("API: API Seek to", positionWanted);
-    this.videoElement.currentTime = positionWanted;
+    this._priv_contentInfos.playbackObserver.setCurrentTime(positionWanted, false);
     return positionWanted;
   }
 
@@ -3376,6 +3383,8 @@ interface IPublicApiContentInfos {
   originalUrl: string | undefined;
   /** `ContentInitializer` used to load the content. */
   initializer: ContentInitializer;
+  /** interface emitting regularly playback observations. */
+  playbackObserver: MediaElementPlaybackObserver;
   /** TaskCanceller triggered when it's time to stop the current content. */
   currentContentCanceller: TaskCanceller;
   /** The default behavior to adopt when switching the audio track. */

src/main_thread/init/utils/get_initial_time.ts

@@ -94,13 +94,19 @@ export default function getInitialTime(
     const min = getMinimumSafePosition(manifest);
     const max = getMaximumSafePosition(manifest);
     if (!isNullOrUndefined(startAt.position)) {
-      log.debug("Init: using startAt.minimumPosition");
+      log.debug("Init: using startAt.position");
+      if (manifest.isDynamic) {
+        return startAt.position;
+      }
       return Math.max(Math.min(startAt.position, max), min);
     } else if (!isNullOrUndefined(startAt.wallClockTime)) {
       log.debug("Init: using startAt.wallClockTime");
       const ast =
         manifest.availabilityStartTime === undefined ? 0 : manifest.availabilityStartTime;
       const position = startAt.wallClockTime - ast;
+      if (manifest.isDynamic) {
+        return position;
+      }
       return Math.max(Math.min(position, max), min);
     } else if (!isNullOrUndefined(startAt.fromFirstPosition)) {
       log.debug("Init: using startAt.fromFirstPosition");

src/main_thread/init/utils/initial_seek_and_play.ts

@@ -93,8 +93,20 @@ export default function performInitialSeekAndPlay(
       let hasAskedForInitialSeek = false;
 
       const performInitialSeek = (initialSeekTime: number) => {
-        playbackObserver.setCurrentTime(initialSeekTime);
+        const pendingSeek = playbackObserver.getPendingSeekInformation();
+
+        /*
+         * NOTE: The user might have asked for a seek before the media element
+         * was ready, in which case we want the seek to be at the user's wanted
+         * position instead.
+         * If multiple internal seeks were asked however, we want to keep the
+         * last one.
+         */
+        if (pendingSeek === null || pendingSeek.isInternal) {
+          playbackObserver.setCurrentTime(initialSeekTime);
+        }
         hasAskedForInitialSeek = true;
+        playbackObserver.unblockSeeking();
       };
 
       // `startTime` defined as a function might depend on metadata to make its
@@ -109,6 +121,8 @@ export default function performInitialSeekAndPlay(
           typeof startTime === "number" ? startTime : startTime();
         if (initiallySeekedTime !== 0 && initiallySeekedTime !== undefined) {
           performInitialSeek(initiallySeekedTime);
+        } else {
+          playbackObserver.unblockSeeking();
         }
         waitForSeekable();
       } else {
@@ -141,6 +155,8 @@ export default function performInitialSeekAndPlay(
                     performInitialSeek(initiallySeekedTime);
                   }, 0);
                 }
+              } else {
+                playbackObserver.unblockSeeking();
               }
               waitForSeekable();
             }

src/playback_observer/media_element_playback_observer.ts

@@ -85,7 +85,7 @@ export default class PlaybackObserver {
    * This position should be seeked to as soon as the HTMLMediaElement is able
    * to handle it.
    */
-  private _pendingSeek: number | null;
+  private _pendingSeek: IPendingSeekInformation | null;
 
   /**
    * The RxPlayer usually wants to differientate when a seek was sourced from
@@ -131,6 +131,8 @@ export default class PlaybackObserver {
    */
   private _expectedSeekingPosition: number | null;
 
+  private _isSeekBlocked: boolean;
+
   /**
    * Create a new `PlaybackObserver`, which allows to produce new "playback
    * observations" on various media events and intervals.
@@ -150,12 +152,13 @@ export default class PlaybackObserver {
     this._observationRef = this._createSharedReference();
     this._expectedSeekingPosition = null;
     this._pendingSeek = null;
+    this._isSeekBlocked = false;
 
     const onLoadedMetadata = () => {
       if (this._pendingSeek !== null) {
-        const positionToSeekTo = this._pendingSeek;
+        const { position: positionToSeekTo, isInternal } = this._pendingSeek;
         this._pendingSeek = null;
-        this._actuallySetCurrentTime(positionToSeekTo);
+        this._actuallySetCurrentTime(positionToSeekTo, isInternal);
       }
     };
     mediaElement.addEventListener("loadedmetadata", onLoadedMetadata);
@@ -205,6 +208,69 @@ export default class PlaybackObserver {
     return this._mediaElement.paused;
   }
 
+  /**
+   * Prevent seeking operations from being performed from inside the
+   * `MediaElementPlaybackObserver` until the `unblockSeeking` method is called.
+   *
+   * You might want to call this method when you want to ensure that the next
+   * seek operation on the media element happens at a specific, controlled,
+   * point in time.
+   */
+  public blockSeeking(): void {
+    this._isSeekBlocked = true;
+  }
+
+  /**
+   * Remove seeking block created by the `blockSeeking` method if it was called.
+   *
+   * If a seek operation was requested while the block was active, the
+   * `MediaElementPlaybackObserver` will seek at the last seeked position as
+   * soon as possible (either right now, or when the `readyState` of the
+   * `HTMLMediaElement` will have at least reached the `"HAVE_METADATA"` state).
+   */
+  public unblockSeeking(): void {
+    if (this._isSeekBlocked) {
+      this._isSeekBlocked = false;
+      if (this._pendingSeek !== null && this._mediaElement.readyState >= 1) {
+        const { position: positionToSeekTo, isInternal } = this._pendingSeek;
+        this._pendingSeek = null;
+        this._actuallySetCurrentTime(positionToSeekTo, isInternal);
+      }
+    }
+  }
+
+  /**
+   * Returns `true` if seeking operations are currently blocked due to a call to
+   * `blockSeeking` that was not yet undone by a call to `unblockSeeking`.
+   * @returns {boolean} - `true` if seeking operations are blocked currently.
+   */
+  public isSeekingBlocked(): boolean {
+    return this._isSeekBlocked;
+  }
+
+  /**
+   * Seek operations, as performed by the `setCurrentTime` method, might be not
+   * yet performed due to either of those reasons:
+   *
+   *   - Seek operations are blocked due to a call to the `blockSeeking` method.
+   *
+   *   - The `HTMLMediaElement`'s `readyState` property has not yet reached the
+   *     `"HAVE_METADATA"` state.
+   *
+   * Under any of those two conditions, this method will return the position
+   * that is planned to be seeked to as soon as both conditions are not met
+   * anymore.
+   *
+   * If seeks are possible right now, no seek should be "pending" and as such
+   * this method will return `null`.
+   *
+   * @returns {Object|null} - If a seek is planned, the position to seek to.
+   * `null` otherwise.
+   */
+  public getPendingSeekInformation(): IPendingSeekInformation | null {
+    return this._pendingSeek;
+  }
+
   /**
    * Update the current position (seek) on the `HTMLMediaElement`, by giving a
    * new position in seconds.
@@ -214,14 +280,18 @@ export default class PlaybackObserver {
    * observation than regular seeks (which most likely comes from the outside,
    * e.g. the user).
    * @param {number} time
+   * @param {boolean} [isInternal=true] - If `false`, the seek was performed by
+   * the user.
    */
-  public setCurrentTime(time: number): void {
-    if (this._mediaElement.readyState >= 1) {
-      this._actuallySetCurrentTime(time);
+  public setCurrentTime(time: number, isInternal: boolean = true): void {
+    if (!this._isSeekBlocked && this._mediaElement.readyState >= 1) {
+      this._actuallySetCurrentTime(time, isInternal);
     } else {
+      this._pendingSeek = { position: time, isInternal };
       this._internalSeeksIncoming = [];
-      this._pendingSeek = time;
-      this._generateObservationForEvent("manual");
+      if (!this._isSeekBlocked) {
+        this._generateObservationForEvent("manual");
+      }
     }
   }
 
@@ -303,9 +373,11 @@ export default class PlaybackObserver {
     return generateReadOnlyObserver(this, transform, this._canceller.signal);
   }
 
-  private _actuallySetCurrentTime(time: number): void {
+  private _actuallySetCurrentTime(time: number, isInternal: boolean): void {
     log.info("API: Seeking internally", time);
-    this._internalSeeksIncoming.push(time);
+    if (isInternal) {
+      this._internalSeeksIncoming.push(time);
+    }
     this._mediaElement.currentTime = time;
   }
 
@@ -385,7 +457,7 @@ export default class PlaybackObserver {
     let isInternalSeeking = false;
 
     /** If set, the position for which we plan to seek to as soon as possible. */
-    let pendingPosition: number | null = this._pendingSeek;
+    let pendingPosition: number | null = this._pendingSeek?.position ?? null;
 
     /** Initially-polled playback observation, before adjustments. */
     const mediaTimings = getMediaInfos(this._mediaElement);
@@ -945,3 +1017,10 @@ function getInitialObservation(mediaElement: IMediaElement): IPlaybackObservatio
     currentRange: null,
   });
 }
+
+interface IPendingSeekInformation {
+  /** Position to seek to. */
+  position: number;
+  /** If `true`, the seek was performed by the RxPlayer's internal logic. */
+  isInternal: boolean;
+}