Skip to content

Commit

Permalink
Allow seeking through seekTo before the HTMLMediaElement is ready
Browse files Browse the repository at this point in the history
As #1600 signalled, there's currently no practical way to update the
initial position (configured through the `startAt` `loadVideo` option)
until `startAt` has actually been applied.

More generally, it's not possible to perform a `seekTo` call until the
initial seek has been performed on the content (the `seekTo` call
will just be ignored).

This commit proposes the following code updates to improve on that
situation:

  1. Tangentially related, we now schedule a JavaScript micro-task right
     at `ContentTimeBoundariesObserver`'s instantiation so the caller
     can catch `events` that were previously synchronously sent (I'm
     thinking here of the warnings for the `MEDIA_TIME_BEFORE_MANIFEST`
     and the `MEDIA_TIME_AFTER_MANIFEST` codes).

     Without this, we would wait for the next playback observation,
     which could happen a whole second later.

  2. I noticed that the position indicated through the `startAt`
     `loadVideo` option was bounded so it's inside the
     `[minimumPosition, maximumPosition]` range (as obtained from the
     Manifest). As [stated
     here](#1600 (comment))
     I personally think this is suboptimal.

     In my opinion, we should let the initial position go outside the
     range of the Manifest and let the application do its thing based on
     `MEDIA_TIME_BEFORE_MANIFEST` / `MEDIA_TIME_AFTER_MANIFEST` events.

     As to not totally change the behavior, I've only done so for
     dynamic contents (contents for which segments are added or removed,
     like live and contents that are being downloaded locally).

     VoD contents continue to have the previous behavior for now.

  3. I added a "seek-blocking" mechanism to the
     `MediaElementPlaybackObserver` and made all seek operations,
     including the one from `seekTo` go through it.

     The idea is that when it is blocked (as it is initially), we'll delay
     any seek operation (by marking it as a "pending seek") and only seek
     to the last one of those once the `unblockSeeking` method is called
     (when the RxPlayer considers that the initial seek should be done).

     I also had to add `getPendingSeekInformation` method, sadly. I feel
     that we could do without it if we're smarter about things, but I
     wanted to avoid changing too much code here.

     I also thought about reworking the initial seek so it is completely
     handled by the `MediaElementPlaybackObserver` - as it could make
     everything simpler - but I chose for now to implement that less
     disruptive "seek-blocking" mechanism for now.

     Note that technically, we could still have an application directly
     updating the `HTMLMediaElement.property.currentTime` property, or
     even a web feature such as picture-in-picture doing that.
     I ensured that this eventuality did not break anything. Still, there
     will be a preference for pending seeks performed through the
     `MediaElementPlaybackObserver` over such "HTML5 seeks" performed
     during that time (if there is a "pending seek", we will apply it
     regardless of if an "HTML5 seek" happened since then).

I'm now unsure if the `getPosition` or `getCurrentBufferGap` API should
now return that planned position. I did nothing for those yet.
  • Loading branch information
peaBerberian committed Dec 23, 2024
1 parent e7277b1 commit ad4cedb
Show file tree
Hide file tree
Showing 7 changed files with 206 additions and 62 deletions.
7 changes: 3 additions & 4 deletions doc/api/Basic_Methods/seekTo.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,16 @@ The argument can be an object with a single `Number` property, either:

- `relative`: seek relatively to the current position

- `position`: seek to the given absolute position (equivalent to
`player.getVideoElement().currentTime = newPosition`)
- `position`: seek to the given absolute position (equivalent to what you would give to
`player.getVideoElement().currentTime)

- `wallClockTime`: seek to the given wallClock position, as returned by
`getWallClockTime`.

The argument can also just be a `Number` property, which will have the same effect than
the `position` property (absolute position).

Seeking should only be done when a content is loaded (i.e. the player isn't in the
`STOPPED`, `LOADING` or `RELOADING` state).
Seeking cannot be done when the content is `STOPPED`.

The seek operation will start as soon as possible, in almost every cases directly after
this method is called.
Expand Down
53 changes: 30 additions & 23 deletions src/core/main/common/content_time_boundaries_observer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import type { IReadOnlyPlaybackObserver } from "../../../playback_observer";
import type { IPlayerError } from "../../../public_types";
import EventEmitter from "../../../utils/event_emitter";
import isNullOrUndefined from "../../../utils/is_null_or_undefined";
import queueMicrotask from "../../../utils/queue_microtask";
import SortedList from "../../../utils/sorted_list";
import TaskCanceller from "../../../utils/task_canceller";

Expand Down Expand Up @@ -94,29 +95,35 @@ export default class ContentTimeBoundariesObserver extends EventEmitter<IContent
this._maximumPositionCalculator = maximumPositionCalculator;

const cancelSignal = this._canceller.signal;
playbackObserver.listen(
({ position }) => {
const wantedPosition = position.getWanted();
if (wantedPosition < manifest.getMinimumSafePosition()) {
const warning = new MediaError(
"MEDIA_TIME_BEFORE_MANIFEST",
"The current position is behind the " +
"earliest time announced in the Manifest.",
);
this.trigger("warning", warning);
} else if (
wantedPosition > maximumPositionCalculator.getMaximumAvailablePosition()
) {
const warning = new MediaError(
"MEDIA_TIME_AFTER_MANIFEST",
"The current position is after the latest " +
"time announced in the Manifest.",
);
this.trigger("warning", warning);
}
},
{ includeLastObservation: true, clearSignal: cancelSignal },
);

// As the following code may send events synchronously, which would not be
// catchable as a caller could not have called `addEventListener` yet,
// we schedule it in a micro-task
queueMicrotask(() => {
playbackObserver.listen(
({ position }) => {
const wantedPosition = position.getWanted();
if (wantedPosition < manifest.getMinimumSafePosition()) {
const warning = new MediaError(
"MEDIA_TIME_BEFORE_MANIFEST",
"The current position is behind the " +
"earliest time announced in the Manifest.",
);
this.trigger("warning", warning);
} else if (
wantedPosition > maximumPositionCalculator.getMaximumAvailablePosition()
) {
const warning = new MediaError(
"MEDIA_TIME_AFTER_MANIFEST",
"The current position is after the latest " +
"time announced in the Manifest.",
);
this.trigger("warning", warning);
}
},
{ includeLastObservation: true, clearSignal: cancelSignal },
);
});

manifest.addEventListener(
"manifestUpdate",
Expand Down
41 changes: 25 additions & 16 deletions src/main_thread/api/public_api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1014,10 +1014,27 @@ class Player extends EventEmitter<IPublicAPIEvent> {
});
}

/** Global "playback observer" which will emit playback conditions */
const playbackObserver = new MediaElementPlaybackObserver(videoElement, {
withMediaSource: !isDirectFile,
lowLatencyMode,
});

/*
* We want to block seeking operations until we know the media element is
* ready for it.
*/
playbackObserver.blockSeeking();

currentContentCanceller.signal.register(() => {
playbackObserver.stop();
});

/** Future `this._priv_contentInfos` related to this content. */
const contentInfos: IPublicApiContentInfos = {
contentId: generateContentId(),
originalUrl: url,
playbackObserver,
currentContentCanceller,
defaultAudioTrackSwitchingMode,
initializer,
Expand Down Expand Up @@ -1108,16 +1125,6 @@ class Player extends EventEmitter<IPublicAPIEvent> {
// content.
this.stop();

/** Global "playback observer" which will emit playback conditions */
const playbackObserver = new MediaElementPlaybackObserver(videoElement, {
withMediaSource: !isDirectFile,
lowLatencyMode,
});

currentContentCanceller.signal.register(() => {
playbackObserver.stop();
});

// Update the RxPlayer's state at the right events
const playerStateRef = constructPlayerStateReference(
initializer,
Expand Down Expand Up @@ -1677,10 +1684,6 @@ class Player extends EventEmitter<IPublicAPIEvent> {
}

const { isDirectFile, manifest } = this._priv_contentInfos;
if (!isDirectFile && manifest === null) {
throw new Error("player: the content did not load yet");
}

let positionWanted: number | undefined;

if (typeof time === "number") {
Expand All @@ -1700,7 +1703,11 @@ class Player extends EventEmitter<IPublicAPIEvent> {
} else if (!isNullOrUndefined(timeObj.wallClockTime)) {
if (manifest !== null) {
positionWanted = timeObj.wallClockTime - (manifest.availabilityStartTime ?? 0);
} else if (isDirectFile && this.videoElement !== null) {
} else if (!isDirectFile) {
throw new Error(
"Cannot seek: wallClockTime asked but Manifest not yet loaded.",
);
} else if (this.videoElement !== null) {
const startDate = getStartDate(this.videoElement);
if (startDate !== undefined) {
positionWanted = timeObj.wallClockTime - startDate;
Expand All @@ -1722,7 +1729,7 @@ class Player extends EventEmitter<IPublicAPIEvent> {
throw new Error("invalid time given");
}
log.info("API: API Seek to", positionWanted);
this.videoElement.currentTime = positionWanted;
this._priv_contentInfos.playbackObserver.setCurrentTime(positionWanted, false);
return positionWanted;
}

Expand Down Expand Up @@ -3376,6 +3383,8 @@ interface IPublicApiContentInfos {
originalUrl: string | undefined;
/** `ContentInitializer` used to load the content. */
initializer: ContentInitializer;
/** interface emitting regularly playback observations. */
playbackObserver: MediaElementPlaybackObserver;
/** TaskCanceller triggered when it's time to stop the current content. */
currentContentCanceller: TaskCanceller;
/** The default behavior to adopt when switching the audio track. */
Expand Down
8 changes: 7 additions & 1 deletion src/main_thread/init/utils/get_initial_time.ts
Original file line number Diff line number Diff line change
Expand Up @@ -94,13 +94,19 @@ export default function getInitialTime(
const min = getMinimumSafePosition(manifest);
const max = getMaximumSafePosition(manifest);
if (!isNullOrUndefined(startAt.position)) {
log.debug("Init: using startAt.minimumPosition");
log.debug("Init: using startAt.position");
if (manifest.isDynamic) {
return startAt.position;
}
return Math.max(Math.min(startAt.position, max), min);
} else if (!isNullOrUndefined(startAt.wallClockTime)) {
log.debug("Init: using startAt.wallClockTime");
const ast =
manifest.availabilityStartTime === undefined ? 0 : manifest.availabilityStartTime;
const position = startAt.wallClockTime - ast;
if (manifest.isDynamic) {
return position;
}
return Math.max(Math.min(position, max), min);
} else if (!isNullOrUndefined(startAt.fromFirstPosition)) {
log.debug("Init: using startAt.fromFirstPosition");
Expand Down
18 changes: 17 additions & 1 deletion src/main_thread/init/utils/initial_seek_and_play.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,20 @@ export default function performInitialSeekAndPlay(
let hasAskedForInitialSeek = false;

const performInitialSeek = (initialSeekTime: number) => {
playbackObserver.setCurrentTime(initialSeekTime);
const pendingSeek = playbackObserver.getPendingSeekInformation();

/*
* NOTE: The user might have asked for a seek before the media element
* was ready, in which case we want the seek to be at the user's wanted
* position instead.
* If multiple internal seeks were asked however, we want to keep the
* last one.
*/
if (pendingSeek === null || pendingSeek.isInternal) {
playbackObserver.setCurrentTime(initialSeekTime);
}
hasAskedForInitialSeek = true;
playbackObserver.unblockSeeking();
};

// `startTime` defined as a function might depend on metadata to make its
Expand All @@ -109,6 +121,8 @@ export default function performInitialSeekAndPlay(
typeof startTime === "number" ? startTime : startTime();
if (initiallySeekedTime !== 0 && initiallySeekedTime !== undefined) {
performInitialSeek(initiallySeekedTime);
} else {
playbackObserver.unblockSeeking();
}
waitForSeekable();
} else {
Expand Down Expand Up @@ -141,6 +155,8 @@ export default function performInitialSeekAndPlay(
performInitialSeek(initiallySeekedTime);
}, 0);
}
} else {
playbackObserver.unblockSeeking();
}
waitForSeekable();
}
Expand Down
Loading

0 comments on commit ad4cedb

Please sign in to comment.