From 0398cced35f289c3dae255abb6d9c45efe47439f Mon Sep 17 00:00:00 2001
From: Paul Berberian <paul.berberian@proton.me>
Date: Fri, 13 Dec 2024 14:50:02 +0100
Subject: [PATCH] Thumbnails: Add supplementary metadata to
 `getAvailableThumbnailTracks`

Based on #1496

Problem
-------

We're currently trying to provide a complete[1] and easy to-use API for
DASH thumbnail tracks in the RxPlayer.

Today the proposal is to have an API called `renderThumbnail`, to which
an application would just provide an HTML element and a timestamp, and
the RxPlayer would do all that's necessary to fetch the corresponding
thumbnail and display it in the corresponding element.

The API is like so:
```js
rxPlayer.renderThumbnail({ element, time })
  .then(() => console.log("The thumbnail is now rendered in the element"));
```

This works and seems to me very simple to understand.

Yet, we've known of advanced use cases where an application might not
just want to display a single thumbnail for a single position. For
example, there's very known examples where an application displays a
window of multiple thumbnails at once on the player's UI to facilitate
navigation inside the content.

To do that under the solution proposed in #1496, an application could
just call `renderThumbnail` with several `element` and `time` values.

Yet for this type of feature, what the interface would want is not really
to indicate a `time` values, it actually wants basically a list of
distinct thumbnails around/before/after a given position.

By just being able to set a `time` value, an application is blind on
which `time` value is going to lead to a different timestamp (i.e. is
the thumbnail for the `time` `11` different than the thumbnail for the
`time` `12`? Nobody - but the RxPlayer - knows).

So we have to find a solution for this

[1] By complete, I here mean that we want to be able to handle its
complexities inside the RxPlayer, to ensure complex DASH situations like
multi-CDN, retry settings for requests and so on while still allowing
all potential use cases for an application.

Solution
--------

In this solution, I experiment with a second thumbnail API,
`getAvailableThumbnailTracks` (it already exists in #1496, but its role
there was only to list the various thumbnail qualities, if there are
several size for example). As this solution build upon yet stays
compatible to #1496, I chose to open this second PR on top of that
previous one.

I profit from the fact that most standardized thumbnail implementations I
know of (BIF, DASH) seem follow the principle of having evenly-spaced
(in terms of time) thumbnails (though I do see a possibility
for that to change, e.g. to have thumbnails corresponding to "important"
scenes instead, so our implementation has to be resilient).

So here, what this commit does is to add the following properties (all
optional) to a track returned by the `getAvailableThumbnailTracks` API:

  - `start`: The initial `time` the first thumbnail of that track will
    apply to

  - `end`: The last `time` the last thumbnail of that track will
    apply to

  - thumbnailsPerSegment: Individual thumbnails may be technically part of
    "segments" containing multiple consecutive thumbnails each.

    `thumbnailsPerSegment` is the number of thumbnails each of those
    segments contain.

    For example you could have stored on the server a segment which is a
    grid of 2 x 3 (2 horizontal rows and * 3 vertical columns)
    thumbnails, which the RxPlayer will load at once then "cut" the right
    way when calling `renderThumbnail`. In that example,
    `thumbnailsPerSegment` would be set to `6` (2*3).

    Note that the last segment of a content may contain less thumbnails as
    anounced here depending on the duration of the content.

  - `segmentDuration`: The "duration" (in seconds) each segments of
    thumbnails applies to (with the exception of the last thumbnail,
    which just fills until `end`)

Then, an application should have all information needed to calculate a
`time` which correspond to a different thumbnail.

Though this solution lead to a minor issue: by letting application make
the `time` operation themselves with `start`, `end`, `segmentDuration`
and so on, there's a risk of rounding errors leading to a `time`
which does not correspond to the thumbnail wanted but the one before or
after. To me, we could just indicate in our API documentation to
application developers that they should be extra careful and may add an
epsilon (or even choose a `time` in the "middle" of thumbnails each time)
if they want that type of thumbnail list feature.

Thoughts?
---
 src/main_thread/api/public_api.ts             |  4 ++
 .../classes/__tests__/adaptation.test.ts      |  3 +
 .../classes/__tests__/representation.test.ts  |  3 +
 .../__tests__/update_period_in_place.test.ts  | 21 +++++++
 src/manifest/classes/period.ts                | 43 +++++++++++++++
 .../classes/representation_index/static.ts    | 18 ++++++
 .../classes/representation_index/types.ts     | 23 ++++++++
 .../classes/update_period_in_place.ts         |  9 +++
 src/manifest/types.ts                         | 37 +++++++++++++
 .../manifest/dash/common/indexes/base.ts      | 23 ++++++++
 .../manifest/dash/common/indexes/list.ts      | 19 +++++++
 .../manifest/dash/common/indexes/template.ts  | 16 ++++++
 .../timeline/timeline_representation_index.ts | 27 +++++++++
 .../dash/common/parse_adaptation_sets.ts      | 15 +++++
 .../manifest/local/representation_index.ts    | 21 +++++++
 .../metaplaylist/representation_index.ts      | 10 ++++
 .../manifest/smooth/representation_index.ts   | 24 ++++++++
 src/parsers/manifest/types.ts                 | 37 +++++++++++++
 .../get_first_time_from_adaptations.test.ts   |  3 +
 .../get_last_time_from_adaptation.test.ts     |  3 +
 src/public_types.ts                           | 55 +++++++++++++++++++
 21 files changed, 414 insertions(+)

diff --git a/src/main_thread/api/public_api.ts b/src/main_thread/api/public_api.ts
index 3e34fda840..47234d9516 100644
--- a/src/main_thread/api/public_api.ts
+++ b/src/main_thread/api/public_api.ts
@@ -799,6 +799,10 @@ class Player extends EventEmitter<IPublicAPIEvent> {
         width: Math.floor(t.width / t.horizontalTiles),
         height: Math.floor(t.height / t.verticalTiles),
         mimeType: t.mimeType,
+        start: t.start,
+        end: t.end,
+        segmentDuration: t.segmentDuration,
+        thumbnailsPerSegment: t.horizontalTiles * t.verticalTiles,
       };
     });
   }
diff --git a/src/manifest/classes/__tests__/adaptation.test.ts b/src/manifest/classes/__tests__/adaptation.test.ts
index 57c6feff37..66916b8233 100644
--- a/src/manifest/classes/__tests__/adaptation.test.ts
+++ b/src/manifest/classes/__tests__/adaptation.test.ts
@@ -51,6 +51,9 @@ const minimalRepresentationIndex: IRepresentationIndex = {
   addPredictedSegments() {
     /* noop */
   },
+  getTargetSegmentDuration() {
+    return undefined;
+  },
   _replace() {
     /* noop */
   },
diff --git a/src/manifest/classes/__tests__/representation.test.ts b/src/manifest/classes/__tests__/representation.test.ts
index 0075702af1..9c4ff5e683 100644
--- a/src/manifest/classes/__tests__/representation.test.ts
+++ b/src/manifest/classes/__tests__/representation.test.ts
@@ -47,6 +47,9 @@ const minimalIndex: IRepresentationIndex = {
   canBeOutOfSyncError(): true {
     return true;
   },
+  getTargetSegmentDuration() {
+    return undefined;
+  },
   _replace() {
     return;
   },
diff --git a/src/manifest/classes/__tests__/update_period_in_place.test.ts b/src/manifest/classes/__tests__/update_period_in_place.test.ts
index 3ddb4da617..108ed08871 100644
--- a/src/manifest/classes/__tests__/update_period_in_place.test.ts
+++ b/src/manifest/classes/__tests__/update_period_in_place.test.ts
@@ -174,6 +174,9 @@ function generateFakeThumbnailTrack({ id }: { id: string }) {
     width: 200,
     horizontalTiles: 5,
     verticalTiles: 3,
+    start: 0,
+    end: 100,
+    segmentDuration: 2,
     index: {
       _update() {
         /* noop */
@@ -1433,6 +1436,9 @@ describe("Manifest - updatePeriodInPlace", () => {
           id: "thumb-2",
           mimeType: "image/png",
           verticalTiles: 3,
+          start: 0,
+          end: 100,
+          segmentDuration: 2,
           width: 200,
         },
       ],
@@ -1443,6 +1449,9 @@ describe("Manifest - updatePeriodInPlace", () => {
           id: "thumb-1",
           mimeType: "image/png",
           verticalTiles: 3,
+          start: 0,
+          end: 100,
+          segmentDuration: 2,
           width: 200,
         },
       ],
@@ -1510,6 +1519,9 @@ describe("Manifest - updatePeriodInPlace", () => {
           id: "thumb-2",
           mimeType: "image/png",
           verticalTiles: 3,
+          start: 0,
+          end: 100,
+          segmentDuration: 2,
           width: 200,
         },
       ],
@@ -1520,6 +1532,9 @@ describe("Manifest - updatePeriodInPlace", () => {
           id: "thumb-1",
           mimeType: "image/png",
           verticalTiles: 3,
+          start: 0,
+          end: 100,
+          segmentDuration: 2,
           width: 200,
         },
       ],
@@ -1588,6 +1603,9 @@ describe("Manifest - updatePeriodInPlace", () => {
           id: "thumb-1",
           mimeType: "image/png",
           verticalTiles: 3,
+          start: 0,
+          end: 100,
+          segmentDuration: 2,
           width: 200,
         },
       ],
@@ -1660,6 +1678,9 @@ describe("Manifest - updatePeriodInPlace", () => {
           id: "thumb-1",
           mimeType: "image/png",
           verticalTiles: 3,
+          start: 0,
+          end: 100,
+          segmentDuration: 2,
           width: 200,
         },
       ],
diff --git a/src/manifest/classes/period.ts b/src/manifest/classes/period.ts
index 05a35b34c4..5050833b4c 100644
--- a/src/manifest/classes/period.ts
+++ b/src/manifest/classes/period.ts
@@ -145,6 +145,9 @@ export default class Period implements IPeriodMetadata {
       width: thumbnailTrack.width,
       horizontalTiles: thumbnailTrack.horizontalTiles,
       verticalTiles: thumbnailTrack.verticalTiles,
+      start: thumbnailTrack.start,
+      end: thumbnailTrack.end,
+      segmentDuration: thumbnailTrack.segmentDuration,
     }));
     this.duration = args.duration;
     this.start = args.start;
@@ -305,6 +308,9 @@ export default class Period implements IPeriodMetadata {
         width: thumbnailTrack.width,
         horizontalTiles: thumbnailTrack.horizontalTiles,
         verticalTiles: thumbnailTrack.verticalTiles,
+        start: thumbnailTrack.start,
+        end: thumbnailTrack.end,
+        segmentDuration: thumbnailTrack.segmentDuration,
       })),
     };
   }
@@ -344,4 +350,41 @@ export interface IThumbnailTrack {
    * images contained vertically in a whole loaded thumbnail resource.
    */
   verticalTiles: number;
+  /**
+   * Starting `position` the first thumbnail of this thumbnail track applies to,
+   * if known.
+   */
+  start: number | undefined;
+  /**
+   * Ending `position` the last thumbnail of this thumbnail track applies to,
+   * if known.
+   */
+  end: number | undefined;
+  /**
+   * When loaded, thumbnails are part of so-called "segments" which may contain
+   * either a single thumbnail or a grid of them.
+   *
+   * This `segmentDuration` property indicates a duration in seconds each
+   * segment applies to. You might then want to divide that value by
+   * `horizontalTiles * verticalTiles` to get the duration each thumbnail
+   * applies to.
+   *
+   * Note that the last segment of a content may have a lower duration depending
+   * on the duration of the content.
+   *
+   * Set to `undefined` either the duration is unknown or if the duration
+   * depends from segment to segments.
+   *
+   * For example, with a `start` set to `10`, an `end` set to `21`, a
+   * `horizontalTiles` set to `2`, a `verticalTiles` set to `1` and a
+   * `SegmentDuration` set to `4`, there should be 3 segments, each with 2
+   * thumbnails:
+   * 1. A segment of 2 thumbnails for the seconds: 10-14
+   *    (The first thumbnail in that segment for 10-12, the second for 12-14)
+   * 2. A segment of 2 thumbnails for the seconds: 14-18
+   *    (The first thumbnail in that segment for 14-16, the second for 16-18)
+   * 3. A segment of 2 thumbnails for the seconds: 18-21 (the end)
+   *    (The first thumbnail in that segment for 18-20, the second for 20-21)
+   */
+  segmentDuration: number | undefined;
 }
diff --git a/src/manifest/classes/representation_index/static.ts b/src/manifest/classes/representation_index/static.ts
index 4eeffc461a..5aa8f9ec08 100644
--- a/src/manifest/classes/representation_index/static.ts
+++ b/src/manifest/classes/representation_index/static.ts
@@ -155,6 +155,24 @@ export default class StaticRepresentationIndex implements IRepresentationIndex {
     log.error("A `StaticRepresentationIndex` does not need to be initialized");
   }
 
+  /**
+   * Returns the `duration` of each segment in the context of its Manifest (i.e.
+   * as the Manifest anounces them, actual segment duration may be different due
+   * to approximations), in seconds.
+   *
+   * NOTE: we could here do a median or a mean but I chose to be lazy (and
+   * more performant) by returning the duration of the first element instead.
+   * As `isPrecize` is `false`, the rest of the code should be notified that
+   * this is only an approximation.
+   * @returns {number}
+   */
+  getTargetSegmentDuration(): { duration: number; isPrecize: boolean } | undefined {
+    return {
+      duration: Number.MAX_VALUE,
+      isPrecize: false,
+    };
+  }
+
   addPredictedSegments(): void {
     log.warn("Cannot add predicted segments to a `StaticRepresentationIndex`");
   }
diff --git a/src/manifest/classes/representation_index/types.ts b/src/manifest/classes/representation_index/types.ts
index e8497bc216..3876a8d475 100644
--- a/src/manifest/classes/representation_index/types.ts
+++ b/src/manifest/classes/representation_index/types.ts
@@ -429,6 +429,29 @@ export interface IRepresentationIndex {
    */
   initialize(segmentList: ISegmentInformation[]): void;
 
+  /**
+   * Returns an approximate for the duration of that `RepresentationIndex`s
+   * segments, in seconds in the context of its Manifest (i.e. as the Manifest
+   * anounces them, actual segment duration may be different due to
+   * approximations), with the exception of the last one (that usually is
+   * shorter).
+   * @returns {number}
+   */
+  getTargetSegmentDuration():
+    | {
+        /** Approximate duration of any segments but the last one in seconds. */
+        duration: number;
+        /**
+         * If `true`, the given duration should be relatively precize for all
+         * segments but the last one.
+         *
+         * If `false`, `duration` indicates only a general idea of what can be
+         * expected.
+         */
+        isPrecize: boolean;
+      }
+    | undefined;
+
   /**
    * Add segments to a RepresentationIndex that were predicted after parsing the
    * segment linked to `currentSegment`.
diff --git a/src/manifest/classes/update_period_in_place.ts b/src/manifest/classes/update_period_in_place.ts
index ebbfa4b127..cbee92ade1 100644
--- a/src/manifest/classes/update_period_in_place.ts
+++ b/src/manifest/classes/update_period_in_place.ts
@@ -75,6 +75,9 @@ export default function updatePeriodInPlace(
       oldThumbnailTrack.width = newThumbnailTrack.width;
       oldThumbnailTrack.horizontalTiles = newThumbnailTrack.horizontalTiles;
       oldThumbnailTrack.verticalTiles = newThumbnailTrack.verticalTiles;
+      oldThumbnailTrack.start = newThumbnailTrack.start;
+      oldThumbnailTrack.end = newThumbnailTrack.end;
+      oldThumbnailTrack.segmentDuration = newThumbnailTrack.segmentDuration;
       oldThumbnailTrack.cdnMetadata = newThumbnailTrack.cdnMetadata;
       if (updateType === MANIFEST_UPDATE_TYPE.Full) {
         oldThumbnailTrack.index._replace(newThumbnailTrack.index);
@@ -88,6 +91,9 @@ export default function updatePeriodInPlace(
         width: oldThumbnailTrack.width,
         horizontalTiles: oldThumbnailTrack.horizontalTiles,
         verticalTiles: oldThumbnailTrack.verticalTiles,
+        start: oldThumbnailTrack.start,
+        end: oldThumbnailTrack.end,
+        segmentDuration: oldThumbnailTrack.segmentDuration,
       });
     }
   }
@@ -105,6 +111,9 @@ export default function updatePeriodInPlace(
         width: t.width,
         horizontalTiles: t.horizontalTiles,
         verticalTiles: t.verticalTiles,
+        start: t.start,
+        end: t.end,
+        segmentDuration: t.segmentDuration,
       })),
     );
     oldPeriod.thumbnailTracks.push(...newThumbnailTracks);
diff --git a/src/manifest/types.ts b/src/manifest/types.ts
index 94497de68f..e42d4e7fd0 100644
--- a/src/manifest/types.ts
+++ b/src/manifest/types.ts
@@ -286,6 +286,43 @@ export interface IThumbnailTrackMetadata {
    * images contained vertically in a whole loaded thumbnail resource.
    */
   verticalTiles: number;
+  /**
+   * Starting `position` the first thumbnail of this thumbnail track applies to,
+   * if known.
+   */
+  start: number | undefined;
+  /**
+   * Ending `position` the last thumbnail of this thumbnail track applies to,
+   * if known.
+   */
+  end: number | undefined;
+  /**
+   * When loaded, thumbnails are part of so-called "segments" which may contain
+   * either a single thumbnail or a grid of them.
+   *
+   * This `segmentDuration` property indicates a duration in seconds each
+   * segment applies to. You might then want to divide that value by
+   * `horizontalTiles * verticalTiles` to get the duration each thumbnail
+   * applies to.
+   *
+   * Note that the last segment of a content may have a lower duration depending
+   * on the duration of the content.
+   *
+   * Set to `undefined` either the duration is unknown or if the duration
+   * depends from segment to segments.
+   *
+   * For example, with a `start` set to `10`, an `end` set to `21`, a
+   * `horizontalTiles` set to `2`, a `verticalTiles` set to `1` and a
+   * `SegmentDuration` set to `4`, there should be 3 segments, each with 2
+   * thumbnails:
+   * 1. A segment of 2 thumbnails for the seconds: 10-14
+   *    (The first thumbnail in that segment for 10-12, the second for 12-14)
+   * 2. A segment of 2 thumbnails for the seconds: 14-18
+   *    (The first thumbnail in that segment for 14-16, the second for 16-18)
+   * 3. A segment of 2 thumbnails for the seconds: 18-21 (the end)
+   *    (The first thumbnail in that segment for 18-20, the second for 20-21)
+   */
+  segmentDuration: number | undefined;
 }
 
 export interface ILoadedThumbnailData {
diff --git a/src/parsers/manifest/dash/common/indexes/base.ts b/src/parsers/manifest/dash/common/indexes/base.ts
index 4fc95adfd2..f2f91b8be1 100644
--- a/src/parsers/manifest/dash/common/indexes/base.ts
+++ b/src/parsers/manifest/dash/common/indexes/base.ts
@@ -433,6 +433,29 @@ export default class BaseRepresentationIndex implements IRepresentationIndex {
     log.warn("Cannot add predicted segments to a `BaseRepresentationIndex`");
   }
 
+  /**
+   * Returns the `duration` of each segment in the context of its Manifest (i.e.
+   * as the Manifest anounces them, actual segment duration may be different due
+   * to approximations), in seconds.
+   *
+   * NOTE: we could here do a median or a mean but I chose to be lazy (and
+   * more performant) by returning the duration of the first element instead.
+   * As `isPrecize` is `false`, the rest of the code should be notified that
+   * this is only an approximation.
+   * @returns {number}
+   */
+  getTargetSegmentDuration(): { duration: number; isPrecize: boolean } | undefined {
+    const { timeline, timescale } = this._index;
+    const firstElementInTimeline = timeline[0];
+    if (firstElementInTimeline === undefined) {
+      return undefined;
+    }
+    return {
+      duration: firstElementInTimeline.duration / timescale,
+      isPrecize: false,
+    };
+  }
+
   /**
    * Replace in-place this `BaseRepresentationIndex` information by the
    * information from another one.
diff --git a/src/parsers/manifest/dash/common/indexes/list.ts b/src/parsers/manifest/dash/common/indexes/list.ts
index 34eac96412..afa5fe9ea4 100644
--- a/src/parsers/manifest/dash/common/indexes/list.ts
+++ b/src/parsers/manifest/dash/common/indexes/list.ts
@@ -348,6 +348,25 @@ export default class ListRepresentationIndex implements IRepresentationIndex {
     log.warn("Cannot add predicted segments to a `ListRepresentationIndex`");
   }
 
+  /**
+   * Returns the `duration` of each segment in the context of its Manifest (i.e.
+   * as the Manifest anounces them, actual segment duration may be different due
+   * to approximations), in seconds.
+   *
+   * NOTE: we could here do a median or a mean but I chose to be lazy (and
+   * more performant) by returning the duration of the first element instead.
+   * As `isPrecize` is `false`, the rest of the code should be notified that
+   * this is only an approximation.
+   * @returns {number}
+   */
+  getTargetSegmentDuration(): { duration: number; isPrecize: boolean } | undefined {
+    const { duration, timescale } = this._index;
+    return {
+      duration: duration / timescale,
+      isPrecize: true,
+    };
+  }
+
   /**
    * @param {Object} newIndex
    */
diff --git a/src/parsers/manifest/dash/common/indexes/template.ts b/src/parsers/manifest/dash/common/indexes/template.ts
index df3dca4586..e5f83cae40 100644
--- a/src/parsers/manifest/dash/common/indexes/template.ts
+++ b/src/parsers/manifest/dash/common/indexes/template.ts
@@ -507,6 +507,22 @@ export default class TemplateRepresentationIndex implements IRepresentationIndex
     log.warn("Cannot add predicted segments to a `TemplateRepresentationIndex`");
   }
 
+  /**
+   * Returns the `duration` of each segment in the context of its Manifest (i.e.
+   * as the Manifest anounces them, actual segment duration may be different due
+   * to approximations), in seconds.
+   * @returns {number}
+   */
+  getTargetSegmentDuration(): {
+    duration: number;
+    isPrecize: boolean;
+  } {
+    return {
+      duration: this._index.duration / this._index.timescale,
+      isPrecize: true,
+    };
+  }
+
   /**
    * @param {Object} newIndex
    */
diff --git a/src/parsers/manifest/dash/common/indexes/timeline/timeline_representation_index.ts b/src/parsers/manifest/dash/common/indexes/timeline/timeline_representation_index.ts
index 284ebaaefc..87724c2435 100644
--- a/src/parsers/manifest/dash/common/indexes/timeline/timeline_representation_index.ts
+++ b/src/parsers/manifest/dash/common/indexes/timeline/timeline_representation_index.ts
@@ -775,6 +775,33 @@ export default class TimelineRepresentationIndex implements IRepresentationIndex
     log.warn("Cannot add predicted segments to a `TimelineRepresentationIndex`");
   }
 
+  /**
+   * Returns the `duration` of each segment in the context of its Manifest (i.e.
+   * as the Manifest anounces them, actual segment duration may be different due
+   * to approximations), in seconds.
+   *
+   * NOTE: we could here do a median or a mean but I chose to be lazy (and
+   * more performant) by returning the duration of the first element instead.
+   * As `isPrecize` is `false`, the rest of the code should be notified that
+   * this is only an approximation.
+   * @returns {number}
+   */
+  getTargetSegmentDuration(): { duration: number; isPrecize: boolean } | undefined {
+    this._refreshTimeline();
+    const { timeline, timescale } = this._index;
+    if (timeline === null) {
+      return undefined;
+    }
+    const firstElementInTimeline = timeline[0];
+    if (firstElementInTimeline === undefined) {
+      return undefined;
+    }
+    return {
+      duration: firstElementInTimeline.duration / timescale,
+      isPrecize: false,
+    };
+  }
+
   /**
    * Returns `true` if the given object can be used as an "index" argument to
    * create a new `TimelineRepresentationIndex`.
diff --git a/src/parsers/manifest/dash/common/parse_adaptation_sets.ts b/src/parsers/manifest/dash/common/parse_adaptation_sets.ts
index 4409fe9436..a1665ac700 100644
--- a/src/parsers/manifest/dash/common/parse_adaptation_sets.ts
+++ b/src/parsers/manifest/dash/common/parse_adaptation_sets.ts
@@ -574,6 +574,18 @@ function createThumbnailTracks(
         log.warn("DASH: Invalid thumbnails Representation, no width information");
         continue;
       }
+
+      const start = representation.index.getFirstAvailablePosition() ?? undefined;
+      const end = representation.index.getEnd() ?? undefined;
+
+      let segmentDuration;
+      const targetDuration = representation.index.getTargetSegmentDuration();
+      if (targetDuration !== undefined && targetDuration.isPrecize) {
+        segmentDuration = targetDuration.duration;
+      } else {
+        log.warn("DASH: Cannot produce duration estimate for thumbnail track");
+      }
+
       tracks.push({
         id: representation.id,
         cdnMetadata: representation.cdnMetadata,
@@ -583,6 +595,9 @@ function createThumbnailTracks(
         width: representation.width,
         horizontalTiles: tileInfo.horizontalTiles,
         verticalTiles: tileInfo.verticalTiles,
+        start,
+        end,
+        segmentDuration,
       });
     }
   }
diff --git a/src/parsers/manifest/local/representation_index.ts b/src/parsers/manifest/local/representation_index.ts
index 3db39bc7ff..338c8fa4a4 100644
--- a/src/parsers/manifest/local/representation_index.ts
+++ b/src/parsers/manifest/local/representation_index.ts
@@ -202,6 +202,27 @@ export default class LocalRepresentationIndex implements IRepresentationIndex {
     log.warn("Cannot add predicted segments to a `LocalRepresentationIndex`");
   }
 
+  /**
+   * Returns the `duration` of each segment in the context of its Manifest (i.e.
+   * as the Manifest anounces them, actual segment duration may be different due
+   * to approximations), in seconds.
+   *
+   * NOTE: we could here do a median or a mean but I chose to be lazy (and
+   * more performant) by returning the duration of the first element instead.
+   * As `isPrecize` is `false`, the rest of the code should be notified that
+   * this is only an approximation.
+   * @returns {number}
+   */
+  getTargetSegmentDuration(): { duration: number; isPrecize: boolean } | undefined {
+    if (this._index.segments.length === 0) {
+      return undefined;
+    }
+    return {
+      duration: this._index.segments[0].duration,
+      isPrecize: false,
+    };
+  }
+
   _replace(newIndex: LocalRepresentationIndex): void {
     this._index.segments = newIndex._index.segments;
     this._index.loadSegment = newIndex._index.loadSegment;
diff --git a/src/parsers/manifest/metaplaylist/representation_index.ts b/src/parsers/manifest/metaplaylist/representation_index.ts
index 6a8643c8a7..3a6bdd7a99 100644
--- a/src/parsers/manifest/metaplaylist/representation_index.ts
+++ b/src/parsers/manifest/metaplaylist/representation_index.ts
@@ -227,6 +227,16 @@ export default class MetaRepresentationIndex implements IRepresentationIndex {
     return this._wrappedIndex.addPredictedSegments(nextSegments, currentSegment);
   }
 
+  /**
+   * Returns the `duration` of each segment in the context of its Manifest (i.e.
+   * as the Manifest anounces them, actual segment duration may be different due
+   * to approximations), in seconds.
+   * @returns {number}
+   */
+  getTargetSegmentDuration(): { duration: number; isPrecize: boolean } | undefined {
+    return this._wrappedIndex.getTargetSegmentDuration();
+  }
+
   /**
    * @param {Object} newIndex
    */
diff --git a/src/parsers/manifest/smooth/representation_index.ts b/src/parsers/manifest/smooth/representation_index.ts
index 72a011e75d..7a33c5bbb7 100644
--- a/src/parsers/manifest/smooth/representation_index.ts
+++ b/src/parsers/manifest/smooth/representation_index.ts
@@ -475,6 +475,30 @@ export default class SmoothRepresentationIndex implements IRepresentationIndex {
     );
   }
 
+  /**
+   * Returns the `duration` of each segment in the context of its Manifest (i.e.
+   * as the Manifest anounces them, actual segment duration may be different due
+   * to approximations), in seconds.
+   *
+   * NOTE: we could here do a median or a mean but I chose to be lazy (and
+   * more performant) by returning the duration of the first element instead.
+   * As `isPrecize` is `false`, the rest of the code should be notified that
+   * this is only an approximation.
+   * @returns {number}
+   */
+  getTargetSegmentDuration(): { duration: number; isPrecize: boolean } | undefined {
+    this._refreshTimeline();
+    const { timeline, timescale } = this._sharedSmoothTimeline;
+    const firstElementInTimeline = timeline[0];
+    if (firstElementInTimeline === undefined) {
+      return undefined;
+    }
+    return {
+      duration: firstElementInTimeline.duration / timescale,
+      isPrecize: false,
+    };
+  }
+
   /**
    * Replace this RepresentationIndex by a newly downloaded one.
    * Check if the old index had more information about new segments and re-add
diff --git a/src/parsers/manifest/types.ts b/src/parsers/manifest/types.ts
index b1171a932a..24855d0de0 100644
--- a/src/parsers/manifest/types.ts
+++ b/src/parsers/manifest/types.ts
@@ -147,6 +147,43 @@ export interface IParsedThumbnailTrack {
    * images contained vertically in a whole loaded thumbnail resource.
    */
   verticalTiles: number;
+  /**
+   * Starting `position` the first thumbnail of this thumbnail track applies to,
+   * if known.
+   */
+  start: number | undefined;
+  /**
+   * Ending `position` the last thumbnail of this thumbnail track applies to,
+   * if known.
+   */
+  end: number | undefined;
+  /**
+   * When loaded, thumbnails are part of so-called "segments" which may contain
+   * either a single thumbnail or a grid of them.
+   *
+   * This `segmentDuration` property indicates a duration in seconds each
+   * segment applies to. You might then want to divide that value by
+   * `horizontalTiles * verticalTiles` to get the duration each thumbnail
+   * applies to.
+   *
+   * Note that the last segment of a content may have a lower duration depending
+   * on the duration of the content.
+   *
+   * Set to `undefined` either the duration is unknown or if the duration
+   * depends from segment to segments.
+   *
+   * For example, with a `start` set to `10`, an `end` set to `21`, a
+   * `horizontalTiles` set to `2`, a `verticalTiles` set to `1` and a
+   * `SegmentDuration` set to `4`, there should be 3 segments, each with 2
+   * thumbnails:
+   * 1. A segment of 2 thumbnails for the seconds: 10-14
+   *    (The first thumbnail in that segment for 10-12, the second for 12-14)
+   * 2. A segment of 2 thumbnails for the seconds: 14-18
+   *    (The first thumbnail in that segment for 14-16, the second for 16-18)
+   * 3. A segment of 2 thumbnails for the seconds: 18-21 (the end)
+   *    (The first thumbnail in that segment for 18-20, the second for 20-21)
+   */
+  segmentDuration: number | undefined;
 }
 
 /** Representation of a "quality" available in an Adaptation. */
diff --git a/src/parsers/manifest/utils/__tests__/get_first_time_from_adaptations.test.ts b/src/parsers/manifest/utils/__tests__/get_first_time_from_adaptations.test.ts
index d4ee312d83..96290da68d 100644
--- a/src/parsers/manifest/utils/__tests__/get_first_time_from_adaptations.test.ts
+++ b/src/parsers/manifest/utils/__tests__/get_first_time_from_adaptations.test.ts
@@ -48,6 +48,9 @@ function generateRepresentationIndex(
     addPredictedSegments(): void {
       return;
     },
+    getTargetSegmentDuration() {
+      return undefined;
+    },
     _replace() {
       /* noop */
     },
diff --git a/src/parsers/manifest/utils/__tests__/get_last_time_from_adaptation.test.ts b/src/parsers/manifest/utils/__tests__/get_last_time_from_adaptation.test.ts
index 2eada3f1c3..11979c12e0 100644
--- a/src/parsers/manifest/utils/__tests__/get_last_time_from_adaptation.test.ts
+++ b/src/parsers/manifest/utils/__tests__/get_last_time_from_adaptation.test.ts
@@ -48,6 +48,9 @@ function generateRepresentationIndex(
     canBeOutOfSyncError(): true {
       return true;
     },
+    getTargetSegmentDuration() {
+      return undefined;
+    },
     _replace() {
       /* noop */
     },
diff --git a/src/public_types.ts b/src/public_types.ts
index 2ae3c3cafa..2b3bd6e628 100644
--- a/src/public_types.ts
+++ b/src/public_types.ts
@@ -1304,6 +1304,61 @@ export interface IThumbnailTrackInfo {
    * `image/jpeg` or `image/png`.
    */
   mimeType: string | undefined;
+  /**
+   * Starting `position` the first thumbnail of this thumbnail track applies to,
+   * if known.
+   */
+  start: number | undefined;
+  /**
+   * Ending `position` the last thumbnail of this thumbnail track applies to,
+   * if known.
+   */
+  end: number | undefined;
+  /**
+   * Individual thumbnails may be technically part of "segments" containing
+   * multiple consecutive thumbnails each.
+   *
+   * `thumbnailsPerSegment` is the number of `thumbnails` each segments have.
+   *
+   * For example you could have stored on the server a segment which is a grid
+   * of 2 x 3 (2 horizontal rows and 3 vertical columns) thumbnails, which the
+   * RxPlayer will load at once then "cut" the right way when calling
+   * `renderThumbnail`. In that example, `thumbnailsPerSegment` would be set to
+   * `6` (2*3).
+   *
+   * Note that the last segment of a content may contain less thumbnails as
+   * anounced here depending on the duration of the content.
+   *
+   * You may want to rely on this information alongside `segmentDuration` to
+   * construct a list of available thumbnails and/or of available segments of
+   * thumbnails.
+   */
+  thumbnailsPerSegment: number | undefined;
+  /**
+   * When loaded, thumbnails are part of so-called "segments" which may contain
+   * either a single thumbnail or a grid of them (@see `thumbnailsPerSegment`).
+   *
+   * This `segmentDuration` property indicates a duration in seconds each
+   * segment applies to. You might then want to divide that value by
+   * `thumbnailsPerSegment` to get the duration each thumbnail applies to.
+   *
+   * Note that the last segment of a content may have a lower duration depending
+   * on the duration of the content.
+   *
+   * Set to `undefined` either the duration is unknown or if the duration
+   * depends from segment to segments.
+   *
+   * For example, with a `start` set to `10`, an `end` set to `21`, a
+   * `thumbnailsPerSegment` set to `2` and a `SegmentDuration` set to
+   * `4`, there should be 3 segments, each with 2 thumbnails:
+   * 1. A segment of 2 thumbnails for the seconds: 10-14
+   *    (The first thumbnail in that segment for 10-12, the second for 12-14)
+   * 2. A segment of 2 thumbnails for the seconds: 14-18
+   *    (The first thumbnail in that segment for 14-16, the second for 16-18)
+   * 3. A segment of 2 thumbnails for the seconds: 18-21 (the end)
+   *    (The first thumbnail in that segment for 18-20, the second for 20-21)
+   */
+  segmentDuration: number | undefined;
 }
 
 /**