Constructors for RTC encoded frames with custom metadata

[guidou]

こんにちは TAG-さん!

I'm requesting a TAG review of [feature name].

[One paragraph summary of idea, ideally copy-pasted from Explainer introduction]

In some use cases, a Web application would like to have control over the metadata attached to RTC encoded frames. One example would be adjusting timestamps to introduce a delay in playout. Another one would be relaying incoming encoded data from one or more input peer connections to one or more output PeerConnections use case 3.2.2, which requires encoded frame metadata to conform with the requirements of its destination. The proposed solution consists of a constructor for encoded frames that takes data from an existing frame + customized metadata.

Note: Fully supporting use case 3.2.2 also requires an additional API for inserting encoded frames into an RTCPeerConnection, for which a separate TAG review will be requested.

• Explainer¹ (minimally containing user needs and example code):
  • Minimal explainer: https://github.com/guidou/webrtc-extensions/blob/main/constructor-explainer.md
  • Explainer for use case 3.2.2
    https://github.com/guidou/webrtc-extensions/blob/main/relay-explainer.md
• User research: N/A
• Security and Privacy self-review²: https://github.com/guidou/webrtc-extensions/blob/main/sp-self-review.md
• GitHub repo (if you prefer feedback filed there): https://github.com/w3c/webrtc-encoded-transform/pull/223/files (please don't provide feedback there)
• Primary contacts (and their relationship to the specification):
  • Guido Urdaneta, (guidou), Google
  • Palak Agarwal, (palak8669), Google
  • Harald Alvestrand, (alvestrand), Google
• Organization/project driving the design: Google
• External status/issue trackers for this feature (publicly visible, e.g. Chrome Status): https://chromestatus.com/feature/5116073827893248

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• The group where the incubation/design work on this is being done (or is intended to be done in the future): W3C WebRTC Working Group
• The group where standardization of this work is intended to be done ("unknown" if not known): W3C WebRTC Working Group
• Existing major pieces of multi-stakeholder review or discussion of this design: Constructor frame w3c/webrtc-encoded-transform#223 (review of PR adding the constructors)
• Major unresolved issues with or opposition to this design: No major issues. Some details about the API surface and precise wording of the spec are under discussion at Constructor frame w3c/webrtc-encoded-transform#223
• This work is being funded by: Google

Please provide feedback here (not on w3c/webrtc-encoded-transform#223).

[martinthomson]

Hi @guidou,

I've looked at this explainer several times and now the pull request. I have to confess that I still do not understand what it is that you are trying to achieve.

Our explainer explainer suggests that you outline use cases for the proposal, starting with end user objectives. I don't see that here, just "we would like to attach metadata to encoded frames".

I see that the WebKit issue was closed as "invalid". Should we be doing the same?