Switch RTCP handling to pion/interceptor

[aler9]

Features that are currently missing in gortsplib's receiver reports and sender reports, and are handled correctly in pion/interceptor:

• support for reordered packets in receiver reports
• NACKs

[yelodevopsi]

Hi @aler9 ! What is the current status on this one?

We're really eager to get some stats for RTSP-sessions (accessible in Prometheus metrics or in API), especially for jitter and package loss.

Do you have any basic overview of the migration process to get this working from the webrtc package, over to to RTSP on this matter? Because if not currently in progress, I/we might try to start the process because of the need for it now. Also, if you could provide a rough overview of the needed work, this would help alot.

Some key features, that might overlap with this endeavour:

• Would it be possible to get some features equivalent to rtpjitterbuffer (gstreamer) functionality here? E.g setting latency manually to compensate for out-of-order packages etc.

• We're using the mediamtx repo currently, and see that the most optimal way would be to avoid using gstreamer all together because of the great cross-platform support blueenviron's go-libs have 😺

Is the gortsplib currently utilizing any optimalisation/adjustments in the videostream based on the RTCP for RTSP-sessions?

[aler9]

Hello Magnus,

What is the current status on this one?

The switch to pion/interceptor was put on hold because it requires an additional layer that decreases performance. Most of its features are already implemented here.

We're really eager to get some stats for RTSP-sessions (accessible in Prometheus metrics or in API), especially for jitter and package loss

This might be a good future enhancement, currently the only two exposed stats are ServerSession.BytesReceived() and ServerSession.BytesSent(), we might add ServerSession.Stats() in order to obtain jitter and packet loss too, that are currently available inside rtpreorderer and rtcpreceiver. I've created a feature request here: #556

Would it be possible to get some features equivalent to rtpjitterbuffer (gstreamer) functionality here?

This is already present in a real-time variant, it's inside rtpreorderer. The maximum number of lost packet is fixed to 64.

Is the gortsplib currently utilizing any optimalisation/adjustments in the videostream based on the RTCP for RTSP-sessions?

Video adjustments like dynamic bitrate cannot be achieved since an RTSP server or client doesn't have access to the video encoder, contrarily to browser-based WebRTC, therefore video packets can only be routed without touching them.

Resending lost video frames through NACKs would be a very good enhancement but there is no client that supports it (VLC, FFmpeg, GStreamer), thus it would be useless to implement it.

The only thing that you can do to improve the transmission rate with RTSP is using Forward Error Compensation (FEC), that consists in sending the same packet multiple times at once. You can generate packets by following the specification and send them through gortsplib.

[yelodevopsi]

Thank you so much for the detailed reply @aler9 ! 👏

We have a lot of unstable (wifi/wireless linked) RTSP-cameras we need to connect to.
Would it be a requirement that the RTSP-cameras support FEC, to stabilize the connection between the source and RTSP-server?

Btw, we're currently benchmarking TCP vs UDP on the streams, but are interested in all ways the gortsplib by default handles instable/bad streams, and what potential features/param adjustments we can use/create to manually improve the streams.

From what you write, I cannot find alot of FEC implementations (only some research-papers) which use this.

Thanks in advance

[yelodevopsi]

From what I've used in gstreamer, mainly three params/features would be nice to manually adjust:

• latency (buffer, and jitter-buffer on the received source-stream): Set manually in ms
• dropOnLatency: true/false
• udp-buffer-size: (in bytes)

Are any if these already implemented as with any default-value?
Also, when you mention: rtpreorderer - What is the default buffer/latency here?

[aler9]

Also, when you mention: rtpreorderer - What is the default buffer/latency here?

There is no latency, because the server doesn't need to emit frames with a constant interval. The server sends out frames to clients as soon as they are available. It's the client responsibility to keep a buffer with a latency and emit frames with a constant interval.

[tmatth]

This might be out of scope of the original issue but is it possible currently to handle PLI RTCP feedback packets (in the context of a mediamtx server being sent them by an RTSP client)? In my case I'm needing to intercept them and forward them upstream (as mediamtx is proxying an RTSP stream from another source).