Support H.265

[xieenming]

Does ExoPlayer support H.265 (a.k.a HEVC) ?

Currently, if I play a RTSP/RTMP H.264 stream, ExoPlayer has no problem.
But when I play a RTSP/RTMP H.265 stream, ExoPlayer reports an error:

missing sprop-vps parameter

E/ExoPlayerImplInternal: Playback error
      com.google.android.exoplayer2.ExoPlaybackException: Source error
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:644)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:620)
        at android.os.Handler.dispatchMessage(Handler.java:102)
        at android.os.Looper.loopOnce(Looper.java:201)
        at android.os.Looper.loop(Looper.java:288)
        at android.os.HandlerThread.run(HandlerThread.java:67)
     Caused by: java.io.IOException: java.lang.IllegalArgumentException: missing sprop-vps parameter
        at com.google.android.exoplayer2.source.rtsp.RtspMediaPeriod$InternalListener.onSessionTimelineRequestFailed(RtspMediaPeriod.java:664)
        at com.google.android.exoplayer2.source.rtsp.RtspClient.dispatchRtspError(RtspClient.java:317)
        at com.google.android.exoplayer2.source.rtsp.RtspClient.access$700(RtspClient.java:75)
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspResponse(RtspClient.java:656)
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspMessage(RtspClient.java:507)
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.lambda$onRtspMessageReceived$0$com-google-android-exoplayer2-source-rtsp-RtspClient$MessageListener(RtspClient.java:500)
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener$$ExternalSyntheticLambda0.run(Unknown Source:4)
        at android.os.Handler.handleCallback(Handler.java:942)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loopOnce(Looper.java:201) 
        at android.os.Looper.loop(Looper.java:288) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 
     Caused by: com.google.android.exoplayer2.source.rtsp.RtspMediaSource$RtspPlaybackException: java.lang.IllegalArgumentException: missing sprop-vps parameter
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspResponse(RtspClient.java:656) 
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspMessage(RtspClient.java:507) 
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.lambda$onRtspMessageReceived$0$com-google-android-exoplayer2-source-rtsp-RtspClient$MessageListener(RtspClient.java:500) 
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener$$ExternalSyntheticLambda0.run(Unknown Source:4) 
        at android.os.Handler.handleCallback(Handler.java:942) 
        at android.os.Handler.dispatchMessage(Handler.java:99) 
        at android.os.Looper.loopOnce(Looper.java:201) 
        at android.os.Looper.loop(Looper.java:288) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 
     Caused by: java.lang.IllegalArgumentException: missing sprop-vps parameter
        at com.google.android.exoplayer2.util.Assertions.checkArgument(Assertions.java:54)
        at com.google.android.exoplayer2.source.rtsp.RtspMediaTrack.processH265FmtpAttribute(RtspMediaTrack.java:421)
        at com.google.android.exoplayer2.source.rtsp.RtspMediaTrack.generatePayloadFormat(RtspMediaTrack.java:266)
        at com.google.android.exoplayer2.source.rtsp.RtspMediaTrack.<init>(RtspMediaTrack.java:162)
        at com.google.android.exoplayer2.source.rtsp.RtspClient.buildTrackList(RtspClient.java:347)
        at com.google.android.exoplayer2.source.rtsp.RtspClient.access$1700(RtspClient.java:75)
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.onDescribeResponseReceived(RtspClient.java:690)
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspResponse(RtspClient.java:598)
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspMessage(RtspClient.java:507) 
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.lambda$onRtspMessageReceived$0$com-google-android-exoplayer2-source-rtsp-RtspClient$MessageListener(RtspClient.java:500) 
        at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener$$ExternalSyntheticLambda0.run(Unknown Source:4) 
        at android.os.Handler.handleCallback(Handler.java:942) 
        at android.os.Handler.dispatchMessage(Handler.java:99) 
        at android.os.Looper.loopOnce(Looper.java:201) 
        at android.os.Looper.loop(Looper.java:288) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 
E/EventLogger: playerFailed [eventTime=0.07, mediaPos=0.00, window=0, period=0, errorCode=ERROR_CODE_IO_UNSPECIFIED

[microkatz]

Hello @xieenming. Thank you for logging your enhancement request! It does help our team reprioritise tasks as we see users requesting features.

Exoplayer does support H265 to a degree. H265 requires encoding information contained in the sprop-vps, sprop_pps, and sprop-pps parameters sets. The RTSP server you are trying to stream from is not providing these values in the SDP layer of the RTSP Describe Response. Exoplayer currently requires these parameters in the Describe Response and does not read them further down the pipeline when handling the stream.

This is similar to the request to read sprop values from within the H264 bitstream. #10971

If you would like to test reading them from the stream, you can clone the repo and try the following changes.

• Comment out the sps related checks and code in RtspMediaTrack::processH265FmtpAttribute. Instead you will add these sprop-related attributes to the RtpPayloadFormat.format variable when parsing the bitstream in RtpH265Reader
• You will need to parse for the vps, sps and pps data in RtpH265Reader. You can use the code in H265Reader as a basis as that extracts the vps, sps, and pps data from the NAL units from the bitstream. You can update payloadFormat.format with the Format.Builder::buildUpon method after extracting the required data.
• Comment out the line trackOutput.format(payloadFormat.format) in RtpH265Reader::createTracks. This method outputs the format onto the SampleQueue which will create the decoder. You want to postpone this method call until after you have extracted the required vps, sps and pps data.
• Just like in H265Reader::endNalUnit, once you have extracted the required vps/sps/pps data in RtpH265Reader, you will then invoke trackOutput.format(payloadFormat.format) to put the format onto the SampleQueue to configure the decoder.

If you are successful, you are more than welcome to make a contribution to Exoplayer with these changes. We consider high quality pull requests. These should merge into the dev-v2 branch. Before a pull request can be accepted you must submit a Contributor License Agreement.

I'll leave this open as an enhancement but please feel free to comment back with questions.

[xieenming]

Hello @xieenming. Thank you for logging your enhancement request! It does help our team reprioritise tasks as we see users requesting features.

Exoplayer does support H265 to a degree. H265 requires encoding information contained in the sprop-vps, sprop_pps, and sprop-pps parameters sets. The RTSP server you are trying to stream from is not providing these values in the SDP layer of the RTSP Describe Response. Exoplayer currently requires these parameters in the Describe Response and does not read them further down the pipeline when handling the stream.

This is similar to the request to read sprop values from within the H264 bitstream. #10971

If you would like to test reading them from the stream, you can clone the repo and try the following changes.

• Comment out the sps related checks and code in RtspMediaTrack::processH265FmtpAttribute. Instead you will add these sprop-related attributes to the RtpPayloadFormat.format variable when parsing the bitstream in RtpH265Reader
• You will need to parse for the vps, sps and pps data in RtpH265Reader. You can use the code in H265Reader as a basis as that extracts the vps, sps, and pps data from the NAL units from the bitstream. You can update payloadFormat.format with the Format.Builder::buildUpon method after extracting the required data.
• Comment out the line trackOutput.format(payloadFormat.format) in RtpH265Reader::createTracks. This method outputs the format onto the SampleQueue which will create the decoder. You want to postpone this method call until after you have extracted the required vps, sps and pps data.
• Just like in H265Reader::endNalUnit, once you have extracted the required vps/sps/pps data in RtpH265Reader, you will then invoke trackOutput.format(payloadFormat.format) to put the format onto the SampleQueue to configure the decoder.

If you are successful, you are more than welcome to make a contribution to Exoplayer with these changes. We consider high quality pull requests. These should merge into the dev-v2 branch. Before a pull request can be accepted you must submit a Contributor License Agreement.

I'll leave this open as an enhancement but please feel free to comment back with questions.

@microkatz Thanks a lot for your quick reply. I will test.

[Vane101]

@xieenming also working with a lot of hikvision camera's all with different parameters being sent, any luck on your side?

[AnonymousWebHacker]

Actually, what happens is that many devices do not have support for HEVC (except for 10bit) at the hardware level, so to play the videos, you need a player capable of transcoding at the software level, such as MPV, MX alternatives. Player..etc

Exoplayer is something universal and used in many other projects, I don't know why it's hard for them to add support for software coding to exoplayer

[icbaker]

@AnonymousWebHacker

Actually, what happens is that many devices do not have support for HEVC (except for 10bit) at the hardware level, so to play the videos, you need a player capable of transcoding at the software level, such as MPV, MX alternatives. Player..etc

All android devices have been required to support HEVC since Android 5.0 (Lollipop, launched in November 2014): https://developer.android.com/guide/topics/media/media-formats#video-codecs

This issue is specifically tracking supporting H.265 in RTSP, which requires changes in ExoPlayer to parse the video bitstream out of the RTSP packets - it's not related to device-level H.265 decoding support, nor related to software decoding with bundled video decoders, so I don't think your comment is particularly relevant.

[AnonymousWebHacker]

@AnonymousWebHacker

Actually, what happens is that many devices do not have support for HEVC (except for 10bit) at the hardware level, so to play the videos, you need a player capable of transcoding at the software level, such as MPV, MX alternatives. Player..etc

All android devices have been required to support HEVC since Android 5.0 (Lollipop, launched in November 2014): https://developer.android.com/guide/topics/media/media-formats#video-codecs

This issue is specifically tracking supporting H.265 in RTSP, which requires changes in ExoPlayer to parse the video bitstream out of the RTSP packets - it's not related to device-level H.265 decoding support, nor related to software decoding with bundled video decoders, so I don't think your comment is particularly relevant.

Either RTSP/RTMP or direct raw playback, without segments, Exoplayer is not able to play an h265 where the device does not support it by hardware

[rup-sarma]

@AnonymousWebHacker

Actually, what happens is that many devices do not have support for HEVC (except for 10bit) at the hardware level, so to play the videos, you need a player capable of transcoding at the software level, such as MPV, MX alternatives. Player..etc

All android devices have been required to support HEVC since Android 5.0 (Lollipop, launched in November 2014): https://developer.android.com/guide/topics/media/media-formats#video-codecs
This issue is specifically tracking supporting H.265 in RTSP, which requires changes in ExoPlayer to parse the video bitstream out of the RTSP packets - it's not related to device-level H.265 decoding support, nor related to software decoding with bundled video decoders, so I don't think your comment is particularly relevant.

Either RTSP/RTMP or direct raw playback, without segments, Exoplayer is not able to play an h265 where the device does not support it by hardware

yes you are saying correct.

[almahirpm]

hello!
Just adding here another request to support H265 as we are working in a project with an IP Camera.
Thanks.

[AnonymousWebHacker]

hello! Just adding here another request to support H265 as we are working in a project with an IP Camera. Thanks.

If you wait for this feature, I don't think they have plans to fix this.

So strongly, I don't know your project, the best thing I can recommend is to use mvp-android and it will solve at least 99% of your problems