Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hardware H264/H265(HEVC) decode acceleration for Raspberry Pi 4 64bit #129

Open
MELSunny opened this issue Apr 20, 2022 · 5 comments
Open

Hardware H264/H265(HEVC) decode acceleration for Raspberry Pi 4 64bit #129

MELSunny opened this issue Apr 20, 2022 · 5 comments
Labels
enhancement New feature or request

Comments

@MELSunny
Copy link

MELSunny commented Apr 20, 2022 •
edited
Loading

Raspberry pi officially announced the 64bit OS on Feb 2022.
On 64bit Raspbian, I found the following hardware acceleration is available on ffmpeg of rpi's distribution RPi-Distro/ffmpeg.
H264 decode/encode
H265 decode

It just need a single commend to enable the V4L2 decoder. (For more informations: jc-kynesim/hello_drmprime)
sudo dtoverlay rpivid-v4l2

I downloaded two sample h265 videos

wget http://www.jell.yfish.us/media/jellyfish-3-mbps-hd-hevc.mkv
wget http://www.jell.yfish.us/media/jellyfish-3-mbps-hd-hevc-10bit.mkv

and tried to convert videos from h265/h265-10bit to h264 with RPi-Distro/ffmpeg, here is the output.

ffmpeg -hwaccel drm -i file:"jellyfish-3-mbps-hd-hevc.mkv" -codec:v h264_v4l2m2m -b:v 16M -codec:a aac -f mp4 jellyfish-3-mbps-hd-hevc_converted.mkv
ffmpeg version 4.3.3-0+rpt3+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --extra-version=0+rpt3+deb11u1 --toolchain=hardened --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom 
--enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq 
--enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx 
--enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-mmal --enable-neon --enable-v4l2-request --enable-libudev --enable-epoxy --enable-sand --libdir=/usr/lib/aarch64-linux-gnu --arch=arm64 --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-vout-drm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, matroska,webm, from 'file:jellyfish-3-mbps-hd-hevc.mkv':
  Metadata:
    COMPATIBLE_BRANDS: iso4hvc1iso6
    MAJOR_BRAND     : iso4
    MINOR_VERSION   : 1
    ENCODER         : Lavf56.3.100
  Duration: 00:00:30.10, start: 0.067000, bitrate: 2827 kb/s
    Stream #0:0(und): Video: hevc (Main), yuv420p(tv), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 1k tbn, 29.97 tbc (default)
    Metadata:
      CREATION_TIME   : 2016-02-04 02:54:49
      LANGUAGE        : und
      HANDLER_NAME    : [email protected]
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_v4l2m2m))
Press [q] to stop, [?] for help
[hevc @ 0x5587ac9df0] Hwaccel V4L2 HEVC stateless V2; devices: /dev/media0,/dev/video19
[h264_v4l2m2m @ 0x5587ade130] Using device /dev/video11
[h264_v4l2m2m @ 0x5587ade130] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
[h264_v4l2m2m @ 0x5587ade130] requesting formats: output=YU12 capture=H264
[h264_v4l2m2m @ 0x5587ade130] Failed to set gop size: Invalid argument
Output #0, mp4, to 'jellyfish-3-mbps-hd-hevc_converted.mkv':
  Metadata:
    COMPATIBLE_BRANDS: iso4hvc1iso6
    MAJOR_BRAND     : iso4
    MINOR_VERSION   : 1
    encoder         : Lavf58.45.100
    Stream #0:0(und): Video: h264 (h264_v4l2m2m) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 16000 kb/s, 29.97 fps, 11988 tbn, 29.97 tbc (default)
    Metadata:
      CREATION_TIME   : 2016-02-04 02:54:49
      LANGUAGE        : und
      HANDLER_NAME    : [email protected]
      encoder         : Lavc58.91.100 h264_v4l2m2m
[mp4 @ 0x5587adcab0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[mp4 @ 0x5587adcab0] Encoder did not produce proper pts, making some up.
frame=  900 fps= 50 q=-0.0 Lsize=   52408kB time=00:00:29.99 bitrate=14312.3kbits/s speed=1.65x
video:52403kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.009842%

ffmpeg -hwaccel drm -i file:"jellyfish-3-mbps-hd-hevc-10bit.mkv" -codec:v h264_v4l2m2m -b:v 16M -pix_fmt yuv420p -codec:a aac -f mp4 jellyfish-3-mbps-hd-hevc-10bit_converted.mkv
ffmpeg version 4.3.3-0+rpt3+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --extra-version=0+rpt3+deb11u1 --toolchain=hardened --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d 
--enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex 
--enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-mmal --enable-neon --enable-v4l2-request 
--enable-libudev --enable-epoxy --enable-sand --libdir=/usr/lib/aarch64-linux-gnu --arch=arm64 --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-vout-drm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, matroska,webm, from 'file:jellyfish-3-mbps-hd-hevc-10bit.mkv':
  Metadata:
    COMPATIBLE_BRANDS: iso4hvc1iso6
    MAJOR_BRAND     : iso4
    MINOR_VERSION   : 1
    ENCODER         : Lavf56.3.100
  Duration: 00:00:30.10, start: 0.067000, bitrate: 2986 kb/s
    Stream #0:0(und): Video: hevc (Main 10), yuv420p10le(tv), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 1k tbn, 29.97 tbc (default)
    Metadata:
      CREATION_TIME   : 2016-02-04 02:39:21
      LANGUAGE        : und
      HANDLER_NAME    : [email protected]
File 'jellyfish-3-mbps-hd-hevc-10bit_converted.mkv' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_v4l2m2m))
Press [q] to stop, [?] for help
[hevc @ 0x5586f19240] Hwaccel V4L2 HEVC stateless V2; devices: /dev/media0,/dev/video19
[h264_v4l2m2m @ 0x5586f2d140] Using device /dev/video11
[h264_v4l2m2m @ 0x5586f2d140] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
[h264_v4l2m2m @ 0x5586f2d140] requesting formats: output=YU12 capture=H264
[h264_v4l2m2m @ 0x5586f2d140] Failed to set gop size: Invalid argument
Output #0, mp4, to 'jellyfish-3-mbps-hd-hevc-10bit_converted.mkv':
  Metadata:
    COMPATIBLE_BRANDS: iso4hvc1iso6
    MAJOR_BRAND     : iso4
    MINOR_VERSION   : 1
    encoder         : Lavf58.45.100
    Stream #0:0(und): Video: h264 (h264_v4l2m2m) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 16000 kb/s, 29.97 fps, 11988 tbn, 29.97 tbc (default)
    Metadata:
      CREATION_TIME   : 2016-02-04 02:39:21
      LANGUAGE        : und
      HANDLER_NAME    : [email protected]
      encoder         : Lavc58.91.100 h264_v4l2m2m
[mp4 @ 0x5586f2bac0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[mp4 @ 0x5586f2bac0] Encoder did not produce proper pts, making some up.
frame=  900 fps= 32 q=-0.0 Lsize=   52526kB time=00:00:29.99 bitrate=14344.7kbits/s speed=1.08x
video:52521kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.009842%

RPI4/4GB(RAM) CPU load usage during the conversion is around 0.9 on average, and the converted videos can be correctly played.
However, jellyfin-ffmpeg is not working with the following error.
I am grateful if jellyfin team could provide hardware encode acceleration for RPI, and make the RPI a cost-effective home media centre with jellyfin.

/usr/lib/jellyfin-ffmpeg/ffmpeg -hwaccel drm -i file:"jellyfish-3-mbps-hd-hevc.mkv" -codec:v h264_v4l2m2m -b:v 16M -codec:a aac -f mp4 jellyfish-3-mbps-hd-hevc_converted.mkv
ffmpeg version 4.4.1-Jellyfin Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-static --enable-gmp --enable-gnutls --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray 
--enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --toolchain=hardened --enable-cross-compile --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu-
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, matroska,webm, from 'file:jellyfish-3-mbps-hd-hevc.mkv':
  Metadata:
    COMPATIBLE_BRANDS: iso4hvc1iso6
    MAJOR_BRAND     : iso4
    MINOR_VERSION   : 1
    ENCODER         : Lavf56.3.100
  Duration: 00:00:30.10, start: 0.067000, bitrate: 2827 kb/s
  Stream #0:0(und): Video: hevc (Main), yuv420p(tv), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 1k tbn, 29.97 tbc (default)
    Metadata:
      CREATION_TIME   : 2016-02-04 02:54:49
      LANGUAGE        : und
      HANDLER_NAME    : [email protected]
File 'jellyfish-3-mbps-hd-hevc_converted.mkv' already exists. Overwrite? [y/N] y
Device creation failed: -14.
[hevc @ 0x559a63efc0] No device available for decoder: device type drm needed for codec hevc.
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_v4l2m2m))
Device setup failed for decoder on input stream #0:0 : Bad address
@MELSunny MELSunny added the enhancement New feature or request label Apr 20, 2022
@nyanmisaka
Copy link
Member

Blocked by jc-kynesim/rpi-ffmpeg#42

@Shadowghost Shadowghost mentioned this issue Jun 7, 2022
Closed
@da-anda
Copy link

da-anda commented Jul 21, 2022 •
edited
Loading

is the linked blocker the reason why there is no "enable h265 HW decoding" setting in JF atm? h264 is already available.

@NickM-27 NickM-27 mentioned this issue Jul 31, 2022
Closed
@PimTournaye
Copy link

Hi, has there been any progress towards this on this side of things? Sadly don't know enough about low level stuff to help out with this, but if I can help any other way, please let me know.

@Arthedian
Copy link

Hi,
@nyanmisaka could you please tell us if there is any progress? It would help a lot.

@nyanmisaka
Copy link
Member

@Arthedian
This has been blocked by the fact that scale_v4l2m2m video scaler it not available in upstream. I'm not versed in Linux V4L2 and the closed source Broadcom GPU so you'd better turn to https://github.com/jc-kynesim/rpi-ffmpeg for help. It seems the author jc-kynesim is an employee of the RPi Foundation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@da-anda @Arthedian @MELSunny @nyanmisaka @PimTournaye