defcom5-rockchip Posted August 13 Posted August 13 (edited) Cross-posting this because it took a while to untangle, and most guides out there are either vendor-BSP/rkmpp (the wrong stack for mainline) or they stop at "it says hardware accelerated" without checking whether it actually is. Tested on an Orange Pi 5B, mainline Debian 13 (trixie), kernel 7.1.3+deb13-arm64, panthor + PanVK, Wayfire. Should apply to any RK3588/RK3588S board on a mainline stateless-decode kernel (Rock 5B, OPi 5, etc.). Corrections welcome; I'd rather this be right than mine. First, the thing that trips everyone up: on mainline, rkmpp is not your path. Rockchip's MPP library (the thing that hardware-decodes at ~9% CPU on the vendor BSP images) talks to /dev/mpp_service. That device only exists on the vendor kernel. On mainline/panthor there is no mpp_service and there never will be, so every rkmpp-based ffmpeg/Chromium build is a dead end here. And no, the GPU doesn't help. PanVK exposes zero VK_KHR_video_decode extensions. Mali-G610 has no video engine. Vulkan composites and scales the decoded frames; it never produces them. The decode blocks are separate silicon (rkvdec, Hantro) behind a separate kernel API. The mainline path is V4L2 stateless (the request API), the work Collabora and Jernej Skrabec carry upstream, with the userspace ffmpeg side maintained by Kwiboo (Jonas Karlman). The kernel drivers are already there; on my box the v4l2_h264/v4l2_vp9 helpers are loaded and bound to rkvdec/hantro_vpu out of the box. What's missing is a userspace that speaks the request API. Stock Debian ffmpeg doesn't, so it silently decodes in software (1080p H.264 is about 600% CPU, six cores pegged). Check what your kernel actually exposes before anything else: for v in /dev/video*; do echo "== $v =="; v4l2-ctl -d $v --list-formats-out 2>/dev/null | grep -oE "'[A-Z0-9]{4}'"; done On my 7.1.3 kernel: /dev/video1 (rkvdec) -> S264 S265 (H.264, HEVC) /dev/video2 (rk3568-vpu) -> S264 VP8F MG2S (H.264, VP8, MPEG-2) /dev/video4 (rk3588-av1-vpu-dec) -> AV1F (AV1, advertises up to 4096x2304) VP9 is NOT exposed on this kernel; that is a beryllium/7.2-rc5+ restore (VP9-on-VDPU381). More on why that barely matters below. Part A: system-wide HW decode (mpv, VLC, GStreamer, everything) Build Kwiboo's FFmpeg fork, branch v4l2-request-n7.1.3. The version choice is deliberate: ffmpeg 7.1.3 is libavcodec.so.61, the same soname as Debian trixie's 7.1.5, so it is a genuine drop-in. You swap one component and every libav app hardware-decodes with zero rebuilds. (The newer n8.x branches are .so.62 and become an island: only the one binary you point at them benefits.) sudo apt install build-essential git nasm pkg-config libdrm-dev libudev-dev v4l-utils \ libx264-dev libx265-dev libvpx-dev libopus-dev libvorbis-dev libmp3lame-dev libdav1d-dev libva-dev git clone -b v4l2-request-n7.1.3 https://github.com/Kwiboo/FFmpeg ffmpeg-7131 && cd ffmpeg-7131 ./configure --prefix=/usr/local --enable-shared --disable-static \ --enable-gpl --enable-version3 --enable-v4l2-request --enable-libdrm \ --enable-libx264 --enable-libx265 --enable-libvpx --enable-libopus \ --enable-libvorbis --enable-libmp3lame --enable-libdav1d --disable-vulkan --disable-doc make -j$(nproc) Build gotcha that cost me an hour: do NOT add --enable-libv4l2. On n7.1.3 its libavdevice/v4l2.c fails to compile against current libv4l headers (the SET_WRAPPERS macro). That is the webcam-capture indev, nothing to do with request-API decode. Leave it off. Test before installing anything system-wide, with stock mpv pointed at the fresh libs: LD_LIBRARY_PATH=/usr/local/lib mpv --hwdec=v4l2request-copy --gpu-api=vulkan yourfile.mp4 You want to see "Using hardware decoding (v4l2request-copy)" and pixfmt drm_prime. My numbers: 1080p H.264 went from about 600% CPU (software) to about 37% of one core. Then package it as a .deb that Provides/Conflicts/Replaces the eight Debian libav* packages (= 7:7.1.5-0+deb13u1) and installs your .so.61 plus symlinks into /usr/lib/aarch64-linux-gnu/. apt treats it as satisfying the deps, so mpv/vlc/gstreamer/qt-multimedia all stay installed and all start hardware-decoding. Cache the stock debs first (apt download the eight) so rollback is one command. Honest tradeoff: this replaces your system libav with a fork, so (a) the blast radius is every media app, and (b) you now own libav security. When Debian bumps 7.1.x for a CVE, you rebuild the one deb. I think that is a fair price for one-component maintenance versus forking every app. Your call. Part B: in-browser HW decode (Chromium), and a myth to kill You do not need a forked Chromium, and you do not need the libva-v4l2-request VA-API shim for the browser. Both are things the vendor/rkmpp world needs; mainline does not. Two facts do the work: 1. Debian trixie already ships Chromium 151, past the upstream 150 ANGLE roll that fixed the old VP9 green-artifact bug. Stock: apt install chromium. 2. Chromium bundles its own ffmpeg, and its AcceleratedVideoDecoder talks V4L2-stateless straight to /dev/videoN. So Part A's libav swap is irrelevant to the browser; the two paths are independent. Debian's chromium launcher sources /etc/chromium.d/*, so drop a flag file: # /etc/chromium.d/50-hwdecode export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --ozone-platform-hint=auto \ --enable-features=AcceleratedVideoDecoder,AcceleratedVideoDecodeLinuxGL,AcceleratedVideoDecodeLinuxZeroCopyGL \ --ignore-gpu-blocklist --enable-gpu-rasterization --enable-zero-copy" Notes: - The flag that actually gates it is AcceleratedVideoDecoder. The older AcceleratedVideoDecodeLinuxV4L2 no longer exists in 150+ and is silently ignored. - It needs a Wayland session. --ozone-platform-hint=auto picks it under Wayfire/wlroots. Result: browser H.264 held /dev/video1 at about 23% of one core. AV1 held /dev/video4. Which brings us to the part that actually stumped me. Part 😄 the 4K@30 gotcha. cma=256M is too small, and it fails silently. This is the reason I wrote the post. In-browser AV1 worked perfectly at 1080p: decoder node held, about 26% of one core, 95% idle. Force the same av01 stream to 2160p and it dropped straight back to software: one Chromium process at about 400%, /dev/video4 idle, and stats-for-nerds still reading av01. Same codec, two outcomes. The tell that it is not the codec: the AV1 block advertises up to 4096x2304, so 4K is in spec. It is not a capability ceiling. The kernel log named it exactly: cma: __cma_alloc_frozen: alloc failed, req-size: 3421 pages, ret: -16 hantro_postproc_alloc -> dma_alloc_attrs -> hantro_start_streaming The Hantro decoder needs a roughly 13 MB contiguous buffer for a 4K frame, and the default cma=256M pool was too fragmented to hand it out. hantro_start_streaming bails, and Chromium does what it is designed to do: falls back to software, silently, with no error on screen. At 1080p the buffer is a quarter the size and fits; at 4K it does not. Fix (a boot argument, not a kernel rebuild): bump the CMA reservation. On the mainline extlinux setup that is: # /boot/extlinux/extlinux.conf -> in the "append ..." line(s), and # /etc/default/u-boot -> U_BOOT_PARAMETERS (the template extlinux regenerates from) cma=256M becomes cma=1G Edit both, or a future u-boot-update reverts you. Reboot. (The other fix you will see mentioned is CONFIG_VSI_IOMMU=y, which lets the AV1 VPU scatter-gather through the IOMMU and sidestep the contiguous requirement, but that is a kernel rebuild. The boot arg is free.) After cma=1G (CmaTotal 1 GB): the same 4K av01 fullscreen stream went from four pegged cores to 58% of one core, 91% idle, decoder node held, zero allocation failures, and it ran cooler (47C to 36C). 256 MB is simply too small a house for 4K decode on this SoC. Ship 512M or more; I use 1G. Codec support, honestly (this kernel): H.264 yes, HEVC yes, AV1 yes (including 4K with the CMA bump), VP8/MPEG-2 yes, VP9 is software (not in 7.1.3; needs beryllium/7.2+). In practice the VP9 gap barely bites: because Chromium now advertises AV1 decode, YouTube negotiates av01 at every resolution I tested (1080p and 4K), and only reaches for VP9 in ads. So I am leaving VP9 to upstream rather than chasing a custom kernel. How to actually verify HW decode (do not trust chrome://gpu): it reports a capability ("Video Decode: Hardware accelerated") that stays true even while a given stream decodes in software. The only honest test is who holds the decode node, sampled during steady playback (not during an ad or a quality ramp; those transients will lie to a CPU snapshot): fuser /dev/video1 /dev/video2 /dev/video4 # whoever holds it is the real decoder Cross-check in the browser at chrome://media-internals: kVideoDecoderName is V4L2VideoDecoder, and kIsPlatformVideoDecoder is true. The build this ran on (for reproducibility) Nothing exotic; the point is it is stock trixie + backports, not a vendor BSP. This is a mainline Debian desktop I put together for the 5B (my own image line, "Pi Phreak"): - Board: Xunlong Orange Pi 5B, RK3588S (4x Cortex-A76 + 4x Cortex-A55), Mali-G610 MC4, 16 GB - OS: Debian 13 (trixie), with trixie-backports enabled - Kernel: 7.1.3+deb13-arm64, mainline (stateless V4L2 decoders, panthor), booted with cma=1G - GPU stack: Mesa 26.1.2 (~bpo13), Vulkan 1.4 via PanVK on Mali-G610 (loader 1.4.309), GLES 3.1 - Compositor: Wayfire 0.11.0 on wlroots 0.20.2 (built from source; trixie ships 0.9), plus swaybg / waybar / wofi / foot - Media: Chromium 151 (stock Debian) + the /etc/chromium.d flags, mpv 0.40, system libav = the Kwiboo v4l2-request 7.1.3 drop-in (7:7.1.3-phreak1, soname-matched to 7.1.5) Two things to flag for anyone reproducing this: Mesa/PanVK come from trixie-backports (stock trixie Mesa is older and PanVK is not 1.4 there, a separate "package version is not hardware capability" trap), and the 16 GB headroom is what makes a 1 GB CMA reservation painless. On an 8 GB board I would still bump past 256M (try 512M) but keep an eye on free RAM. Credits: Kwiboo (Jonas Karlman) for the FFmpeg v4l2-request fork; Collabora / Jernej Skrabec for the mainline stateless decoders; the dongioia/rock5bplus-rkvdec2 writeup for the Chromium-150 flag findings; beryllium-org for the packaging reference and the VP9/VDPU381 kernel note. I only assembled and measured; the hard part is theirs. Happy to answer questions or hand over the packaging scripts if it is useful. Peace, Defcom5-Rockchip "Got 2 Claudes and A Turntable" Sorry I got my Flagship baking and excited working on my new Debian build with the above outlined and the honey its f in due list! 2 things I've learned about AI. 1. Don't Trust Them, you have to tell them like they are children. (They act like they know). 2. Don't Trust Them. Edited August 15 by defcom5-rockchip 2 Quote
Blind55 Posted August 23 Posted August 23 (edited) @defcom5-rockchip I am very grateful for your explanations here. This has been extremely helpful. Cheers! Edited August 23 by Blind55 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.