Understanding Hardware-Accelerated Video Decoding

robertoj · April 17, 2023

I have been trying to get ffmpeg patched and compiled, with compilation errors.

But the armbian default ffmpeg has this:

ffmpeg -hide_banner -h decoder=h264

Decoder h264 [H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10]:
    General capabilities: dr1 delay threads
    Threading capabilities: frame and slice
    Supported hardware devices: vaapi vdpau
H264 Decoder AVOptions:
-enable_er         <boolean>    .D.V...... Enable error resilience on damaged frames (unsafe) (default auto)
-x264_build        <int>        .D.V...... Assume this x264 version if no x264 version found in any SEI (from -1 to INT_MAX) (default -1)

What should it look like, when H3-hardware acceleration is enabled?

With ffplay and mpv, I am able to play a 480p H264 video with 85% CPU (across 4 cores), and a 720p H264 video with 300% CPU... so I am pretty sure it is CPU-decoded

I tried gstreamer and it still decodes with the cpu:

sudo apt install gstreamer1.0-x gstreamer1.0-vaapi gstreamer1.0-libav gstreamer1.0-plugins-base gstreamer1.0-plugins-base-apps gstreamer1.0-alsa gstreamer1.0-x

gst-launch-1.0 playbin uri=flie:~/myfile.mp4

Edited April 17, 2023 by robertoj

usual user · April 17, 2023

13 hours ago, robertoj said:

What should it look like, when H3-hardware acceleration is enabled?

When the request-api is patched in properly, you are looking for:

 ffmpeg -hide_banner -decoders | grep v4l2
 V..... h263_v4l2m2m         V4L2 mem2mem H.263 decoder wrapper (codec h263)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 decoder wrapper (codec h264)
 V..... hevc_v4l2m2m         V4L2 mem2mem HEVC decoder wrapper (codec hevc)
 V..... mpeg1_v4l2m2m        V4L2 mem2mem MPEG1 decoder wrapper (codec mpeg1video)
 V..... mpeg2_v4l2m2m        V4L2 mem2mem MPEG2 decoder wrapper (codec mpeg2video)
 V..... mpeg4_v4l2m2m        V4L2 mem2mem MPEG4 decoder wrapper (codec mpeg4)
 V..... vc1_v4l2m2m          V4L2 mem2mem VC1 decoder wrapper (codec vc1)
 V..... vp8_v4l2m2m          V4L2 mem2mem VP8 decoder wrapper (codec vp8)
 V..... vp9_v4l2m2m          V4L2 mem2mem VP9 decoder wrapper (codec vp9)

 ffmpeg -hide_banner -hwaccels
 Hardware acceleration methods:
 vdpau
 cuda
 vaapi
 drm
 opencl
 vulkan

And mpv (shift i) will use it via the drm hwaccel:

mpv.png.f260ba67de9db62214880f440e655e88.png

13 hours ago, robertoj said:

I tried gstreamer and it still decodes with the cpu

Either your used gstreamer version is too old to have out-of-the-box support at all, or the build of a current mainline version has not activated the necessary components.

robertoj · April 17, 2023

Thank you 👍🏽

How should I check, whether my Linux would enable the hardware acceleration? My only datapoint is that I have the sunxi-cedrus.ko kernel module, and I can load it fine.

what libraries are on top of it?

what ffmpeg patches did you use?

Is that a vanilla, or patched MPV?

(sorry, probably those questions were asked already a couple of times… but … new year… gotta ask again)

robertoj · April 18, 2023

Thank you 👍🏽

How should I check, whether my Linux would enable the hardware acceleration? My only datapoint is that I have the sunxi-cedrus.ko kernel module, and I can load it fine.

what libraries are on top of it?

what ffmpeg patches did you use?

Is that a vanilla, or patched MPV?

(sorry, probably those questions were asked already a couple of times… but … new year… gotta ask again)

update: I Checked my ffmpeg and I have all the decoders and almost all the hwaccels. I am using ffmpeg 4.3.5-deb11u1 from an updated armbian 23.05

update: I tried ffplay -vcodec h264_v4l2m2m -i myfile.m4v

and it gave the warnings:

[h264_v4l2m2m @ 0xa4b2bca0] could not find a valid device

[h264_v4l2m2m @ 0xa4b2bca0] can’t configure decoder

and when I try: ffplay -hwaccel vaapi -i myfile.m4v

failed to set value vaapi for option hwaccel: option not found

update: I found that my ffmpeg does not have “—enable-v4l2request” in its compilation time configuration.

I found these instructions, which I will follow when I have time:

Edited April 18, 2023 by robertoj

usual user · April 18, 2023

2 hours ago, robertoj said:

How should I check, whether my Linux would enable the hardware acceleration?

To check if the kernel exposes a suitable decoder run a script like this:

#!/bin/bash
for I in {0..12} ; do
  printf "################################################################################\n\n"
  v4l2-compliance --device=/dev/video${I}
  EXITSTATUS="${?}"
  [ "${EXITSTATUS}" == "0" ] || printf "Compliance test for device /dev/video${I} failed, exitstatus: ${EXITSTATUS}\n"
  printf "\n"
done

You need v4l2-compliance from the v4l-utils-tools.

robertoj · April 18, 2023

Thank you!

Indeed, my /dev/video0 exists, but it failed the test

Spoiler

roberto@orangepizero:~$ ./check_linux_decoder.sh
################################################################################

v4l2-compliance SHA: not available
, 32 bits, 32-bit time_t

Compliance test for cedrus device /dev/video0:

Driver Info:
   Driver name      : cedrus
   Card type        : cedrus
   Bus info         : platform:cedrus
   Driver version   : 6.1.15
   Capabilities     : 0x84208000
       Video Memory-to-Memory
       Streaming
       Extended Pix Format
       Device Capabilities
   Device Caps      : 0x04208000
       Video Memory-to-Memory
       Streaming
       Extended Pix Format
Media Driver Info:
   Driver name      : cedrus
   Model            : cedrus
   Serial           :
   Bus info         : platform:cedrus
   Media version    : 6.1.15
   Hardware revision: 0x00000000 (0)
   Driver version   : 6.1.15
Interface Info:
   ID               : 0x0300000c
   Type             : V4L Video
Entity Info:
   ID               : 0x00000001 (1)
   Name             : cedrus-source
   Function         : V4L2 I/O
   Pad 0x01000002   : 0: Source
      Link 0x02000008: to remote pad 0x1000004 of entity 'cedrus-proc': Data, Enabled, Immutable

Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
   test second /dev/video0 open: OK
   test VIDIOC_QUERYCAP: OK
   test VIDIOC_G/S_PRIORITY: OK
   test for unlimited opens: OK

   test invalid ioctls: OK
Debug ioctls:
   test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
   test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
   test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
   test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
   test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
   test VIDIOC_ENUMAUDIO: OK (Not Supported)
   test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
   test VIDIOC_G/S_AUDIO: OK (Not Supported)
   Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
   test VIDIOC_G/S_MODULATOR: OK (Not Supported)
   test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
   test VIDIOC_ENUMAUDOUT: OK (Not Supported)
   test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
   test VIDIOC_G/S_AUDOUT: OK (Not Supported)
   Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
   test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
   test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
   test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
   test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
   test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
   test VIDIOC_QUERYCTRL: OK
   test VIDIOC_G/S_CTRL: OK
       fail: v4l2-test-controls.cpp(679): try_ext_ctrls returned an error (22)
   test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
   test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
   test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
   Standard Controls: 23 Private Controls: 0

Format ioctls:
   test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
   test VIDIOC_G/S_PARM: OK (Not Supported)
   test VIDIOC_G_FBUF: OK (Not Supported)
   test VIDIOC_G_FMT: OK
   test VIDIOC_TRY_FMT: OK
   test VIDIOC_S_FMT: OK
   test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
   test Cropping: OK (Not Supported)
   test Composing: OK (Not Supported)
   test Scaling: OK (Not Supported)

Codec ioctls:
   test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
   test VIDIOC_G_ENC_INDEX: OK (Not Supported)
       fail: v4l2-test-codecs.cpp(123): ret != 0
   test VIDIOC_(TRY_)DECODER_CMD: FAIL

Buffer ioctls:
   test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
   test VIDIOC_EXPBUF: OK
   test Requests: OK

Total for cedrus device /dev/video0: 46, Succeeded: 44, Failed: 2, Warnings: 0
Compliance test for device /dev/video0 failed, exitstatus: 1

At this point, I have zero clues about what I could do next.

The origin of my *.ko's was just to use what came in armbian, and remove them from the blacklist

cat /etc/modprobe.d/blacklist-orangepizero.conf

#blacklist lima
#blacklist sunxi_cedrus

usual user · April 18, 2023

14 hours ago, robertoj said:

Indeed, my /dev/video0 exists, but it failed the test

That report doesn't look that bad.

14 hours ago, robertoj said:

At this point, I have zero clues about what I could do next.

If the decoder is exported correctly, i.e. it is correctly described in the DTB and the driver has been loaded automatically, you only need a corresponding video framework that uses it. Depending on the application you plan to use, this is either ffmpeg or gstreamer. For ffmpeg you need one with a properly patched in request-api. For gstreamer, it is sufficient to use a current mainline release with these built elements:

 gst-inspect-1.0 | grep v4l2
 v4l2codecs:  v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
 v4l2codecs:  v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder
 v4l2codecs:  v4l2slvp8alphadecodebin: VP8 Alpha Decoder
 v4l2codecs:  v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder

gst-play-1.0 --use-playbin3 bbb_sunflower_1080p_60fps_normal.mp4

will assemble a video pipeline that uses the hardware decoder out-of-the-box.

mpv --hwdec=auto --hwdec-codecs=all bbb_sunflower_1080p_60fps_normal.mp4

will use it via "hwdec: drm" if a proper patched ffmpeg framework is available.
See mpv.log for reference.

robertoj · April 19, 2023

Thank you for the advice

My gstreamer did not include those codecs.

Now I am trying to get my meson updated, so I can compile gstreamer

Same with ffmpeg... I will look for the code and patches

Update: if your debian installs an old meson and ninja, install them through pip with

python3 -m pip install meson

python3 -m pip install ninja

While doing the meson setup builddir, I found this:

sudo apt install gstreamer1.0-plugins-bad

roberto@orangepizero:~$ gst-inspect-1.0 | grep v4l2
v4l2codecs: v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder
v4l2codecs: v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
video4linux2: v4l2deviceprovider (GstDeviceProviderFactory)
video4linux2: v4l2radio: Radio (video4linux2) Tuner
video4linux2: v4l2sink: Video (video4linux2) Sink
video4linux2: v4l2src: Video (video4linux2) Source

Then I tried:

gst-play-1.0 --use-playbin3 myvideo.mp4

But then I got:

ERROR Driver did not report framing and start code method for file myvideo.mp4

ERROR debug information: ../sys/v4l2codecs/gstv4l2codec.c(134): gst_v4l2_codec_h264_dec_open () ...

gst_v4l2_decoder_get_controls() failed: invalid argument

Reached end of playlist

update: while compiling mainline gstreamer, I got

../subprojects/FFmpeg/libavutil/arm/float_dsp_neon.S:268: Error: selected processor does not support `vpadd.f32 d0,d0,d0' in Thumb mode

I think I need some special compilation options... I will look later

Edited April 19, 2023 by robertoj

usual user · April 20, 2023

On 4/19/2023 at 5:51 AM, robertoj said:

ERROR debug information: ../sys/v4l2codecs/gstv4l2codec.c(134): gst_v4l2_codec_h264_dec_open () ...

On 4/19/2023 at 5:51 AM, robertoj said:

../subprojects/FFmpeg/libavutil/arm/float_dsp_neon.S:268: Error: selected processor does not support `vpadd.f32 d0,d0,d0' in Thumb mode

Sorry, I don't know at what "mainline" gstreamer you are. I can't see a line 134 in gstv4l2codec.c nor has gstreamer a subprojects/FFmpeg.

robertoj · April 20, 2023

I noticed i was using development version:

https://gstreamer.freedesktop.org/documentation/installing/building-from-source-using-meson.html?gi-language=c

git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git

I will try again with git clone https://gitlab.freedesktop.org/gstreamer/gst-build.git

robertoj · April 21, 2023

Update:

I installed armbian-bookworm. The gstreamer in that distro has the same v4l2 codecs… but can’t play the h264 because of a dkms and drm issue… I was starting it from within Lterm in LXDE.

then I tried turning off lightdm (the graphical desktop), and repeated, from SSH, with export display 0

the orangepi zero became unresponsive, as if it had crashed.

I will do other tests tonight.

I left it building vanilla ffmpeg, as a practice run to build the patched ffmpeg.

robertoj · April 23, 2023

Yesterday, I was able to build the unpatched source of ffmpeg (the snapshot available in this date), and it ran fine, with CPU H264 decoding.

https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu

Then, I downloaded this libreelec patch.

https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-request/ffmpeg-001-v4l2-request.patch

with:

cd ffmpeg_sources/ffmpeg

patch -p1 < ffmpeg-001-v4l2-request.patch

There was only 1 hunk error.

patching file libavcodec/h264_slice.c
Hunk #1 FAILED at 808.
Hunk #2 succeeded at 861 (offset 7 lines).
1 out of 2 hunks FAILED -- saving rejects to file libavcodec/h264_slice.c.rej

I am working on fixing this patch correctly.

Edited April 23, 2023 by robertoj

robertoj · April 24, 2023

The patch correction is to replace the hunk:

Spoiler

diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c

index 3d0d45b2a3..f7af51b28e 100644

--- a/libavcodec/h264_slice.c

+++ b/libavcodec/h264_slice.c

@@ -808,10 +808,17 @@ static enum AVPixelFormat get_pixel_format(H264Context *h, int force_callback)

*fmt++ = AV_PIX_FMT_GBRP10;

} else

*fmt++ = AV_PIX_FMT_YUV444P10;

- } else if (CHROMA422(h))

+ } else if (CHROMA422(h)) {

+#if CONFIG_H264_V4L2REQUEST_HWACCEL

+ *fmt++ = AV_PIX_FMT_DRM_PRIME;

+#endif

*fmt++ = AV_PIX_FMT_YUV422P10;

- else

+ } else {

+#if CONFIG_H264_V4L2REQUEST_HWACCEL

+ *fmt++ = AV_PIX_FMT_DRM_PRIME;

+#endif

*fmt++ = AV_PIX_FMT_YUV420P10;

+ }

break;

case 12:

if (CHROMA444(h)) {

with:

Spoiler

diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c
index 3d0d45b2a3..f7af51b28e 100644
--- a/libavcodec/h264_slice.c
+++ b/libavcodec/h264_slice.c
@@ -808,8 +808,14 @@ static enum AVPixelFormat get_pixel_format(H264Context *h, int force_callback)
                 *fmt++ = AV_PIX_FMT_GBRP10;
             } else
                 *fmt++ = AV_PIX_FMT_YUV444P10;
-        } else if (CHROMA422(h))
+        } else if (CHROMA422(h)) {
+#if CONFIG_H264_V4L2REQUEST_HWACCEL
+            *fmt++ = AV_PIX_FMT_DRM_PRIME;
+#endif
             *fmt++ = AV_PIX_FMT_YUV422P10;
-        else {
+        } else {
+#if CONFIG_H264_V4L2REQUEST_HWACCEL
+            *fmt++ = AV_PIX_FMT_DRM_PRIME;
+#endif
#if CONFIG_H264_VAAPI_HWACCEL
             // Just add as candidate. Whether VAProfileH264High10 usable or
             // not is decided by vaapi_decode_make_config() defined in FFmpeg

Then, h264_slice.c compiles, but the whole make process fails this way:

Spoiler

C   libavcodec/v4l2_m2m_enc.o
CC   libavcodec/v4l2_request.o
libavcodec/v4l2_request.c: In function ‘v4l2_request_controls’:
libavcodec/v4l2_request.c:79:12: error: implicit declaration of function ‘ioctl’ [-Werror=implicit-function-declaration]
   79 |     return ioctl(ctx->video_fd, type, &controls);
      |            ^~~~~
libavcodec/v4l2_request.c: In function ‘v4l2_request_probe_video_device’:
libavcodec/v4l2_request.c:609:9: error: implicit declaration of function ‘close’; did you mean ‘pclose’? [-Werror=implicit-function-declaration]
609 |         close(ctx->video_fd);
      |         ^~~~~
      |         pclose
cc1: some warnings being treated as errors
make: *** [ffbuild/common.mak:81: libavcodec/v4l2_request.o] Error 1

Should I replace ioctl with something, and close with pclose?

(it looks like ioctl gets used 30 times in this file, and close 6 times)

I found that the other v4l2_*.c use this line, so I will add it and try again: #include <sys/ioctl.h>

Also, the complaint about close(), was because it needed #include <unistd.h>

After making the change in v4l2_request.c, I ran make again, then it restarted in that same file

Update: the compilation failed in the last file linking:

Spoiler

CC   fftools/cmdutils.o
CC   fftools/opt_common.o
CC   fftools/ffmpeg.o
LD   ffmpeg_g
/usr/bin/ld: libavcodec/libavcodec.a(utvideodec.o): in function `decode_init':
/home/roberto/ffmpeg_sources/ffmpeg/libavcodec/utvideodec.c:889: undefined reference to `ff_utvideodsp_init'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x24): undefined reference to `ff_ayuv_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x120): undefined reference to `ff_utvideo_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x124): undefined reference to `ff_v210_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x128): undefined reference to `ff_v308_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x12c): undefined reference to `ff_v408_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x130): undefined reference to `ff_v410_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x2e4): undefined reference to `ff_h263_v4l2m2m_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x2e8): undefined reference to `ff_h264_v4l2m2m_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x2f0): undefined reference to `ff_hevc_v4l2m2m_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x300): undefined reference to `ff_mpeg4_v4l2m2m_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x304): undefined reference to `ff_vp8_v4l2m2m_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x360): undefined reference to `ff_ayuv_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x464): undefined reference to `ff_h263_v4l2m2m_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x46c): undefined reference to `ff_h264_v4l2m2m_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x478): undefined reference to `ff_hevc_v4l2m2m_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x50c): undefined reference to `ff_mpeg4_v4l2m2m_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x514): undefined reference to `ff_mpeg1_v4l2m2m_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x518): undefined reference to `ff_mpeg2_v4l2m2m_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x690): undefined reference to `ff_v210_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x694): undefined reference to `ff_v210x_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x698): undefined reference to `ff_v308_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x69c): undefined reference to `ff_v408_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x6a0): undefined reference to `ff_v410_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x6b8): undefined reference to `ff_vc1_v4l2m2m_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x6e8): undefined reference to `ff_vp8_v4l2m2m_decoder'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0x6f0): undefined reference to `ff_vp9_v4l2m2m_decoder'
collect2: error: ld returned 1 exit status
make: *** [Makefile:133: ffmpeg_g] Error 1

I am going to try other versions of ffmpeg

Edited April 24, 2023 by robertoj

usual user · April 24, 2023

22 hours ago, robertoj said:

There was only 1 hunk error.

The patch set should be applied to the source for which it was designed, even if it can be applied cleanly to another source, there is no guarantee that a change in a non-contradictory place will not affect functionality.
The specific ffmpeg version is not decisive here, since the request-api can be patched in for quite some time, e.g. I am currently still using a 5.x for other reasons.
So if you've cloned the ffmpeg git repository, it's just a "git checkout <commit>" away to have a corresponding version.
I grabbed the patches from @jernej's Github at the time, as his branch names indicate which version they belong to. Of course, the 5.x ones are long gone.

Edited April 24, 2023 by usual user
corrected patches link

robertoj · April 25, 2023

I succeeded at compiling ffmpeg 6.0 with libreelec's patch:

wget https://ffmpeg.org/releases/ffmpeg-6.0.tar.bz2

wget https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-request/ffmpeg-001-v4l2-request.patch

unzip and go inside ffmpeg dir

patch -p1 < ffmpeg-001-v4l2-request.patch

./configure, make and install as per https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu, plus --enable-v4l2-request --enable-libdrm --enable-libudev

The compilation finished with no errors

Tests:

Spoiler

roberto@orangepizero:~$ bin/ffmpeg -hwaccels
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
configuration: --prefix=/home/roberto/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/roberto/ffmpeg_build/include --extra-ldflags=-L/home/roberto/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/roberto/bin --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-v4l2-request --enable-libdrm --enable-libudev
libavutil      58. 2.100 / 58. 2.100
libavcodec     60. 3.100 / 60. 3.100
libavformat    60. 3.100 / 60. 3.100
libavdevice    60. 1.100 / 60. 1.100
libavfilter     9. 3.100 / 9. 3.100
libswscale      7. 1.100 / 7. 1.100
libswresample   4. 10.100 / 4. 10.100
libpostproc    57. 1.100 / 57. 1.100
Hardware acceleration methods:
vdpau
vaapi
drm

roberto@orangepizero:~$ bin/ffmpeg -hide_banner -decoders|grep v4l2
V..... h263_v4l2m2m         V4L2 mem2mem H.263 decoder wrapper (codec h263)
V..... h264_v4l2m2m         V4L2 mem2mem H.264 decoder wrapper (codec h264)
V..... hevc_v4l2m2m         V4L2 mem2mem HEVC decoder wrapper (codec hevc)
V..... mpeg1_v4l2m2m        V4L2 mem2mem MPEG1 decoder wrapper (codec mpeg1video)
V..... mpeg2_v4l2m2m        V4L2 mem2mem MPEG2 decoder wrapper (codec mpeg2video)
V..... mpeg4_v4l2m2m        V4L2 mem2mem MPEG4 decoder wrapper (codec mpeg4)
V..... vc1_v4l2m2m          V4L2 mem2mem VC1 decoder wrapper (codec vc1)
V..... vp8_v4l2m2m          V4L2 mem2mem VP8 decoder wrapper (codec vp8)
V..... vp9_v4l2m2m          V4L2 mem2mem VP9 decoder wrapper (codec vp9)

I checked that my ffmpeg source had the lines of code, so it is compatible with mpv:

https://github.com/Kwiboo/FFmpeg/commit/c2a9396e0c1b302a88f82d6a420111c9d495c322

Then, to test it

mpv --vo=gpu --gpu-context=drm --hwdec=auto video.mkv

It fails like this:

Spoiler

[vo/gpu] VT_GETMODE failed: inappropriate ioctl for device

...

Error: opening/initializing the selected video_out (--vo) device

Video: no video

Next I disabled the lightdm window desktop manager. Then to play from ssh: export DISPLAY=:0.0

I tried the same mpv command, and it played, with CPU decoding, and these video errors:

Spoiler

[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
[vo/gpu/opengl/kms] Could not find any preferred mode. Picking the first mode.
[ffmpeg/video] h264_v4l2m2m: Could not find a valid device
[ffmpeg/video] h264_v4l2m2m: can't configure decoder
Could not open codec.
[W][00397.740564] pw.conf | [ conf.c: 939 try_load_conf()] can't load config client.conf: No such file or directory
[E][00397.741094] pw.conf | [ conf.c: 963 pw_conf_load_conf_for_context()] can't load default config client.conf: No such file or directory

And just to be sure, I share my v4l2-compliance... and my linux is 6.1.15-sunxi, with updated Debian Bookworm

Spoiler

roberto@orangepizero:~$ v4l2-compliance --device=/dev/video0
v4l2-compliance 1.22.1, 32 bits, 32-bit time_t

Compliance test for cedrus device /dev/video0:

Driver Info:
   Driver name      : cedrus
   Card type        : cedrus
   Bus info         : platform:cedrus
   Driver version   : 6.1.15
   Capabilities     : 0x84208000
       Video Memory-to-Memory
       Streaming
       Extended Pix Format
       Device Capabilities
   Device Caps      : 0x04208000
       Video Memory-to-Memory
       Streaming
       Extended Pix Format
Media Driver Info:
   Driver name      : cedrus
   Model            : cedrus
   Serial           :
   Bus info         : platform:cedrus
   Media version    : 6.1.15
   Hardware revision: 0x00000000 (0)
   Driver version   : 6.1.15
Interface Info:
   ID               : 0x0300000c
   Type             : V4L Video
Entity Info:
   ID               : 0x00000001 (1)
   Name             : cedrus-source
   Function         : V4L2 I/O
   Pad 0x01000002   : 0: Source
      Link 0x02000008: to remote pad 0x1000004 of entity 'cedrus-proc' (Video Decoder): Data, Enabled, Immutable

Required ioctls:
   test MC information (see 'Media Driver Info' above): OK
   test VIDIOC_QUERYCAP: OK
   test invalid ioctls: OK

Allow for multiple opens:
   test second /dev/video0 open: OK
   test VIDIOC_QUERYCAP: OK
   test VIDIOC_G/S_PRIORITY: OK
   test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
   test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
   test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
   test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
   test VIDIOC_ENUMAUDIO: OK (Not Supported)
   test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
   test VIDIOC_G/S_AUDIO: OK (Not Supported)
   Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
   test VIDIOC_G/S_MODULATOR: OK (Not Supported)
   test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
   test VIDIOC_ENUMAUDOUT: OK (Not Supported)
   test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
   test VIDIOC_G/S_AUDOUT: OK (Not Supported)
   Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
   test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
   test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
   test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
   test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
   test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
   test VIDIOC_QUERYCTRL: OK
   test VIDIOC_G/S_CTRL: OK
       fail: v4l2-test-controls.cpp(717): try_ext_ctrls returned an error (22)
   test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
   test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
   test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
   Standard Controls: 7 Private Controls: 0
   Standard Compound Controls: 16 Private Compound Controls: 0

Format ioctls:
       fail: v4l2-test-formats.cpp(263): fmtdesc.description mismatch: was 'Y/UV 4:2:0 (32x32 Linear)', expected 'Y/CbCr 4:2:0 (32x32 Linear)'
   test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: FAIL
   test VIDIOC_G/S_PARM: OK (Not Supported)
   test VIDIOC_G_FBUF: OK (Not Supported)
       fail: v4l2-test-formats.cpp(460): pixelformat 32315453 (ST12) for buftype 1 not reported by ENUM_FMT
   test VIDIOC_G_FMT: FAIL
       fail: v4l2-test-formats.cpp(460): pixelformat 32315453 (ST12) for buftype 1 not reported by ENUM_FMT
   test VIDIOC_TRY_FMT: FAIL
       fail: v4l2-test-formats.cpp(460): pixelformat 32315453 (ST12) for buftype 1 not reported by ENUM_FMT
   test VIDIOC_S_FMT: FAIL
   test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
   test Cropping: OK (Not Supported)
   test Composing: OK (Not Supported)
   test Scaling: OK (Not Supported)

Codec ioctls:
   test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
   test VIDIOC_G_ENC_INDEX: OK (Not Supported)
       fail: v4l2-test-codecs.cpp(122): ret != 0
   test VIDIOC_(TRY_)DECODER_CMD: FAIL

Buffer ioctls:
   test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
   test VIDIOC_EXPBUF: OK
   test Requests: OK (Not Supported)
   test TIME32/64: OK

Total for cedrus device /dev/video0: 47, Succeeded: 41, Failed: 6, Warnings: 0

Is this normal?

Spoiler

roberto@orangepizero:~$ sudo vainfo
error: XDG_RUNTIME_DIR is invalid or not set in the environment.
error: can't connect to X server!
libva info: VA-API version 1.17.0
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/lima_drv_video.so
libva info: va_openDriver() returns -1
vaInitialize failed with error code -1 (unknown libva error),exit

And that file /usr/lib/arm-linux-gnueabihf/dri/lima_drv_video.so

does not exist

only /usr/lib/arm-linux-gnueabihf/dri/lima_dri.so exists

Edited April 25, 2023 by robertoj

jernej · April 25, 2023

*_v4l2m2m codecs are for v4l2 stateful drivers, not request api. Actually, you have to grep for "request" word. Check that you have 6.1 kernel headers installed. IIRC last important kernel patches went into 6.2, so ideally you would run that. Last but not least, to know which codecs are supported, you have to use v4l2-ctl command. Easiest way is to use --all parameter and check for supported formats on cedrus device.

robertoj · April 25, 2023

OH

roberto@orangepizero:~$ ffmpeg -hide_banner -decoders|grep request

... returns NOTHING

Also:

Spoiler

roberto@orangepizero:~$ v4l2-ctl --device /dev/video0 -L

Codec Controls

                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=2 value=2 (Main)
               0: Baseline
               1: Constrained Baseline
               2: Main
               4: High

Stateless Codec Controls

               h264_decode_mode 0x00a40900 (menu)   : min=0 max=0 default=0 value=0 (Slice-Based)
               0: Slice-Based
                h264_start_code 0x00a40901 (menu)   : min=0 max=0 default=0 value=0 (No Start Code)
               0: No Start Code
    h264_sequence_parameter_set 0x00a40902 (h264-sps): value=unsupported payload type flags=has-payload
     h264_picture_parameter_set 0x00a40903 (h264-pps): value=unsupported payload type flags=has-payload
            h264_scaling_matrix 0x00a40904 (h264-scaling-matrix): value=unsupported payload type flags=has-payload
   h264_prediction_weight_table 0x00a40905 (h264-pred-weights): value=unsupported payload type flags=has-payload
          h264_slice_parameters 0x00a40906 (h264-slice-params): value=unsupported payload type flags=has-payload
         h264_decode_parameters 0x00a40907 (h264-decode-params): value=unsupported payload type flags=has-payload
           vp8_frame_parameters 0x00a409c8 (vp8-frame): value=unsupported payload type flags=has-payload
         mpeg_2_sequence_header 0x00a409dc (mpeg2-sequence): value=unsupported payload type flags=has-payload
          mpeg_2_picture_header 0x00a409dd (mpeg2-picture): value=unsupported payload type flags=has-payload
   mpeg_2_quantisation_matrices 0x00a409de (mpeg2-quantisation): value=unsupported payload type flags=has-payload
    hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 value=unsupported payload type flags=has-payload
     hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 value=unsupported payload type flags=has-payload
          hevc_slice_parameters 0x00a40a92 (unknown): type=272 dims=[1] flags=has-payload, 0x00000800
            hevc_scaling_matrix 0x00a40a93 (unknown): type=273 value=unsupported payload type flags=has-payload
         hevc_decode_parameters 0x00a40a94 (unknown): type=274 value=unsupported payload type flags=has-payload
               hevc_decode_mode 0x00a40a95 (menu)   : min=0 max=0 default=0 value=0 (Slice-Based)
               0: Slice-Based
                hevc_start_code 0x00a40a96 (menu)   : min=0 max=0 default=0 value=0 (No Start Code)
               0: No Start Code
       hevc_entry_point_offsets 0x00a40a97 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[256] flags=has-payload, 0x00000800

roberto@orangepizero:~$ v4l2-ctl --device /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'ST12' (Y/UV 4:2:0 (32x32 Linear))
[1]: 'NV12' (Y/UV 4:2:0)

roberto@orangepizero:~$ v4l2-ctl --device=/dev/video0 --all
Driver Info:
   Driver name      : cedrus
   Card type        : cedrus
   Bus info         : platform:cedrus
   Driver version   : 6.1.15
   Capabilities     : 0x84208000
       Video Memory-to-Memory
       Streaming
       Extended Pix Format
       Device Capabilities
   Device Caps      : 0x04208000
       Video Memory-to-Memory
       Streaming
       Extended Pix Format
Media Driver Info:
   Driver name      : cedrus
   Model            : cedrus
   Serial           :
   Bus info         : platform:cedrus
   Media version    : 6.1.15
   Hardware revision: 0x00000000 (0)
   Driver version   : 6.1.15
Interface Info:
   ID               : 0x0300000c
   Type             : V4L Video
Entity Info:
   ID               : 0x00000001 (1)
   Name             : cedrus-source
   Function         : V4L2 I/O
   Pad 0x01000002   : 0: Source
      Link 0x02000008: to remote pad 0x1000004 of entity 'cedrus-proc' (Video Decoder): Data, Enabled, Immutable
Priority: 2
Format Video Capture:
   Width/Height      : 16/32
   Pixel Format      : 'ST12' (Y/UV 4:2:0 (32x32 Linear))
   Field             : None
   Bytes per Line    : 32
   Size Image        : 2048
   Colorspace        : Default
   Transfer Function : Default (maps to Rec. 709)
   YCbCr/HSV Encoding: Default (maps to ITU-R 601)
   Quantization      : Default (maps to Full Range)
   Flags             :
Format Video Output:
   Width/Height      : 16/32
   Pixel Format      : 'MG2S' (MPEG-2 Parsed Slice Data)
   Field             : None
   Bytes per Line    : 0
   Size Image        : 1024
   Colorspace        : Default
   Transfer Function : Default (maps to Rec. 709)
   YCbCr/HSV Encoding: Default (maps to ITU-R 601)
   Quantization      : Default (maps to Full Range)
   Flags             :

Codec Controls

                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=2 value=2 (Main)
               0: Baseline
               1: Constrained Baseline
               2: Main
               4: High

Stateless Codec Controls

               h264_decode_mode 0x00a40900 (menu)   : min=0 max=0 default=0 value=0 (Slice-Based)
               0: Slice-Based
                h264_start_code 0x00a40901 (menu)   : min=0 max=0 default=0 value=0 (No Start Code)
               0: No Start Code
    h264_sequence_parameter_set 0x00a40902 (h264-sps): value=unsupported payload type flags=has-payload
     h264_picture_parameter_set 0x00a40903 (h264-pps): value=unsupported payload type flags=has-payload
            h264_scaling_matrix 0x00a40904 (h264-scaling-matrix): value=unsupported payload type flags=has-payload
   h264_prediction_weight_table 0x00a40905 (h264-pred-weights): value=unsupported payload type flags=has-payload
          h264_slice_parameters 0x00a40906 (h264-slice-params): value=unsupported payload type flags=has-payload
         h264_decode_parameters 0x00a40907 (h264-decode-params): value=unsupported payload type flags=has-payload
           vp8_frame_parameters 0x00a409c8 (vp8-frame): value=unsupported payload type flags=has-payload
         mpeg_2_sequence_header 0x00a409dc (mpeg2-sequence): value=unsupported payload type flags=has-payload
          mpeg_2_picture_header 0x00a409dd (mpeg2-picture): value=unsupported payload type flags=has-payload
   mpeg_2_quantisation_matrices 0x00a409de (mpeg2-quantisation): value=unsupported payload type flags=has-payload
    hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 value=unsupported payload type flags=has-payload
     hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 value=unsupported payload type flags=has-payload
          hevc_slice_parameters 0x00a40a92 (unknown): type=272 dims=[1] flags=has-payload, 0x00000800
            hevc_scaling_matrix 0x00a40a93 (unknown): type=273 value=unsupported payload type flags=has-payload
         hevc_decode_parameters 0x00a40a94 (unknown): type=274 value=unsupported payload type flags=has-payload
               hevc_decode_mode 0x00a40a95 (menu)   : min=0 max=0 default=0 value=0 (Slice-Based)
               0: Slice-Based
                hevc_start_code 0x00a40a96 (menu)   : min=0 max=0 default=0 value=0 (No Start Code)
               0: No Start Code
       hevc_entry_point_offsets 0x00a40a97 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[256] flags=has-payload, 0x00000800

checking linux headers, there's only these relevant available

Spoiler

roberto@orangepizero:~$ apt search linux-headers
Sorting... Done
Full Text Search... Done

...

linux-headers-current-sunxi/bookworm 23.02.2 armhf
Linux kernel headers for 5.15.93-sunxi on armhf current

...

linux-headers-edge-sunxi/bookworm 23.02.2 armhf
Linux kernel headers for 6.1.11-sunxi on armhf edge

And I found this tip:

INSTALL_HEADERS="yes" within armbian build... I will try it, if the headers from 6.1.11 don't work

I didn't have them installed at all... this means I need to rebuild my ffmpeg+libreelec patch... to get the v4l2_request codec?

I am currently rebuilding ffmpeg, after installing the kernel-headers.

Do I need to set CMA=128M in bootargs?

update: after I ran ./configure, I saw:

Spoiler

...

External libraries providing hardware acceleration:
v4l2_m2m v4l2_request vaapi vdpau

...

Enabled hwaccels:
av1_vaapi               h264_vdpau              mpeg1_vdpau             mpeg4_vdpau             vp9_v4l2request
av1_vdpau               hevc_v4l2request        mpeg2_v4l2request       vc1_vaapi               vp9_vaapi
h263_vaapi              hevc_vaapi              mpeg2_vaapi             vc1_vdpau               vp9_vdpau
h264_v4l2request        hevc_vdpau              mpeg2_vdpau             vp8_v4l2request         wmv3_vaapi
h264_vaapi              mjpeg_vaapi             mpeg4_vaapi             vp8_vaapi               wmv3_vdpau

...

But I never looked at this before... I hope it means that h264_v4l2request is available to use (previously, the hwaccels were only vdpau, vaapi and drm)

While waiting that ffmpeg compiles, I check that the latest libreelec, with linux 6.1.19, runs great, using the cedrus driver, although not in module form, but incorporated in the kernel.

LibreELEC:~ # uname -a
Linux LibreELEC 6.1.19 #1 SMP PREEMPT Sat Mar 18 16:09:53 UTC 2023 armv7l GNU/Linux

LibreELEC:~ # dmesg |grep cedrus
[ 0.784939] cedrus 1c0e000.video-codec: Device registered as /dev/video0

Checking the libreelec patches... I see that these linux files are patched:

https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Allwinner/patches/linux/0061-media--cedrus--Don-t-CPU-map-source-buffers.patch

https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Allwinner/patches/linux/0020-media-cedrus-Add-callback-for-buffer-cleanup.patch

https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Allwinner/patches/linux/0021-media-cedrus-hevc-Improve-buffer-management.patch

https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Allwinner/patches/linux/0022-media-cedrus-h264-Improve-buffer-management.patch

Are they needed?

Edited April 25, 2023 by robertoj

robertoj · April 25, 2023

Negative results

Test:

Spoiler

roberto@orangepizero:~$ ffmpeg -hwaccels
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
configuration: --prefix=/home/roberto/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/roberto/ffmpeg_build/include --extra-ldflags=-L/home/roberto/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/roberto/bin --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-v4l2-request --enable-libdrm --enable-libudev
libavutil      58. 2.100 / 58. 2.100
libavcodec     60. 3.100 / 60. 3.100
libavformat    60. 3.100 / 60. 3.100
libavdevice    60. 1.100 / 60. 1.100
libavfilter     9. 3.100 / 9. 3.100
libswscale      7. 1.100 / 7. 1.100
libswresample   4. 10.100 / 4. 10.100
libpostproc    57. 1.100 / 57. 1.100
Hardware acceleration methods:
vdpau
vaapi
drm

roberto@orangepizero:~$ ffmpeg -hide_banner -decoders|grep req
roberto@orangepizero:~$

roberto@orangepizero:~$ ffmpeg -hide_banner -decoders|grep v4l
V..... h263_v4l2m2m         V4L2 mem2mem H.263 decoder wrapper (codec h263)
V..... h264_v4l2m2m         V4L2 mem2mem H.264 decoder wrapper (codec h264)
V..... hevc_v4l2m2m         V4L2 mem2mem HEVC decoder wrapper (codec hevc)
V..... mpeg1_v4l2m2m        V4L2 mem2mem MPEG1 decoder wrapper (codec mpeg1video)
V..... mpeg2_v4l2m2m        V4L2 mem2mem MPEG2 decoder wrapper (codec mpeg2video)
V..... mpeg4_v4l2m2m        V4L2 mem2mem MPEG4 decoder wrapper (codec mpeg4)
V..... vc1_v4l2m2m          V4L2 mem2mem VC1 decoder wrapper (codec vc1)
V..... vp8_v4l2m2m          V4L2 mem2mem VP8 decoder wrapper (codec vp8)
V..... vp9_v4l2m2m          V4L2 mem2mem VP9 decoder wrapper (codec vp9)
roberto@orangepizero:~$

Run (both starting from SSH without X running):

Spoiler

roberto@orangepizero:~$ mpv --vo=gpu --gpu-context=drm --hwdec=auto youtube720p.mp4
(+) Video --vid=1 (*) (h264 1280x720 30.000fps)
(+) Audio --aid=1 --alang=eng (*) (aac 2ch 44100Hz)
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
[vo/gpu/opengl/kms] Could not find any preferred mode. Picking the first mode.
[vo/gpu] Failed to create GBM surface.
[vo/gpu] Failed to setup GBM.
[vo/gpu] Failed initializing any suitable GPU context!
Error opening/initializing the selected video_out (--vo) device.
Video: no video

Exiting... (Errors when loading file)
roberto@orangepizero:~$

roberto@orangepizero:~$ ffplay youtube720p.mp4
ffplay version 6.0 Copyright (c) 2003-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
configuration: --prefix=/home/roberto/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/roberto/ffmpeg_build/include --extra-ldflags=-L/home/roberto/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/roberto/bin --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-v4l2-request --enable-libdrm --enable-libudev
libavutil      58. 2.100 / 58. 2.100
libavcodec     60. 3.100 / 60. 3.100
libavformat    60. 3.100 / 60. 3.100
libavdevice    60. 1.100 / 60. 1.100
libavfilter     9. 3.100 / 9. 3.100
libswscale      7. 1.100 / 7. 1.100
libswresample   4. 10.100 / 4. 10.100
libpostproc    57. 1.100 / 57. 1.100

...

It plays with CPU decoding

From the terminal in the CVBS console

Spoiler

roberto@orangepizero:~$ mpv --vo=gpu --gpu-context=drm --hwdec=auto youtube720p.mp4

[gpu/opengl/DKMS] could not find any preferred mode. Picking first mode

PLays video, with 225% CPU

roberto@orangepizero:~$ ffplay youtube720p.mp4

PLays video, with 225% CPU

I was able to make an MPV log:

Spoiler

...

[   1.706][v][vo/gpu] Loaded extension GL_ARB_invalidate_subdata.
[   1.708][v][vo/gpu] Loaded extension GL_ARB_get_program_binary.
[   1.711][v][vo/gpu] Loaded extension GL_ARB_buffer_storage.
[   1.713][v][vo/gpu] Loaded extension GL_ARB_debug_output.
[   1.715][v][vo/gpu] Loaded extension GL_KHR_debug.
[   1.718][v][vo/gpu/opengl] Preparing framebuffer
[   1.720][v][vo/gpu/opengl] GBM surface using modifier 0x0
[   1.722][v][vo/gpu] crtc is not VRR capable
[   1.736][v][vo/gpu/opengl] Could not find path to render node. VAAPI hwdec will be disabled
[   1.739][v][vo/gpu] Monitor pixel aspect: 1
[   1.740][d][vo/gpu/opengl] Texture formats:
[   1.740][d][vo/gpu/opengl]   NAME       COMP*TYPE SIZE           DEPTH PER COMP.
[   1.740][d][vo/gpu/opengl]   l8         1*unorm   1B    LF    ST {8}
[   1.740][d][vo/gpu/opengl]   la8        2*unorm   2B LA LF    ST {8 8}

...

[   1.747][d][vo/gpu/opengl]   p416
[   1.752][v][vo/gpu] Disabling HDR peak computation (one or more of the following is not supported: compute shaders=0, SSBO=0).
[   1.754][v][vo/gpu] No advanced processing required. Enabling dumb mode.
[   1.757][v][vo/gpu] Assuming 50.000000 FPS for display sync.
[   1.760][v][vd] Container reported FPS: 30.000000
[   1.766][v][vd] Codec list:
[   1.768][v][vd]     h264 - H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
[   1.770][v][vd]     h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper
[   1.772][v][vd] Opening decoder h264
[   1.775][v][vd] Looking at hwdec h264-vaapi...
[   1.778][v][vo/gpu] Loading hwdec drivers for format: 'vaapi'
[   1.780][v][vo/gpu] Loading hwdec driver 'vaapi'
[   1.782][v][vo/gpu/vaapi] using EGL dmabuf interop
[   1.785][v][vo/gpu/vaapi] Trying to open a x11 VA display...
[   1.787][v][vo/gpu/vaapi] Trying to open a wayland VA display...
[   1.789][v][vo/gpu/vaapi] Trying to open a drm VA display...
[   1.791][v][vo/gpu/vaapi] Could not create a VA display.
[   1.793][v][vo/gpu] Loading failed.
[   1.796][v][vd] Could not create device.
[   1.798][v][vd] Looking at hwdec h264-vdpau...
[   1.800][v][vo/gpu] Loading hwdec drivers for format: 'vdpau'
[   1.802][v][vd] Could not create device.
[   1.805][v][vd] Looking at hwdec h264-vaapi-copy...
[   1.806][d][vaapi] libva: VA-API version 1.17.0
[   1.806][d][vaapi] libva: Trying to open /usr/lib/arm-linux-gnueabihf/dri/lima_drv_video.so
[   1.807][d][vaapi] libva: va_openDriver() returns -1
[   1.810][v][vd] Could not create device.
[   1.812][v][vd] Looking at hwdec h264-vdpau-copy...
[   1.814][v][vd] Could not create device.
[   1.817][v][vd] Looking at hwdec h264_v4l2m2m-v4l2m2m-copy...
[   1.819][v][vd] Trying hardware decoding via h264_v4l2m2m-v4l2m2m-copy.
[   1.822][v][vd] Using underlying hw-decoder 'h264_v4l2m2m'
[   1.826][e][ffmpeg/video] h264_v4l2m2m: Could not find a valid device
[   1.829][e][ffmpeg/video] h264_v4l2m2m: can't configure decoder
[   1.831][e][vd] Could not open codec.
[   1.834][v][vd] Falling back to software decoding.
[   1.839][v][vd] Detected 4 logical cores.

...

Tomorrow I will try with jernej's and kiwiboo's github source

update: jernejs code cant compile, because the compiler thinks there's no FPU

Spoiler

...

BEGIN /tmp/ffconf.YqkEhv90/test.c
    1   #include <stddef.h>
    2   #if !(defined __ARM_ARCH_7A__ || defined __TARGET_ARCH_7_A)
    3   #error "unsatisfied condition: defined __ARM_ARCH_7A__ || defined __TARGET_ARCH_7_A"
    4   #endif
END /tmp/ffconf.YqkEhv90/test.c
gcc -I/home/roberto/ffmpeg_build/include -E -o /tmp/ffconf.YqkEhv90/test.o /tmp/ffconf.YqkEhv90/test.c
test_ld cc
test_cc
BEGIN /tmp/ffconf.YqkEhv90/test.c
    1   int main(void){ return 0; }
END /tmp/ffconf.YqkEhv90/test.c
gcc -I/home/roberto/ffmpeg_build/include -march=armv7-a -c -o /tmp/ffconf.YqkEhv90/test.o /tmp/ffconf.YqkEhv90/test.c
cc1: error: '-mfloat-abi=hard': selected architecture lacks an FPU
C compiler test failed.

But I see some people solving this problem: https://github.com/openssl/openssl/issues/18818

Edited April 25, 2023 by robertoj

jernej · April 25, 2023

9 hours ago, robertoj said:

Do I need to set CMA=128M in bootargs?

Not sure what are current defaults, but that's generally on the low side. LE uses CMA size of 384 MiB, mainly due to 4k.

9 hours ago, robertoj said:

Checking the libreelec patches... I see that these linux files are patched:

https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Allwinner/patches/linux/0061-media--cedrus--Don-t-CPU-map-source-buffers.patch

https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Allwinner/patches/linux/0020-media-cedrus-Add-callback-for-buffer-cleanup.patch

https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Allwinner/patches/linux/0021-media-cedrus-hevc-Improve-buffer-management.patch

https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Allwinner/patches/linux/0022-media-cedrus-h264-Improve-buffer-management.patch

Are they needed?

They fix one buffer management issue which may cause incorrect decoding but most importantly, they massively lower amount of memory that's used during decoding. Those fixes are included in 6.2 and newer kernels.

Use following command for testing:

ffmpeg -loglevel debug -hwaccel drm -hwaccel_output_format drm_prime -i video.mkv -f null -

It will be pretty obvious from debug log if request api is used or not.

Not sure which version of mpv you're using, but very latest (not sure if it's in released version or in development branch) should allow using request api ffmpeg also when window manager is running.

EDIT: make sure that this ffmpeg patch is included: https://github.com/jernejsk/FFmpeg/commit/5057eb96b2adbff022a1abd8d5b06f369f908d51

usual user · April 25, 2023

15 hours ago, jernej said:

*_v4l2m2m codecs are for v4l2 stateful drivers, not request api.

ffmpeg -hide_banner -decoders | grep request

returns for me ... NOTHING also, but hwdec is working as expected. So your statement seems not to be correct.

From "man mpv":

NOTE:

Even if enabled, hardware decoding is still only white-listed for some codecs. See --hwdec-codecs to enable hardware decoding in more cases.
Which method to choose?
If you only want to enable hardware decoding at runtime, don't set the parameter, or put hwdec=no into your mpv.conf (relevant on distros which force-enable it by default, such as on Ubuntu). Use the Ctrl+h default binding to enable it at runtime.
If you're not sure, but want hardware decoding always enabled by default, put hwdec=auto-safe into your mpv.conf, and acknowledge that this may cause problems.
If you want to test available hardware decoding methods, pass --hwdec=auto --hwdec-codecs=all and look at the terminal output.

As the hacked in request-api codecs are not white-listed in mpv proper by default ==> so "--hwdec=auto --hwdec-codecs=all"

Spoiler

mpv --hwdec=auto --hwdec-codecs=all bbb_sunflower_1080p_60fps_normal.mp4

 (+) Video --vid=1 (*) (h264 1920x1080 60.000fps)
 (+) Audio --aid=1 (*) (mp3 2ch 48000Hz)
     Audio --aid=2 (*) (ac3 6ch 48000Hz)
File tags:
 Artist: Blender Foundation 2008, Janus Bager Kristensen 2013
 Comment: Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
 Composer: Sacha Goedegebure
 Genre: Animation
 Title: Big Buck Bunny, Sunflower version
Cannot load libcuda.so.1
[ffmpeg] AVHWDeviceContext: Cannot load libcuda.so.1
[ffmpeg] AVHWDeviceContext: Could not dynamically load CUDA
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
Using hardware decoding (drm).
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [gpu] 1920x1080 drm_prime[nv12]
AV: 00:00:04 / 00:10:34 (1%) A-V:  0.011 Dropped: 131

Exiting... (Quit)

Verbose log | grep vd]:

[vd] Container reported FPS: 60.000000
[vd] Codec list:
[vd]     h264 - H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
[vd]     h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper
[vd]     h264_cuvid (h264) - Nvidia CUVID H264 decoder
[vd] Opening decoder h264
[vd] Looking at hwdec h264-nvdec...
[vd] Could not create device.
[vd] Looking at hwdec h264-vaapi...
[vd] Could not create device.
[vd] Looking at hwdec h264-vdpau...
[vd] Could not create device.
[vd] Looking at hwdec h264-nvdec-copy...
[vd] Could not create device.
[vd] Looking at hwdec h264-vaapi-copy...
[vd] Could not create device.
[vd] Looking at hwdec h264-vdpau-copy...
[vd] Error when calling vdp_device_create_x11: 1
[vd] Could not create device.
[vd] Looking at hwdec h264-drm...
[vd] Trying hardware decoding via h264-drm.
[vd] Selected codec: h264 (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)
[vd] Pixel formats supported by decoder: vdpau cuda vaapi drm_prime yuv420p
[vd] Codec profile: High (0x64)
[vd] Requesting pixfmt 'drm_prime' from decoder.
[vd] Using hardware decoding (drm).
[vd] Decoder format: 1920x1080 drm_prime[nv12] auto/auto/auto/auto/auto CL=mpeg2/4/h264

jernej · April 25, 2023

No, that part is correct, but I must admit that since I have no system with patched ffmpeg at hand currently, I misremember how request api is expressed in ffmpeg decoders list. In any case, request api is covered by drm hwaccel and use drmprime pixel formats. Anyway, it seems you succeeded in reaching your goal.

robertoj · April 25, 2023

Usual user… what do you get with “vainfo” … does it matter?

Are there other components at play, more required Linux patches, libraries sandwiched between Linux and ffmpeg?

installing Linux-headers (armbian sunxi edge deb) before repeating ./configure and make, for ffmpeg, did nothing for me… no difference. Should it make a difference?

update: I am back home, and I checked that the "hwcontext_drm: do not require drm device" patch is already included in ffmpeg 6.0... so it was already in my ffmpeg last night.

In the libreelec forum, someone has added this patch:

https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-drmprime/ffmpeg-001-v4l2-drmprime.patch

So, I am adding this patch and compiling again.

He aims to make it work for Rockchip and still hasn't been successful yet, by the way.

Edited April 26, 2023 by robertoj

robertoj · April 26, 2023

Jernej, I am trying the ffmpeg with just the v4l2_request patch... while the other additional patch is compiling

Running ffmpeg -loglevel debug -hwaccel drm -hwaccel_output_format drm_prime -i video.mkv -f null -

It looks like some decoding is happening, I see that v4l2_request is being used:

Spoiler

roberto@orangepizero:~$ ffmpeg -loglevel debug -hwaccel drm -hwaccel_output_format drm_prime -i youtube720p.mp4 -f null -
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
configuration: --prefix=/home/roberto/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/roberto/ffmpeg_build/include --extra-ldflags=-L/home/roberto/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/roberto/bin --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-v4l2-request --enable-libdrm --enable-libudev
libavutil      58. 2.100 / 58. 2.100
libavcodec     60. 3.100 / 60. 3.100
libavformat    60. 3.100 / 60. 3.100
libavdevice    60. 1.100 / 60. 1.100
libavfilter     9. 3.100 / 9. 3.100
libswscale      7. 1.100 / 7. 1.100
libswresample   4. 10.100 / 4. 10.100
libpostproc    57. 1.100 / 57. 1.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-hwaccel' ... matched as option 'hwaccel' (use HW accelerated decoding) with argument 'drm'.
Reading option '-hwaccel_output_format' ... matched as option 'hwaccel_output_format' (select output format used with HW accelerated decoding) with argument 'drm_prime'.
Reading option '-i' ... matched as input url with argument 'youtube720p.mp4'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'null'.
Reading option '-' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url youtube720p.mp4.
Applying option hwaccel (use HW accelerated decoding) with argument drm.
Applying option hwaccel_output_format (select output format used with HW accelerated decoding) with argument drm_prime.
Successfully parsed a group of options.
Opening an input file: youtube720p.mp4.
[NULL @ 0x25a7ac0] Opening 'youtube720p.mp4' for reading
[file @ 0x25a7f40] Setting default whitelist 'file,crypto,data'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x25a7ac0] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x25a7ac0] ISO: File Type Major Brand: mp42
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x25a7ac0] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x25a7ac0] Processing st: 0, edit list 0 - media time: 512, duration: 4531200
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x25a7ac0] Offset DTS by 512 to make first pts zero.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x25a7ac0] Setting codecpar->delay to 1 for stream st: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x25a7ac0] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x25a7ac0] Before avformat_find_stream_info() pos: 127312 bytes read:131072 seeks:0 nb_streams:2
[h264 @ 0x25a8780] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x25a8780] Decoding VUI
[h264 @ 0x25a8780] nal_unit_type: 8(PPS), nal_ref_idc: 3
For transform of length 64, inverse, mdct_float, flags: [aligned, out_of_place], found 2 matches:
    1: mdct_inv_float_c - type: mdct_float, len: [2, ∞], factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only], prio: 96
    2: mdct_naive_inv_float_c - type: mdct_float, len: [2, ∞], factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only], prio: -130976
For transform of length 32, inverse, fft_float, flags: [aligned, inplace, preshuf], found 2 matches:
    1: fft_pfa_ns_float_c - type: fft_float, len: [6, ∞], factors[2]: [7, 5, 3, 2, any], flags: [unaligned, inplace, out_of_place, preshuf], prio: 112
    2: fft32_ns_float_c - type: fft_float, len: 32, factor: 2, flags: [unaligned, inplace, out_of_place, preshuf], prio: 96
Transform tree:
    mdct_inv_float_c - type: mdct_float, len: 64, factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only]
        fft32_ns_float_c - type: fft_float, len: 32, factor: 2, flags: [unaligned, inplace, out_of_place, preshuf]
For transform of length 64, inverse, mdct_float, flags: [aligned, out_of_place], found 2 matches:
    1: mdct_inv_float_c - type: mdct_float, len: [2, ∞], factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only], prio: 96
    2: mdct_naive_inv_float_c - type: mdct_float, len: [2, ∞], factors[2]: [2, any], flags: [unaligned, out_of_place, inv_only], prio: -130976
For transform of length 32, inverse, fft_float, flags: [aligned, inplace, preshuf], found 2 matches:
    1: fft_pfa_ns_float_c - type: fft_float, len: [6, ∞], factors[2]: [7, 5, 3, 2, any], flags: [unaligned, inplace, out_of_place, preshuf], prio: 112
    2: fft32_ns_float_c - type: fft_float, len: 32, factor: 2, flags: [unaligned, inplace, out_of_place, preshuf], prio: 96
Transform tree:
    mdct_inv_float_c - type: mdct_float, le

....

th 512, forward, fft_float, flags: [aligned, inplace, preshuf], found 2 matches:
    1: fft_pfa_ns_float_c - type: fft_float, len: [6, ∞], factors[2]: [7, 5, 3, 2, any], flags: [unaligned, inplace, out_of_place, preshuf], prio: 112
    2: fft512_ns_float_c - type: fft_float, len: 512, factor: 2, flags: [unaligned, inplace, out_of_place, preshuf], prio: 96
Transform tree:
    mdct_fwd_float_c - type: mdct_float, len: 1024, factors[2]: [2, any], flags: [unaligned, out_of_place, fwd_only]
        fft512_ns_float_c - type: fft_float, len: 512, factor: 2, flags: [unaligned, inplace, out_of_place, preshuf]
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[vost#0:0/wrapped_avframe @ 0x25c5b60] cur_dts is invalid [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
    Last message repeated 1 times
[h264 @ 0x25ccb10] nal_unit_type: 5(IDR), nal_ref_idc: 1
[h264 @ 0x25ccb10] Format drm_prime chosen by get_format().
[h264 @ 0x25ccb10] Format drm_prime requires hwaccel initialisation.
[h264 @ 0x25ccb10] ff_v4l2_request_init: avctx=0x25ccb10 hw_device_ctx=0x270fd10 hw_frames_ctx=(nil)
[h264 @ 0x25ccb10] v4l2_request_probe_media_device: avctx=0x25ccb10 ctx=0xb0e61530 path=/dev/media0 driver=cedrus
[h264 @ 0x25ccb10] v4l2_request_probe_video_device: avctx=0x25ccb10 ctx=0xb0e61530 path=/dev/video0 capabilities=69238784
[h264 @ 0x25ccb10] v4l2_request_init_context: pixelformat=842094158 width=1280 height=720 bytesperline=1280 sizeimage=1382400
[h264 @ 0x25ccb10] ff_v4l2_request_frame_params: avctx=0x25ccb10 ctx=0xb0e61530 hw_frames_ctx=0xb0e71980 hwfc=0xb0e72900 pool=0xb0e729d0 width=1280 height=720 initial_pool_size=3
[h264 @ 0x25ccb10] v4l2_request_frame_alloc: avctx=0x25ccb10 size=512 data=0xb0e72c30
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: avctx=0x25ccb10 buf=0xb0e72d64 type=2
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: pixelformat=875967059 width=1280 height=720 bytesperline=0 sizeimage=4194304
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: buf=0xb0e72d64 index=0 fd=-1 addr=0xb0a00000 width=1280 height=720 size=4194304
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: avctx=0x25ccb10 buf=0xb0e72dc8 type=1
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: pixelformat=842094158 width=1280 height=720 bytesperline=1280 sizeimage=1382400
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: buf=0xb0e72dc8 index=0 fd=6 addr=(nil) width=1280 height=720 size=1382400
[h264 @ 0x25ccb10] v4l2_request_frame_alloc: avctx=0x25ccb10 size=512 data=0xb0e72c30 request_fd=7
[h264 @ 0x25ccb10] v4l2_request_frame_alloc: avctx=0x25ccb10 size=512 data=0xb0e61960
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: avctx=0x25ccb10 buf=0xb0e61a94 type=2
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: pixelformat=875967059 width=1280 height=720 bytesperline=0 sizeimage=4194304
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: buf=0xb0e61a94 index=1 fd=-1 addr=0xb0600000 width=1280 height=720 size=4194304
[h264 @ 0x25ccb10] v4l2_request_buffer_alloc: avctx=

...

I will try it with mpv

mpv --hwdec=drm --hwdec-codecs=all youtube720p.mp4 decodes with CPU

mpv --hwdec=gpu --hwdec-codecs=all --drm-drmprime-videoplane=primary youtube720p.mp4 decodes with CPU

ffmpeg -loglevel debug -hwaccel drm -hwaccel_output_format drm_prime -i youtube720p.mp4 -vcodec copy -acodec copy -f matroska - | ffplay -i - decodes with CPU

ffmpeg -loglevel debug -hwaccel drm -hwaccel_output_format drm_prime -i youtube720p.mp4 -vcodec copy -acodec copy -f h264 - | ffplay -i - decodes with CPU

ffmpeg -loglevel debug -hwaccel drm -hwaccel_output_format drm_prime -i youtube720p.mp4 -vcodec copy -acodec copy -f drm_prime - | ffplay -i - does not work

I keep trying

Edited April 26, 2023 by robertoj

usual user · April 26, 2023

6 hours ago, robertoj said:

Usual user… what do you get with “vainfo” … does it matter?

I don't even know what "vainfo" is.

6 hours ago, robertoj said:

Are there other components at play, more required Linux patches, libraries sandwiched between Linux and ffmpeg?

I' m running on a system that is build from pure mainline not even special designed for my devices but runs to it full capacity what mainline provides.
I have to just patch in the request-api as that is missing in mainline.

1 hour ago, robertoj said:

mpv --hwdec=drm --hwdec-codecs=all youtube720p.mp4 decodes with CPU

In a terminal run:

mpv -v --hwdec=drm --hwdec-codecs=all youtube720p.mp4

Quit the playing and provide the log to see what is going on.

robertoj · April 26, 2023

Thank you...

This is what I get: (running from SSH worked the same way as running from the console, without Xserver or desktop)

Spoiler

roberto@orangepizero:~$ export DISPLAY=:0.0
roberto@orangepizero:~$ mpv -v --hwdec=drm --hwdec-codecs=all youtube720p.mp4
[cplayer] Command line options: '-v' '--hwdec=drm' '--hwdec-codecs=all' 'youtube720p.mp4'
[cplayer] mpv 0.35.1 Copyright © 2000-2023 mpv/MPlayer/mplayer2 projects
[cplayer] built on UNKNOWN
[cplayer] FFmpeg library versions:
[cplayer]    libavutil       57.28.100
[cplayer]    libavcodec      59.37.100
[cplayer]    libavformat     59.27.100
[cplayer]    libswscale      6.7.100
[cplayer]    libavfilter     8.44.100
[cplayer]    libswresample   4.7.100
[cplayer] FFmpeg version: 5.1.2-3
[cplayer]
[cplayer] Configuration: meson configure build -Dprefix=/usr -Dbuildtype=plain -Doptimization=plain
[cplayer] List of enabled features: alsa av-channel-layout caca cdda cplayer cplugins dmabuf-interop-gl dmabuf-interop-pl dmabuf-wayland drm drm-is-kms dvdnav egl egl-drm egl-helpers egl-x11 ffmpeg gbm gl gl-wayland glibc-thread-name glob glob-posix gpl iconv jack javascript jpeg jpegxl lcms2 libarchive libass libavdevice libbluray libdl libm libmpv-shared libplacebo libplacebo-next librt linux-fstatfs lua52 manpage-build memfd_create noexecstack pipewire plain-gl posix pulse rubberband rubberband-3 sdl2 sdl2-audio sdl2-gamepad sdl2-video sixel spirv-cross stdatomic threads uchardet vaapi vaapi-drm vaapi-egl vaapi-libplacebo vaapi-wayland vaapi-x-egl vaapi-x11 vdpau vector vt.h vulkan wayland wayland_protocols_1_24 x11 xv zimg zlib
[cplayer] Reading config file /etc/mpv/encoding-profiles.conf
[cplayer] Applying profile 'default'...
[cplayer] Reading config file /home/roberto/.config/mpv/mpv.conf
[cplayer] Applying profile 'default'...
[cplayer] Setting option 'v' = '' (flags = 😎
[cplayer] Setting option 'hwdec' = 'drm' (flags = 😎
[cplayer] Setting option 'hwdec-codecs' = 'all' (flags = 😎
[cplayer] Waiting for scripts...
[osd/libass] libass API version: 0x1701000
[osd/libass] libass source: tarball: 0.17.1
[osd/libass] Shaper: FriBidi 1.0.8 (SIMPLE) HarfBuzz-ng 6.0.0 (COMPLEX)
[osd/libass] Setting up fonts...
[osd/libass] Using font provider fontconfig
[osd/libass] Done.
[cplayer] Set property: shared-script-properties -> 1
[cplayer] Set property: shared-script-properties -> 1
[cplayer] Done loading scripts.
[cplayer] Running hook: ytdl_hook/on_load
[ytdl_hook] ytdl:// hook
[ytdl_hook] not a ytdl:// url
[ifo_dvdnav] Opening youtube720p.mp4
[bdmv/bluray] Opening youtube720p.mp4
[file] Opening youtube720p.mp4
[demux] Trying demuxers for level=normal.
[cplayer] Set property: shared-script-properties -> 1
[osd/libass] libass API version: 0x1701000
[osd/libass] libass source: tarball: 0.17.1
[osd/libass] Shaper: FriBidi 1.0.8 (SIMPLE) HarfBuzz-ng 6.0.0 (COMPLEX)
[osd/libass] Setting up fonts...
[lavf] Found 'mov,mp4,m4a,3gp,3g2,mj2' at score=100 size=2048.
[osd/libass] Using font provider fontconfig
[osd/libass] Done.
[demux] Detected file format: mov,mp4,m4a,3gp,3g2,mj2 (libavformat)
[cplayer] Opening done: youtube720p.mp4
[find_files] Loading external files in .
[cplayer] Running hook: ytdl_hook/on_preloaded
[lavf] select track 0
[lavf] select track 1
[cplayer] (+) Video --vid=1 (*) (h264 1280x720 30.000fps)
[cplayer] (+) Audio --aid=1 --alang=eng (*) (aac 2ch 44100Hz)
[vo/gpu] Probing for best GPU context.
[vo/gpu/opengl] Initializing GPU context 'wayland'
[vo/gpu/opengl] Initializing GPU context 'x11egl'
[vo/gpu/x11] X11 opening display: :0.0
[vo/gpu/x11] couldn't open the X11 display (:0.0)!
[vo/gpu/opengl] Initializing GPU context 'drm'
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
[vo/gpu/opengl] Initializing KMS
[vo/gpu/opengl] Picked DRM card 1, primary node /dev/dri/card0 as the default.
[vo/gpu/opengl] Driver: sun4i-drm 1.0.0 (20150629)
[vo/gpu/opengl/kms] Connector 62 currently connected to encoder 61
[vo/gpu/opengl/kms] Selected Encoder 61 with CRTC 60
[vo/gpu/opengl/kms] Could not find any preferred mode. Picking the first mode.
[vo/gpu/opengl/kms] Selected mode: PAL (720x576@50.00Hz)
[vo/gpu/opengl] DRM Atomic support found
[vo/gpu/opengl/kms] Using primary plane 56 as draw plane
[vo/gpu/opengl/kms] Using overlay plane 50 as drmprime plane
[vo/gpu] GBM_FORMAT_ARGB8888 supported by draw plane.
[vo/gpu] Supported modifier: 0x0
[vo/gpu] Creating GBM device
[vo/gpu] Initializing GBM surface (720 x 576)
[vo/gpu/opengl] Initializing EGL
[vo/gpu/opengl] EGL_VERSION=1.4
[vo/gpu/opengl] EGL_VENDOR=Mesa Project
[vo/gpu/opengl] EGL_CLIENT_APIS=OpenGL OpenGL_ES
[vo/gpu/opengl] Trying to create Desktop OpenGL context.
[vo/gpu/opengl] Attempting to find EGLConfig matching GBM_FORMAT_ARGB8888
[vo/gpu/opengl] Found matching EGLConfig for GBM_FORMAT_ARGB8888
[vo/gpu/opengl] Initializing EGL surface
[vo/gpu] GL_VERSION='2.1 Mesa 22.3.6'
[vo/gpu] Detected desktop OpenGL 2.1.
[vo/gpu] GL_VENDOR='lima'
[vo/gpu] GL_RENDERER='Mali400'
[vo/gpu] GL_SHADING_LANGUAGE_VERSION='1.20'
[vo/gpu] Loaded extension GL_ARB_map_buffer_range.
[vo/gpu] Loaded extension GL_ARB_vertex_array_object.
[vo/gpu] Loaded extension GL_ARB_texture_rg.
[vo/gpu] Loaded extension GL_ARB_sync.
[vo/gpu] Loaded extension GL_ARB_invalidate_subdata.
[vo/gpu] Loaded extension GL_ARB_get_program_binary.
[vo/gpu] Loaded extension GL_ARB_buffer_storage.
[vo/gpu] Loaded extension GL_ARB_debug_output.
[vo/gpu] Loaded extension GL_KHR_debug.
[vo/gpu/opengl] Preparing framebuffer
[vo/gpu/opengl] GBM surface using modifier 0x0
[vo/gpu] crtc is not VRR capable
[vo/gpu/opengl] Could not find path to render node. VAAPI hwdec will be disabled
[vo/gpu] Monitor pixel aspect: 1
[vo/gpu] Disabling HDR peak computation (one or more of the following is not supported: compute shaders=0, SSBO=0).
[vo/gpu] No advanced processing required. Enabling dumb mode.
[vo/gpu] Assuming 50.000000 FPS for display sync.
[vd] Container reported FPS: 30.000000
[vd] Codec list:
[vd]     h264 - H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
[vd]     h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper
[vd] Opening decoder h264
[vd] Unsupported hwdec: drm
[vd] No hardware decoding available for this codec.
[vd] Using software decoding.
[vd] Detected 4 logical cores.
[vd] Requesting 5 threads for decoding.
[vd] Selected codec: h264 (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)
[vf] User filter list:
[vf]   (empty)
[ad] Codec list:
[ad]     aac - AAC (Advanced Audio Coding)
[ad]     aac_fixed (aac) - AAC (Advanced Audio Coding)
[ad] Opening decoder aac
[ad] Requesting 1 threads for decoding.
[ad] Selected codec: aac (AAC (Advanced Audio Coding))
[af] User filter list:
[af]   (empty)
[cplayer] Starting playback...
[cplayer] Set property: shared-script-properties -> 1
[vd] DR failed - disabling.
[af] [in] 44100Hz stereo 2ch floatp
[af] [userspeed] 44100Hz stereo 2ch floatp
[af] [userspeed] (disabled)
[af] [convert] 44100Hz stereo 2ch floatp
[vd] Using software decoding.
[vd] Decoder format: 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/auto CL=mpeg2/4/h264
[vf] [in] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [userdeint] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [userdeint] (disabled)
[vf] [autorotate] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [autorotate] (disabled)
[vf] [convert] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [convert] (disabled)
[vf] [out] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[ao] Trying audio driver 'pipewire'
[ao/pipewire] requested format: 44100 Hz, stereo channels, floatp
[ao/pipewire] Headers version: 0.3.65
[ao/pipewire] Library version: 0.3.65
[W][00368.799358] pw.conf      | [          conf.c: 939 try_load_conf()] can't load config client.conf: No such file or directory
[E][00368.799931] pw.conf      | [          conf.c: 963 pw_conf_load_conf_for_context()] can't load default config client.conf: No such file or directory
[ao] Trying audio driver 'pulse'
[ao/pulse] requested format: 44100 Hz, stereo channels, floatp
xcb_connection_has_error() returned true
[ao/pulse] Library version: 16.1.0
[ao/pulse] Proto: 35
[ao/pulse] Server proto: 4294967295
[ao] Trying audio driver 'alsa'
[ao/alsa] requested format: 44100 Hz, stereo channels, floatp
[ao/alsa] using ALSA version: 1.2.8
[ao/alsa] opening device 'default'
ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM default
[ao/alsa] Playback open error: No such file or directory
[ao] Trying audio driver 'jack'
[ao/jack] requested format: 44100 Hz, stereo channels, floatp
[ao/jack] Channel layouts:
[ao/jack] - waveext
[ao/jack] result: stereo
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
[ao/jack] cannot open server
[ao] Trying audio driver 'sdl'
[ao/sdl] requested format: 44100 Hz, stereo channels, floatp
xcb_connection_has_error() returned true
[ao/sdl] Channel layouts:
[ao/sdl] - mono
[ao/sdl] - stereo
[ao/sdl] - 2.1
[ao/sdl] - 4.0
[ao/sdl] - 5.0
[ao/sdl] - 5.1
[ao/sdl] - 6.1
[ao/sdl] - 7.1
[ao/sdl] - unknown9
[ao/sdl] - unknown10
[ao/sdl] - unknown11
[ao/sdl] - unknown12
[ao/sdl] - unknown13
[ao/sdl] - unknown14
[ao/sdl] - unknown15
[ao/sdl] - unknown16
[ao/sdl] result: stereo
[ao/sdl] requested format: 44100 Hz, 2 channels, 8120, buffer size: 0 samples
ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM default
[ao/sdl] could not open audio: ALSA: Couldn't open audio device: No such file or directory
[ao] Failed to initialize audio driver 'sdl'
[cplayer] Could not open/initialize audio device -> no sound.
[lavf] deselect track 1
[cplayer] Audio: no audio
[cplayer] VO: [gpu] 1280x720 yuv420p
[cplayer] VO: Description: Shader-based GPU Renderer
[vo/gpu] DR path suspected slow/uncached, disabling..reconfig to 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vo/gpu] Resize: 720x576
[vo/gpu] Window size: 720x576 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1280x720 (1:1)
[vo/gpu] Video display: (0, 0) 1280x720 -> (0, 85) 720x405
[vo/gpu] Video scale: 0.562500/0.562500
[vo/gpu] OSD borders: l=0 t=85 r=0 b=86
[vo/gpu] Video borders: l=0 t=85 r=0 b=86
[vo/gpu] Texture for plane 0: 1280x720
[vo/gpu] Texture for plane 1: 640x360
[vo/gpu] Texture for plane 2: 640x360
[vo/gpu] No advanced processing required. Enabling dumb mode.
[cplayer] first video frame after restart shown
[cplayer] playback restart complete @ 0.000000, audio=eof, video=playing
[statusline] V: 00:00:00 / 00:04:55 (0%)
[cplayer] Set property: shared-script-properties -> 1
[vo/gpu/opengl] GBM surface using modifier 0x0
[cplayer] Set property: shared-script-properties -> 1
[vo/gpu/opengl] GBM surface using modifier 0x0
[statusline] V: 00:00:00 / 00:04:55 (0%)
[vo/gpu/opengl] GBM surface using modifier 0x0
[statusline] V: 00:00:12 / 00:04:55 (4%)
[cplayer] EOF code: 5
[cplayer] finished playback, success (reason 3)
[cplayer]
[cplayer] Exiting... (Quit)
[cplayer] Set property: shared-script-properties -> 1
roberto@orangepizero:~$

And when I run it from within the lxde desktop manager:

Spoiler

roberto@orangepizero:~$ mpv -v --hwdec=drm -hwdec-codecs=all youtube720p.mp4
[cplayer] Command line options: '-v' '--hwdec=drm' '-hwdec-codecs=all' 'youtube720p.mp4'
[cplayer] mpv 0.35.1 Copyright © 2000-2023 mpv/MPlayer/mplayer2 projects
[cplayer] built on UNKNOWN
[cplayer] FFmpeg library versions:
[cplayer]    libavutil       57.28.100
[cplayer]    libavcodec      59.37.100
[cplayer]    libavformat     59.27.100
[cplayer]    libswscale      6.7.100
[cplayer]    libavfilter     8.44.100
[cplayer]    libswresample   4.7.100
[cplayer] FFmpeg version: 5.1.2-3
[cplayer]
[cplayer] Configuration: meson configure build -Dprefix=/usr -Dbuildtype=plain -Doptimization=plain
[cplayer] List of enabled features: alsa av-channel-layout caca cdda cplayer cplugins dmabuf-interop-gl dmabuf-interop-pl dmabuf-wayland drm drm-is-kms dvdnav egl egl-drm egl-helpers egl-x11 ffmpeg gbm gl gl-wayland glibc-thread-name glob glob-posix gpl iconv jack javascript jpeg jpegxl lcms2 libarchive libass libavdevice libbluray libdl libm libmpv-shared libplacebo libplacebo-next librt linux-fstatfs lua52 manpage-build memfd_create noexecstack pipewire plain-gl posix pulse rubberband rubberband-3 sdl2 sdl2-audio sdl2-gamepad sdl2-video sixel spirv-cross stdatomic threads uchardet vaapi vaapi-drm vaapi-egl vaapi-libplacebo vaapi-wayland vaapi-x-egl vaapi-x11 vdpau vector vt.h vulkan wayland wayland_protocols_1_24 x11 xv zimg zlib
[cplayer] Reading config file /etc/mpv/encoding-profiles.conf
[cplayer] Applying profile 'default'...
[cplayer] Reading config file /home/roberto/.config/mpv/mpv.conf
[cplayer] Applying profile 'default'...
[cplayer] Setting option 'v' = '' (flags = 😎
[cplayer] Setting option 'hwdec' = 'drm' (flags = 😎
[cplayer] Setting option 'hwdec-codecs' = 'all' (flags = 😎
[cplayer] Waiting for scripts...
[osd/libass] libass API version: 0x1701000
[osd/libass] libass source: tarball: 0.17.1
[osd/libass] Shaper: FriBidi 1.0.8 (SIMPLE) HarfBuzz-ng 6.0.0 (COMPLEX)
[osd/libass] Setting up fonts...
[osd/libass] Using font provider fontconfig
[osd/libass] Done.
[cplayer] Set property: shared-script-properties -> 1
[cplayer] Set property: shared-script-properties -> 1
[cplayer] Done loading scripts.
[cplayer] Running hook: ytdl_hook/on_load
[ytdl_hook] ytdl:// hook
[ytdl_hook] not a ytdl:// url
[cplayer] Set property: shared-script-properties -> 1
[ifo_dvdnav] Opening youtube720p.mp4
[bdmv/bluray] Opening youtube720p.mp4
[file] Opening youtube720p.mp4
[demux] Trying demuxers for level=normal.
[cplayer] Set property: shared-script-properties -> 1
[osd/libass] libass API version: 0x1701000
[osd/libass] libass source: tarball: 0.17.1
[osd/libass] Shaper: FriBidi 1.0.8 (SIMPLE) HarfBuzz-ng 6.0.0 (COMPLEX)
[osd/libass] Setting up fonts...
[lavf] Found 'mov,mp4,m4a,3gp,3g2,mj2' at score=100 size=2048.
[demux] Detected file format: mov,mp4,m4a,3gp,3g2,mj2 (libavformat)
[cplayer] Opening done: youtube720p.mp4
[osd/libass] Using font provider fontconfig
[osd/libass] Done.
[find_files] Loading external files in .
[cplayer] Running hook: ytdl_hook/on_preloaded
[lavf] select track 0
[lavf] select track 1
[cplayer] (+) Video --vid=1 (*) (h264 1280x720 30.000fps)
[cplayer] (+) Audio --aid=1 --alang=eng (*) (aac 2ch 44100Hz)
[vo/gpu] Probing for best GPU context.
[vo/gpu/opengl] Initializing GPU context 'wayland'
[vo/gpu/opengl] Initializing GPU context 'x11egl'
[vo/gpu/x11] X11 opening display: :0
[vo/gpu/x11] Display 0 (Composite-1): [0, 0, 720, 576] @ 50.000000 FPS
[vo/gpu/x11] Current display FPS: 50.000000
[vo/gpu/opengl] EGL_VERSION=1.4
[vo/gpu/opengl] EGL_VENDOR=Mesa Project
[vo/gpu/opengl] EGL_CLIENT_APIS=OpenGL OpenGL_ES
[vo/gpu/opengl] Trying to create Desktop OpenGL context.
[vo/gpu/opengl] Choosing visual EGL config 0x7, visual ID 0x21
[vo/gpu/x11] The XPresent extension was found.
[vo/gpu/x11] XPresent disabled.
[vo/gpu/opengl] GL_VERSION='2.1 Mesa 22.3.6'
[vo/gpu/opengl] Detected desktop OpenGL 2.1.
[vo/gpu/opengl] GL_VENDOR='lima'
[vo/gpu/opengl] GL_RENDERER='Mali400'
[vo/gpu/opengl] GL_SHADING_LANGUAGE_VERSION='1.20'
[vo/gpu/opengl] Loaded extension GL_ARB_map_buffer_range.
[vo/gpu/opengl] Loaded extension GL_ARB_vertex_array_object.
[vo/gpu/opengl] Loaded extension GL_ARB_texture_rg.
[vo/gpu/opengl] Loaded extension GL_ARB_sync.
[vo/gpu/opengl] Loaded extension GL_ARB_invalidate_subdata.
[vo/gpu/opengl] Loaded extension GL_ARB_get_program_binary.
[vo/gpu/opengl] Loaded extension GL_ARB_buffer_storage.
[vo/gpu/opengl] Loaded extension GL_ARB_debug_output.
[vo/gpu/opengl] Loaded extension GL_KHR_debug.
[vo/gpu] Disabling HDR peak computation (one or more of the following is not supported: compute shaders=0, SSBO=0).
[vo/gpu] No advanced processing required. Enabling dumb mode.
[vo/gpu] Assuming 50.000000 FPS for display sync.
[vd] Container reported FPS: 30.000000
[vd] Codec list:
[vd]     h264 - H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
[vd]     h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper
[vd] Opening decoder h264
[vd] Unsupported hwdec: drm
[vd] No hardware decoding available for this codec.
[vd] Using software decoding.
[vd] Detected 4 logical cores.
[vd] Requesting 5 threads for decoding.
[vd] Selected codec: h264 (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)
[vf] User filter list:
[vf]   (empty)
[ad] Codec list:
[ad]     aac - AAC (Advanced Audio Coding)
[ad]     aac_fixed (aac) - AAC (Advanced Audio Coding)
[ad] Opening decoder aac
[ad] Requesting 1 threads for decoding.
[ad] Selected codec: aac (AAC (Advanced Audio Coding))
[af] User filter list:
[af]   (empty)
[cplayer] Starting playback...
[vd] DR failed - disabling.
[af] [in] 44100Hz stereo 2ch floatp
[af] [userspeed] 44100Hz stereo 2ch floatp
[af] [userspeed] (disabled)
[af] [convert] 44100Hz stereo 2ch floatp
[vd] Using software decoding.
[vd] Decoder format: 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/auto CL=mpeg2/4/h264
[vf] [in] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [userdeint] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [userdeint] (disabled)
[vf] [autorotate] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [autorotate] (disabled)
[vf] [convert] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [convert] (disabled)
[vf] [out] 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[ao] Trying audio driver 'pipewire'
[ao/pipewire] requested format: 44100 Hz, stereo channels, floatp
[ao/pipewire] Headers version: 0.3.65
[ao/pipewire] Library version: 0.3.65
[W][01007.514280] pw.conf      | [          conf.c: 939 try_load_conf()] can't load config client.conf: No such file or directory
[E][01007.515644] pw.conf      | [          conf.c: 963 pw_conf_load_conf_for_context()] can't load default config client.conf: No such file or directory
[ao] Trying audio driver 'pulse'
[ao/pulse] requested format: 44100 Hz, stereo channels, floatp
[ao/pulse] Library version: 16.1.0
[ao/pulse] Proto: 35
[ao/pulse] Server proto: 4294967295
[ao] Trying audio driver 'alsa'
[ao/alsa] requested format: 44100 Hz, stereo channels, floatp
[ao/alsa] using ALSA version: 1.2.8
[ao/alsa] opening device 'default'
ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM default
[ao/alsa] Playback open error: No such file or directory
[ao] Trying audio driver 'jack'
[ao/jack] requested format: 44100 Hz, stereo channels, floatp
[ao/jack] Channel layouts:
[ao/jack] - waveext
[ao/jack] result: stereo
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
[ao/jack] cannot open server
[ao] Trying audio driver 'sdl'
[ao/sdl] requested format: 44100 Hz, stereo channels, floatp
[ao/sdl] Channel layouts:
[ao/sdl] - mono
[ao/sdl] - stereo
[ao/sdl] - 2.1
[ao/sdl] - 4.0
[ao/sdl] - 5.0
[ao/sdl] - 5.1
[ao/sdl] - 6.1
[ao/sdl] - 7.1
[ao/sdl] - unknown9
[ao/sdl] - unknown10
[ao/sdl] - unknown11
[ao/sdl] - unknown12
[ao/sdl] - unknown13
[ao/sdl] - unknown14
[ao/sdl] - unknown15
[ao/sdl] - unknown16
[ao/sdl] result: stereo
[ao/sdl] requested format: 44100 Hz, 2 channels, 8120, buffer size: 0 samples
ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name
ALSA lib conf.c:5180:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5703:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM default
[ao/sdl] could not open audio: ALSA: Couldn't open audio device: No such file or directory
[ao] Failed to initialize audio driver 'sdl'
[cplayer] Could not open/initialize audio device -> no sound.
[lavf] deselect track 1
[cplayer] Audio: no audio
[cplayer] VO: [gpu] 1280x720 yuv420p
[cplayer] VO: Description: Shader-based GPU Renderer
[vo/gpu] DR path suspected slow/uncached, disabling..reconfig to 1280x720 yuv420p bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vo/gpu/x11] not waiting for MapNotify
[vo/gpu] Resize: 716x402
[vo/gpu] Window size: 716x402 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1280x720 (1:1)
[vo/gpu] Video display: (0, 0) 1280x720 -> (1, 0) 714x402
[vo/gpu] Video scale: 0.557813/0.558333
[vo/gpu] OSD borders: l=1 t=0 r=1 b=0
[vo/gpu] Video borders: l=1 t=0 r=1 b=0
[vo/gpu] Texture for plane 0: 1280x720
[vo/gpu] Texture for plane 1: 640x360
[vo/gpu] Texture for plane 2: 640x360
[vo/gpu] No advanced processing required. Enabling dumb mode.
[cplayer] first video frame after restart shown
[cplayer] playback restart complete @ 0.000000, audio=eof, video=playing
[statusline] V: 00:00:00 / 00:04:55 (0%)
[vo/gpu/x11] Disabling screensaver.
[statusline] V: 00:00:00 / 00:04:55 (0%)
[cplayer] Set property: shared-script-properties -> 1
[cplayer] Set property: shared-script-properties -> 1
[statusline] V: 00:00:07 / 00:04:55 (3%) Dropped: 9
[cplayer] Set property: shared-script-properties -> 1
[statusline] V: 00:00:07 / 00:04:55 (3%) Dropped: 9
[osd/libass] fontselect: (sans-serif, 400, 0) -> /usr/share/fonts/truetype/dejavu/DejaVuSans.ttf, 0, DejaVuSans
[osd/libass] fontselect: (mpv-osd-symbols, 400, 0) -> mpv-osd-symbols-Regular, 0, mpv-osd-symbols-Regular
[vo/gpu] Reallocating OSD texture to 1024x128.
[statusline] V: 00:00:08 / 00:04:55 (3%) Dropped: 9
[cplayer] Set property: shared-script-properties -> 1
[statusline] V: 00:00:20 / 00:04:55 (7%) Dropped: 66
[vo/gpu] Resize: 704x396
[vo/gpu] Window size: 704x396 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1280x720 (1:1)
[vo/gpu] Video display: (0, 0) 1280x720 -> (0, 0) 704x396
[vo/gpu] Video scale: 0.550000/0.550000
[vo/gpu] OSD borders: l=0 t=0 r=0 b=0
[vo/gpu] Video borders: l=0 t=0 r=0 b=0
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 66
[cplayer] Set property: shared-script-properties -> 1
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 68
[vo/gpu] Resize: 553x311
[vo/gpu] Window size: 553x311 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1280x720 (1:1)
[vo/gpu] Video display: (0, 0) 1280x720 -> (0, 0) 552x311
[vo/gpu] Video scale: 0.431250/0.431944
[vo/gpu] OSD borders: l=0 t=0 r=1 b=0
[vo/gpu] Video borders: l=0 t=0 r=1 b=0
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 68
[cplayer] Set property: shared-script-properties -> 1
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 68
[vo/gpu] Resize: 460x258
[vo/gpu] Window size: 460x258 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1280x720 (1:1)
[vo/gpu] Video display: (0, 0) 1280x720 -> (1, 0) 458x258
[vo/gpu] Video scale: 0.357812/0.358333
[vo/gpu] OSD borders: l=1 t=0 r=1 b=0
[vo/gpu] Video borders: l=1 t=0 r=1 b=0
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 69
[cplayer] Set property: shared-script-properties -> 1
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 73
[vo/gpu] Resize: 424x238
[vo/gpu] Window size: 424x238 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1280x720 (1:1)
[vo/gpu] Video display: (0, 0) 1280x720 -> (0, 0) 423x238
[vo/gpu] Video scale: 0.330469/0.330556
[vo/gpu] OSD borders: l=0 t=0 r=1 b=0
[vo/gpu] Video borders: l=0 t=0 r=1 b=0
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 75
[cplayer] Set property: shared-script-properties -> 1
[vo/gpu] Resize: 422x237
[vo/gpu] Window size: 422x237 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1280x720 (1:1)
[vo/gpu] Video display: (0, 0) 1280x720 -> (0, 0) 421x237
[vo/gpu] Video scale: 0.328906/0.329167
[vo/gpu] OSD borders: l=0 t=0 r=1 b=0
[vo/gpu] Video borders: l=0 t=0 r=1 b=0
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 78
[cplayer] Set property: shared-script-properties -> 1
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 80
[vo/gpu] Resize: 405x227
[vo/gpu] Window size: 405x227 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1280x720 (1:1)
[vo/gpu] Video display: (0, 0) 1280x720 -> (1, 0) 403x227
[vo/gpu] Video scale: 0.314844/0.315278
[vo/gpu] OSD borders: l=1 t=0 r=1 b=0
[vo/gpu] Video borders: l=1 t=0 r=1 b=0
[statusline] V: 00:00:21 / 00:04:55 (7%) Dropped: 81
[cplayer] Set property: shared-script-properties -> 1
[cplayer] Set property: shared-script-properties -> 1
[statusline] V: 00:03:29 / 00:04:55 (71%) Dropped: 121
[cplayer] EOF code: 5
[vo/gpu/x11] Enabling screensaver.
[cplayer] finished playback, success (reason 3)
[cplayer]
[cplayer] Exiting... (Quit)
[cplayer] Set property: shared-script-properties -> 1
roberto@orangepizero:~$

In both cases, the CPU use is 200%

i am using Debian bookworm’s default MPV 0.35.1…

what ubuntu or debian did you build with armbian-build?

When you built ffmpeg with the v4l2_request patch, did you add only "--enable-v4l2-request --enable-libdrm --enable-libudev".. can you share your ffmpeg banner (the text identifying itself and the ./configure options)?

Did you install ffmpeg with these instructions? https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu

The special part is that ffmpeg stays in ~/bin, and many other codecs are enabled.

Edited April 27, 2023 by robertoj

usual user · April 27, 2023

I don't know much about the ffmpeg framework and even less about the interpretation of its protocols, but IMHO the display subsystem ([vo/gpu/opengl/kms] Selected mode: PAL (720x576@50.00Hz)) does not support the necessary prerequisites to serve the hardware decoder output. That is the reason why the VPU is not selected. I don't know if this is a hardware limitation, or just lack of display driver support for your device. As @jernej has confirmed, drm_prime is necessary for VPU usage and this is not mentioned in your logs.

Xwindow is also only based on drm/kms and therefore cannot bring any improvement. Even if the display subsystem provides all the prerequisites, it is not a good choice, as it does not work efficiently with VPUs by design. Here, a desktop environment based on a Wayland backend delivers better results.

On 4/26/2023 at 9:10 AM, robertoj said:

what ubuntu or debian did you build with armbian-build?

Neither, I haven't even built my own system. Debian based systems are IMHO way too stable - outdated. I prefer a system that follows mainline releases more closely.
I'm using an RPM-based system and I just need to recompile the ffmpeg source package that has been supplemented with my extensions.

All I have to do is to install the ffmpeg-5.1.2-3.fc37.src.rpm, copy the patches into the ~/rpmbuild/SOURCES directory and make these changes to the ~/rpmbuild/SPECS/ffmpeg.spec file:

Spoiler

diff ffmpeg.spec.orig ffmpeg.spec
126a127,131
>.
> Patch100:       v4l2-request-n5.1.2.patch
> Patch101:       v4l2-drmprime-n5.1.2.patch
> Patch102:       vf-deinterlace-v4l2m2m-n5.1.2.patch
>.
175a181
> BuildRequires:  pkgconfig(libudev)
366a373,374
>     --enable-libudev \\\
>     --enable-v4l2-request \\\
523a532,538
> * Thu Dec 29 2022 usual user <usual.user@nowhere.org> - [5.1.2-3]
> - BuildRequires:  pkgconfig(libudev)
> - Apply v4l2-request-n5.1.2.patch
> - Apply v4l2-drmprime-n5.1.2.patch
> - Apply vf-deinterlace-v4l2m2m-n5.1.2.patch

After that, a "dnf buildep ~/rpmbuild/SPECS/ffmpeg.spec" installs all the necessary dependencies and I can leave everything else to my build system with "rpmbuild -ba ~/rpmbuild/SPECS/ffmpeg.spec".
Because this procedure can be applied to any RPM package, I can rebuild any package in my distribution to make changes if necessary.
Since only a small proportion of packages are device-dependent, it is absolutely not necessary to compile a distribution completely by myself. I have to build only the kernel and firmware (uboot-tools) packages. In addition, there are mesa, gstreamer1 and ffmpeg to be up to date, but these are not really device-specific. I don't have to build mesa and gstreamer1 myself anymore because current releases already contain everything I need. So I have a generic system that works for all devices of an architecture, as long as there is sufficient mainline support in kernel and firmware.
I use this system for my CuBox, my HummingBord, my NanoPC-T4, my Odroid-N2+ and my Odroid-M1. All devices of the same architecture can use the same SD card. And since I can build firmware for Odroid-C4, Odroid-HC4 and ROCK PI 4B, whose functionality is confirmed, my SD card will also work there in the same way. In order to be able to support other devices, only a suitable firmware and a corresponding DTB is necessary.
As soon as the rkvdec2 driver support for an rk35xx SOC becomes available in the kernel, my Odroid M1 will also work with it without further intervention, since all userspace programs are already available and already use the other decoder IPs today. They just need to select a different dev/videoX.

On 4/26/2023 at 9:10 AM, robertoj said:

can you share your ffmpeg banner (the text identifying itself and the ./configure options)?

This is the outcome of my build:

Spoiler

ffmpeg version 5.1.2 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12 (GCC)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg
 --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64
 --mandir=/usr/share/man --arch=aarch64 --optflags='-O2 -flto=auto
 -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall
 -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
 -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mbranch-protection=standard
 -fasynchronous-unwind-tables -fstack-clash-protection' --extra-ldflags='-Wl,-z,relro
 -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 '
 --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb
 --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3
 --enable-bzlib --enable-chromaprint --disable-crystalhd --enable-fontconfig
 --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom
 --enable-libdav1d --enable-libass --enable-libbluray --enable-libbs2b
 --enable-libcdio --enable-libdrm --enable-libjack --enable-libfreetype
 --enable-libfribidi --enable-libgsm --enable-libilbc --enable-libmp3lame
 --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl
 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse
 --enable-librsvg --enable-librav1e --enable-librubberband --enable-libsmbclient
 --enable-version3 --enable-libsnappy --enable-libsoxr --enable-libspeex
 --enable-libsrt --enable-libssh --enable-libtesseract --enable-libtheora
 --enable-libtwolame --enable-libvorbis --enable-libudev --enable-v4l2-request
 --enable-libv4l2 --enable-libvidstab --enable-libvpx --enable-vulkan
 --enable-libshaderc --enable-libwebp --enable-libx264 --enable-libx265
 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi
 --enable-lv2 --enable-avfilter --enable-libmodplug --enable-postproc
 --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug
 --disable-stripping --shlibdir=/usr/lib64 --enable-lto
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

It is a generic build that serves all devices of the architecture for which it was built.

robertoj · April 28, 2023

Thank you. I havent tried anything today, but tomorrow I will try a lower ffmpeg version.

Maybe ffmpeg 6.0 is too new for mpv (although kodi is able to use it)

update: I tried it with ffmpeg 5.1.3… and i am still getting the same error in mpv: hwdec not supported drm, and falling into software decoding.

is there any need to “—enable-v4l2”?… (I can’t because my Debian does not have libv4l2-dev… only libv4l-dev)

i will document this better today.

I will poke around kodi once again (same orangepi zero model, with the same CVBS pal/ntsc output, does hardware h264 everyday)… I would need to know what the internal ffmpeg and gbm are doing together)

Edited April 28, 2023 by robertoj

usual user · April 29, 2023

17 hours ago, robertoj said:

is there any need to “—enable-v4l2”?

I don't know the inner structure of ffmpeg and I don't know what actions trigger corresponding compiler switches, but since the request-api is only an extension in v4l2, I would guess that the base v4l2 support is also needed.

17 hours ago, robertoj said:

same orangepi zero model, with the same CVBS pal/ntsc output, does hardware h264 everyday

Does this system run on current mainline software or does it use legacy code?

robertoj · April 29, 2023

The previous LibreElec+Kodi, and the latest LibreElec+Kodi, with the latest linux kernel version, with the LibreElec patches (for Linux, ffmpeg, etc) run on the OrangePiZero, and provides hardware H264, in the analog video output.

usual user · April 29, 2023

8 hours ago, robertoj said:

run on the OrangePiZero, and provides hardware H264, in the analog video output.

If everything is based on mainline code, it is only a matter of reverse engineering the build configuration to replicate it in your own build system. However, you should check if LE is still applying patches to the drm/kms display subsystem in the kernel, as I don't know what the out-of-the-box support is for your device in this regard. This could also be a reason why the VPU cannot be used. And that's not something where userspace code is involved.

Sign In

Understanding Hardware-Accelerated Video Decoding

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Join the conversation

Important Information