Can't use FFmpeg hardware accelration (v4l2) - Orange Pi 4 LTS

[Marc512]

Hello everyone,

 

Thank you for stopping buy and reading this thread. I have Orange Pi 4 LTS with the latest Armbian Jammy image (Armbian_23.8.1_Orangepi4-lts_jammy_current_6.1.50). But I also want to mention that I've tried the same thing on the "official" Ubuntu image from OrangePi website and it was the same.

I need to decode and show RTSP video stream, h264 encoded. First of all, I've tried gstreamer with v4l2 codecs. I was using v4l2slh264dec from gst_plugins_bad. This is the only hardware codec in gstreamer for my device (at least available by default) and it worked fine until I started encountering a strange bug when my video getting stuck into 0.5-1 second loop forever. Anyway, I thought that I could bypass this issue by building libav/FFmpeg with v4l2 features and using it instead. However, I couldn't, and this is my problem.
I am building FFmpeg from source like this:
First of all, I made sure that there is no FFmpeg on my system.

Then:
 

sudo apt install libv4l-dev libsdl2-dev -y
git clone https://github.com/FFmpeg/FFmpeg.git
cd FFmpeg/
./configure --enable-libv4l2 --enable-libdrm

No errors/warnings. h264_v4l2m2m is in "Enabled decoders".

make -j6
sudo make install
sudo ldconfig
ffplay -vcodec h264_v4l2m2m rtsp://192.168.0.125:8554/stream

It gives me the following error:

If I do not specify v4l2m2m decoder - it works just fine, but with software decoding.

Drivers are present:

And I have no idea how to fix this issue
I will appreciate a lot any help!

Edited November 21, 2023 by Marc512

[Werner]

moved

[jock]

hardware video decoding is still a complex beast, mostly because ffmpeg is still missing the necessary bits which are available as separate patches.

v4l2m2m is not the right codec: those are suitable for stataful decoders (rpi and amlogic), but rockchip has a stateless decoder and you have to use v4l2_request decoders.

 

I'm currently working on bringing a ubuntu and debian apt repository which should ease the pain with ffmpeg. It is in an early state and works better in debian bookworm rather than ubuntu jammy currently. I would not disclose the repository yet because it is very early and hosted in my personal lan, but if you're interested in give it a chance I may give you some instructions via private message.

[Marc512]

@jock hi! Thank you for info. Yes, of course, I would like to give it a try. Appreciate that!

[Marc512]

I also decided to give a try to vulkan: HWAccelIntro – FFmpeg. But it seems like there is no proper vulkan driver, so it uses llvmpipe. But it works.
I also found this fork of FFmpeg that introduces v4l2-request hwaccel support, and it SEEMS to work if I launch ffmpeg with -hwaccel drm

[Dave Jones]

@jockHi, your ffmpeg-v4l2request is 5.1.4, Debian bookworm's ffmpeg now is 5.1.6.

Would you share your patches about v4l2request support for ffmpeg so I could build 5.1.6 myself, thank you.