What to use when programming with ffmpeg to do HW video decode!?

[fredrum]

Hello!! 

I just got my Orange Pi 5 the other day and am trying to 'port' a program I've been working on for RPi4b over to this new board.

The program is using ffmpeg to decode h264 and hevc video streams.

 

My main question is how do I need to configure the ffmpeg codex context and formats to make use of (ideally zero copy) HW accelerated video decode on Orange Pi 5?
For the raspberry code I was using these:

 

avcodec_find_decoder_by_name("hevc");

and

avcodec_find_decoder_by_name("h264_v4l2m2m");



together with 

codec_context->pix_fmt = AV_PIX_FMT_DRM_PRIME;   // for zero copy video path

 

But this does not work when compiling on the Orange Pi 5 and I get an error message saying:

"Failed to create specified HW device."

 

But maybe on this board/distro it looks like h264_v4l2m2m is disabled.

 

Does anyone know how to set up ffmpeg for Orange Pi 5?
Does it support V4L2 at all? 

 

EDIT:  VLC seems to use hw accel when playing back some h264/hevc file I have.  I'm guessing because I see a pretty low CPU use.

 

 

Cheers!

 

 

Here's some info about my system:

 

Linux orangepi5 5.10.110-rockchip-rk3588 #23.02.2 SMP Fri Feb 17 23:59:20 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

 

Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Panfrost (0xffffffff)
    Device: Mali-G610 (Panfrost) (0xffffffff)
    Version: 23.0.0
    Accelerated: yes
    Video memory: 7689MB
    Unified memory: yes
    Preferred profile: compat (0x2)
    Max core profile version: 0.0
    Max compat profile version: 3.0
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: Panfrost
OpenGL renderer string: Mali-G610 (Panfrost)
OpenGL version string: 3.0 Mesa 23.0.0-devel
OpenGL shading language version string: 1.30
OpenGL context flags: (none)

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 23.0.0-devel
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

 

 

OS: Armbian (23.02.2) aarch64

Host: Orange Pi 5

Kernel: 5.10.110-rockchip-rk3588

 

 

 

 

Edited February 26, 2023 by fredrum

[fredrum]

 

I did make a little progress yesterday but I seem to only be allowed to post here once every 24h so I'm going to ask a whole bunch of questions related to this all at once now. 
 

I found that I can use these codecs instead of the ones I mentioned above,

 

avcodec_find_decoder_by_name("hevc_rkmpp");
or
avcodec_find_decoder_by_name("h264_rkmpp");

 

Those both now decode and even if I haven't yet been able to display them (I'll get to that tonight I think) I have confirmed decode success by converting and writing out a single frame to an png image.

 

 

Q1:  But how do I do zero copy drm display with this setup?  My program on RPi4 does zero copy by sending a dma buf to the opengl call,

const EGLImage image = eglCreateImageKHR( egl_display,
                                              EGL_NO_CONTEXT,
                                              EGL_LINUX_DMA_BUF_EXT,
                                              NULL, attribs);

But here when I ask for AV_PIX_FMT_DRM_PRIME I get a pix_fmt value of 0.  (drm prime should have 181)

codec_context->pix_fmt = AV_PIX_FMT_DRM_PRIME;  // Does not work

 

The program does still on the OPi5 display the GUI using kmsdrm when I run it on the tty.  (using SDL2)

But that does not seem to extend to ffmpeg.

 

I have figured out that mpp is a thing for Rockchip and that does seem to have its own kmsdrm display techniques.

Is that what I'll need to use?

 

 

Q2:  I read someone say that with a future mainline kernel support and upgrade to 6.x that v4l2 and standard drm display should become available.

Can anyone confirm if this is true?  And also if this is likely to happen in near future or is a quite some time yet?

 

 

Q3:  What are the libavcodec-dev et al on the OPi5 Armbian distro's coming from?

Are they some special Rockchip forks or are they the ffmpeg master with just config options?

 

Cheers!

Edited February 28, 2023 by fredrum
avcodec-dev to libavcodec-dev