Jump to content

Repository for v4l2request hardware video decoding (rockchip, allwinner)

jock
 Share

Recommended Posts

jock

jock

Posted
  • Author
Posted
  On 7/23/2024 at 4:35 PM, alejor said:

Hi, I tried this with armbian bookworm and orange pi zero 3 worked flawlessly with h264 hwdec. Now... Armbian image are Ubuntu Noble 24.04, can you please create or recompile packages for that 24.04 distro? or give some instructions to create those packages? thank you!

Expand  

That will take some time to set the whole setup; unfortunately it is not easy task to bring up a system which could be used easily to rebuild the packages, but will do in the future. Can't say when though.

0
Guest

Guest

Posted
Posted

Hi @jock , I would like to request a Noble build as well. If hardware is an issue, I can help compile this (remotely).

Thanks!

0
robertoj

robertoj

Posted
Posted

Hello all,

 

I tried the apt repo install instructions in the first post... but it seems that my armbian bookworm is still installing its own .deb package and not the one from the apt.undo.it repo

 

Is there an obvious step that I am missing?

 

What version of mpv should I have if I installed successfully?

 

Thank you in advance!!

 

(my target is orange pi zero 3)

 

P.S. I see that the debian repo (through my web browser) does not provide mpv.deb, but Ubuntu Jammy does... does this mean that the armbian-bookworm mpv.deb does not need the patches?

I also see that the available debs are 11 months old... perhaps if you could point to some instructions that work, we could do it ourselves and share the outcome.

0
jock

jock

Posted
  • Author
Posted
  On 10/24/2024 at 9:24 PM, robertoj said:

P.S. I see that the debian repo (through my web browser) does not provide mpv.deb, but Ubuntu Jammy does... does this mean that the armbian-bookworm mpv.deb does not need the patches?

Expand  

Exacty, mpv in debian bookworm already has full support for drm-prime, so there is no need for a patched version of that

 

  On 10/24/2024 at 9:24 PM, robertoj said:

I also see that the available debs are 11 months old... perhaps if you could point to some instructions that work, we could do it ourselves and share the outcome.

Expand  

the instructions that work are there, in the meantime it can be that mpv changed some default, actually I didn't check recently

0
HiperDoo

HiperDoo

Posted
Posted
  Quote

Hi, I tried this with armbian bookworm and orange pi zero 3 worked flawlessly with h264 hwdec.

Expand  

I haven't been able to get it working, I also have the Orange Pi Zero 3 using the Debian 12 (Bookworm) [Build Date: Oct 14, 2024] image with the Minimal/IOT option.

I tried using mpv with the instructions at the beginning of the post on the Gnome and XFCE desktops, but both have the same negative result (sometimes the screen is black or shows a frame of the video, it can never be put into play and I can move through the time scroll to see other frames, but it is unusable).
 

The worst of all is that I do get the message "Using hardware decoding (drm)".
 

The only configurations I do after installing the image are: 

sudo apt update && sudo apt upgrade -y

sudo apt-get install armbian-config mesa-utils chromium

# Install a Desktop
sudo armbian-config

# After rebooting and entering the desktop
sudo wget http://apt.undo.it:7241/apt.undo.it.asc -O /etc/apt/trusted.gpg.d/apt.undo.it.asc
echo "deb http://apt.undo.it:7241/debian bookworm main" | sudo tee /etc/apt/sources.list.d/apt.undo.it.list
sudo apt update
sudo apt install ffmpeg-v4l2request mpv
sudo mkdir -p /etc/mpv
echo -e "hwdec=drm\ndrm-drmprime-video-plane=primary\ndrm-draw-plane=overlay" | sudo tee /etc/mpv/mpv.conf

mpv video.mp4

 

This is the output on the XFCE desktop (but with the same negative result): 

hiperdoo@orangepizero3:~/Downloads$ mpv --ao=pulse video.mp4
 (+) Video --vid=1 (*) (h264 1920x1080 60.000fps)
 (+) Audio --aid=1 (*) (aac 2ch 44100Hz)
File tags:
 Artist: LG Global
 Title: 2021 LG OLED l  Ink Art 4K HDR 60fps
Using hardware decoding (drm).
AO: [pulse] 44100Hz stereo 2ch float
VO: [gpu] 1920x1080 drm_prime[nv12]
AV: 00:00:00 / 00:00:45 (1%) A-V:  0.006 Dropped: 7


I would be very grateful if someone could enlighten me on what other steps are required.

0
robertoj

robertoj

Posted
Posted
  On 10/29/2024 at 10:42 PM, HiperDoo said:

I haven't been able to get it working, I also have the Orange Pi Zero 3 using the Debian 12 (Bookworm) [Build Date: Oct 14, 2024] image with the Minimal/IOT option.

I tried using mpv with the instructions at the beginning of the post on the Gnome and XFCE desktops, but both have the same negative result (sometimes the screen is black or shows a frame of the video, it can never be put into play and I can move through the time scroll to see other frames, but it is unusable).

Expand  

The last reported success was with Debian Bookworm, in a comment from July 23... can you repeat the experiment with an OS image from before that date?

(I don't have access to my orange pi zero 3 for a few days)

0
robertoj

robertoj

Posted
Posted

I confirm that ffmpeg-v4l2request works for opiz3, with OS image Armbian_community_24.8.0-trunk.6_Orangepizero3_bookworm_current_6.6.31_minimal.img.xz

 

I started with the freshly booted OS image, added my DTS for ili9341 SPI LCD (see other thread in community-allwinner), rebooted, and followed all the steps in the original post.

 

The final steps to play an mp4 in tty:

  Reveal hidden contents

 

Also works with mplayer (more efficient)

  Reveal hidden contents

 

Next up, I try again with the latest armbian OS image

 

Please ask what information I can get from my working setup, and see what's the roadblock in the latest armbian

 

Update: I confirm that the ffmpeg-v4l2request package is not working for the latest armbian Armbian_community_24.11.0-trunk.351_Orangepizero3_bookworm_current_6.6.54_minimal

 

  Reveal hidden contents

 

mplayer does not work either

  Reveal hidden contents

 

I can only think of using apt to install an older mpv.deb (+dependencies), if starting from the newest Armbian OS...

 

Or forcing apt to never upgrade the mpv.deb and related deb dependencies, if I start with the older Armbian OS

 

As a temporary measure, only needed while ffpmeg gets the patches officially, and the Debian servers include that new version too, right?

 

Reference: https://unix.stackexchange.com/questions/242014/how-to-install-the-previous-version-of-a-deb-package-and-pin-it

 

Another thing to try is to compile an Armbian image (latest), with bleeding edge linux kernel, as I see success for other people in Linux 6.7

https://forum.armbian.com/topic/29794-how-to-install-armbian-in-h618/?do=findComment&comment=204697

 

Jock, if I want to build my own ffmpeg with v4l2-request, is this a good recommendation?

"You also need to build with --enable-v4l2-request --enable-libudev"

https://artemis.sh/2023/03/06/ffmpeg-v4l2-requests-4.4.3.html

0
HiperDoo

HiperDoo

Posted
Posted

Thank you very much for your reply, but I can't find the old images of the Orange Pi Zero 3 anywhere. I found this archive link, but it doesn't have the OPi Zero 3, do you know where I can get the version of the image you mentioned?

0
robertoj

robertoj

Posted
Posted

Fortunately, I was able to upload this file:

Armbian_community_24.8.0-trunk.6_Orangepizero3_bookworm_current_6.6.31_minimal.img.xz

https://drive.google.com/file/d/1nlvUPLOdIz706EWWv4c2BzPQxkzilZHG/view?usp=drive_link

 

If anyone has a similar file for the orange pi zero 2w, please share it too

 

I found that mpv.deb remains the same version from armbian 24.8 to 24.11 (from may to october 2024)... and the only thing that got upgraded is linux 6.6.31 to 6.6.54

Freezing the kernel in 6.6.31 could be a good move too

 

I also just finished compiling armbian with the latest kernel... I will try this next time I can.

0
robertoj

robertoj

Posted
Posted

If you start with the old image with linux 6.6.31 and:

  Reveal hidden contents

It will update everything except the kernel (remains in 6.6.31)

 

And mpv, mplayer continue working with h264 hardware acceleration :)

 

 

When trying the newest kernel version 6.10.9, it didn't work for me:

  Reveal hidden contents

mplayer didn't work either

 

I will try compiling and installing ffmpeg from source...

  Reveal hidden contents

 

0
jock

jock

Posted
  • Author
Posted

I just had the chance to test an old Allwinner H3 (OrangePi One) with kernel 6.6.44 and can confirm that mpv fails to work correctly with hardware decoding, both via terminal and also in weston, both on kernel 6.6.44 and 6.10.8.

On rockchip64 instead it works pretty well in both terminal and weston on kernel 6.6.57.

 

I guess something broke in the kernel for allwinner platforms 😕

0
jock

jock

Posted
  • Author
Posted

@robertoj well a 240p video is easy enough even for H3 without hardware decoding. The repository is intended for hardware decoding, and my tests were done with 720p30 h.264 and 1080p30 h.265 videos

0
robertoj

robertoj

Posted
Posted

I will test again with my banana pi m2 zero (my only H3 SBC with HDMI)...

 

I am currently trying to build ffmpeg, but there's always a compilation error somewhere 🤣

0
robertoj

robertoj

Posted
Posted

You are right jock

 

I tried my bananapi m2 zero, playing a 240p H264 mp4 with the HDMI output.

 

With mpv, the rendering is scaled full screen, at 3 fps and the cpu is 100%

With mplayer, the 240p shows as 240p, 24fps (1/4 the height of my LCD) and the cpu is 25%

with mplayer, a 360p video plays also, with 35% cpu

720p and 1080p videos are played very choppy with mpv and mplayer

 

It turns out i was never using hardware acceleration in my H3 boards

 

Next I will try with the orange pi zero 3... with kernel 6.6.31, where I could play in my 240p LCD, but in my HDMI LCD this time

...I can play 1080p video smoothly in the tty, but the cpu is at 100% in 1 core

 

 

I finished compiling ffmpeg-v4l2request from https://github.com/Kwiboo/FFmpeg in my orangepi zero 3, under 6.10.9, but still could not play to my 240p LCD

[can someone tell me how to activate HDMI in the orange pi zero 3, in an arbian OS which I built myself?]

0
jock

jock

Posted
  • Author
Posted

@robertoj I may guess there are some missing pieces in the armbian kernel for allwinner: for rockchip, there are some patches borrowed from libreelec to make it work correctly with all formats. I don't know what is the status for allwinner and if there are similar patches and fixes that are not include right now, you won't be able to get anything.

 

ffmpeg with v4l2request patches applied should work with any device which has a working v4l2request compliant drivers in the kernel, plus it also requires a properly working presentation framework for the DRM/DRMPrime and EGL parts, so there are several players involved: ffmpeg has to talk via v4l2-request to the kernel, the hardware decoding happens in the V4L2 drivers, but the presentation on screen happens within the kernel (DRM) and/or Mesa (EGL/OpenGL), bridged by DRMPRIME buffer sharing.

 

I guess Mesa is pretty ok, since both Lima and Panfrost have the necessary bits in place to present things on screen, yet the Cedrus hardware decoder driver has to properly support hardware decoding and buffer sharing via drmprime with the GPU driver. I made some simplifications here and omitted details, but as you see it is already a fairly complex communication setup, of which ffmpeg is just the "user" of all those other services.

0
robertoj

robertoj

Posted
Posted

The kernel fix in NickA’s Armbian, allows ffmpeg-v4l2 play 1080p with hardware acceleration, under X11 (openbox wm), with “mpv filename.mp4”

 

I git-cloned his repo, selected orangepizero3, changed nothing, kept all his repo’s patches, chose bookworm minimal, Linux edge 6.7.12

 

i installed lightdm and xorg, then jock’s package repo for ffmpeg-v4l2requedt, then installed mpv (official bookworm package).

 

the 720p and 1080p videos played fine, with 5% cpu

 

:)

 

It is weird for me

 

update: to play in a screen not being used by x11, like a LCD TFT, use ctrl-alt-f2 to switch the keyboard into the tty in the LCD

 

Play any low res video with vlc, then quit it (I see that mpv can't play without this previous step, for me)

Play with mpv --vo=drm --video-connector=2.SPI-1 yourvideo.mp4

 

many thanks to both!!

0
joeculler

joeculler

Posted
Posted

@jock Hi, thanks for your work about ffmpeg-v4l2request.  Your ffmpeg-v4l2request is 5.1.4, Debian bookworm's ffmpeg now is 5.1.6.
Would you like to share your patches about v4l2request support for ffmpeg so I could build 5.1.6 myself, thank you.

 

0
robertoj

robertoj

Posted
Posted

Jock, Doug,

Do I only need this 1 patch file?

https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-request/ffmpeg-001-v4l2-request.patch

 

Is ffmpeg 5.1.6 recommended?

I will follow these straightforward compilation guides:

  Reveal hidden contents

 

0
David Kibble

David Kibble

Posted
Posted

Hi All - really hoping someone here can help me move forward in the right direction. I've been using Allwinner A13 hardware (Olimex SOM) for some years for small HMI applications without issue. Recently I've had the opportunity to explore a request to incorporate h264 video. I'm very aware of the A13's single core limits and therefore have been trying to get the VPU working. I don't use any sort of window manager (X11 / Wayland etc) so this DRM only. My current setup is:

 

Olimex A13 SOM with 512MB RAM

Fanscoo F43TAT007 touch LCD (RGB 666 interface)

Kernel 6.2 with a few mods for the target LCD and all the necessary updates to the DT to get both the mali 400 GPU and the CEDRUS driver working

Debian Bookworm

 

Here are the relevant kernel modules loaded:
lima                   57344  0
sun4i_backend          20480  0
gpu_sched              32768  1 lima
drm_shmem_helper       20480  1 lima
sun4i_ss               32768  0
libdes                 28672  1 sun4i_ss
sunxi_cedrus           45056  0
v4l2_mem2mem           24576  1 sunxi_cedrus
videobuf2_dma_contig    24576  1 sunxi_cedrus
videobuf2_memops       20480  1 videobuf2_dma_contig
videobuf2_v4l2         24576  2 sunxi_cedrus,v4l2_mem2mem
videodev              200704  3 sunxi_cedrus,v4l2_mem2mem,videobuf2_v4l2
videobuf2_common       49152  5 sunxi_cedrus,videobuf2_dma_contig,videobuf2_memops,v4l2_mem2mem,videobuf2_v4l2
mc                     49152  5 sunxi_cedrus,videobuf2_common,videodev,v4l2_mem2mem,videobuf2_v4l2
sun4i_drm              20480  0
sun4i_frontend         20480  2 sun4i_drm,sun4i_backend
drm_dma_helper         16384  3 sun4i_drm,sun4i_backend,sun4i_frontend
panel_simple           69632  0
sun4i_tcon             32768  1 sun4i_drm
pwm_bl                 24576  0
sun8i_tcon_top         16384  2 sun4i_drm,sun4i_tcon

 

Initially I tried the cedrus v4l2-request VAAPI library route, but post kernel 5.9 that's pretty broken and looked like a lot of work. I then went through a process of getting up to kernel 6.2 so that I could try out gstreamer that claimed to have native v4l2 backend capability. That DOES play, but the video is very corrupt. Then I tried mpv which worked, but without h/w decoding so I began looking into compiling my own ffmpeg with v4l2 hardware decoding enabled. While digging into that I found this page!

 

I can play a h264 RTSP video stream ok at 640x480 with s/w decoding which seems to prove the main parts of the video pipeline. I can see v4l2 drivers etc present in the kernel via lsmod along with the sunxi display frontend and backend etc. But - when I try to use your modified mpv with hwdec=drm the hardware decoder is detected, but then I get errors. Here's a bit of the mpv gpu trace log:

vo/gpu/opengl] Initializing KMS
[vo/gpu/opengl] Picked DRM card 0, primary node /dev/dri/card1 as the default.
[vo/gpu/opengl] Driver: sun4i-drm 1.0.0 (20150629)
[vo/gpu/opengl/kms] Connector 49 currently connected to encoder 48
[vo/gpu/opengl/kms] Selected Encoder 48 with CRTC 47
[vo/gpu/opengl/kms] Selected mode: 480x800 (480x800@60.01Hz)
[vo/gpu/opengl] DRM Atomic support found
[vo/gpu/opengl/kms] Using overlay plane 35 as draw plane
[vo/gpu/opengl/kms] Using primary plane 31 as drmprime plane
[vo/gpu] GBM_FORMAT_ARGB8888 supported by draw plane.
[vo/gpu] Supported modifier: 0x0
[vo/gpu] Creating GBM device
[vo/gpu] Initializing GBM surface (480 x 800)
[vo/gpu/opengl] Initializing EGL
[vo/gpu/opengl] EGL_VERSION=1.4
[vo/gpu/opengl] EGL_VENDOR=Mesa Project
[vo/gpu/opengl] EGL_CLIENT_APIS=OpenGL OpenGL_ES 
[vo/gpu/opengl] Trying to create Desktop OpenGL context.

....

[vo/gpu/opengl] Initializing EGL surface
[vo/gpu] GL_VERSION='2.1 Mesa 22.3.6'
[vo/gpu] Detected desktop OpenGL 2.1.
[vo/gpu] GL_VENDOR='lima'
[vo/gpu] GL_RENDERER='Mali400'
[vo/gpu] GL_SHADING_LANGUAGE_VERSION='1.20'
...

[vo/gpu] Loaded extension GL_ARB_map_buffer_range.
[vo/gpu] Loaded extension GL_ARB_vertex_array_object.
[vo/gpu] Loaded extension GL_ARB_texture_rg.
[vo/gpu] Loaded extension GL_ARB_sync.
[vo/gpu] Loaded extension GL_ARB_invalidate_subdata.
[vo/gpu] Loaded extension GL_ARB_get_program_binary.
[vo/gpu] Loaded extension GL_ARB_buffer_storage.
[vo/gpu] Loaded extension GL_ARB_debug_output.
[vo/gpu] Loaded extension GL_KHR_debug.
[vo/gpu/opengl] Preparing framebuffer
[vo/gpu/opengl] GBM surface using modifier 0x0
[vo/gpu] crtc is not VRR capable
[vo/gpu/opengl] Could not find path to render node. VAAPI hwdec will be disabled
[vo/gpu] Monitor pixel aspect: 1

...

[vo/gpu] No advanced processing required. Enabling dumb mode.
[vo/gpu] Assuming 60.006001 FPS for display sync.
[vo/gpu] Loading hwdec drivers for format: 'drm_prime'
[vo/gpu] Loading hwdec driver 'drmprime-overlay'
[vo/gpu/drmprime-overlay] Using overlay plane 35 as draw plane
[vo/gpu/drmprime-overlay] Using primary plane 31 as drmprime plane
[vo/gpu] Loading hwdec driver 'drmprime'
[vo/gpu/drmprime] using EGL dmabuf interop
[vo/gpu/drmprime] Using DRM device: /dev/dri/renderD128
Using hardware decoding (drm).
VO: [gpu] 480x270 drm_prime[nv12]
[vo/gpu] reconfig to 480x270 drm_prime[nv12] bt.601/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264 rot=270
[vo/gpu] Resize: 480x800
[vo/gpu] Window size: 480x800 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 480x270 (1:1)
[vo/gpu] Video display: (0, 0) 270x480 -> (15, 0) 450x800
[vo/gpu] Video scale: 1.666667/1.666667
[vo/gpu] OSD borders: l=15 t=0 r=15 b=0
[vo/gpu] Video borders: l=15 t=0 r=15 b=0
[vo/gpu] Using HW-overlay mode. No GL filtering is performed on the video!
[vo/gpu] No advanced processing required. Enabling dumb mode.
[vo/gpu/drmprime-overlay] Failed to create framebuffer with drmModeAddFB2 on layer 0: Invalid argument

 

I've highlighted the two obvious errors. The first one about the render node is odd as I do have a /dev/dri/renderD128 device

 

I have enabled DRM debug via kernel cmd line, but nothing new showed up there, just a -22 for the drmModeAddFB2 call.

 

I'm more than happy to go away and build / test / investigate, but this is now beyond my current limits of knowledge, so hoped someone here would be able to point me in the right direction?

 

Many thanks

0
David Kibble

David Kibble

Posted
Posted

Thanks robertoj - I'll have a look at Armbian and give it a shot. However - the slight problem here is the that our HMI apps are all built using a native DRM framework, (no X11) and they drive an RGB LCD, so the X11/HDMI solution isn't going to work for us. I think the modified MPV is very close, it looks to me like it's just the framebuffer output that failing rather than the h/w decode, which is a shame as the standard mpv outputs just fine to the DRM plane.

 

Thanks for the advice, will give it a go, but maybe will also have a look at the modified ffmpeg. I'm still a bit confused about one thing though, maybe you can help? Does the modified ffmpeg (and mpv) drive the kernel v4l2-m2m driver directly or does it still rely on the userspace v4l2-request layer?

 

Many thanks

0
jock

jock

Posted
  • Author
Posted

Just a note: there is no modified mpv. Ubuntu Jammy just requires an updated version which my repository provides. Debian bookworm works fine with the packaged version it comes with.

 

0
David Kibble

David Kibble

Posted
Posted

Thanks Jock - so it's just a modified ffmpeg you've created? Just trying to get my head around what I need to tackle here to make this work. Feels so close and getting h264 decode on the A13 with mainline would I'm sure be v-useful for lots of folks out there with the old hardware.

 

Thanks again for a great forum!

0

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept