Jump to content

Hardware Graphic/Video Acceleration in H3 Mainline


ATK

Recommended Posts

Kodi with KMS/DRM/GBM backend only

Tried MPV, but it didn't seem to work . Looks like MPV is not supported http://linux-sunxi.org/Sunxi-Cedrus#Player_Support

we're tweaking the sources and scripts to release a new RetrOrangePi version , so hopefully a working tutorial will be available soon

Link to comment
Share on other sites

@Alerino Reis If you're using ffmpeg patches from LibreELEC, then you need only this additional patch to make it compatible with mpv. I tested yesterday and it works for me when running without any window manager running with either of these commands:

mpv --vo=gpu --gpu-context=drm --hwdec=auto video.mkv
mpv --vo=drm --hwdec=auto video.mkv

You can append "-v" parameter to check if mpv really uses HW decoding.

Link to comment
Share on other sites

Took some time to resume working on it...

 

At the moment, i'm only able to get Kodi working fine with hw decoding.

MPV and VLC are a no-go.

 

MPV log

 

Spoiler

pi@RetrOrangePi:~$ mpv -v --vo=gpu --gpu-context=drm --gpu-hwdec-interop=drmprime-drm --hwdec=auto Videos/BigBuckBunny-1080p.mp4 
[cplayer] Command line options: '-v' '--vo=gpu' '--gpu-context=drm' '--gpu-hwdec-interop=drmprime-drm' '--hwdec=auto' 'Videos/BigBuckBunny-1080p.mp4'
[cplayer] mpv 0.32.0-328-gc5f8ec76b1-dirty Copyright © 2000-2020 mpv/MPlayer/mplayer2 projects
[cplayer]  built on Wed Apr  8 18:24:45 UTC 2020
[cplayer] FFmpeg library versions:
[cplayer]    libavutil       56.14.100
[cplayer]    libavcodec      58.18.100
[cplayer]    libavformat     58.12.100
[cplayer]    libswscale      5.1.100
[cplayer]    libavfilter     7.16.100
[cplayer]    libswresample   3.1.100
[cplayer] FFmpeg version: 4.0.4-Kodi
[cplayer] 
[cplayer] Configuration: ./waf configure
[cplayer] List of enabled features: aligned_alloc alsa asm build-date caca cplayer cplugins debug-build drm egl egl-drm egl-helpers egl-x11 ffmpeg gbm gbm.h gl gl-x11 glibc-thread-name glob glob-posix gpl iconv jpeg libass libavdevice libbluray libdl libm librt linux-fstatfs memfd_create optimize posix posix-or-mingw posix-spawn posix-spawn-native pthreads pulse stdatomic vaapi vaapi-drm vaapi-egl vaapi-x-egl vaapi-x11 vdpau vdpau-gl-x11 vt.h x11 xv zlib
[cplayer] Reading config file /usr/local/etc/mpv/encoding-profiles.conf
[cplayer] Applying profile 'default'...
[cplayer] Setting option 'v' = '' (flags = 8)
[cplayer] Setting option 'vo' = 'gpu' (flags = 8)
[cplayer] Setting option 'gpu-context' = 'drm' (flags = 8)
[cplayer] Setting option 'gpu-hwdec-interop' = 'drmprime-drm' (flags = 8)
[cplayer] Setting option 'hwdec' = 'auto' (flags = 8)
[bdmv/bluray] Opening Videos/BigBuckBunny-1080p.mp4
[file] Opening Videos/BigBuckBunny-1080p.mp4
[demux] Trying demuxers for level=normal.
[lavf] Found 'mov,mp4,m4a,3gp,3g2,mj2' at score=100 size=2048.
[file] stream level seek from 131072 to 22698752
[demux] Detected file format: mov,mp4,m4a,3gp,3g2,mj2 (libavformat)
[cplayer] Opening done: Videos/BigBuckBunny-1080p.mp4
[find_files] Loading external files in Videos/
[lavf] select track 0
[lavf] select track 1
[cplayer]  (+) Video --vid=1 (*) (h264 1920x1080 24.000fps)
[cplayer]  (+) Audio --aid=1 (*) (aac 6ch 48000Hz)
[vo/gpu/opengl] Initializing GPU context 'drm'
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
[vo/gpu/opengl] Initializing KMS
[vo/gpu/opengl] Device name: /dev/dri/card0
[vo/gpu/opengl] Driver: sun4i-drm 1.0.0 (20150629)
[vo/gpu/opengl/kms] Connector 43 currently connected to encoder 42
[vo/gpu/opengl/kms] Selected Encoder 42 with CRTC 41
[vo/gpu/opengl/kms] Selected mode: 1920x1080 (1920x1080@60.00Hz)
[vo/gpu/opengl] DRM Atomic support found
[vo/gpu/opengl/kms] Using primary plane 35 as draw plane
[vo/gpu/opengl/kms] Using overlay plane 31 as drmprime plane
[vo/gpu] GBM_FORMAT_ARGB8888 supported by draw plane.
[vo/gpu] Creating GBM device
[vo/gpu] Initializing GBM surface (1920 x 1080)
[vo/gpu/opengl] Initializing EGL
[vo/gpu/opengl] EGL_VERSION=1.4 Linux-r6p2-01rel0
[vo/gpu/opengl] EGL_VENDOR=ARM
[vo/gpu/opengl] EGL_CLIENT_APIS=OpenGL_ES
[vo/gpu/opengl] Trying to create Desktop OpenGL context.
[vo/gpu/opengl] Could not bind API!
[vo/gpu/opengl] Trying to create GLES 3.x context.
[vo/gpu/opengl] Could not choose EGLConfig for GLES 3.x!
[vo/gpu/opengl] Trying to create GLES 2.x context.
[vo/gpu/opengl] Attempting to find EGLConfig matching GBM_FORMAT_ARGB8888
[vo/gpu/opengl] No matching EGLConfig for GBM_FORMAT_ARGB8888
[vo/gpu/opengl] Attempting to find EGLConfig matching GBM_FORMAT_XRGB8888
[vo/gpu/opengl] Found matching EGLConfig for GBM_FORMAT_XRGB8888
[vo/gpu/opengl] Initializing EGL surface
[vo/gpu] GL_VERSION='OpenGL ES 2.0'
[vo/gpu] Detected GLES 2.0.
[vo/gpu] GL_VENDOR='ARM'
[vo/gpu] GL_RENDERER='Mali-400 MP'
[vo/gpu] GL_SHADING_LANGUAGE_VERSION='OpenGL ES GLSL ES 1.00'
[vo/gpu/opengl] Preparing framebuffer
[vo/gpu/opengl] Could not find path to render node. VAAPI hwdec will be disabled
[vo/gpu/opengl] 16 bit UNORM textures not available.
[vo/gpu] Disabling HDR peak computation (one or more of the following is not supported: compute shaders=0, SSBO=0).
[vo/gpu] No advanced processing required. Enabling dumb mode.
[vo/gpu] Loading hwdec driver 'drmprime-drm'
[vo/gpu/drmprime-drm] Using primary plane 35 as draw plane
[vo/gpu/drmprime-drm] Using overlay plane 31 as drmprime plane
[vo/gpu] Assuming 60.000000 FPS for display sync.
[decode_wrapper/vd] Container reported FPS: 24.000000
[decode_wrapper/vd] Codec list:
[decode_wrapper/vd]     h264 - H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
[decode_wrapper/vd]     h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper
[decode_wrapper/vd] Opening decoder h264
[decode_wrapper/vd] Looking at hwdec h264-vaapi...
[decode_wrapper/vd] Could not create device.
[decode_wrapper/vd] Looking at hwdec h264-vaapi-copy...
[decode_wrapper/vd] Could not create device.
[decode_wrapper/vd] Looking at hwdec h264-drm...
[decode_wrapper/vd] Trying hardware decoding via h264-drm.
[decode_wrapper/vd] Selected codec: h264 (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)
[vf] User filter list:
[vf]   (empty)
[decode_wrapper/ad] Codec list:
[decode_wrapper/ad]     aac - AAC (Advanced Audio Coding)
[decode_wrapper/ad]     aac_fixed (aac) - AAC (Advanced Audio Coding)
[decode_wrapper/ad] Opening decoder aac
[decode_wrapper/ad] Requesting 1 threads for decoding.
[decode_wrapper/ad] Selected codec: aac (AAC (Advanced Audio Coding))
[af] User filter list:
[af]   (empty)
[cplayer] Starting playback...
[file] stream level seek from 22718509 to 48
[decode_wrapper/vd] Pixel formats supported by decoder: vaapi_vld drm_prime yuv420p
[decode_wrapper/vd] Codec profile: High (0x64)
[decode_wrapper/vd] Requesting pixfmt 'drm_prime' from decoder.
[af] [in] 48000Hz 5.1 6ch floatp
[af] [userspeed] 48000Hz 5.1 6ch floatp
[af] [userspeed] (disabled)
[af] [convert] 48000Hz 5.1 6ch floatp
[decode_wrapper/vd] Using hardware decoding (drm).
[decode_wrapper/vd] Decoder format: 1920x1080 drm_prime[nv12] auto/auto/auto/auto/auto CL=mpeg2/4/h264
[vf] [in] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [userdeint] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [userdeint] (disabled)
[vf] [autorotate] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [autorotate] (disabled)
[vf] [convert] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [convert] (disabled)
[vf] [out] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[ao] Trying audio driver 'pulse'
[ao/pulse] requested format: 48000 Hz, 5.1 channels, floatp
[ao/pulse] Library version: 11.1.0
[ao/pulse] Proto: 32
[ao/pulse] Server proto: 4294967295
[ao] Trying audio driver 'alsa'
[ao/alsa] requested format: 48000 Hz, 5.1 channels, floatp
[ao/alsa] using ALSA version: 1.1.3
[ao/alsa] opening device 'default'
[ao/alsa] trying format float/14
[ao/alsa] Channel layouts:
[ao/alsa]  - stereo
[ao/alsa] result: stereo
[ao/alsa] channel map reported by ALSA: FL FR
[ao/alsa] which we understand as: stereo
[ao/alsa] which is what we requested.
[ao/alsa] hw pausing supported: no
[ao/alsa] buffersize: 4096 samples
[ao/alsa] period size: 1024 samples
[ao/alsa] device buffer: 4096 samples.
[ao/alsa] using soft-buffer of 8820 samples.
[cplayer] AO: [alsa] 44100Hz stereo 2ch float
[cplayer] AO: Description: ALSA audio output
[autoconvert] inserting resampler
[swresample] format change, reinitializing resampler
[swresample] 48000Hz 5.1 floatp -> 44100Hz stereo float
[swresample] Remix: 5.1 -> stereo
[af] [out] 44100Hz stereo 2ch float
[cplayer] VO: [gpu] 1920x1080 drm_prime[nv12]
[cplayer] VO: Description: Shader-based GPU Renderer
[vo/gpu] reconfig to 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vo/gpu] Resize: 1920x1080
[vo/gpu] Window size: 1920x1080 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1920x1080 (1:1)
[vo/gpu] Video display: (0, 0) 1920x1080 -> (0, 0) 1920x1080
[vo/gpu] Video scale: 1.000000/1.000000
[vo/gpu] OSD borders: l=0 t=0 r=0 b=0
[vo/gpu] Video borders: l=0 t=0 r=0 b=0
[vo/gpu] Using HW-overlay mode. No GL filtering is performed on the video!
[vo/gpu] No advanced processing required. Enabling dumb mode.
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[cplayer] first video frame after restart shown
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[cplayer] audio sync: sync_to_video=1, offset=0.000000
[cplayer] playback restart complete @ 0.000000, audio=playing, video=playing
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V: -0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V: -0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V: -0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[cplayer] EOF code: 5  
[cplayer] finished playback, success (reason 3)
[cplayer] 
[cplayer] Exiting... (Quit)
[ao/alsa] draining...
 

 

Link to comment
Share on other sites

Huh..it seams that it is not so simple for everyone to reach this poing to have a HW acceleration on H3.

My goal is to reach transcoding from h264 full HD as input stream (satillite stream from enigma2 receivers) and lower down the resolution to SD in h264 with FFmpeg.

I compiled myself the ffmpeg with h264 and libfdk-aac.

All this working even without HW but from SD to SD.

Is there some detail tutorial how can I reach this point?

Thx!

Link to comment
Share on other sites

Trying to add VA-API into actual Armbian Focal (Linux orangepipc 5.4.45-sunxi #20.05.4)

 

Install instruction

Spoiler

$ sudo apt-get install libtool pkg-config vainfo libva2 libva-x11-2 libva-drm2 libva-utils libva-dev libdrm-dev

$ git clone https://github.com/bootlin/libva-v4l2-request

$ cd libva-v4l2-request

$ ./autogen.sh

$ make

$ sudo make install

 

Next, export driver:

$ export LIBVA_DRIVER_NAME=v4l2_request

and check it with vainfo tool. Have an error:

$ vainfo 
libva info: VA-API version 1.7.0
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
libva info: va_openDriver() returns -1
vaInitialize failed with error code -1 (unknown libva error),exit

 

To fix, add symbolic link to installed driver:

$ sudo ln -s /usr/lib/dri/v4l2_request_drv_video.so /usr/lib/arm-linux-gnueabihf/dri/

 

And seems everything is ok

$ vainfo 
libva info: VA-API version 1.7.0
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.7 (libva 2.6.0)
vainfo: Driver version: v4l2-request
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264MultiviewHigh      :	VAEntrypointVLD
      VAProfileH264StereoHigh         :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointVLD

 

Edited by Sash0k
fixed by myself
Link to comment
Share on other sites

4 hours ago, Dreyk said:

@Sash0k, it looks interesting.
What about the real video playback in video players (like mpv, vlc, etc...)?

 

Unfortunately, no success.

 

First, mainline bootlin backend is outdated, h264 broken. It has some pull requests, that not merged.

Tryng to recompile Philipp Zabel's version, and merge his work for h264

Spoiler

$ git clone https://github.com/pH5/libva-v4l2-request.git
$ cd libva-v4l2-request
$ git merge origin/h264
$ ./autogen.sh
$ make
$ sudo make install

 

 

This version works with artifacts in VLC:

U24j7fV.png

 

Link to comment
Share on other sites

Finally got it! No kernel modifications needed, only v4l2-request.


Key notes:

  • Use bootlin code, latest master (not release-2019.03 tag)
  • I merged just one small patch from https://github.com/bootlin/libva-v4l2-request/pull/30/files (seems, it's unecessary)
  • Download kernel sources with corresponding version. For my armbian is: https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.4.45.tar.xz
  • Extract 2 files from kernel/include/media mpeg2-ctrls.h and h264-ctrls.h and replace ones in v4l2-request
  • Replace V4L2_PIX_FMT_H264_SLICE_RAW to V4L2_PIX_FMT_H264_SLICE in v4l2-request source code
  • Compile and install (instruction is as 2 posts above)
  • Don't forget to set VLC as in https://linux-sunxi.org/Sunxi-Cedrus
    Spoiler

    Tools > Preferences > Input / Codecs > Codecs > Hardware-accelerated decoding > VA-API video decoder

    Tools > Preferences > Video > Display > Output > X11 video output (XCB)

 

Tested with VLC, usable with issues:

  • Artifacts in some videos h264 720p and higher, for example: https://imgur.com/nYFArT4 (360/480 works fine)
  • Scaling (fullscreen, resizing) not works, slowdown with message
    [a310cb88] main filter error: Failed to create video converter
  • Minor issues in console output on playback (see bold)
    Spoiler

    $ vlc 3-big_buck_bunny_480p_H264_AAC_25fps_1800K.MP4
    VLC media player 3.0.9.2 Vetinari (revision 3.0.9.2-0-gd4c1aefe4d)
    [02287b98] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
    libEGL warning: DRI2: failed to authenticate
    libva info: VA-API version 1.7.0
    libva info: User environment variable requested driver 'v4l2_request'
    libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
    libva info: Found init function __vaDriverInit_1_7
    libva info: va_openDriver() returns 0
    [a272d350] avcodec decoder: Using v4l2-request for hardware decoding
    [a3018b98] blend blend error: no matching alpha blending routine (chroma: YUVA -> VAOP)
    [a3018b98] main blend error: blending YUVA to VAOP failed

 

Thanks to: @jernej for this post:

 

Edited by Sash0k
thanks
Link to comment
Share on other sites

Well, kernel modification is needed for better decoding (less glitches). Probably new *-ctrl.h files contain new fields (not 100% sure) which also need to be filled.

 

There is no any post processing implemented here like scaling. It would be possible to do that via SoC specific peripherals but that wouldn't be universal and thus it's out of scope of this library.

 

Anyway, I'm glad you succeeded.

Link to comment
Share on other sites

Hi,

i'm interested in this topic, but since i use wayland i would like to get it working on mpv (for example). I got vaainfo detecting the library, but when launched using mpv i got green screen and a lot of "failed to destoy surface". I also have noticed that after a some try, i got an error about failed resource allocation (i imagine because it does not free correctly buffers). Someone have managed to get mpv working?

For the context, i'm not using Armbian device, but a Pinephone with PostmarketOS (it's the only Allwinner device i have), but i think a lot of people could benefit from get this driver working.

Link to comment
Share on other sites

Sadly noone maintain vaapi layer, so you're mostly on your own there, which means learning a ton of internal details of vaapi, v4l2 request api and supported codecs. On the other hand, ffmpeg fork with v4l2 request api is pretty well maintained. I know this is not ideal, but for example mpv (with one patch) can work with that modified ffmpeg.

Link to comment
Share on other sites

Hi guys, i did what you say above but still doesnt work on both vlc and ffmpeg running on Orangepi PC.

 

I am using armbian latest stable version : Linux orangepipc 5.10.60-sunxi #21.08.1 SMP Wed Aug 25 18:19:32 UTC 2021 armv7l armv7l armv7l GNU/Linux

 

Umit

Link to comment
Share on other sites

1 hour ago, umtkyck said:

I am using armbian latest stable version : Linux orangepipc 5.10.60-sunxi #21.08.1 SMP Wed Aug 25 18:19:32 UTC 2021 armv7l armv7l armv7l GNU/Linux

5.10 is ancient for video decoding purposes. Codecs started stabilizing after 5.11. With 5.14 you'll have stable and fully working MPEG2, VP8 and H264. If you need low effort solution, build latest gstreamer 1.20 RC with gst-plugins-bad with one of the latest kernels. With proper pipeline, it works. If you need out of the box solution, use LibreELEC distro.

Link to comment
Share on other sites

13 hours ago, umtkyck said:

how can i find armbian kernel 5.14 for orangepi pc?

There is no 5.14 kernel available in Armbian, I talked about minimum version. In general, codecs are improved with (almost) every release, so latest version is best. I only see images with kernel 5.13 on download page, which should in theory work. You can build your own Armbian image with kernel 5.15 (edge).

Link to comment
Share on other sites

Jernej, what if i want to make backporting the 5.14 codecs to 5.4 release coming with openwrt for orange pi pc. Is that possible, what do i need to take care doing this operation.  Just taking the cedrus drivers will not help generally, right?

 

I wanna port the working librelec libraries to latest openwrt stable distro. 

 

Umit

Link to comment
Share on other sites

On 2/5/2022 at 4:49 PM, umtkyck said:

Just taking the cedrus drivers will not help generally, right?

You'll have to also port headers and maybe more. This is generally not something I'd spend time on. It should be easier to update Linux package on openwrt.

Link to comment
Share on other sites

@jernej my main approach is backport drivers to kernel 4.19, due to the limitations they dont want to change it. What if i use ubarobov's images , i guess he used 4.20 for the tests. Also bootlin guys also says 4.20 is working for their toolchains.

 

Do you see any risk on that? Can i run hw decode on kernel 4.19?

 

Umit

Link to comment
Share on other sites

Kernel 4.19 was one of the first which has anything of request API framework IIRC. Task of porting mainlined patches there will be even harder. You can use WIP patches from Bootlin and accompanying libs, but then expect that many videos won't work, especially interlaced H264. Also gstreamer or patched ffmpeg won't be compatible with that API. That's even more futile IMO and don't expect any help from me.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines