Hardware Graphic/Video Acceleration in H3 Mainline


ATK
 Share

12 12

Recommended Posts

Donate and support the project!

@Alerino Reis If you're using ffmpeg patches from LibreELEC, then you need only this additional patch to make it compatible with mpv. I tested yesterday and it works for me when running without any window manager running with either of these commands:

mpv --vo=gpu --gpu-context=drm --hwdec=auto video.mkv
mpv --vo=drm --hwdec=auto video.mkv

You can append "-v" parameter to check if mpv really uses HW decoding.

Link to post
Share on other sites

Took some time to resume working on it...

 

At the moment, i'm only able to get Kodi working fine with hw decoding.

MPV and VLC are a no-go.

 

MPV log

 

Spoiler

pi@RetrOrangePi:~$ mpv -v --vo=gpu --gpu-context=drm --gpu-hwdec-interop=drmprime-drm --hwdec=auto Videos/BigBuckBunny-1080p.mp4 
[cplayer] Command line options: '-v' '--vo=gpu' '--gpu-context=drm' '--gpu-hwdec-interop=drmprime-drm' '--hwdec=auto' 'Videos/BigBuckBunny-1080p.mp4'
[cplayer] mpv 0.32.0-328-gc5f8ec76b1-dirty Copyright © 2000-2020 mpv/MPlayer/mplayer2 projects
[cplayer]  built on Wed Apr  8 18:24:45 UTC 2020
[cplayer] FFmpeg library versions:
[cplayer]    libavutil       56.14.100
[cplayer]    libavcodec      58.18.100
[cplayer]    libavformat     58.12.100
[cplayer]    libswscale      5.1.100
[cplayer]    libavfilter     7.16.100
[cplayer]    libswresample   3.1.100
[cplayer] FFmpeg version: 4.0.4-Kodi
[cplayer] 
[cplayer] Configuration: ./waf configure
[cplayer] List of enabled features: aligned_alloc alsa asm build-date caca cplayer cplugins debug-build drm egl egl-drm egl-helpers egl-x11 ffmpeg gbm gbm.h gl gl-x11 glibc-thread-name glob glob-posix gpl iconv jpeg libass libavdevice libbluray libdl libm librt linux-fstatfs memfd_create optimize posix posix-or-mingw posix-spawn posix-spawn-native pthreads pulse stdatomic vaapi vaapi-drm vaapi-egl vaapi-x-egl vaapi-x11 vdpau vdpau-gl-x11 vt.h x11 xv zlib
[cplayer] Reading config file /usr/local/etc/mpv/encoding-profiles.conf
[cplayer] Applying profile 'default'...
[cplayer] Setting option 'v' = '' (flags = 8)
[cplayer] Setting option 'vo' = 'gpu' (flags = 8)
[cplayer] Setting option 'gpu-context' = 'drm' (flags = 8)
[cplayer] Setting option 'gpu-hwdec-interop' = 'drmprime-drm' (flags = 8)
[cplayer] Setting option 'hwdec' = 'auto' (flags = 8)
[bdmv/bluray] Opening Videos/BigBuckBunny-1080p.mp4
[file] Opening Videos/BigBuckBunny-1080p.mp4
[demux] Trying demuxers for level=normal.
[lavf] Found 'mov,mp4,m4a,3gp,3g2,mj2' at score=100 size=2048.
[file] stream level seek from 131072 to 22698752
[demux] Detected file format: mov,mp4,m4a,3gp,3g2,mj2 (libavformat)
[cplayer] Opening done: Videos/BigBuckBunny-1080p.mp4
[find_files] Loading external files in Videos/
[lavf] select track 0
[lavf] select track 1
[cplayer]  (+) Video --vid=1 (*) (h264 1920x1080 24.000fps)
[cplayer]  (+) Audio --aid=1 (*) (aac 6ch 48000Hz)
[vo/gpu/opengl] Initializing GPU context 'drm'
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
[vo/gpu/opengl] Initializing KMS
[vo/gpu/opengl] Device name: /dev/dri/card0
[vo/gpu/opengl] Driver: sun4i-drm 1.0.0 (20150629)
[vo/gpu/opengl/kms] Connector 43 currently connected to encoder 42
[vo/gpu/opengl/kms] Selected Encoder 42 with CRTC 41
[vo/gpu/opengl/kms] Selected mode: 1920x1080 (1920x1080@60.00Hz)
[vo/gpu/opengl] DRM Atomic support found
[vo/gpu/opengl/kms] Using primary plane 35 as draw plane
[vo/gpu/opengl/kms] Using overlay plane 31 as drmprime plane
[vo/gpu] GBM_FORMAT_ARGB8888 supported by draw plane.
[vo/gpu] Creating GBM device
[vo/gpu] Initializing GBM surface (1920 x 1080)
[vo/gpu/opengl] Initializing EGL
[vo/gpu/opengl] EGL_VERSION=1.4 Linux-r6p2-01rel0
[vo/gpu/opengl] EGL_VENDOR=ARM
[vo/gpu/opengl] EGL_CLIENT_APIS=OpenGL_ES
[vo/gpu/opengl] Trying to create Desktop OpenGL context.
[vo/gpu/opengl] Could not bind API!
[vo/gpu/opengl] Trying to create GLES 3.x context.
[vo/gpu/opengl] Could not choose EGLConfig for GLES 3.x!
[vo/gpu/opengl] Trying to create GLES 2.x context.
[vo/gpu/opengl] Attempting to find EGLConfig matching GBM_FORMAT_ARGB8888
[vo/gpu/opengl] No matching EGLConfig for GBM_FORMAT_ARGB8888
[vo/gpu/opengl] Attempting to find EGLConfig matching GBM_FORMAT_XRGB8888
[vo/gpu/opengl] Found matching EGLConfig for GBM_FORMAT_XRGB8888
[vo/gpu/opengl] Initializing EGL surface
[vo/gpu] GL_VERSION='OpenGL ES 2.0'
[vo/gpu] Detected GLES 2.0.
[vo/gpu] GL_VENDOR='ARM'
[vo/gpu] GL_RENDERER='Mali-400 MP'
[vo/gpu] GL_SHADING_LANGUAGE_VERSION='OpenGL ES GLSL ES 1.00'
[vo/gpu/opengl] Preparing framebuffer
[vo/gpu/opengl] Could not find path to render node. VAAPI hwdec will be disabled
[vo/gpu/opengl] 16 bit UNORM textures not available.
[vo/gpu] Disabling HDR peak computation (one or more of the following is not supported: compute shaders=0, SSBO=0).
[vo/gpu] No advanced processing required. Enabling dumb mode.
[vo/gpu] Loading hwdec driver 'drmprime-drm'
[vo/gpu/drmprime-drm] Using primary plane 35 as draw plane
[vo/gpu/drmprime-drm] Using overlay plane 31 as drmprime plane
[vo/gpu] Assuming 60.000000 FPS for display sync.
[decode_wrapper/vd] Container reported FPS: 24.000000
[decode_wrapper/vd] Codec list:
[decode_wrapper/vd]     h264 - H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
[decode_wrapper/vd]     h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper
[decode_wrapper/vd] Opening decoder h264
[decode_wrapper/vd] Looking at hwdec h264-vaapi...
[decode_wrapper/vd] Could not create device.
[decode_wrapper/vd] Looking at hwdec h264-vaapi-copy...
[decode_wrapper/vd] Could not create device.
[decode_wrapper/vd] Looking at hwdec h264-drm...
[decode_wrapper/vd] Trying hardware decoding via h264-drm.
[decode_wrapper/vd] Selected codec: h264 (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)
[vf] User filter list:
[vf]   (empty)
[decode_wrapper/ad] Codec list:
[decode_wrapper/ad]     aac - AAC (Advanced Audio Coding)
[decode_wrapper/ad]     aac_fixed (aac) - AAC (Advanced Audio Coding)
[decode_wrapper/ad] Opening decoder aac
[decode_wrapper/ad] Requesting 1 threads for decoding.
[decode_wrapper/ad] Selected codec: aac (AAC (Advanced Audio Coding))
[af] User filter list:
[af]   (empty)
[cplayer] Starting playback...
[file] stream level seek from 22718509 to 48
[decode_wrapper/vd] Pixel formats supported by decoder: vaapi_vld drm_prime yuv420p
[decode_wrapper/vd] Codec profile: High (0x64)
[decode_wrapper/vd] Requesting pixfmt 'drm_prime' from decoder.
[af] [in] 48000Hz 5.1 6ch floatp
[af] [userspeed] 48000Hz 5.1 6ch floatp
[af] [userspeed] (disabled)
[af] [convert] 48000Hz 5.1 6ch floatp
[decode_wrapper/vd] Using hardware decoding (drm).
[decode_wrapper/vd] Decoder format: 1920x1080 drm_prime[nv12] auto/auto/auto/auto/auto CL=mpeg2/4/h264
[vf] [in] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [userdeint] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [userdeint] (disabled)
[vf] [autorotate] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [autorotate] (disabled)
[vf] [convert] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vf] [convert] (disabled)
[vf] [out] 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[ao] Trying audio driver 'pulse'
[ao/pulse] requested format: 48000 Hz, 5.1 channels, floatp
[ao/pulse] Library version: 11.1.0
[ao/pulse] Proto: 32
[ao/pulse] Server proto: 4294967295
[ao] Trying audio driver 'alsa'
[ao/alsa] requested format: 48000 Hz, 5.1 channels, floatp
[ao/alsa] using ALSA version: 1.1.3
[ao/alsa] opening device 'default'
[ao/alsa] trying format float/14
[ao/alsa] Channel layouts:
[ao/alsa]  - stereo
[ao/alsa] result: stereo
[ao/alsa] channel map reported by ALSA: FL FR
[ao/alsa] which we understand as: stereo
[ao/alsa] which is what we requested.
[ao/alsa] hw pausing supported: no
[ao/alsa] buffersize: 4096 samples
[ao/alsa] period size: 1024 samples
[ao/alsa] device buffer: 4096 samples.
[ao/alsa] using soft-buffer of 8820 samples.
[cplayer] AO: [alsa] 44100Hz stereo 2ch float
[cplayer] AO: Description: ALSA audio output
[autoconvert] inserting resampler
[swresample] format change, reinitializing resampler
[swresample] 48000Hz 5.1 floatp -> 44100Hz stereo float
[swresample] Remix: 5.1 -> stereo
[af] [out] 44100Hz stereo 2ch float
[cplayer] VO: [gpu] 1920x1080 drm_prime[nv12]
[cplayer] VO: Description: Shader-based GPU Renderer
[vo/gpu] reconfig to 1920x1080 drm_prime[nv12] bt.709/bt.709/bt.1886/limited/display SP=1.000000 CL=mpeg2/4/h264
[vo/gpu] Resize: 1920x1080
[vo/gpu] Window size: 1920x1080 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu] Video source: 1920x1080 (1:1)
[vo/gpu] Video display: (0, 0) 1920x1080 -> (0, 0) 1920x1080
[vo/gpu] Video scale: 1.000000/1.000000
[vo/gpu] OSD borders: l=0 t=0 r=0 b=0
[vo/gpu] Video borders: l=0 t=0 r=0 b=0
[vo/gpu] Using HW-overlay mode. No GL filtering is performed on the video!
[vo/gpu] No advanced processing required. Enabling dumb mode.
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[cplayer] first video frame after restart shown
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[cplayer] audio sync: sync_to_video=1, offset=0.000000
[cplayer] playback restart complete @ 0.000000, audio=playing, video=playing
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V: -0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V: -0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[statusline] AV: 00:00:00 / 00:00:30 (0%) A-V: -0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[cplayer] EOF code: 5  
[cplayer] finished playback, success (reason 3)
[cplayer] 
[cplayer] Exiting... (Quit)
[ao/alsa] draining...
 

 

Link to post
Share on other sites

Huh..it seams that it is not so simple for everyone to reach this poing to have a HW acceleration on H3.

My goal is to reach transcoding from h264 full HD as input stream (satillite stream from enigma2 receivers) and lower down the resolution to SD in h264 with FFmpeg.

I compiled myself the ffmpeg with h264 and libfdk-aac.

All this working even without HW but from SD to SD.

Is there some detail tutorial how can I reach this point?

Thx!

Link to post
Share on other sites

Trying to add VA-API into actual Armbian Focal (Linux orangepipc 5.4.45-sunxi #20.05.4)

 

Install instruction

Spoiler

$ sudo apt-get install libtool pkg-config vainfo libva2 libva-x11-2 libva-drm2 libva-utils libva-dev libdrm-dev

$ git clone https://github.com/bootlin/libva-v4l2-request

$ cd libva-v4l2-request

$ ./autogen.sh

$ make

$ sudo make install

 

Next, export driver:

$ export LIBVA_DRIVER_NAME=v4l2_request

and check it with vainfo tool. Have an error:

$ vainfo 
libva info: VA-API version 1.7.0
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
libva info: va_openDriver() returns -1
vaInitialize failed with error code -1 (unknown libva error),exit

 

To fix, add symbolic link to installed driver:

$ sudo ln -s /usr/lib/dri/v4l2_request_drv_video.so /usr/lib/arm-linux-gnueabihf/dri/

 

And seems everything is ok

$ vainfo 
libva info: VA-API version 1.7.0
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.7 (libva 2.6.0)
vainfo: Driver version: v4l2-request
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264MultiviewHigh      :	VAEntrypointVLD
      VAProfileH264StereoHigh         :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointVLD

 

Edited by Sash0k
fixed by myself
Link to post
Share on other sites

4 hours ago, Dreyk said:

@Sash0k, it looks interesting.
What about the real video playback in video players (like mpv, vlc, etc...)?

 

Unfortunately, no success.

 

First, mainline bootlin backend is outdated, h264 broken. It has some pull requests, that not merged.

Tryng to recompile Philipp Zabel's version, and merge his work for h264

Spoiler

$ git clone https://github.com/pH5/libva-v4l2-request.git
$ cd libva-v4l2-request
$ git merge origin/h264
$ ./autogen.sh
$ make
$ sudo make install

 

 

This version works with artifacts in VLC:

U24j7fV.png

 

Link to post
Share on other sites

Finally got it! No kernel modifications needed, only v4l2-request.


Key notes:

  • Use bootlin code, latest master (not release-2019.03 tag)
  • I merged just one small patch from https://github.com/bootlin/libva-v4l2-request/pull/30/files (seems, it's unecessary)
  • Download kernel sources with corresponding version. For my armbian is: https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.4.45.tar.xz
  • Extract 2 files from kernel/include/media mpeg2-ctrls.h and h264-ctrls.h and replace ones in v4l2-request
  • Replace V4L2_PIX_FMT_H264_SLICE_RAW to V4L2_PIX_FMT_H264_SLICE in v4l2-request source code
  • Compile and install (instruction is as 2 posts above)
  • Don't forget to set VLC as in https://linux-sunxi.org/Sunxi-Cedrus
    Spoiler

    Tools > Preferences > Input / Codecs > Codecs > Hardware-accelerated decoding > VA-API video decoder

    Tools > Preferences > Video > Display > Output > X11 video output (XCB)

 

Tested with VLC, usable with issues:

  • Artifacts in some videos h264 720p and higher, for example: https://imgur.com/nYFArT4 (360/480 works fine)
  • Scaling (fullscreen, resizing) not works, slowdown with message
    [a310cb88] main filter error: Failed to create video converter
  • Minor issues in console output on playback (see bold)
    Spoiler

    $ vlc 3-big_buck_bunny_480p_H264_AAC_25fps_1800K.MP4
    VLC media player 3.0.9.2 Vetinari (revision 3.0.9.2-0-gd4c1aefe4d)
    [02287b98] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
    libEGL warning: DRI2: failed to authenticate
    libva info: VA-API version 1.7.0
    libva info: User environment variable requested driver 'v4l2_request'
    libva info: Trying to open /usr/lib/arm-linux-gnueabihf/dri/v4l2_request_drv_video.so
    libva info: Found init function __vaDriverInit_1_7
    libva info: va_openDriver() returns 0
    [a272d350] avcodec decoder: Using v4l2-request for hardware decoding
    [a3018b98] blend blend error: no matching alpha blending routine (chroma: YUVA -> VAOP)
    [a3018b98] main blend error: blending YUVA to VAOP failed

 

Thanks to: @jernej for this post:

 

Edited by Sash0k
thanks
Link to post
Share on other sites

Well, kernel modification is needed for better decoding (less glitches). Probably new *-ctrl.h files contain new fields (not 100% sure) which also need to be filled.

 

There is no any post processing implemented here like scaling. It would be possible to do that via SoC specific peripherals but that wouldn't be universal and thus it's out of scope of this library.

 

Anyway, I'm glad you succeeded.

Link to post
Share on other sites

Hi,

i'm interested in this topic, but since i use wayland i would like to get it working on mpv (for example). I got vaainfo detecting the library, but when launched using mpv i got green screen and a lot of "failed to destoy surface". I also have noticed that after a some try, i got an error about failed resource allocation (i imagine because it does not free correctly buffers). Someone have managed to get mpv working?

For the context, i'm not using Armbian device, but a Pinephone with PostmarketOS (it's the only Allwinner device i have), but i think a lot of people could benefit from get this driver working.

Link to post
Share on other sites

Sadly noone maintain vaapi layer, so you're mostly on your own there, which means learning a ton of internal details of vaapi, v4l2 request api and supported codecs. On the other hand, ffmpeg fork with v4l2 request api is pretty well maintained. I know this is not ideal, but for example mpv (with one patch) can work with that modified ffmpeg.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

12 12