4kp30 video on Orange Pi Lite and mainline hardware acceleration


Recommended Posts

21 hours ago, jernej said:

Note that ffmpeg patches for request API are important - without patched ffmpeg, all kernel patches have no meaning. Second important thing is that LibreELEC runs Kodi without X11 for ARM boards - this allows to use display more efficiently. On X11 it would be needed to render video through GPU which is less efficient. Note that I never actually tried that.

Am I understanding it correctly that it works as follows?

LibreElec patches the kernel to make use of the SoC's VPU. (Some of these patches make their way upstream eventually, but that's a slow process.) It also forks ffmpeg, which is patched to recognise these new kernel features & to make use of them. Kodi then refers to ffmpeg to access them.

What would be necessary for players other than Kodi—mpv or VLC for example—to benefit from this as well? And what is the role of libva, VAAPI, and these other things in this process?

Link to post
Share on other sites
Donate and support the project!

Basically yes. Note that kernel patches are only for improvements - fixing decoding issues. At least H264 API should be promoted to stable with kernel 5.10, maybe others too.

 

Any player which uses ffmpeg directly should be able to benefit from those patches. AFAIK mpv needs only one simple patch in order to use this chain. I have no idea about VLC internals so I can't say much about it. About VAAPI - request API (Cedrus implements it for decoding) is similar in terms of requirements and features to VAAPI so Bootlin created simple library to expose it. That way they could use any VAAPI capable player. However, some codec interfaces changed considerably from those times and Bootlin didn't invest any time to update this library, so it's not very useful ATM unless you know how to fix it. I know that some people has still interest in that library so I imagine it will be forked/updated once api stabilizes further.

 

Note that there is one hidden issue. As long as you play video in native size, all is well. If you want to scale it then you might get a lot of dropped frames. Issue here is how you do scaling. It's pretty intense process for CPU so you should avoid that at any cost. Unfortunately, VLC with VAAPI translation library did just that. Second option is to go with GPU. IIUC it was impossible to do with binary drivers due to missing import buffer functions, but with mesa it should work now (didn't try it myself). Note that mali400 may still be limiting factor for some cases. Another approach is to use dedicated HW cores. For example, deinterlace core on H3 can be used for scaling only (with few driver tweaks). Downside of this is that scaling process depends on SoC. This would be different even for Allwinner SoCs (not all SoCs have deinterlacer). Most efficient approach (used in LibreELEC) is to use DRM planes - you instruct display engine how to scale and crop it and where on screen to render it. However, I'm not sure if this can work on X11 at all. Wayland should be able to pull this off IIUC.

 

Disclaimer: I don't care about desktop on ARM boards so I didn't do many experiments in this regard and I don't plan to work on improving desktop experience. LE runs without any desktop on ARM boards. Note that I also don't plan to work on any player except ffmpeg/Kodi combo.

Link to post
Share on other sites

I see; thanks for your detailed response.

VAAPI is of interest to me since IIRC it's what jellyfin interfaces with to provide transcoding to clients, but it's more of an abstract interest since I just watch everything on my desktop anyway.

In this context I'm pleased to hear that getting mpv to work may not be so difficult after all. When I have time, I'm going to try and look into into how mpv interfaces with ffmpeg.

Link to post
Share on other sites

Here are the Armbian bionic images with installed RTSP streamer using H264 HW encoding on Allwinner H3 boards

Orange Pi One

Orange Pi Zero

Images are able to run on any board with Allwinner H3

 

Spoiler

Description:
The image is Armbian bionic 20.08 mainline kernel 5.7.15 with installed RTSP server sourced from the HW encoded H264 bitstream.

H264 HW encoder uses sunxi cedrus driver and takes video from the video device /dev/videoN.
Any USB camera with output format YUV 4:2:2 or NV12 can be used as the image source for the encoder.
The system uses UCI from the OpenWrt project (https://openwrt.org/docs/guide-user/base-system/uci) to manage settings of the RTSP streamer.

RTSP server is starting on boot as a service named supervisor (systemctl start/stop/restart supervisor)
To watch RTSP stream from the board use mpv or vlc:

mpv --profile=low-latency --no-cache --untimed rtsp://YUR_BOARD_IP/live.cam
or
vlc --sout-udp-caching=0 --clock-jitter=500 rtsp://YUR_BOARD_IP/live.cam

 

Installation:
1. Unpack the image
2. Instal the image to the SD card (linux: sudo dd if=armbian_bionic_rtsp_h264_opi_one.img of=/dev/sdX bs=1M)
3. Insert and power up

username: root
password: 123


Settings:
To display RTSP settings use uci show:

uci show system.rtsp

  system.rtsp=rtsp
  system.rtsp.proto='udp'
  system.rtsp.device='/dev/video0'
  system.rtsp.format='YUYV'
  system.rtsp.port='554'
  system.rtsp.resolution='640x480'
  system.rtsp.channel='live.cam'
  system.rtsp.framerate='25'
  system.rtsp.server='rtsp'
  system.rtsp.uclnt='0.0.0.0'
 
To change any settings use command uci set:
...
uci set system.rtsp.device=/dev/video1
uci set system.rtsp.format=UYVY
...

To save changes use command uci commit:
uci commit

Settings description:
  system.rtsp.proto - rtsp server network protocol (udp/tcp)
  system.rtsp.device - input video device for h264 encoder (/dev/videoN)
  system.rtsp.format - input image format (YUYV, UYVY, H264, NV12) from the /dev/videoN
  system.rtsp.port - RTSP server port
  system.rtsp.resolution - input image resolution
  system.rtsp.framerate - input stream framerate
  system.rtsp.server - must be rtsp
  system.rtsp.uclnt - not used

To apply config changes use restart service:
systemctl restart supervisor
 
Troubleshooting:
If there is no RTSP stream after the board is started:
* check the Ethernet cable connected
* check the USB camera connected
* check the USB camera valid name setting (/dev/video0,1,2...N)
* check the USB camera image format supported (YUYV, UYVY, NV12, H264)
* check your router RTSP port unblocked
* check valid command line arguments for playing remote RTSP stream

 

Link to post
Share on other sites
On 9/4/2020 at 12:25 PM, ubobrov said:

Here are the Armbian bionic images with installed RTSP streamer using H264 HW encoding on Allwinner H3 boards

Orange Pi One

Orange Pi Zero

Images are able to run on any board with Allwinner H3

 

  Reveal hidden contents

Description:
The image is Armbian bionic 20.08 mainline kernel 5.7.15 with installed RTSP server sourced from the HW encoded H264 bitstream.

H264 HW encoder uses sunxi cedrus driver and takes video from the video device /dev/videoN.
Any USB camera with output format YUV 4:2:2 or NV12 can be used as the image source for the encoder.
The system uses UCI from the OpenWrt project (https://openwrt.org/docs/guide-user/base-system/uci) to manage settings of the RTSP streamer.

RTSP server is starting on boot as a service named supervisor (systemctl start/stop/restart supervisor)
To watch RTSP stream from the board use mpv or vlc:

mpv --profile=low-latency --no-cache --untimed rtsp://YUR_BOARD_IP/live.cam
or
vlc --sout-udp-caching=0 --clock-jitter=500 rtsp://YUR_BOARD_IP/live.cam

 

Installation:
1. Unpack the image
2. Instal the image to the SD card (linux: sudo dd if=armbian_bionic_rtsp_h264_opi_one.img of=/dev/sdX bs=1M)
3. Insert and power up

username: root
password: 123


Settings:
To display RTSP settings use uci show:

uci show system.rtsp

  system.rtsp=rtsp
  system.rtsp.proto='udp'
  system.rtsp.device='/dev/video0'
  system.rtsp.format='YUYV'
  system.rtsp.port='554'
  system.rtsp.resolution='640x480'
  system.rtsp.channel='live.cam'
  system.rtsp.framerate='25'
  system.rtsp.server='rtsp'
  system.rtsp.uclnt='0.0.0.0'
 
To change any settings use command uci set:
...
uci set system.rtsp.device=/dev/video1
uci set system.rtsp.format=UYVY
...

To save changes use command uci commit:
uci commit

Settings description:
  system.rtsp.proto - rtsp server network protocol (udp/tcp)
  system.rtsp.device - input video device for h264 encoder (/dev/videoN)
  system.rtsp.format - input image format (YUYV, UYVY, H264, NV12) from the /dev/videoN
  system.rtsp.port - RTSP server port
  system.rtsp.resolution - input image resolution
  system.rtsp.framerate - input stream framerate
  system.rtsp.server - must be rtsp
  system.rtsp.uclnt - not used

To apply config changes use restart service:
systemctl restart supervisor
 
Troubleshooting:
If there is no RTSP stream after the board is started:
* check the Ethernet cable connected
* check the USB camera connected
* check the USB camera valid name setting (/dev/video0,1,2...N)
* check the USB camera image format supported (YUYV, UYVY, NV12, H264)
* check your router RTSP port unblocked
* check valid command line arguments for playing remote RTSP stream

 

None of these images work on Orange Pi Zero +2 H3 :(
Can you please upload version for Zero +2?

Link to post
Share on other sites
9 hours ago, ubobrov said:

None of these images work on H5. 

Unfortunetely, encoder doesn't work on H5

 

17 hours ago, martinayotte said:

OPiZeroPlus2 is using H5 SoC, not H3 ...

Guys, Orange Pi Zero +2 H3 is based on H3 chip! There are 2 different types of OPiZeroPlus2.
I need image for H3 version.

Watch this:
https://www.armbian.com/orange-pi-zero-plus-2-h3/
https://www.orangepi.su/content.php?p=122&c=Orange Pi Zero Plus 2 H3

Link to post
Share on other sites
On 9/19/2019 at 6:29 AM, jernej said:

It's tricky and needed kernel header files are not exposed in UAPI (userspace API) yet because they are still evolving and changing.

 

You can try following:

1. Apply Linux patches 1 (you can filter out only media related patches), 5, 7-11 on kernel 5.3: https://github.com/jernejsk/LibreELEC.tv/tree/cedrus_update/projects/Allwinner/patches/linux

2. Apply all FFmpeg patches on FFmpeg 4.0.4 (it is easy to port those patches on newer version, to avoid rebuilding mpv): https://github.com/jernejsk/LibreELEC.tv/tree/cedrus_update/packages/multimedia/ffmpeg/patches/v4l2-request-api

3. Compile FFmpeg with at least following options


--enable-v4l2-request --enable-libudev --enable-libdrm

4. Recompile mpv if needed - depends on FFmpeg version used. If it matches to what was was used to compile mpv with, then there is no need.

5. Add your user to appropriate group ("video" I think) so you can use VPU without root privileges.

 

Now you should have working mpv with HW accelerated decoding. As I don't use any desktop on SBCs, I run mpv with following options:


mpv --hwdec=auto --vo=drm video.mkv

This gives you most speed - HW decoded and directly rendered by DRM driver, so CPU is not involved. Only downside is that it is fullscreen only. Not sure how big performance hit (CPU usage) you would get with X11.

Hi,

I've got an OrangePi Zero2 with the Allwinner H616 chip on.  Is this patch likely to work with that, and is there a base Armbian image I can use to start from?  Currently I have an OrangePi image and video decoding is not working fast enough.  I need to be able to decode h.264 1080p60 reliably without using all the CPU power.

As an aside, if I try and follow those links given above to github.com it gives me a 404 error.  Are they git clone links, rather than webpage links?

 

thanks,

skb13

Link to post
Share on other sites
1 hour ago, skb13 said:

As an aside, if I try and follow those links given above to github.com it gives me a 404 error. 

I regularly delete my branches because all work is eventually merged to  https://github.com/LibreELEC/LibreELEC.tv (you quoted quiet old post).

 

But as @Werner said, H616 is not exactly the same as H6 regarding display stuff so it needs quiet a lot of work. Cedrus will be probably easy, but that comes after display stack.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...