Gediz

It’s been 3 months since we posted on our blog. While we have been pretty active on Armbian/Kobol forum for support and still working at improving the software support and stability, we have been developing in parallel the new iteration of Helios64 around the latest Rockchip SoC RK3568.
 
However things haven’t been progressing as fast as we would have wished. Looking back, 2020 has been a very challenging year to deliver a new product and it took quite a toll on the small team we are. Our energy level is a bit low and we still haven’t really recovered. Now with electronic part prices surge and crazy lead time, it’s even harder to have business visibility in an already challenging market.
 
In light of the above, we decided to go on a full break for the next 2 months, to recharge our battery away from Kobol and come back with a refocused strategy and pumped up energy.
 
Until we are back, we hope you will understand that communication on the different channels (blog, wiki, forum, support email) will be kept to a minimum for the next 2 months.
 
Thanks again all for your support.

I've taken peeks at threads related to hardware decoding (of H.264 and HEVC, mainly) on Allwinner and Rockchip platforms on and off, sometimes dabbled in trying and failing to implement solutions recommended there. Being a complete amateur, I find the topic very opaque and confusing, with various different components that need to interface with each other, be patched in sync, and sometimes change drastically between kernel versions, etc. Today I sat down and read up on these subjects, scouring wikis, documentation, this forum, and assorted other sources to try and understand how this works. In this post I will attempt to compile what I've learned on the different software components involved, their relationships, their current status, and solutions to the problem. I hope people more knowledgeable will correct me when I get something wrong or cite outdated information. Stuff which I am highly uncertain of I will print in italics.
(This post is going to focus on mainline implementations of Cedrus/Allwinner, I haven't looked into Hantro/Rkvdec/Rockchip specifics yet. I will speak only of H.264 and H.265/HEVC; I don't understand the high/low stuff and didn't pay attention to other codecs.)
 
Components:
 
Basics: Video codecs like H.264, H.265/HEVC, MPEG-2, etc. are standardised methods which serve to more efficiently encode and decode videos, reducing their filesize. Software en-/decoding is very CPU-intensive. Modern GPUS and ARM SoCs therefore contain specialised hardware (VPUs) to delegate these tasks to. Working hardware decoding is particularly important for underpowered ARM CPUs.
Drivers: Topmost in the stack are the VPU drivers. These are Sunxi-Cedrus/Cedrus V4L2 M2M and Cedar [Is this the legacy one?] on Allwinner; Hantro and Rkvdec on Rockchip. These are all still in development, but Cedrus already fully supports H.264, and partially supports HEVC, and is already usable in the mainline kernel.
APIs: In order for anything (userspace APIs, libraries) to make use of the VPU drivers, you need backends/APIs. For Cedrus, there is the unmaintained libva-v4l2-request backend which implements VA-API, the legacy VDPAU implementation libvdpau-sunxi, and as of kernel version 5.11, H.264 has been merged into the uAPI headers. Different applications may make recourse to one or another of these APIs.
Libraries: FFmpeg and GStreamer. provide libraries and APIs of their own to other applications but can (importantly!) also output directly to the framebuffer. FFmpeg must be patched to access either libva-v4l2-request or the Cedrus driver headers. GStreamer directly accesses kernel headers since 1.18 (works on 5.9, not on 5.10; 1.20 will support 5.11.)
Media players: mpv and depends on FFmpeg for hardware acceleration (and must be patched together with it). VLC can be set to access libva-v4l2-request directly. Kodi 19.0+ supports hardware acceleration out of the box without any out-of-tree patches.
Display server: An additional complication is drawing the output of any of the above on screen. Most successful implementations I've seen bypass X11 and either output directly to the framebuffer or force a plane/display layer on top of any X windows. Wayland apparently makes this easier by allowing applications to use their own DRM planes, but this hasn't been explored much yet. Kodi 19.0 works with all three windowing systems (X, Wayland, and gbm).
 
Codec status:
 
Taken from the LibreElec thread (which reflects LibreElec's status and is ahead of what works elsewhere, but outlines hardware limitations):
 
 
Approaches:
 
Many people have managed to make it work on their machines using different approaches. Note that some of these solutions are one or two years old, and kernel developments since may have changed the situation. Ordered from newer to older:
 
LibreElec – kernel + ffmpeg + Kodi: LibreElec is a Just-Enough-OS with the sole purpose of running Kodi, a media player. It's at the bleeding edge and usually implements codecs and features well before mainline or other distros. It achieves this by heavily patching everything up and down the stack, from the Linux kernel over FFmpeg to Kodi itself. These patches could all be applied to an Armbian build, but there are a lot of them, they're poorly documented, and you'd need to dig into their github to understand what they all do. LibreElec runs Kodi directly without a desktop. kodi-gbm is a package that can be installed on Armbian and functions similarly.
Key contributors to the project are @jernej and @Kwiboo, who sometimes post about their work here (and have been very helpful with questions, thank you). @balbes150 includes some of LibreElec's patches in his Armbian-TV builds, but I don't know which.
 
Kodi 19.0: 
 
LibreElec patches + mpv:
 
 
@megous – Kernel 5.11 + GStreamer: This implementation, done here on a PinePhone (A64), patches the 5.9 kernel and uses a recent version of GStreamer (1.18 and up), whose output is rendered directly to a DRM plane via kmssink. (No X or Wayland.)
GStreamer 1.18 works with the 5.9 kernel. It does not work with 5.10, because of numerous changes to the kernel headers in this version. In 5.11 the H.264 headers were moved into the uAPI; the master branch of GStreamer reflects this, but there haven't been any releases with these patches yet. It'll probably be in repositories with GStreamer 1.20; until then you can build it from source.
 
@Sash0k – patched libva-v4l2-request + VLC: This updates bootlin's libva-v4l2-request and follows the Sunxi wiki's instructions for enabling VLC to make use of it. It works on the desktop. This only works for H.264 and breaks HEVC. When I tried to replicate this approach on a recent Armbian build, I discovered that the h264.c files in the kernel (that libva-v4l2-request draws on) have changed considerably between 5.8 and 5.10, and I lack the understanding to reconcile libva-v4l2-request with them.
 
@ubobrov – old kernel + libcedrus + libvdpau-sunxi + ffmpeg + mpv: This approach, which supports encoding decoding of H.264 uses the libvdpau-sunxi API and ports the legacy driver to mainline as a loadable kernel module and if I understand it correctly, ubobrov ported a legacy feature to mainline. In the post quoted below the kernel is 4.20, but the same method has been successfully applied to 5.7.8 by another user. It requires that the board's device tree be patched, as documented in ubobrov's github repository.
 
 
The summary seems to be that none of the current implementations on Allwinner boards really play nice with X or desktop sessions, and it's best to output directly to the framebuffer. Kwiboo has forked FFmpeg and mpv to make good use of new and unstable kernel features/hardware acceleration which will take a while to make their way upstream. The recent 5.11 move of stateless H.264 out of staging and into the uAPI should facilitate further developments.
I intend to try some of these things in the nearer future. Thanks to everyone who works on mainlining all of this VPU stuff, and to users here who contribute solutions and readily & patiently answer questions (Jernej especially). I hope I didn't post any falsehoods out of ignorance, and welcome any corrections.
 
Other related threads here:
https://forum.armbian.com/topic/11551-4kp30-video-on-orange-pi-lite-and-mainline-hardware-acceleration/
https://forum.armbian.com/topic/16804-orange-pi-pc-h3-armbian-focal-5104-sunxi-av-tv-out-cvbs-enable/page/2/
https://forum.armbian.com/topic/11184-hardware-graphicvideo-acceleration-in-h3-mainline/
https://forum.armbian.com/topic/13622-mainline-vpu/

Wherein I explain my conversion... 
 
Pine64 was barely on my radar in the past.  There are so many SBC and companies just trying to sell crap it can be hard to tell who is who some times, until you look into one company / board / device or another, a little more in depth.
 
I have been following Librem 5 development for a long time, so I was aware of PinePhone, but they seemed in my mind back then a bit like a "Johnny come lately" who were just trying to capitalize on whatever Purism were trying to do.
 
Then @lanefu was raving about his PineBookPro in IRC, it was only then I started really poking around their forums the last month or so, and now I must say my opinion of them have really changed a lot.
 
Now they seem to me like a really hacker mindset, hardware oriented company.  They really seem to just enjoy making and selling interesting/compelling hardware (for cheap! to get the stuff out there).  They sell spare parts for example, and have a wiki with lots of good information, and schematics, etc.  And they have a nice community forming around their devices, who are in turn, creating spin-off projects...  In short, they seem to be the genuine article (in a time where many other companies are riding on the wave of popularity of "open source" whilst in actuality being nothing of the sort).
 
Purism seem more committed to pushing the supply chain in a F/LOSS direction, but how can we know that Pine64 are not (without being mind readers)?  Well, I have always realized Pine64 are simply taking a different approach.  Some times I fear Purism may have bit off more than they can chew.  They are shipping now, but only in low numbers.  r/Purism is a cesspool of nasty, disgruntled people (maybe I expect too much, it is Reddit, after all).  But if Purism fail, I fear that will set back GNU/Linux on phones overall.  Which is the only thing I really care about.  At any rate, even if I were to order one now, it seems I would be waiting at least a year, also with a less than nil chance of never receiving the device at all, unfortunately.
 
The Pine64 approach, whilst admittedly not as appealing to my inner rms zealot , is perhaps more realistic and attainable.  The older I get, and more real world project / life experience, I start to wonder if this may be the better approach.  Smaller steps, and iteration.  Who knows where it will lead, if we can get more numbers of Linux phones actually in the hands of more people.  For example, apparently some guy already reverse engineered the PinePhone modem OS, and has it running with plain Linux, without blobs!  Amazing!
 
At some point I guess I realized that PinePhone and Librem 5 are not competing with one another, and that none of this is an either/or proposition.  In fact, both projects are advancing the state of the art of GNU/Linux phones; just in different ways.
 
Very recently, I was trying to upgrade to the latest LineageOS on my Galaxy s5.  Of course this is a pain in the arse and why I only get around to really doing it every year or two it seems.  And every time, it seems Google / carriers / whoever have tightened the noose a little bit more.  So, this time around, I finally reached the point where I became so disgusted, that I was willing to actually cancel my cellular service altogether, rather than continue dealing with this nonsense (specifically: VoLTE is becoming mandatory on US carriers, and LineageOS have said they cannot / will not support VoLTE for whatever reason).
 
Importantly, I think it's important to point out that I think any human being needs to reach some tipping point, where our disgust for ${current platform} exceeds the hassle and/or learning curve of switching to ${new platform}.  I have said this many times to people who are curious about switching from Windows -> Linux for example.  Also, this point varies for different people (I get there sooner as I care more about Freedom and have less patience for dinosaur sociopathic proprietary business models, etc.).
 
Anyway, reading some recent threads/posts at Pine64 forums revealed that the PinePhone is currently actually apparently quite usable as a "daily driver" at least for the bare minimum of things I would say that entails: phone/voice (including VoLTE(!)), SMS, and data (on most carriers in US, anyway[0]).  Which was honestly better than I was expecting.  Of course, there are many, many programs in the repositories that do not work properly (or at all) on a phone interface.  And what any individual will consider "bare minimum" acceptable will vary greatly!  But for me at least, that basic phone/data/SMS functionality is enough.  All the rest will not only come in time, if I have a PinePhone in hand I will be able to actively contribute to making it happen, sooner.
 
So, yeah, I pulled the trigger on a PinePhone.      And I plan to use it (more or less) as my "daily driver", likely with my old s5 along side for some period of time, just in case (I figure I should be able to tether it if needed, as tethering is also already working on PinePhone).
 
Long term, I think we still need to continue to push the supply chain in a more explicitly F/LOSS/H direction.  Therefore (assuming they don't go bankrupt before then) I will likely also purchase one or more Librem devices from Purism in the future to support that goal.
 
But buying a PinePhone now gets me in the game, sooner.  And I really started to like what they have going on over there.
 
Please discuss.
 
Cheers,
TRS-80
 
[0] some times requiring workarounds

You're welcome. If you don't mind, I'd recommend you to use a more recent U-Boot version if there isn't any special purpose to use v2015.04.
 
By the way, I had to spend a little bit amount of time to integrate a custom U-Boot and Linux to Armbian for my own use case. I thought that it'd help to see what/where should i add/modify back in then. Maybe this may help you a bit. By the way, do not mind FEX. It was for a really old kernel.
 
. ├── config │   ├── boards │   │   └── myboard-a13.csc │   ├── fex │   │   └── olinux-som-a13.fex │   ├── kernel │   │   └── linux-sun5i-default.config │   └── sources │   └── sun5i.conf ├── defconfig │   └── u-boot │   └── myboard-a13_defconfig └── userpatches ├── config-myboard.conf └── u-boot └── u-boot-sunxi-legacy └── myboard-a13_defconfig.patch  
Just after copying an overlay like this to your Armbian build directory, then ./compile.sh myboard
 
This configuration may be outdated to an extent. It's been some months.

I do not know about the other issues but if i recall correctly CONFIG_VIDEO_LCD_MODE is actually an option in U-Boot, not kernel.

Assembled - first boot of Ubuntu 20.04 looks very very nice  But it was tense ... "what if it will not work?"
 
Also NVME and SATA DOMs are recognized but 10Gb NIC. Which probably need more recent kernel ... and water cooling doesn't fit into the case. Currently mounted outside, which might even not be that bad.
 

There is something important I should add. I am sure big corporations can easily contribute money to meet the demand goal, however this is a community project and it is therefore imperative that we as individuals contribute to the project. 
 
If only big corporations contribute they will probably want to have a say in what direction the project develops which could lead to only a small fraction of devices being developed for or it could lead to a situation where they ask that only their devices are prioritised.
 
If you want that a wider range of devices are going to be worked on it is important that we also contribute, even if we just donate small amounts. At the end of the day there are a lot of us who have an interest in running armbian on our devices.
 
Essentially we have to be fast and donate faster than the companies who only have a commercial interest! 
 
If you are not able to donate there are also other ways to contribute towards the project. Writing documentation, translating documentation, programming and even fan artwork is important (desktop background images, forum banners, forum signatures and other artwork). 
 
Do you love armbian as much as I do? 
 
Don't wait and contribute to the project now by any means you wish! 
 
 

Okay, make sure to set BOOT flag on the second partition.
 
Adding setenv rootdev "/dev/mmcblk0p2" to armbianEnv.txt to the second partition's /boot/armbianEnv.txt doesn't work.
 
Moving on...
 

Thought you might want to know :-)