giddy

armbian-firmware = minimal blobs that are needed that on-board wireless function + few popular ones = https://github.com/armbian/firmware 
- full = + everything else https://kernel.googlesource.com/pub/scm/linux/kernel/git/firmware/linux-firmware.git
 
You can install one of those, while source package, headers and armbian-config are optional. Also armbian-firmware if your will not be running any wireless.

Kernel = image + dtb package. Those are the core.

Those are developers preview builds, where you can check what works and at some point it will be good enough for some uses cases. In a couple of years, it will be functional on the level of kernel 5.10.y. 
 
Download is possible from CI pipeline: https://github.com/armbian/build/releases
 

 
Boot log:
 
This board is looking for maintainer(s) and (this) forum moderator (contact @Werner).

https://www.armbian.com/rock-5b/

I received at least one PM so far that I did not know the answer to.
 
So, rather than all of us spend a lot of effort duplicating info, I thought maybe good idea to compile these in one place.  Especially as we have an influx of new Board Maintainers at the moment.
 
I also encourage people to stop by IRC and say hello.  I shall also endeavor to copy the most useful bits from IRC into this thread as they come up.  Maybe later can be distilled even further into some docs page or ...?  Anyway...
 
So, first up:
 
Q: How /what checks/tests to run?
 
A: OK, I guess I thought armbian/autotests was more about that (long in development hardware), however looking again now it seems it's actually just a script which can be run on its own, just directly on SBC (even without that test rig).  Amirite?

Yes, I do believe so. It is probably best to just start from scratch. I made a new repository, named QA, since entire testings goes under quality assurance. https://github.com/armbian/qa I have also added you more privileges.
 

Keep it generic, perhaps left in XML / JSON format we can import elsewhere or into some generic XML 2 HTML function.
 
 
The goal is that we can test simultaneously on many devices, run as much tests we like and get some results. Also check if it is possible to make a bridge with @William Bonnetwork. To have some a bit more generic testings with an option to have per hardware family. Not needed per board. But perhaps. we can prepare its possible.

Hi Igor, i started to write such a tool for the same ûrpose as yours, it was a log time ago, for exactly the same target as yours (many things to talk;) )   it under Apache License and github, usung python and simple unit test. BTW i know verwell Ansible, and i'll be very happy to help you
 
The tool is available under https://github.com/wbonnet/sbit
 
It looks empty a first i need to push several things if you are interested, i will provide doc and some board auto test. The simpliest is maybe to start talkinng about it here or on IRC. deb packages are also availables online from the DFT project (githhub) also

i'll look forward to you about this subject in the next days, i my have several other thing to share and contribute
cheers William

Wonderful news! I briefly checked your project and it seems to match our ideas on (unit) testing. I am Ansible newbie, know basics, while @lanefu (US) is more experienced in this and we should all come together on chat - we are hanging out here https://docs.armbian.com/Community_IRC/. Our most active hours starts from lunch break, when US people gets up and evenings, late night, CET.
 
Thanks!

I will just add a bit more. 
 
We want to rewrite / make this from scratch with Ansible. Basically we seek Ansible expert or someone who wants to become one.  We have this know-how but we are simply too overloaded to move on. This is yet another job for common good! To make Armbian support better. To make Debian support better. To make Arch better, OpenWrt, ... To make every Linux distro out there running better on your board.
 
The person(s) should focus only on testing. Creating and maintaining scripts for automated testing and later, when our support hardware is out of beta testing, implement that as well. It's a continuous project and will not end tomorrow. Also you don't need to stay on it forever. Do what you can. Help us get going ...
 
We would like to automate:
 
- initial board setup (after you flash the image, automated 1st login, setup network, different local repository, ...)
- simple tasks such as upgrade, change to beta repository, downgrade to specific kernel, etc.
- run various of tasks which can be added without limits
- run various of benchmarks which can be added without limits
- run tasks in parallel
- make reporting in HTML, XML (common and per board)

Most of those ideas are covered in some basic form in our first try: https://github.com/armbian/autotests
 
More ideas:
https://github.com/SoInteractive/ansible-benchmark
 

It worked, I have my eMMC system back!
 
Here are all the steps in case someone else has the problem and wants to repair his eMMC image...
 
1. download a previous Helios64 image from here...
https://wiki.kobol.io/download/
2. install the image on a microSD e.g. for Windows and macOS there is Etcher...
https://www.balena.io/etcher/
3a. boot your Helios64 with the microSD and Jumper 10, see here...
https://wiki.kobol.io/helios64/troubleshoot/#how-to-force-boot-from-microsd
3b. Alternative: from the serial console press a key on the keyboard while u-boot start. You will get the u-boot prompt.
From this prompt write and press enter. Helios64 will boot from SD... (thx @prahal)
run bootcmd_mmc1  
4. if you are logged in as root user (normal users must put a sudo in front of the commands) execute the following commands...
root@helios64:~# mkdir -p /mnt/system root@helios64:~# mount /dev/mmcblk2p1 /mnt/system root@helios64:~# cd /mnt/system/
5. The contents of the /mnt/system directory should now be filled with the contents of your eMMC....
root@helios64:/mnt/system# ll total 80 lrwxrwxrwx 1 root root 7 Aug 30 2020 bin -> usr/bin drwxr-xr-x 3 root root 4096 Sep 1 03:01 boot drwxr-xr-x 2 root root 4096 Oct 15 2020 dev drwxr-xr-x 110 root root 12288 Sep 1 03:01 etc drwxr-xr-x 2 root root 4096 Sep 22 2020 export drwxr-xr-x 5 root root 4096 Jun 10 04:12 home lrwxrwxrwx 1 root root 7 Aug 30 2020 lib -> usr/lib drwx------ 5 root root 4096 Oct 5 2020 lost+found drwxr-xr-x 4 root root 4096 Oct 15 2020 media drwxr-xr-x 2 root root 4096 Feb 9 2021 mnt drwxr-xr-x 4 root root 4096 Dec 2 2020 opt dr-xr-xr-x 2 root root 4096 Oct 15 2020 proc drwx------ 7 root root 4096 Aug 31 23:36 root drwxr-xr-x 2 root root 4096 Oct 15 2020 run lrwxrwxrwx 1 root root 8 Aug 30 2020 sbin -> usr/sbin drwxrwxr-x 2 root root 4096 Oct 5 2020 selinux drwxr-xr-x 6 root root 4096 Jun 10 03:18 srv dr-xr-xr-x 2 root root 4096 Oct 15 2020 sys lrwxrwxrwx 1 root root 42 Sep 1 03:01 thermal_zone0 -> /sys/devices/virtual/thermal/thermal_zone0 drwxrwxrwt 2 root root 4096 Oct 15 2020 tmp drwxr-xr-x 12 root root 4096 Nov 27 2020 usr drwxr-xr-x 14 root root 4096 Oct 6 2020 var  
6. now download the 3 old packages...
root@helios64:/mnt/system# wget http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/linux-dtb-current-rockchip64_21.05.4_arm64.deb root@helios64:/mnt/system# wget http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/linux-headers-current-rockchip64_21.05.4_arm64.deb root@helios64:/mnt/system# wget http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/linux-image-current-rockchip64_21.05.4_arm64.deb  
7. now changes the root directory...
root@helios64:/mnt/system# chroot /mnt/system root@helios64:/# pwd / root@helios64:/# ll total 50996 lrwxrwxrwx 1 root root 7 Aug 30 2020 bin -> usr/bin drwxr-xr-x 3 root root 4096 Sep 1 14:56 boot drwxr-xr-x 2 root root 4096 Sep 1 16:15 dev drwxr-xr-x 110 root root 12288 Sep 1 03:01 etc drwxr-xr-x 2 root root 4096 Sep 22 2020 export drwxr-xr-x 5 root root 4096 Jun 10 04:12 home lrwxrwxrwx 1 root root 7 Aug 30 2020 lib -> usr/lib -rw-r--r-- 1 root root 314304 Jul 8 19:32 linux-dtb-current-rockchip64_21.05.4_arm64.deb -rw-r--r-- 1 root root 11527696 Jul 8 19:32 linux-headers-current-rockchip64_21.05.4_arm64.deb -rw-r--r-- 1 root root 40290884 Jul 8 19:33 linux-image-current-rockchip64_21.05.4_arm64.deb drwx------ 5 root root 4096 Oct 5 2020 lost+found drwxr-xr-x 4 root root 4096 Oct 15 2020 media drwxr-xr-x 2 root root 4096 Feb 9 2021 mnt drwxr-xr-x 4 root root 4096 Dec 2 2020 opt dr-xr-xr-x 2 root root 4096 Oct 15 2020 proc drwx------ 7 root root 4096 Aug 31 23:36 root drwxr-xr-x 2 root root 4096 Oct 15 2020 run lrwxrwxrwx 1 root root 8 Aug 30 2020 sbin -> usr/sbin drwxrwxr-x 2 root root 4096 Oct 5 2020 selinux drwxr-xr-x 6 root root 4096 Jun 10 03:18 srv dr-xr-xr-x 2 root root 4096 Oct 15 2020 sys lrwxrwxrwx 1 root root 42 Sep 1 03:01 thermal_zone0 -> /sys/devices/virtual/thermal/thermal_zone0 drwxrwxrwt 2 root root 4096 Oct 15 2020 tmp drwxr-xr-x 12 root root 4096 Nov 27 2020 usr drwxr-xr-x 14 root root 4096 Oct 6 2020 var  
8. now installs the downloaded packages...
root@helios64:/# dpkg -i *.deb dpkg: warning: downgrading linux-dtb-current-rockchip64 from 21.08.1 to 21.05.4 (Reading database ... 62558 files and directories currently installed.) Preparing to unpack linux-dtb-current-rockchip64_21.05.4_arm64.deb ... Unpacking linux-dtb-current-rockchip64 (21.05.4) over (21.08.1) ... Selecting previously unselected package linux-headers-current-rockchip64. Preparing to unpack linux-headers-current-rockchip64_21.05.4_arm64.deb ... Unpacking linux-headers-current-rockchip64 (21.05.4) ... dpkg: warning: downgrading linux-image-current-rockchip64 from 21.08.1 to 21.05.4 Preparing to unpack linux-image-current-rockchip64_21.05.4_arm64.deb ... update-initramfs: Deleting /boot/initrd.img-5.10.60-rockchip64 Removing obsolete file uInitrd-5.10.60-rockchip64 stat: cannot stat '/proc/1/root/.': No such file or directory Unpacking linux-image-current-rockchip64 (21.05.4) over (21.08.1) ... Setting up linux-dtb-current-rockchip64 (21.05.4) ... Setting up linux-headers-current-rockchip64 (21.05.4) ... Compiling headers - please wait ... grep: /proc/cpuinfo: No such file or directory grep: /proc/cpuinfo: No such file or directory Setting up linux-image-current-rockchip64 (21.05.4) ... update-initramfs: Generating /boot/initrd.img-5.10.43-rockchip64 W: Couldn't identify type of root file system for fsck hook update-initramfs: Converting to u-boot format  
9. reset the root change with exit and restart your helios64 with reboot...
root@helios64:/# exit exit root@helios64:/mnt/system# reboot  
10. after a reboot you should now boot from the eMMC again...
root@192.168.180.5's password: _ _ _ _ __ _ _ | | | | ___| (_) ___ ___ / /_ | || | | |_| |/ _ \ | |/ _ \/ __| '_ \| || |_ | _ | __/ | | (_) \__ \ (_) |__ _| |_| |_|\___|_|_|\___/|___/\___/ |_| Welcome to Armbian 21.08.1 Buster with Linux 5.10.43-rockchip64 No end-user support: built from trunk System load: 74% Up time: 3 min Memory usage: 34% of 3.77G IP: 172.18.0.1 172.19.0.1 172.20.0.1 172.17.0.1 192.168.180.5 CPU temp: 66°C Usage of /: 87% of 15G [ 0 security updates available, 3 updates total: apt upgrade ] Last check: 2021-09-01 16:38 [ General system configuration (beta): armbian-config ] Last login: Wed Sep 1 16:26:23 2021 root@helios64:~# root@helios64:~# uname -a Linux helios64 5.10.43-rockchip64 #21.05.4 SMP PREEMPT Wed Jun 16 08:02:12 UTC 2021 aarch64 GNU/Linux root@helios64:~#  
11. if the eMMC was used you can see with mount...
root@helios64:~# mount | grep /dev/mmc /dev/mmcblk2p1 on / type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600) /dev/mmcblk2p1 on /var/folder2ram/var/log type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600) /dev/mmcblk2p1 on /var/folder2ram/var/tmp type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600) /dev/mmcblk2p1 on /var/folder2ram/var/lib/openmediavault/rrd type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600) /dev/mmcblk2p1 on /var/folder2ram/var/spool type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600) /dev/mmcblk2p1 on /var/folder2ram/var/lib/rrdcached type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600) /dev/mmcblk2p1 on /var/folder2ram/var/lib/monit type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600) /dev/mmcblk2p1 on /var/folder2ram/var/cache/samba type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)  

thank you @aprayoga that was helpful.
 
TL:DR

my problem was, docker was starting BEFORE my mergefs volumes mounted, in one of my containers I am mounting a docker volume. because docker does not see the volume on disk, it "helpfully" creates that volume on disk. now my mergefs cannot mount because there are files created by docker in that location.
 

Assumptions:
you have a computer connected via USB-SERIAL and you can see the Helios console. inline code/commands important variables are wrapped with `` - I am used to markdown. I assume you know that `vim` and `nano` are text editors and you know how to exit `vim`.  
 
How I found out and investigated:

1. I followed the above advice to boot from sd-card (balena etcher + latest stable armbian buster). 
2. I fsck emmc = no issues.
3. I mounted the emmc, chrooted in, changed root password. Now I can login past the `Give root password for maintenance (or press Control-D to continue):`
4. I cannot start services, system is dead, `journalctl` does not have any useful info. `/var/log/messages` looks boring.
5. I found some instructions to edit `/boot/armbianEnv.txt` (while still chrooted to emmc install) I bumped up the verbosity to `9` (make sure you only edit verbosity and do not change other things in your armbianEnv.txt).
`vim /boot/armbianEnv.txt`
verbosity=9 bootlogo=false overlay_prefix=rockchip rootdev=UUID=b31229b9-40ab-441c-95be-66666 rootfstype=ext4 console=serial usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u
 
6. I reboot, and I notice this in the boot logs (I am able to see boot logs because I am still connected via USBC SERIAL cable):
[FAILED] Failed to mount /srv/f95ca…b-439d-450e-b700-4444. See 'systemctl status "srv-f95ca73b\\x2…0\\x2d4444.mount"' for details.  
After this the system "hangs" with the message we saw before:
Starting kernel ... Give root password for maintenance (or press Control-D to continue):
7. Because I changed the password, I am able to get in to recovery mode on the emmc install.
I do a `cat /etc/fstab` and notice that `f95ca…b-439d-450e-b700-4444` is my mergefs volume. 
I do an `ls -alsht /srv/f95ca…b-439d-450e-b700-4444` and I see some directories there, both of these cannot be true, it's either mounted and files or NOT mounted and NO directories.
These directories match up with the docker volume mounts I specified for one of my containers.
 
8. I do a `systemctl docker stop` `systemctl docker disable` so docker does not do a mess again (for now). I do a `du -hs /srv/f95ca…b-439d-450e-b700-4444` to make sure it is only empty directories created by docker, not my actual data. The output shows only empty dirs, (I am expecting gigabytes). Only AFTER I verified there is no data to lose, I do a `rm -rf /srv/f95ca…b-439d-450e-b700-4444`.
 
9. Now I need to make mergefs mount the volume BEFORE docker starts. I run `systemctl list-units --type=mount` this shows me ALL THE MOUNTS, for simplicity I am only including the drives we care about.
srv-dev\x2ddisk\x2dby\x2dlabel\x2dsda.mount                loaded active mounted /srv/dev-disk-by-label-sda srv-dev\x2ddisk\x2dby\x2dlabel\x2dsdb.mount                loaded active mounted /srv/dev-disk-by-label-sdb srv-dev\x2ddisk\x2dby\x2dlabel\x2dsdc.mount                loaded active mounted /srv/dev-disk-by-label-sdc srv-dev\x2ddisk\x2dby\x2dlabel\x2dsdd.mount                loaded active mounted /srv/dev-disk-by-label-sdd srv-f95ca73b\x2d439d\x2d450e\x2db700\x2d4444.mount loaded active mounted /srv/f95ca73b-439d-450e-b700-4444
these are the disks and volumes matching up with the failed mount in the boot logs.
10. Now I edit the systemd docker override with `systemctl edit docker` and add this block:
[Unit] After=srv-dev\x2ddisk\x2dby\x2dlabel\x2dsda.mount srv-dev\x2ddisk\x2dby\x2dlabel\x2dsdb.mount srv-dev\x2ddisk\x2dby\x2dlabel\x2dsdc.mount srv-dev\x2ddisk\x2dby\x2dlabel\x2dsdd.mount srv-f95ca73b\x2d439d\x2d450e\x2db700\x2d4444.mount
I save, I exit nano.

I want to check if systemctl sees my changes... I run this: `systemctl cat docker`
the output shows the override (look at the last three lines, one of them includes my override to wait until mounts are done):
# /lib/systemd/system/docker.service [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com BindsTo=containerd.service After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket [Service] Type=notify # the default is not to use systemd for cgroups because the delegate issues still # exists and systemd currently does not support the cgroup feature set required # for containers run by docker ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ExecReload=/bin/kill -s HUP $MAINPID TimeoutSec=0 RestartSec=2 Restart=always # Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229. # Both the old, and new location are accepted by systemd 229 and up, so using the old location # to make them work for either version of systemd. StartLimitBurst=3 # Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230. # Both the old, and new name are accepted by systemd 230 and up, so using the old name to make # this option work for either version of systemd. StartLimitInterval=60s # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Comment TasksMax if your systemd version does not support it. # Only systemd 226 and above support this option. TasksMax=infinity # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target # /etc/systemd/system/docker.service.d/mount-disks-before-docker.conf [Unit] After=srv-dev\x2ddisk\x2dby\x2dlabel\x2dsda.mount srv-dev\x2ddisk\x2dby\x2dlabel\x2dsdb.mount srv-dev\x2ddisk\x2dby\x2dlabel\x2dsdc.mount srv-dev\x2ddisk\x2dby\x2dlabel\x2dsdd.mount srv-f95ca73b\x2d439d\x2d450e\x2db700\x2d4444.mount  
11. I start docker, I enable the service `systemctl start docker` `systemctl enable docker`. I reboot, and it is all working. My filesystems mount properly BEFORE docker starts, ensuring docker does not create docker volumes because my filesystem is not ready.

 

To continue discussion from Helios64 Support
  
 
Could you boot from micro SD card? Please use Armbian 20.08.10 image, version 20.08.13 known to have stability problem on many users.
After setting up Armbian on the micro SD, reboot (to finish the setup) and login as root, you can check eMMC for filesystem error
fsck -p /dev/mmcblk1p1  
You can try to change the password of Armbian on the eMMC:
 
After finished you can poweroff the system, remove the micro SD card and power on the system again. See if you can boot to your system on eMMC.