ebin-dev

Thank you @prahalfor digging into this !! @aprayoga@piter75 Could the emmc issue be solved now with this input ?

If you could boot from SD - the current Buster image should be fine - mount your ssd, replace linux on ssd (same procedure), and check the UUIDs at the two locations hopwfully solves the problem.

Hi @Flolm, what you could do to rescue your system on emmc is to boot from sd (i.e. using a fresh Armbian Buster) and reinstall linux on emmc from there (using chroot as described here). You also need to check whether the UUIDs are set correctly in /mnt/system/etc/fstab and /mnt/system/boot/armbianEnv.txt (assuming that your emmc is mounted to /mnt/system). Good luck! P.S.: in any case you should maintain a backup of your emmc (on sd).

@piter75 commited the emmc fix into Armbian and built Armbian 21.08.2 which is already available through the Armbian mirrors (via 'apt update && apt upgrade'). It is save to upgrade your Buster or Bullseye installation to Armbian 21.08.2 - but emmc read speed will be about 50% what it used to be. You can roll back to the previous linux 5.10.43 kernel if you don't like that - just install those files with 'dpkg -i *.deb'.

Really ? If you are on macOS Big Sur all the drivers and apps are already there. If you need to access the serial console of your Helios64, just connect your mac to the USB-C port of your Helios64. A device such as /dev/tty.usbserial-XXXXXXXX will be recognised by macOS. All you need to do then is to enter the following into the Terminal of your mac (X to be replaced): screen /dev/tty.usbserial-XXXXXXXX 1500000 -L Edit: If the 'screen' command on macOS does not show the output of the bootloader (scrambled characters) you could use the app serial. The only thing you need to configure is to set the serial speed manually to 1500000 bps. If someone would like to contribute in kernel testing - the Armbian build system is very easy to install and use. All you need is an Ubuntu 21.04 (amd64) installation. All the rest is explained here. To compile your own kernel is then as simple as this: ./compile.sh BOARD=helios64 BRANCH=current KERNEL_ONLY=yes RELEASE=bullseye

Sorry - but it is difficult to help because the current state of your system is not known. You could format your system partition on emmc, rsync your backup to it and refresh the bootloader on emmc as described earlier in this thread.

See the parallel thread (I have just changed the thread title). You need to downgrade linux on emmc to 5.10.43.

I can confirm that your temporary fix works on my system. To test it one just needs to fully upgrade emmc to the latest Armbian 21.08.1 release and apply the above fix before a reboot. The system partition on emmc remains writable - even with kernel 5.10.60 - but with the performance penalty. (However, I switched back to kernel 5.10.43 again) So - from my point of view you could provide an updated release including this temporary fix. Please keep us informed if the underlying issue is solved. Thank you again !

Hi Peter, thank you for the analysis. If someone confirms that hs200 mode is a temporary solution for linux kernels 60+ you could release an update with this fix. So everyone will have the choice to stay with linux-5.10.43 for the time being or to upgrade to newer kernels .60+ with a performance penalty on emmc (but with a writable file system).

All you need to do is this (see my earlier message): The easiest way to downgrade linux on emmc now would be to copy those files to /mnt/system (root directory of your emmc) - and then change root with 'chroot /mnt/system' and install the packages with 'dpkg -i *.deb' (while your active system is on SD). You may also need to refresh your bootloader on emmc - as described in this thread. P.S.: It would appear that you have mounted your SD card (/dev/mmcblk1p1) to /mnt/system and not your emmc (/dev/mmcblk2p1). You need to boot off SD.

As the topic of this thread is upgrading Buster to Bullseye, I did some further testing in this context: I would like to confirm that upgrading Buster installations to Bullseye also works fine if you use network-manager to manage your network interfaces (default), even if you have a bridge configured (using bridge-slave; binutils-bridge). Using a Buster image from the Helios64 download page and following the instructions provided in the first message in this thread, the Buster image was successfully upgraded to Bullseye and the bridge configured is also working (the first message in this thread is updated accordingly): _ _ _ _ __ _ _ | | | | ___| (_) ___ ___ / /_ | || | | |_| |/ _ \ | |/ _ \/ __| '_ \| || |_ | _ | __/ | | (_) \__ \ (_) |__ _| |_| |_|\___|_|_|\___/|___/\___/ |_| Welcome to Armbian 21.08.1 Buster with Linux 5.10.60-rockchip64 No end-user support: built from trunk System load: 3% Up time: 1 min Memory usage: 4% of 3.77G IP: xx.xx.xx.xx 10.0.3.1 CPU temp: 42°C Usage of /: 8% of 29G RX today: 518.9 MiB [ General system configuration (beta): armbian-config ] # nmcli br0: connected to br0 "br0" bridge, xx:xx:xx:xx:xx:B1, sw, mtu 1500 ip4 default, ip6 default inet4 xx.xx.xx.55/24 route4 0.0.0.0/0 route4 xx.xx.xx.0/24 inet6 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx/64 inet6 fe80::xxxx:xxxx:xxxx:xxxx/64 route6 2a02:xxxx:xxxx:xxxx::/62 route6 2a02:xxxx:xxxx:xxxx::/64 route6 ::/0 route6 fe80::/64 lxcbr0: connected (externally) to lxcbr0 "lxcbr0" bridge, xx:xx:xx:xx:xx:xx, sw, mtu 1500 inet4 10.0.3.1/24 route4 10.0.3.0/24 route4 169.254.0.0/16 eth0: connected to bridge-slave-eth0 "eth0" ethernet (rk_gmac-dwmac), xx:xx:xx:xx:xx:xx, hw, mtu 1500 master br0 eth1: unavailable "Realtek 10/100/1G/2.5G" ethernet (r8152), 6xx:xx:xx:xx:xx:xx, hw, mtu 1500 vethpivccu: unmanaged "vethpivccu" ethernet (veth), xx:xx:xx:xx:xx:xx, sw, mtu 1500 lo: unmanaged "lo" loopback (unknown), xx:xx:xx:xx:xx:xx, sw, mtu 65536 DNS configuration: servers: xx.xx.xx.xx domains: fritz.box interface: br0 servers: fd00::xxxx:xxxx:xxxx:xxxx interface: br0 Use "nmcli device show" to get complete information about known devices and "nmcli connection show" to get an overview on active connection profiles. Consult nmcli(1) and nmcli-examples(7) manual pages for complete usage details. The resulting system works on SD, but linux needs to be downgraded to 5.10.43 (as described above) before you transfer the system from SD to emmc.

Hi @lanefu - do you mean whether or not a buster or bullseye system on emmc that is updated from Armbian 21.05.4 to 21.08.1 has non matching UUIDs in the three locations ? What I can tell is that boot.scr does not contain a UUID at all. The device /dev/mmcblk0p1 is specified instead (in boot.cmd) - also in the fresh Buster image on the download page. UUIDs are present in /boot/armbianEnv.txt and /etc/fstab. If they are correctly set to the UUID of the root filesystem, a previous Buster 21.05.4 installation was running fine on SD and on emmc. The "emmc issue" occurs after the update from 21.05.4 to 21.08.1 even if the UUIDs in /boot/armbianEnv.txt and /etc/fstab are correctly set to the UUID of the root file system on emmc.

I own a Helios64 too and I store data on it I really do not want to loose. I love this thing ! But It is in fact ridiculous if updates are provided having the potential to render your setup unusable or even to loose your data. After Kobol stopped operations my immediate reaction a was to close the Armbian update channel and to test potential Armbian updates myself - and not on my main system. Instead of turning away from Armbian I try to support Helios64 on this platform. As long as Kobol operations are stopped we need to build and test linux kernels ourselves before updating to any new linux version. However, there is a stable kernel 5.10.43 for Helios64 (in particular with the ondemand governor active) and it is the kernel branch Debian Bullseye is using. With this setup it should be already possible to operate Helios64 for the next 2-5 years in a stable manner by just receiving updates through the Debian Bullseye channel - with occasional linux updates compiled and tested by members of this forum. Everybody is invited to contribute.

Armbian is a community driven project. This is not about trying to find someone to blame, but trying to solve issues by pointing to alleged sources of the trouble. And indeed community support is there as you can see in this thread - it could be stronger though.

@aprayoga Thank you for looking into the emmc issue in the latest Armbian release ! So far there were at least two observations: @alchemist observed a month ago, that several Armbian patches did not compile with newer kernels and he assumed that this may have be the reason. @Igor states that someone may have pushed bad code upstream.

The easiest way to downgrade linux on emmc now would be to copy those files to /mnt/system (root directory of your emmc) - and then change root with 'chroot /mnt/system' and install the packages with 'dpkg -i *.deb' (while your active system is on SD). You can leave the chrooted environment by typing 'exit'. emmc should now be bootable again. If not, you need to update the bootloader on emmc as described earlier in this thread.

This was already answered in this thread.

emmc is accessible with linux 5.10.43. The emmc partition is not write protected in any way - it is simply not writable with linux 5.10.60. In order to recover you will need to boot a system from SD with linux 5.10.43. If you have a valid backup of your emmc on SD - just boot from that one and use armbian-config to copy it to emmc (System&security -> Install to -> system on emmc, boot from emmc).

There is a possibility discussed in the parallel thread link. You also could boot a fresh Armbian 21.05.4 off SD and rsync with it the content from emmc to another bootable SD. Then you continue to downgrade linux on that second SD (booted) ... and rsync the result back to emmc. Maybe somebody else could explain how to downgrade the kernel on emmc using a chrooted environment.

@bunducafeYou could try to update the bootloader on emmc first. @IcerJo claims that this would make it writable again. Then you could downgrade linux on emmc to 21.05.4. There is no need to format emmc - alternatively you could also repair errors in the filesystem with 'fsck -f /dev/mmcblk2p1'. You will see that something needs to be corrected. If you trust that everything is fine after that you are done. EDIT: This does not work - since rewriting the bootloader does not solve the issue. The intention of this thread was to show that Buster can be upgraded to Bullseye and that it is stable with Armbian 21.05.4 with linux-5.10.43. Issues arise if Buster or Bullseye installations are updated from 21.05.4 to Armbian 21.08.1 with linux-5.10.60 against the recommendations earlier in this thread, because emmc will not be accessible anymore without i/o errors. Edit: You also could boot a fresh Armbian 21.05.4 off SD and rsync with it the content from emmc to another bootable SD. Then you continue to downgrade linux on that second SD (booted) ... and rsync the result back to emmc. Maybe somebody else could explain how to downgrade the kernel on emmc using a chrooted environment.

First I would downgrade the system on SD to 21.05.4 (you can access emmc again without i/o errors), then format the partition on emmc 'mkfs.ext4 /dev/mmcblk2p1' (to get rid of potentially corrupt content) and rsync the content from sd to emmc (assuming that the sd contains a valid copy of your system on emmc). For the last step you can adapt the script I use (it is a modified Armbian script). I did no use any armbian-config routines this time. You may need to update the bootloader on emmc too. # cat copytoemmc.sh #!/bin/bash # Check if user is root if [ $(id -u) != "0" ]; then echo "Error: You must be root to run this script." exit 1 fi cat > install-exclude <<EOF /dev/* /proc/* /sys/* /media/data1/* /media/data2/* /media/data3/* /media/data4/* /media/data5/* /mnt/sd/* /mnt/emmc/* /mnt/ssd/* /mnt/usb/* /mnt/hd/* /run/* # /tmp/* # /root/* EOF exec 2>/dev/null umount /mnt/emmc exec 2>&1 mount /dev/mmcblk2p1 /mnt/emmc rsync -avxSE --delete --exclude-from="install-exclude" / /mnt/emmc # change fstab sed -e 's/UUID=< insert uuid of sd >/UUID=< insert uuid of emmc >/g' -i /mnt/emmc/etc/fstab sed -e 's/UUID=< insert uuid of sd >/UUID=< insert uuid of emmc >/g' -i /mnt/emmc/boot/armbianEnv.txt umount /mnt/emmc rm install-exclude echo "All done."

@bunducafe dpkg -i *.deb did the job; @IcerJo I would not trust a "temporary" fix I don't really understand. Helios64 saves your data...

In order to roll back the kernel you just need to install those packages (on sd): http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/

You are using kernel 5.10.60 (Armbian 21.08.1). Several Armbian patches did not compile with this version of the kernel - it is therefore unstable (see the parallel thread - upgrading to Bullseye). The kernel panic occurred after 150372 seconds = 41.77 hours of operation !

There is a severe issue in Armbian 21.08.1 with kernel 5.10.60 on Helios64 - both in Buster (according to @BipBip1981) and in Bullseye: emmc is not accessible anymore - i/o errors occur, and all those installations on emmc are therefore broken after the update. The same bug was observed weeks ago by @alchemist - the reason was that some Armbian patches do not compile anymore with recent kernels. If linux-5.10.43 is kept, Bullseye is perfectly stable on Helios64. Would you therefore roll back the current kernel versions to 5.10.43 in Buster and Bullseye images ? In the meantime I had a look at the bootloader images in the file system after the update to 21.08.1. They are still dated July 24th and can be used to update the bootloader on emmc: # cd /usr/lib/linux-u-boot-helios64-current_21.08.1_arm64 # ls -la total 4192 drwxrwxr-x 2 root root 4096 Jul 25 19:59 . drwxr-xr-x 80 root root 4096 Aug 27 12:50 .. -rw-rw-r-- 1 root root 206844 Jul 24 14:11 idbloader.bin -rw-rw-r-- 1 root root 4194304 Jul 24 14:11 trust.bin -rw-rw-r-- 1 root root 4194304 Jul 24 14:11 uboot.img # dd if=idbloader.bin of=/dev/mmcblk2 seek=64 conv=notrunc # dd if=uboot.img of=/dev/mmcblk2 seek=16384 conv=notrunc # dd if=trust.bin of=/dev/mmcblk2 seek=24576 conv=notrunc # reboot now P.S.: I am happy to test any release candidates (in particular new kernel packages and Armbian Bullseye images).