Upgrading to Bullseye (troubleshooting Armbian 21.08.1)


ebin-dev
 Share

11 11

Recommended Posts

vor 16 Stunden schrieb ebin-dev:

@bunducafeYou could try to update the bootloader on emmc first. @IcerJo claims that this would make it writable again. Then you could downgrade linux on emmc to 21.05.4. There is no need to format emmc - alternatively you could also repair errors in the filesystem with 'fsck -f /dev/mmcblk2p1'. You will see that something needs to be corrected. If you trust that everything is fine after that you are done.

 

The intention of this thread was to show that Buster can be upgraded to Bullseye and that it is stable with Armbian 21.05.4 with linux-5.10.43.

 

Issues arise if Buster or Bullseye installations are updated from 21.05.4 to Armbian 21.08.1 with linux-5.10.60 against the recommendations earlier in this thread, because emmc will not be accessible anymore without i/o errors.

 

Edit: You also could boot a fresh Armbian 21.05.4 off  SD and rsync with it the content from emmc to another bootable SD. Then you continue to downgrade linux on that second SD (booted)  ... and rsync the result back to emmc.

Maybe somebody else could explain how to downgrade the kernel on emmc using a chrooted environment.

 

 

I unfortunately ran fsck last night as mentioned by you, which probably does not boot cleanly or I no longer have ssh access.
Maybe I have to connect it to the monitor and check what I see in the bootlog.

If someone could offer a step by step guide on how to recover I would be very grateful!

Link to post
Share on other sites

Armbian is a community driven open source project. Do you like to contribute your code?

49 minutes ago, TDCroPower said:

If someone could offer a step by step guide on how to recover I would be very grateful!

 

emmc is accessible with linux 5.10.43. The emmc partition is not write protected in any way - it is simply not writable with linux 5.10.60.

In order to recover you will need to boot a system from SD with linux 5.10.43. If you have a valid backup of your emmc on SD - just boot from that one and use armbian-config to copy it to emmc (System&security -> Install to -> system on emmc, boot from emmc).

Link to post
Share on other sites

vor 17 Minuten schrieb ebin-dev:

 

emmc is accessible with linux 5.10.43. The emmc partition is not write protected in any way - it is simply not writable with linux 5.10.60.

In order to recover you will need to boot a system from SD with linux 5.10.43. If you have a valid backup of your emmc on SD - just boot from that one and use armbian-config to copy it to emmc (System&security -> Install to -> system on emmc, boot from emmc).

I don't have a clean backup of the eMMC unfortunately, will probably have to introduce in the future to have a backup for such a step.

I found the following tutorial to clone the system from the emmc to a connected media....
https://averagelinuxuser.com/backup-and-restore-your-linux-system-with-rsync/

After that I would have to somehow downgrade my emmc backup on the media?
Does anyone have a tip which files have to be replaced?

After that I should clone from the fixed medium to a new SD and boot from it, now I should see my system again or?
And then in the armbian-config with...
System&security -> Install to -> system on emmc, boot from emmc
... to clone the image again on the emmc...

https://wiki.kobol.io/helios64/install/transfer/

Link to post
Share on other sites

46 minutes ago, TDCroPower said:

I don't have a clean backup of the eMMC unfortunately, will probably have to introduce in the future to have a backup for such a step.

I found the following tutorial to clone the system from the emmc to a connected media....
https://averagelinuxuser.com/backup-and-restore-your-linux-system-with-rsync/

After that I would have to somehow downgrade my emmc backup on the media?
Does anyone have a tip which files have to be replaced?

After that I should clone from the fixed medium to a new SD and boot from it, now I should see my system again or?
And then in the armbian-config with...
System&security -> Install to -> system on emmc, boot from emmc
... to clone the image again on the emmc...

https://wiki.kobol.io/helios64/install/transfer/

Thank you for the Rsync guide, I'm going to give this a try real quick, I had to modify the parameters since i had my usb SD card reader mounted as /media/USB, the rsync just finished. I'm going to shutdown, and pop the SD Card in and see what happens :)

 

Link to post
Share on other sites

ok i got a little further.
I wrote the armbian image 21.02.3 with 5.10.21-rockchip64 on a SD and triggered the boot from microSD via jumper 10...

https://wiki.kobol.io/helios64/troubleshoot/#how-to-force-boot-from-microsd

 

i am back on the box and was able to mount the emmc under /dev/mmcblk2p1...

 

root@helios64:~# ll /mnt/
total 0
root@helios64:~# mkdir -p /mnt/system
root@helios64:~# mount /dev/mmcblk2p1 /mnt/system
root@helios64:~# cd /mnt/system/
root@helios64:/mnt/system# ll
total 80
lrwxrwxrwx   1 root root     7 Aug 30  2020 bin -> usr/bin
drwxr-xr-x   3 root root  4096 Sep  1 03:01 boot
drwxr-xr-x   2 root root  4096 Oct 15  2020 dev
drwxr-xr-x 110 root root 12288 Sep  1 03:01 etc
drwxr-xr-x   2 root root  4096 Sep 22  2020 export
drwxr-xr-x   5 root root  4096 Jun 10 04:12 home
lrwxrwxrwx   1 root root     7 Aug 30  2020 lib -> usr/lib
drwx------   5 root root  4096 Oct  5  2020 lost+found
drwxr-xr-x   4 root root  4096 Oct 15  2020 media
drwxr-xr-x   2 root root  4096 Feb  9  2021 mnt
drwxr-xr-x   4 root root  4096 Dec  2  2020 opt
dr-xr-xr-x   2 root root  4096 Oct 15  2020 proc
drwx------   7 root root  4096 Aug 31 23:36 root
drwxr-xr-x   2 root root  4096 Oct 15  2020 run
lrwxrwxrwx   1 root root     8 Aug 30  2020 sbin -> usr/sbin
drwxrwxr-x   2 root root  4096 Oct  5  2020 selinux
drwxr-xr-x   6 root root  4096 Jun 10 03:18 srv
dr-xr-xr-x   2 root root  4096 Oct 15  2020 sys
lrwxrwxrwx   1 root root    42 Sep  1 03:01 thermal_zone0 -> /sys/devices/virtual/thermal/thermal_zone0
drwxrwxrwt   2 root root  4096 Oct 15  2020 tmp
drwxr-xr-x  12 root root  4096 Nov 27  2020 usr
drwxr-xr-x  14 root root  4096 Oct  6  2020 var
root@helios64:/mnt/system#

 

remains only the question how/where I install the packages that were described here...
https://forum.armbian.com/topic/18855-upgrading-to-bullseye/?tab=comments#comment-128189

 

maybe it has something to do with the boot/ directory....

root@helios64:/mnt/system# ll boot/
total 64696
-rw-r--r-- 1 root root      166 Sep  1 03:01 armbianEnv.txt
-rw-r--r-- 1 root root     1536 Oct 15  2020 armbian_first_run.txt.template
-rw-r--r-- 1 root root    38518 Oct 15  2020 boot.bmp
-rw-r--r-- 1 root root     3113 Oct 15  2020 boot.cmd
-rw-r--r-- 1 root root     3185 Oct 15  2020 boot.scr
-rw-r--r-- 1 root root   221489 Aug 25 20:56 config-5.10.60-rockchip64
lrwxrwxrwx 1 root root       22 Sep  1 02:31 dtb -> dtb-5.10.60-rockchip64
drwxr-xr-x 6 root root     4096 Sep  1 02:30 dtb-5.10.60-rockchip64
lrwxrwxrwx 1 root root       26 Sep  1 02:31 Image -> vmlinuz-5.10.60-rockchip64
-rw-r--r-- 1 root root 15767106 Sep  1 02:32 initrd.img-5.10.60-rockchip64
-rw-r--r-- 1 root root  5839373 Aug 25 20:56 System.map-5.10.60-rockchip64
lrwxrwxrwx 1 root root       26 Sep  1 02:32 uInitrd -> uInitrd-5.10.60-rockchip64
-rw-r--r-- 1 root root 15767170 Sep  1 02:32 uInitrd-5.10.60-rockchip64
-rw-r--r-- 1 root root 28580352 Aug 25 20:56 vmlinuz-5.10.60-rockchip64
root@helios64:/mnt/system#

 

Link to post
Share on other sites

1 hour ago, TDCroPower said:

remains only the question how/where I install the packages that were described here...

 

The easiest way to downgrade linux on emmc now would be to copy those files to /mnt/system (root directory of your emmc) - and then change root with  'chroot /mnt/system' and install the packages with 'dpkg -i *.deb'   (while your active system is on SD).

 

You can leave the chrooted environment by typing 'exit'. emmc should now be bootable again. If not, you need to update the bootloader on emmc as described earlier in this thread.

Link to post
Share on other sites

I opted to just setup an SD Card with the latest, and redo my OMV install on it, since that's technically all I use it for at this time, I was able to get the old system booted with no problems by pushing re-pushing the bootloader but wasn't able to install the Deb files, and I am not that confident in my linux skill set, though i learned a few new tips and tricks during the troubleshooting! I'll repush to EMMC once issues have eventually been resolved, or i may downgrade the kernel and push, i'll see later.

 

Thank you @ebin-dev for your advice and assistance and thank you @TDCroPower for the RSync article!

Link to post
Share on other sites

It worked, I have my eMMC system back!

 

Here are all the steps in case someone else has the problem and wants to repair his eMMC image...

 

1. download a previous Helios64 image from here...
https://wiki.kobol.io/download/
2. install the image on a microSD e.g. for Windows and macOS there is Etcher...
https://www.balena.io/etcher/
3a. boot your Helios64 with the microSD and Jumper 10, see here...
https://wiki.kobol.io/helios64/troubleshoot/#how-to-force-boot-from-microsd
3b. Alternative: from the serial console press a key on the keyboard while u-boot start. You will get the u-boot prompt.

From this prompt write and press enter. Helios64 will boot from SD... (thx @prahal)

run bootcmd_mmc1

 

4. if you are logged in as root user (normal users must put a sudo in front of the commands) execute the following commands...

root@helios64:~# mkdir -p /mnt/system
root@helios64:~# mount /dev/mmcblk2p1 /mnt/system
root@helios64:~# cd /mnt/system/


5. The contents of the /mnt/system directory should now be filled with the contents of your eMMC....

root@helios64:/mnt/system# ll
total 80
lrwxrwxrwx   1 root root     7 Aug 30  2020 bin -> usr/bin
drwxr-xr-x   3 root root  4096 Sep  1 03:01 boot
drwxr-xr-x   2 root root  4096 Oct 15  2020 dev
drwxr-xr-x 110 root root 12288 Sep  1 03:01 etc
drwxr-xr-x   2 root root  4096 Sep 22  2020 export
drwxr-xr-x   5 root root  4096 Jun 10 04:12 home
lrwxrwxrwx   1 root root     7 Aug 30  2020 lib -> usr/lib
drwx------   5 root root  4096 Oct  5  2020 lost+found
drwxr-xr-x   4 root root  4096 Oct 15  2020 media
drwxr-xr-x   2 root root  4096 Feb  9  2021 mnt
drwxr-xr-x   4 root root  4096 Dec  2  2020 opt
dr-xr-xr-x   2 root root  4096 Oct 15  2020 proc
drwx------   7 root root  4096 Aug 31 23:36 root
drwxr-xr-x   2 root root  4096 Oct 15  2020 run
lrwxrwxrwx   1 root root     8 Aug 30  2020 sbin -> usr/sbin
drwxrwxr-x   2 root root  4096 Oct  5  2020 selinux
drwxr-xr-x   6 root root  4096 Jun 10 03:18 srv
dr-xr-xr-x   2 root root  4096 Oct 15  2020 sys
lrwxrwxrwx   1 root root    42 Sep  1 03:01 thermal_zone0 -> /sys/devices/virtual/thermal/thermal_zone0
drwxrwxrwt   2 root root  4096 Oct 15  2020 tmp
drwxr-xr-x  12 root root  4096 Nov 27  2020 usr
drwxr-xr-x  14 root root  4096 Oct  6  2020 var

 

6. now download the 3 old packages...

root@helios64:/mnt/system# wget http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/linux-dtb-current-rockchip64_21.05.4_arm64.deb
root@helios64:/mnt/system# wget http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/linux-headers-current-rockchip64_21.05.4_arm64.deb
root@helios64:/mnt/system# wget http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/linux-image-current-rockchip64_21.05.4_arm64.deb

 

7. now changes the root directory...

root@helios64:/mnt/system# chroot /mnt/system
root@helios64:/# pwd
/
root@helios64:/# ll
total 50996
lrwxrwxrwx   1 root root        7 Aug 30  2020 bin -> usr/bin
drwxr-xr-x   3 root root     4096 Sep  1 14:56 boot
drwxr-xr-x   2 root root     4096 Sep  1 16:15 dev
drwxr-xr-x 110 root root    12288 Sep  1 03:01 etc
drwxr-xr-x   2 root root     4096 Sep 22  2020 export
drwxr-xr-x   5 root root     4096 Jun 10 04:12 home
lrwxrwxrwx   1 root root        7 Aug 30  2020 lib -> usr/lib
-rw-r--r--   1 root root   314304 Jul  8 19:32 linux-dtb-current-rockchip64_21.05.4_arm64.deb
-rw-r--r--   1 root root 11527696 Jul  8 19:32 linux-headers-current-rockchip64_21.05.4_arm64.deb
-rw-r--r--   1 root root 40290884 Jul  8 19:33 linux-image-current-rockchip64_21.05.4_arm64.deb
drwx------   5 root root     4096 Oct  5  2020 lost+found
drwxr-xr-x   4 root root     4096 Oct 15  2020 media
drwxr-xr-x   2 root root     4096 Feb  9  2021 mnt
drwxr-xr-x   4 root root     4096 Dec  2  2020 opt
dr-xr-xr-x   2 root root     4096 Oct 15  2020 proc
drwx------   7 root root     4096 Aug 31 23:36 root
drwxr-xr-x   2 root root     4096 Oct 15  2020 run
lrwxrwxrwx   1 root root        8 Aug 30  2020 sbin -> usr/sbin
drwxrwxr-x   2 root root     4096 Oct  5  2020 selinux
drwxr-xr-x   6 root root     4096 Jun 10 03:18 srv
dr-xr-xr-x   2 root root     4096 Oct 15  2020 sys
lrwxrwxrwx   1 root root       42 Sep  1 03:01 thermal_zone0 -> /sys/devices/virtual/thermal/thermal_zone0
drwxrwxrwt   2 root root     4096 Oct 15  2020 tmp
drwxr-xr-x  12 root root     4096 Nov 27  2020 usr
drwxr-xr-x  14 root root     4096 Oct  6  2020 var

 

8. now installs the downloaded packages...

root@helios64:/# dpkg -i *.deb
dpkg: warning: downgrading linux-dtb-current-rockchip64 from 21.08.1 to 21.05.4
(Reading database ... 62558 files and directories currently installed.)
Preparing to unpack linux-dtb-current-rockchip64_21.05.4_arm64.deb ...
Unpacking linux-dtb-current-rockchip64 (21.05.4) over (21.08.1) ...
Selecting previously unselected package linux-headers-current-rockchip64.
Preparing to unpack linux-headers-current-rockchip64_21.05.4_arm64.deb ...
Unpacking linux-headers-current-rockchip64 (21.05.4) ...
dpkg: warning: downgrading linux-image-current-rockchip64 from 21.08.1 to 21.05.4
Preparing to unpack linux-image-current-rockchip64_21.05.4_arm64.deb ...
update-initramfs: Deleting /boot/initrd.img-5.10.60-rockchip64
Removing obsolete file uInitrd-5.10.60-rockchip64
stat: cannot stat '/proc/1/root/.': No such file or directory
Unpacking linux-image-current-rockchip64 (21.05.4) over (21.08.1) ...
Setting up linux-dtb-current-rockchip64 (21.05.4) ...
Setting up linux-headers-current-rockchip64 (21.05.4) ...
Compiling headers - please wait ...
grep: /proc/cpuinfo: No such file or directory
grep: /proc/cpuinfo: No such file or directory
Setting up linux-image-current-rockchip64 (21.05.4) ...
update-initramfs: Generating /boot/initrd.img-5.10.43-rockchip64
W: Couldn't identify type of root file system for fsck hook
update-initramfs: Converting to u-boot format

 

9. reset the root change with exit and restart your helios64 with reboot...

root@helios64:/# exit
exit
root@helios64:/mnt/system# reboot

 

10. after a reboot you should now boot from the eMMC again...

root@192.168.180.5's password:
 _   _      _ _            __   _  _
| | | | ___| (_) ___  ___ / /_ | || |
| |_| |/ _ \ | |/ _ \/ __| '_ \| || |_
|  _  |  __/ | | (_) \__ \ (_) |__   _|
|_| |_|\___|_|_|\___/|___/\___/   |_|

Welcome to Armbian 21.08.1 Buster with Linux 5.10.43-rockchip64

No end-user support: built from trunk

System load:   74%           	Up time:       3 min
Memory usage:  34% of 3.77G  	IP:	       172.18.0.1 172.19.0.1 172.20.0.1 172.17.0.1 192.168.180.5
CPU temp:      66°C           	Usage of /:    87% of 15G

[ 0 security updates available, 3 updates total: apt upgrade ]
Last check: 2021-09-01 16:38

[ General system configuration (beta): armbian-config ]

Last login: Wed Sep  1 16:26:23 2021
root@helios64:~#
root@helios64:~# uname -a
Linux helios64 5.10.43-rockchip64 #21.05.4 SMP PREEMPT Wed Jun 16 08:02:12 UTC 2021 aarch64 GNU/Linux
root@helios64:~#

 

11. if the eMMC was used you can see with mount...

root@helios64:~# mount | grep /dev/mmc
/dev/mmcblk2p1 on / type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/log type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/tmp type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/lib/openmediavault/rrd type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/spool type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/lib/rrdcached type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/lib/monit type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/cache/samba type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)

 

Link to post
Share on other sites

For step 3 there is a way out of tweaking Jumper 10 if you want to boot force boot on SD.

From the serial console press a key on the keyboard while u-boot start. You will get the u-boot prompt.

from this prompt write:

run bootcmd_mmc1

and press enter. Helios64 will boot from SD.

To force boot from eMMC instead of SD, do the opposite:

run bootcmd_mmc0

 

This saved me from opening the box.

Link to post
Share on other sites

Since today 7 AM, I read a huge amount of "read-only" error messages in the syslog and cannot do anything on the root system. I installed Armbian 21.08.1 onto /dev/mmcblk2p1, with just 24% of the space in use. Rebooting doesn't help and the system is basically unable to do any operation like getting updates or creating files.

 

Any idea what's going on? Is the built-in storage dying?

Link to post
Share on other sites

49 minutes ago, Schroedingers Cat said:

Any idea what's going on? Is the built-in storage dying?

No, the storage is fine - Armbian 21.08.1 is yet another update that hoses something on the Helios64. Last time it was the fans, the time before that the 2.5Gb Ethernet port (twice), and this time it's the eMMC.

 

You'll need to follow the instructions upthread to downgrade the kernel to get the eMMC working again. Then cross your fingers it gets fixed at some point in the future.

Link to post
Share on other sites

16 hours ago, Gareth Halfacree said:

Armbian 21.08.1 is yet another update that hoses something on the Helios64

Something that's probably gonna keep happening if nobody is using the beta repository (nightly/edge) and reporting errors before they make it to the current version. 

FYI there is no Helios64 in Armbians testing "facility"

Link to post
Share on other sites

@aprayoga Thank you for looking into the emmc issue in the latest Armbian release ! 

 

So far there were at least two observations:

@alchemist observed a month ago, that several Armbian patches did not compile with newer kernels and he assumed that this may have be the reason.

@Igor states that someone may have pushed bad code upstream.

Link to post
Share on other sites

13 minutes ago, ebin-dev said:

@aprayoga Thank you for looking into the emmc issue in the latest Armbian release ! 

 

So far there were at least two observations:

@alchemist observed a month ago, that several Armbian patches did not compile with newer kernels and he assumed that this may have be the reason.

@Igor states that someone may have pushed bad code upstream.

 

Hi!

 

If I remove the patches that don't compile and adapt the other ones, the resulting kernel is having issues with eMMC or unbootable kernel (u-boot does bootloops).

 

I even tried some older versions that did work (i.e. recompile one of my ancient kernels with vanilla sources + gentoo patches +  armbian patches) and it fails. But I didn't do it with method, I wanted to have a stable kernel to have back my "production" home server.

 

The kernel I run is now " 5.10.34-gentoo #1 SMP Mon May 3 10:47:06 CEST 2021 aarch64 GNU/Linux", the version I had in one backup. After May 3 2021 I had issues with u-boot (kernel did not boot) or not compiling patches.

 

I can help you (armbian poeple) to point the problems. In fact, I would like to rebuild all the provides binaries (u-boot, kernel) myself in order to understand how they are working and configured.

 

 

Link to post
Share on other sites

3 hours ago, Heisath said:

Something that's probably gonna keep happening if nobody is using the beta repository (nightly/edge) and reporting errors before they make it to the current version.

Oh, I've tried reporting problems. Igor told me (and many, many others) that if I wasn't paying €50 a month to Armbian he wasn't interested, so I stopped.

Link to post
Share on other sites

1 hour ago, ebin-dev said:

 

@Igor states that someone may have pushed bad code upstream.

 

I can remember that an upstream change from mainline rendered my ODROID-C2 and Khadas Vim / Vim2 unusable due to emmc timing / frequency changes in the corresponding dts files. I'm not sure if these changes (in rc? revisions) even made it into the release branch. So it's not easy to find the right one to blame.

Link to post
Share on other sites

30 minutes ago, umiddelb said:

So it's not easy to find the right one to blame.

 

Armbian is a community driven project. This is not about trying to find someone to blame, but trying to solve issues by pointing to alleged sources of the trouble.

 

And indeed community support is there as you can see in this thread - it could be stronger though.

 

Link to post
Share on other sites

On 9/1/2021 at 5:08 PM, TDCroPower said:

It worked, I have my eMMC system back!

 

Here are all the steps in case someone else has the problem and wants to repair his eMMC image...

 

1. download a previous Helios64 image from here...
https://wiki.kobol.io/download/
2. install the image on a microSD e.g. for Windows and macOS there is Etcher...
https://www.balena.io/etcher/
3a. boot your Helios64 with the microSD and Jumper 10, see here...
https://wiki.kobol.io/helios64/troubleshoot/#how-to-force-boot-from-microsd
3b. Alternative: from the serial console press a key on the keyboard while u-boot start. You will get the u-boot prompt.

From this prompt write and press enter. Helios64 will boot from SD... (thx @prahal)

run bootcmd_mmc1

 

4. if you are logged in as root user (normal users must put a sudo in front of the commands) execute the following commands...

root@helios64:~# mkdir -p /mnt/system
root@helios64:~# mount /dev/mmcblk2p1 /mnt/system
root@helios64:~# cd /mnt/system/


5. The contents of the /mnt/system directory should now be filled with the contents of your eMMC....

root@helios64:/mnt/system# ll
total 80
lrwxrwxrwx   1 root root     7 Aug 30  2020 bin -> usr/bin
drwxr-xr-x   3 root root  4096 Sep  1 03:01 boot
drwxr-xr-x   2 root root  4096 Oct 15  2020 dev
drwxr-xr-x 110 root root 12288 Sep  1 03:01 etc
drwxr-xr-x   2 root root  4096 Sep 22  2020 export
drwxr-xr-x   5 root root  4096 Jun 10 04:12 home
lrwxrwxrwx   1 root root     7 Aug 30  2020 lib -> usr/lib
drwx------   5 root root  4096 Oct  5  2020 lost+found
drwxr-xr-x   4 root root  4096 Oct 15  2020 media
drwxr-xr-x   2 root root  4096 Feb  9  2021 mnt
drwxr-xr-x   4 root root  4096 Dec  2  2020 opt
dr-xr-xr-x   2 root root  4096 Oct 15  2020 proc
drwx------   7 root root  4096 Aug 31 23:36 root
drwxr-xr-x   2 root root  4096 Oct 15  2020 run
lrwxrwxrwx   1 root root     8 Aug 30  2020 sbin -> usr/sbin
drwxrwxr-x   2 root root  4096 Oct  5  2020 selinux
drwxr-xr-x   6 root root  4096 Jun 10 03:18 srv
dr-xr-xr-x   2 root root  4096 Oct 15  2020 sys
lrwxrwxrwx   1 root root    42 Sep  1 03:01 thermal_zone0 -> /sys/devices/virtual/thermal/thermal_zone0
drwxrwxrwt   2 root root  4096 Oct 15  2020 tmp
drwxr-xr-x  12 root root  4096 Nov 27  2020 usr
drwxr-xr-x  14 root root  4096 Oct  6  2020 var

 

6. now download the 3 old packages...

root@helios64:/mnt/system# wget http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/linux-dtb-current-rockchip64_21.05.4_arm64.deb
root@helios64:/mnt/system# wget http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/linux-headers-current-rockchip64_21.05.4_arm64.deb
root@helios64:/mnt/system# wget http://armbian.hosthatch.com/apt/pool/main/l/linux-5.10.43-rockchip64/linux-image-current-rockchip64_21.05.4_arm64.deb

 

7. now changes the root directory...

root@helios64:/mnt/system# chroot /mnt/system
root@helios64:/# pwd
/
root@helios64:/# ll
total 50996
lrwxrwxrwx   1 root root        7 Aug 30  2020 bin -> usr/bin
drwxr-xr-x   3 root root     4096 Sep  1 14:56 boot
drwxr-xr-x   2 root root     4096 Sep  1 16:15 dev
drwxr-xr-x 110 root root    12288 Sep  1 03:01 etc
drwxr-xr-x   2 root root     4096 Sep 22  2020 export
drwxr-xr-x   5 root root     4096 Jun 10 04:12 home
lrwxrwxrwx   1 root root        7 Aug 30  2020 lib -> usr/lib
-rw-r--r--   1 root root   314304 Jul  8 19:32 linux-dtb-current-rockchip64_21.05.4_arm64.deb
-rw-r--r--   1 root root 11527696 Jul  8 19:32 linux-headers-current-rockchip64_21.05.4_arm64.deb
-rw-r--r--   1 root root 40290884 Jul  8 19:33 linux-image-current-rockchip64_21.05.4_arm64.deb
drwx------   5 root root     4096 Oct  5  2020 lost+found
drwxr-xr-x   4 root root     4096 Oct 15  2020 media
drwxr-xr-x   2 root root     4096 Feb  9  2021 mnt
drwxr-xr-x   4 root root     4096 Dec  2  2020 opt
dr-xr-xr-x   2 root root     4096 Oct 15  2020 proc
drwx------   7 root root     4096 Aug 31 23:36 root
drwxr-xr-x   2 root root     4096 Oct 15  2020 run
lrwxrwxrwx   1 root root        8 Aug 30  2020 sbin -> usr/sbin
drwxrwxr-x   2 root root     4096 Oct  5  2020 selinux
drwxr-xr-x   6 root root     4096 Jun 10 03:18 srv
dr-xr-xr-x   2 root root     4096 Oct 15  2020 sys
lrwxrwxrwx   1 root root       42 Sep  1 03:01 thermal_zone0 -> /sys/devices/virtual/thermal/thermal_zone0
drwxrwxrwt   2 root root     4096 Oct 15  2020 tmp
drwxr-xr-x  12 root root     4096 Nov 27  2020 usr
drwxr-xr-x  14 root root     4096 Oct  6  2020 var

 

8. now installs the downloaded packages...

root@helios64:/# dpkg -i *.deb
dpkg: warning: downgrading linux-dtb-current-rockchip64 from 21.08.1 to 21.05.4
(Reading database ... 62558 files and directories currently installed.)
Preparing to unpack linux-dtb-current-rockchip64_21.05.4_arm64.deb ...
Unpacking linux-dtb-current-rockchip64 (21.05.4) over (21.08.1) ...
Selecting previously unselected package linux-headers-current-rockchip64.
Preparing to unpack linux-headers-current-rockchip64_21.05.4_arm64.deb ...
Unpacking linux-headers-current-rockchip64 (21.05.4) ...
dpkg: warning: downgrading linux-image-current-rockchip64 from 21.08.1 to 21.05.4
Preparing to unpack linux-image-current-rockchip64_21.05.4_arm64.deb ...
update-initramfs: Deleting /boot/initrd.img-5.10.60-rockchip64
Removing obsolete file uInitrd-5.10.60-rockchip64
stat: cannot stat '/proc/1/root/.': No such file or directory
Unpacking linux-image-current-rockchip64 (21.05.4) over (21.08.1) ...
Setting up linux-dtb-current-rockchip64 (21.05.4) ...
Setting up linux-headers-current-rockchip64 (21.05.4) ...
Compiling headers - please wait ...
grep: /proc/cpuinfo: No such file or directory
grep: /proc/cpuinfo: No such file or directory
Setting up linux-image-current-rockchip64 (21.05.4) ...
update-initramfs: Generating /boot/initrd.img-5.10.43-rockchip64
W: Couldn't identify type of root file system for fsck hook
update-initramfs: Converting to u-boot format

 

9. reset the root change with exit and restart your helios64 with reboot...

root@helios64:/# exit
exit
root@helios64:/mnt/system# reboot

 

10. after a reboot you should now boot from the eMMC again...

root@192.168.180.5's password:
 _   _      _ _            __   _  _
| | | | ___| (_) ___  ___ / /_ | || |
| |_| |/ _ \ | |/ _ \/ __| '_ \| || |_
|  _  |  __/ | | (_) \__ \ (_) |__   _|
|_| |_|\___|_|_|\___/|___/\___/   |_|

Welcome to Armbian 21.08.1 Buster with Linux 5.10.43-rockchip64

No end-user support: built from trunk

System load:   74%           	Up time:       3 min
Memory usage:  34% of 3.77G  	IP:	       172.18.0.1 172.19.0.1 172.20.0.1 172.17.0.1 192.168.180.5
CPU temp:      66°C           	Usage of /:    87% of 15G

[ 0 security updates available, 3 updates total: apt upgrade ]
Last check: 2021-09-01 16:38

[ General system configuration (beta): armbian-config ]

Last login: Wed Sep  1 16:26:23 2021
root@helios64:~#
root@helios64:~# uname -a
Linux helios64 5.10.43-rockchip64 #21.05.4 SMP PREEMPT Wed Jun 16 08:02:12 UTC 2021 aarch64 GNU/Linux
root@helios64:~#

 

11. if the eMMC was used you can see with mount...

root@helios64:~# mount | grep /dev/mmc
/dev/mmcblk2p1 on / type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/log type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/tmp type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/lib/openmediavault/rrd type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/spool type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/lib/rrdcached type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/lib/monit type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/mmcblk2p1 on /var/folder2ram/var/cache/samba type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)

 

 

For info, my MMC device includes 2 partitions. mmcblk2p1 includes /boot, and mmcblk2p2 contains linux folders.

As I performed the steps above by mounting mmcblk2p2 on /mnt/system, new linux image 5.10.43 was actually available in /mnt/system/boot.

However, I had to copy the files to mmcblk2p1 partition. Otherwise, my system starts on MMC without any change in linux version.

After generated the image, I apply the instructions below:

root@helios64:~# mkdir -p /mnt/boot
root@helios64:~# mount /dev/mmcblk1p2 /mnt/boot
root@helios64:~# cp -r /mnt/system/boot/* /mnt/boot/boot
root@helios64:~# reboot

 

Link to post
Share on other sites

11 hours ago, Gareth Halfacree said:

Oh, I've tried reporting problems. Igor told me

 

I have expected you will think about why I told you that. You can skip my rant below and read https://docs.armbian.com/User-Guide_FAQ/ Most of the answers are there.

 

I am commonly sad because I can't help people that are genuine helpful, friendly, respectful and are willing to repay when asking for help. This is the base to form a healthy relationship. People that understand they are asking for a favour.

 

As a person in any role I can't support putting a pressure to developers / doers / makers that are around, nor support suicidal behaviour. If I notice something like that is happening I have to take a side and support one, trying to reason the other. Since I can't afford to spent indefinite time (=money) for reasoning "customers" which shows to little respect to the platform we maintain and support (I understand anger from users because something failed)  ... while crushing is cheap and usually effective. If I pick your side, you will lost people that creates you value pretty quickly. Nevertheless we wrote some community rules which should help us all avoiding having conversations derailed in first place. "Bumping topic is not a nice thing to do, seeking for attention - keep your ego to yourself, ask politely without pressure, you who came here on this forum.". 

 

I would gladly deal with "you" on the level you expect, but there are too many of "you" any "you" are already having a huge https://en.wikipedia.org/wiki/Debt:_The_First_5000_Years while contributing 1000 x too little to match your wishes and demands. This is nothing to do with you personally. This is just a perception users have. 

Asking what you can do for us is probably far better strategy to get something in return. We have many known bugs, and no resources to deal with them. We have much much too many requests for attention and way too little time to deal with. Some ideas: https://forum.armbian.com/forum/54-help-wanted/ When I saw someone that is also busy and inpatient, asking just for some money is perhaps best? Even chances to get costs covered by a person who is making it are close to 0%, the same goes with an interest to help that person.

Link to post
Share on other sites

11 hours ago, Igor said:

We have many known bugs, and no resources to deal with them.

As I can see, which is why I am making sure to move away from Armbian - and I recommend anyone reading this to do the same. It's simply not a project you can rely upon for daily use. For messing around with as a hobbyist, sure, why not. But to actually rely upon? Impossible.

 

Especially when the founder doesn't understand that bug reports - especially bug reports from beta/nightly users, which have been specifically requested right here in this thread - have value to the project. With an attitude like that, Armbian will never become anything more than a hobbyist's curiosity.

 

I'll repeat for clarity, so others reading this thread don't misunderstand my meaning: this is going to keep happening. Future updates will continue to break the Helios64, and other devices, and there is nothing you can do about it. Armbian, as Igor says, simply doesn't have the resources to properly test its updates before release, much less actually respond to bug reports. You will not be able to rely on it to keep your Helios64 running and your data safe. If you're happy with that, by all means continue to use it; otherwise, I'd advise looking into alternative software and/or moving to a new NAS altogether.

Link to post
Share on other sites

1 hour ago, Gareth Halfacree said:

Future updates will continue to break the Helios64, and other devices, and there is nothing you can do about it.

 

I own a Helios64  too and I store data on it I really do not want to loose.  I love this thing !

But It is in fact ridiculous if updates are provided having the potential to render your setup unusable or even to loose your data.

 

After Kobol stopped operations my immediate reaction a was to close the Armbian update channel and to test potential Armbian updates myself -  and not on my main system.

 

Instead of turning away from Armbian I try to support Helios64 on this platform. As long as Kobol operations are stopped we need to build and test linux kernels ourselves before updating to any new linux version.

 

However, there is a stable kernel 5.10.43 for Helios64 (in particular with the ondemand governor active) and it is the kernel branch Debian Bullseye is using.

With this setup it should be already possible to operate Helios64 for the next 2-5 years in a stable manner by just receiving updates through the Debian Bullseye channel - with occasional linux updates compiled and tested by members of this forum.

 

Everybody is invited to contribute.

Link to post
Share on other sites

19 minutes ago, ebin-dev said:

But It is in fact ridiculous if updates are provided having the potential to render your setup unusable or even to loose your data.


I would like to prevent this to happen, but this is not as simple as it might looks like. Luckily I do have experiences from professional / corporate world and I know how they are approaching and what kind of resources are needed to move a little up. In the place I have cooperated we were working on a few of different devices, which complexity was a bit higher then our most complicated SBCs, but we had more then 100 full time people just for quality control. They were conducting and improving testing procedures round the clock, but well paying customers which developers never had chance to be pressured from, were keep sending bugs ...

Armbian has some primitive bug detection tooling and nobody dedicated on this. Also no budget to expand - I have very little time to do more then this, We would need at least a full time person of my know-how level and strong dedication just for testing areas plus help of a few volunteers. Just to make a small noticeable difference.

Link to post
Share on other sites

8 hours ago, lanefu said:

Is anybody able to tell if after the update the UUID for the rootfs no longer matches between boot.scr, /etc/fstab and the init image?

 

Hi @lanefu - do you mean whether or not a buster or bullseye system on emmc that is updated from Armbian 21.05.4 to 21.08.1 has non matching UUIDs in the three locations ?

 

What I can tell is that boot.scr does not contain a UUID at all. The device /dev/mmcblk0p1 is specified instead (in boot.cmd) - also in the fresh Buster image on the download page.

 

UUIDs are present in /boot/armbianEnv.txt and /etc/fstab. If they are correctly set to the UUID of the root filesystem, a previous Buster 21.05.4 installation was running fine on SD and on emmc.

 

The "emmc issue" occurs after the update from 21.05.4 to 21.08.1 even if the UUIDs in /boot/armbianEnv.txt and /etc/fstab are correctly set to the UUID of the root file system on emmc.

 

Link to post
Share on other sites

As the topic of this thread is upgrading Buster to Bullseye, I did some further testing in this context:

 

I would like to confirm that upgrading Buster installations to Bullseye also works fine if you use network-manager to manage your network interfaces (default), even if you have a bridge configured (using bridge-slave; binutils-bridge).

 

Using a Buster image from the Helios64 download page and following the instructions provided in the first message in this thread, the Buster image was successfully upgraded to Bullseye and the bridge configured is also  working (the first message in this thread is updated accordingly):

 

 
 _   _      _ _            __   _  _   
| | | | ___| (_) ___  ___ / /_ | || |  
| |_| |/ _ \ | |/ _ \/ __| '_ \| || |_ 
|  _  |  __/ | | (_) \__ \ (_) |__   _|
|_| |_|\___|_|_|\___/|___/\___/   |_|  
                                       
Welcome to Armbian 21.08.1 Buster with Linux 5.10.60-rockchip64

No end-user support: built from trunk

System load:   3%           	Up time:       1 min	
Memory usage:  4% of 3.77G  	IP:	       xx.xx.xx.xx 10.0.3.1
CPU temp:      42°C           	Usage of /:    8% of 29G    	
RX today:      518.9 MiB  	

[ General system configuration (beta): armbian-config ]

# nmcli
br0: connected to br0
        "br0"
        bridge, xx:xx:xx:xx:xx:B1, sw, mtu 1500
        ip4 default, ip6 default
        inet4 xx.xx.xx.55/24
        route4 0.0.0.0/0
        route4 xx.xx.xx.0/24
        inet6 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx/64
        inet6 fe80::xxxx:xxxx:xxxx:xxxx/64
        route6 2a02:xxxx:xxxx:xxxx::/62
        route6 2a02:xxxx:xxxx:xxxx::/64
        route6 ::/0
        route6 fe80::/64

lxcbr0: connected (externally) to lxcbr0
        "lxcbr0"
        bridge, xx:xx:xx:xx:xx:xx, sw, mtu 1500
        inet4 10.0.3.1/24
        route4 10.0.3.0/24
        route4 169.254.0.0/16

eth0: connected to bridge-slave-eth0
        "eth0"
        ethernet (rk_gmac-dwmac), xx:xx:xx:xx:xx:xx, hw, mtu 1500
        master br0

eth1: unavailable
        "Realtek 10/100/1G/2.5G"
        ethernet (r8152), 6xx:xx:xx:xx:xx:xx, hw, mtu 1500

vethpivccu: unmanaged
        "vethpivccu"
        ethernet (veth), xx:xx:xx:xx:xx:xx, sw, mtu 1500

lo: unmanaged
        "lo"
        loopback (unknown), xx:xx:xx:xx:xx:xx, sw, mtu 65536

DNS configuration:
        servers: xx.xx.xx.xx
        domains: fritz.box
        interface: br0

        servers: fd00::xxxx:xxxx:xxxx:xxxx
        interface: br0

Use "nmcli device show" to get complete information about known devices and
"nmcli connection show" to get an overview on active connection profiles.

Consult nmcli(1) and nmcli-examples(7) manual pages for complete usage details.

 

The resulting system works on SD, but linux needs to be downgraded to 5.10.43 (as described above) before you transfer the system from SD to emmc.

Link to post
Share on other sites

Hello,

 

I had updates freezed in armbian-config... and decided to de-freeze without looking further :wacko:.

Tried upgrading to latest kernel... and now cannot write to EMMC

Now I'm trying to downgrade, but cannot get there. I need please some help, 'cos don't want loose all my configurations (OMV, PLEX server, etc etc...)

I've written a new SD Card to boot from, using 5.10.35 armbian 21.05.1, and I'm able to mount the EMMC, see all files, and use dpkg to downgrade, but when I try to reboot, nothing happens: fans don't come up, ssh is unreachable... 

I have also access by serial port, if its needed.

 

My EMMC is now mounted under /mnt/system, and /mnt/system/boot shows:

 

root@helios64:/# ll /mnt/system/boot/
total 63708
-rw-r--r-- 1 root root      166 sep  5 14:27 armbianEnv.txt
-rw-r--r-- 1 root root     1536 oct 13  2020 armbian_first_run.txt.template
-rw-r--r-- 1 root root    38518 oct 13  2020 boot.bmp
-rw-r--r-- 1 root root     3113 oct 13  2020 boot.cmd
-rw-rw-r-- 1 root root     3185 oct 13  2020 boot.scr
-rw-r--r-- 1 root root   221432 may  7 15:53 config-5.10.35-rockchip64
lrwxrwxrwx 1 root root       22 sep  5 12:41 dtb -> dtb-5.10.35-rockchip64
drwxr-xr-x 6 root root     4096 sep  5 12:40 dtb-5.10.35-rockchip64
lrwxrwxrwx 1 root root       26 sep  5 12:42 Image -> vmlinuz-5.10.35-rockchip64
-rw-r--r-- 1 root root 15261102 sep  5 12:42 initrd.img-5.10.35-rockchip64
-rw-r--r-- 1 root root  5839770 may  7 15:53 System.map-5.10.35-rockchip64
lrwxrwxrwx 1 root root       26 sep  5 12:42 uInitrd -> uInitrd-5.10.35-rockchip64
-rw-r--r-- 1 root root 15261166 sep  5 12:42 uInitrd-5.10.35-rockchip64
-rw-r--r-- 1 root root 28582400 may  7 15:53 vmlinuz-5.10.35-rockchip64

 

Please note I have also updated armbian-firmware, but don't I don't know if that needs to be downgraded also.

 

Any ideas are appreciated.

 

Note: I have posted here as a response to @TDCroPower's instructions to downgrade, I hope not to be much off-topic.

 

Thank you

Link to post
Share on other sites

30 minutes ago, eClapton said:

My EMMC is now mounted under /mnt/system, and /mnt/system/boot shows:

 

All you need to do is this (see my earlier message):

 

The easiest way to downgrade linux on emmc now would be to copy those files to /mnt/system (root directory of your emmc) - and then change root with  'chroot /mnt/system' and install the packages with 'dpkg -i *.deb'   (while your active system is on SD).

 

You may also need to refresh your bootloader on emmc - as described in this thread.

 

P.S.: It would appear that you have mounted your SD card (/dev/mmcblk1p1) to /mnt/system and not your emmc (/dev/mmcblk2p1). You need to boot off SD.

Link to post
Share on other sites

  • ebin-dev changed the title to Upgrading to Bullseye (troubleshooting Armbian 21.08.1)
 Share

11 11