Does upgrading from Buster to Bullseye still break the system?

[mrjpaxton]

Hey guys, I figured it was time to ask again...

 

Does doing an `apt full-upgrade` from Buster to Bullseye break anything like before a year ago? I kind of want to do it, but not if the eMMC issues are still around. I really prefer not using the slower SD card method.

 

And i also want to ask about stability for people that still use a Kobol with Armbian on Bullseye. Any pertinent issues with anything? A few people on here have claimed that they have gotten the vanilla Debian kernel to run on the Helios64, but I'm not sure how... Sure would be great if the vanilla Debian distro worked on this board, but I doubt that'll ever happen...

 

I love my Kobol, and I am going to have a hard time finding a good replacement for it. i just really like having full CLI access, complete understanding of all the hardware, little to no proprietary bits, and also the open-ness of the hardware. It's all great. But man, finding alternatives NASes is going to be hard for me now. Hahaha.

 

(Please ignore the fact that I posted this on April Fools Day. That's just a coincidence).

[ebin-dev]

Hi there,

 

the emmc issues were solved by reducing read/write speed by a factor of 2 (thanks to @piter75). Unfortunately this is still the case with newer kernels 5.15.xx.

I am still using linux 5.10.43 without any issues and without the emmc speed limitation (Armbian 21.08.2 Bullseye with Linux 5.10.43-rockchip64).

 

Upgrading from Buster to Bullseye should work, but your mileage may vary depending on your specific Buster installation. The upgrade will install the newest kernel 5.15.25. I had troubles at least with rsync in this context.

 

Alternatively you can grab the latest Armbian Bullseye Image (Armbian_21.08.2_Helios64_bullseye_current_5.10.63.img.xz) from the archive and downgrade to linux 5.10.43.

To do that you just need to download those files and install them with 'dpkg -i *.deb' and disable any further Armbian updates in /etc/apt/sources.list.d/armbian.list . Debian updates will still arrive ...

The system on SD can be transferred to emmc.

 

Cheers!

[Werner]

Just FYI: Userspace dist-upgrades (Bionic->Focal, Buster->Bullseye and so on) are neither tested nor supported by Armbian.

[TRS-80]

9 hours ago, mrjpaxton said:

vanilla Debian kernel

 

I see this a lot, and I am afraid it's a common misunderstanding.  Debian is a distribution, they distribute (mostly user space) packages from various places.  The kernels they get wholesale (from kernel.org, usually).

 

Which is why the kernels in vanilla Debian (generally speaking) are going to be so far behind Armbian's (heavily patched and customized) kernels.  In fact, this is essentially one of the main raisons d'être for Armbian in the first place.  SBC/ARM is very different from x86, which is much more standardized on a hardware level.

 

And that's also why, the user space comes pretty much straight from upstream vanilla Debian.  Because that's not what we do here.  Here we are focused on the (much more difficult) low level things, like kernels, bootloaders, etc.  Where vanilla Debian do not.

 

It will be nice if one day this is not the case, but I just don't see that happening in foreseeable future.

[mrjpaxton]

11 hours ago, ebin-dev said:

Hi there,

 

the emmc issues were solved by reducing read/write speed by a factor of 2 (thanks to @piter75). Unfortunately this is still the case with newer kernels 5.15.xx.

I am still using linux 5.10.43 without any issues and without the emmc speed limitation (Armbian 21.08.2 Bullseye with Linux 5.10.43-rockchip64).

 

Upgrading from Buster to Bullseye should work, but your mileage may vary depending on your specific Buster installation. The upgrade will install the newest kernel 5.15.25. I had troubles at least with rsync in this context.

 

Alternatively you can grab the latest Armbian Bullseye Image (Armbian_21.08.2_Helios64_bullseye_current_5.10.63.img.xz) from the archive and downgrade to linux 5.10.43.

To do that you just need to download those files and install them with 'dpkg -i *.deb' and disable any further Armbian updates in /etc/apt/sources.list.d/armbian.list . Debian updates will still arrive ...

The system on SD can be transferred to emmc.

 

Cheers!

 

So I continued to keep Buster by putting in "buster", "buster-upgrades", etc.. in my /etc/apt/sources.list files, then did an `apt full-upgrade`. It pulled in the latest kernel 5.15.25 and some other updates, and the rsync issues you were talking about did not happen to me. So it sounds like it is possible to only get kernel 5.15.25 working on Buster well enough. Either that, or I'm just lucky. But I am going to check stability for at least a week before I even consider re-flashing with Bullseye (I won't even upgrade from Buster to Bullseye within Buster, I suppose.)

 

But if Bullseye still has issues with things on Kobol, even when re-flashing it new and copying over my config files that I've backed up, I guess I will be sticking with Buster... I have no idea how long updates to Buster are going to be supported by Armbian anyway.

[ebin-dev]

16 hours ago, mrjpaxton said:

It pulled in the latest kernel 5.15.25 and some other updates, and the rsync issues you were talking about did not happen to me.

 

Interesting !

 

I tried again with a fresh Armbian Bullseye Image (Armbian_21.08.2_Helios64_bullseye_current_5.10.63.img.xz) on SD and used apt update && apt upgrade to install the latest kernel 5.15.25.

After a reboot, an rsync from SD to another SD attached to the USB port worked perfectly.

 

I have noticed today that SD and emmc device names have changed between linux 5.10.xx and linux 5.15.yy from /dev/mmcblk2p1 (emmc) and /dev/mmcblk1p1 (SD) to /dev/mmcblk1p1 and /dev/mmcblk0p1 respectively. Since my backup scripts use the device names they could not work correctly anymore with kernel 5.15.yy. MEA CULPA.

 

All device names are now replaced by their corresponding UUIDs in the backup scripts - so that this can't happen again.

 

So you can update your buster or bullseye installation to the latest Armbian kernel (no rsync issues) - but be aware that there are still some other issues to be resolved, in particular if you are using zfs-dkms (see the parallel threads).

 

[mrjpaxton]

7 hours ago, ebin-dev said:

 

Interesting !

 

I tried again with a fresh Armbian Bullseye Image (Armbian_21.08.2_Helios64_bullseye_current_5.10.63.img.xz) on SD and used apt update && apt upgrade to install the latest kernel 5.15.25.

After a reboot, an rsync from SD to another SD attached to the USB port worked perfectly.

 

I have noticed today that SD and emmc device names have changed between linux 5.10.xx and linux 5.15.yy from /dev/mmcblk2p1 (emmc) and /dev/mmcblk1p1 (SD) to /dev/mmcblk1p1 and /dev/mmcblk0p1 respectively. Since my backup scripts use the device names they could not work correctly anymore with kernel 5.15.yy. MEA CULPA.

 

All device names are now replaced by their corresponding UUIDs in the backup scripts - so that this can't happen again.

 

So you can update your buster or bullseye installation to the latest Armbian kernel (no rsync issues) - but be aware that there are still some other issues to be resolved, in particular if you are using zfs-dkms (see the parallel threads).

 

I am really happy to hear that you were able to narrow down the rsync issue!

 

Yeah, for me, rsync is one of those essential utilities I couldn't function without to do my backups properly. Using scp just doesn't cut it anymore! And I am not hacking it to use a Btrfs root FS on my Armbian install, so I can't just use Btrfs send and receive on snapshots to backup (or, for example, the equivalent way for ZFS).

 

My NAS is a Btrfs RAID 1, just set up with pure Armbian and NFS (no OMV) so I think I won't run into any ZFS issues... I might be good to go and try it out next week, maybe..

 

Like I said, I can always just re-flash a Bullseye Armbian image and migrate over my config files.

 

As a side note, the halved performance slowdown on eMMC is a minor concern. Sometimes I want to use the NAS with other services. For example, I also have it set up as my print server (CUPS), so it needs to access the spool, etc... and I'm also thinking about trying some other services on it in the future. But anyway, even `apt cache search` runs very, very slow after it gets to "50%" for some reason, and I have no idea why. I think someone on here had a fix for this, but I can't remember what it was...

[TDCroPower]

if who has installed OMV 5, this should be updated before update to Bullseye!
OMV 5 still uses pam_tally2.so which is no longer present with the lib package in Bullseye.
With OMV 6 this is only corrected !

[rafaledevnull]

Hello here,

 

I also have a helios 64 with, Armbian 22.02.1 (Debian 10 buster), kernel 5.15.25 and OMV5

/ is mount on eMMC and I have a raid5 on sd{a,b,c,d,e} and after it's lvm; all lv are mount on /srv/dev-disk-by-uuid-*

 

On 4/2/2022 at 11:30 PM, mrjpaxton said:

As a side note, the halved performance slowdown on eMMC is a minor concern. Sometimes I want to use the NAS with other services. For example, I also have it set up as my print server (CUPS), so it needs to access the spool, etc... and I'm also thinking about trying some other services on it in the future. But anyway, even `apt cache search` runs very, very slow after it gets to "50%" for some reason, and I have no idea why. I think someone on here had a fix for this, but I can't remember what it was...

 

Yes, i have the same problem with apt search, it's very very slow.

But other than that I did not detect a problem.

 

On 4/3/2022 at 9:22 PM, TDCroPower said:

if who has installed OMV 5, this should be updated before update to Bullseye!
OMV 5 still uses pam_tally2.so which is no longer present with the lib package in Bullseye.
With OMV 6 this is only corrected !

 

 

I'm not sure I understand...
To understand two things in fact.

1) when we talk about Armbian 22.02.1, it's "two versions" in reality:
- Armbian 22.02.1 on Debian 10
- Armbian 22.02.1 on Debian 11

 

2)
If I want to switch to on armbian Debian 11 Bullseye, I must update before OMV (5 -> 6) ?
So:

Update OMV 5 -> 6
 

sed -i -e 's/usul/shaitan/g' /etc/apt/sources.list.d/*

apt update

apt upgrade

 

Upgrade armbian Debian 10 -> armbian Debian 11

 

sed -i -e 's/buster/bullseye/g' /etc/apt/sources.list.d/*

sed -i -e 's/buster/bullseye/g' /etc/apt/sources.list

apt update

apt full-upgrade

 

thx

[ebin-dev]

On 4/2/2022 at 4:19 PM, ebin-dev said:

So you can update your buster or bullseye installation to the latest Armbian kernel (no rsync issues) - but be aware that there are still some other issues to be resolved,

 

I just had an issue with the 2.5G interface after the latest update to Armbian Bullseye 22.02.01 (linux 5.15.25):  The 2.5G Interface would not appear to work as expected anymore.

 

A 12.225 GB test file was transferred from Helios64 to a client via a 2.5G switch using netatalk with only about 120 MB/s.  The system I am normally using ( Armbian 21.08.2 Bullseye, linux downgraded to 5.10.43), transfers the file with 271 MB/s (as expected).

 

Here are the related dmesg messages :

[   11.159252] r8152 2-1.4:1.0: Direct firmware load for rtl_nic/rtl8156a-2.fw failed with error -2
[   11.159293] r8152 2-1.4:1.0: unable to load firmware patch rtl_nic/rtl8156a-2.fw (-2)
[   11.199745] r8152 2-1.4:1.0 (unnamed net_device) (uninitialized): netif_napi_add() called with weight 256
[   11.215517] r8152 2-1.4:1.0 eth1: v1.12.12
[   11.215734] usbcore: registered new interface driver r8152
[   14.606490] r8152 2-1.4:1.0 eth1: Promiscuous mode enabled
[   14.606732] r8152 2-1.4:1.0 eth1: carrier on

 

Is there anyone observing the same issue ?

[jmanes]

Reviving this thread, as I, too, have a Helios64 and I am looking into longevity of the device. I am currently on Buster and I have been taking every update as it comes along. Looking into getting Bullseye soon.

For some extra info, here is my current install from uname.

Linux helios 5.15.80-rockchip64 #22.11.1 SMP PREEMPT Wed Nov 30 11:12:47 UTC 2022 aarch64 GNU/Linux

Here is output of an iperf3 test on 2.5gbe port, as a response to user ebin-dev above.

Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.0.0.3, port 54162
[  5] local 10.0.1.10 port 5201 connected to 10.0.0.3 port 54164
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   208 MBytes  1.75 Gbits/sec                  
[  5]   1.00-2.00   sec   226 MBytes  1.89 Gbits/sec                  
[  5]   2.00-3.00   sec   191 MBytes  1.61 Gbits/sec                  
[  5]   3.00-4.00   sec   220 MBytes  1.84 Gbits/sec                  
[  5]   4.00-5.00   sec   227 MBytes  1.91 Gbits/sec                  
[  5]   5.00-6.00   sec   227 MBytes  1.91 Gbits/sec                  
[  5]   6.00-7.00   sec   227 MBytes  1.91 Gbits/sec                  
[  5]   7.00-8.00   sec   228 MBytes  1.91 Gbits/sec                  
[  5]   8.00-9.00   sec   226 MBytes  1.90 Gbits/sec

 

No complaints or instability as of now. I use my device explicitly as a NAS and I have an Intel NUC that then mounts the NAS via SFTP over 2.5gbe  to transcode/serve up the data. I'm feeling a bit adventurous, so I may install Bullseye soon and report back. I don't use eMMC or SD cards though, I use one of the bay slots with a 1tb SSD while the other 4 slots are occupied in RAID10 with 4TB western digital spinning drives.

[ebin-dev]

@privilegejunkie I am still on Linux 5.10.43-rockchip64 #21.05.4 SMP PREEMPT Wed Jun 16 08:02:12 UTC 2021 aarch64 GNU/Linux / debian bullseye 11.7.

 

Would you provide a download link to the kernel you are using ? I could easily test it with debian bullseye.

 

However, debian bookworm is about to be released and it comes with linux 6.1 LTS. So it would make some sense to test 6.1 kernels too for the ones who intend to upgrade.

Edited May 30, 2023 by ebin-dev

[mrjpaxton]

This board still does not have a maintainer, but I'm still using and relying on it.

 

Since Debian announced Bookworm recently, are there any new Armbian Bookworm builds that can be used for this?

[Igor]

On 6/15/2023 at 10:36 PM, mrjpaxton said:

This board still does not have a maintainer, but I'm still using and relying on it.

 

We set maintainership conditions pretty low - he/she doesn't need to fix any low level troubles, keeping a list of them is already a great value or taking such burden away. As I am not paying attention to this hardware (to remain sane) I didn't notice there were no images at all, while we usually provide images for community/no supported targets too.

 

Images would be up, but one bug prevent them from building correctly - you can build then on your own, but they fail to build in automation.

[mrjpaxton]

Great, thanks! I'll try the build too when I have the time and opportunity.

 

EDIT: Actually, I see that there are now downloads for Bookworm listed on the Helios64 page. Someone's already 10 steps ahead. Time to text it out soon then.

Edited August 11, 2023 by mrjpaxton

[ebin-dev]

@mrjpaxton I am about to upgrade linux from 5.10 to 6.1 now and if that works to dist-upgrade bullseye to bookworm. Just to prevent someone from telling us that this is not supported by Armbian - we already know that.

Just in case you would like to try, here is a link to the most recent Armbian 23.08.0 - 6.1.45 bookworm image including all the linux-6.1.45 deb packages.

You could also start with the fresh bookworm image, in case dist-upgrade to bookworm does not complete successfully with your installation.

 

Armbian 23.08.0 - 6.1.45 bookworm image was compiled for Helios64 using the Armbian build system as mentioned in the parallel thread .

 

[mrjpaxton]

@ebin-dev It is probably a better idea to back up all your configuration (home dir, root FS) with rsync or whatever backup utility you prefer before that. So yeah, I'm using the fresh Bookworm image because I'm 2 stable releases behind. That's because there was never a Bullseye image released to my knowledge. You'd have to build it. I think I have some time to try it out tomorrow if I remember to do it. If this was vanilla Debian, i would so it. OpenWrt is the same way, where they prefer the backup config and re-flash method. A little annoying, but whatever works I guess.

[ebin-dev]

@mrjpaxton Dist-upgrade(ing) from bullseye to bookworm did finally complete successfully. However, one should consider that device names have changed (otherwise your system may end up offline 🙂) the new interface names are:

 

# interface names (bookworm)
sd:	/dev/mmcblk0
emmc:	/dev/mmcblk1
eth0:	end0		(1GBase-T ethernet)
eth1:	end1		(2.5GBase-T ethernet)

 

P.S.: I am currently setting up bookworm from scratch starting from the fresh image to get rid of the stuff that accumulated during the last years.

Edited August 14, 2023 by ebin-dev

[mrjpaxton]

Okay, so I finally did it this Friday. I did the expose eMMC steps - https://wiki.kobol.io/helios64/install/emmc/

 

(Somebody will probably have to mirror that link and the eMMC SD-Card exposure image if those instructions and files ever come Offline for good.)

 

I'm also going to report that it all works just fine. I connected it with Picocom afterwards over USB-C with `picocom -b 1500000 /dev/ttyUSB0`, boots up fine, got a login prompt, did the Armbian initial setup stuff like setting up root password, a user password, which was different from last time I think, and then shell, etc...

 

Now I'm in the long-haul process of meticulously restoring my config backups one by one from my old install, got SSH quickly set up again so I could disconnect the serial cable, double-checking them to make sure my static mounts will still work, my personal user scripts still run, and everything else still works without too many errors. I'm going to install more packages, set up Systemd's networking stack (networkd/resolved) the way I want, the Btrfs array, then NFS, and finally, maybe I'll even try to set up extras that I can't remember right now, like maybe disabling the ZRAM if it becomes a problem. Because usually the 4 GB of regular RAM is enough.

 

So far, yes, everything is working great. I'll report back with any problems I run into.

[mrjpaxton]

Okay, well I am running into one problem with the CLI (not minimal) install. So I uninstalled Network Manager and Netplan, because I prefer straight-up Systemd networkd. Did `sudo systemctl unmask systemd-networkd.service`, double checked my files in `/etc/systemd/network` and did a `networkctl` to check everything. Works fine once I enable and start the service, but after rebooting, the service does not start up upon boot, even though the service is enabled, and I can't seem to find any related error messages associated with it. What's going on there? Is it an Armbian thing?

 

EDIT: never mind, I just tried `sudo systemctl enable systemd-networkd.service`, and now it's working again. Although `systemd-networkd-wait-online.service` is failing for some reason, but at least the rest of networkd still works.

Edited August 19, 2023 by mrjpaxton

[ebin-dev]

How to switch from network-manager to systemd-networkd is explained i.e here.

 

Bookworm is up and running from SD for a few days now, but it would appear that emmc is still being accessed with hs200 speed.

[mrjpaxton]

The topic of this thread is kind of old and dated now. I suppose I could either update it, or maybe a mod can close it for now. I don't know how to do it.

 

Most of my general questions have been answered thankfully with Bookworm. I'm so happy the Armbian community is continuing to support this board.

 

I'll probably start posting in some other Helios64 threads now. Thanks, everyone.

Edited August 19, 2023 by mrjpaxton