Ancient Orange PI PC+ blank screen after update focal->jammy

[Domas]

Armbianmonitor:

https://paste.armbian.com/abacatidin

Hi there,

 

Since being only a novice in linux, there are some things that are difficult to understand. Like - which configs to update or keep old versions when doing the upgrade. Last time I said N (keep original version) to everything. Really the only things I care about are vnc configs and samba setup. I basically use it as a network drive for backups. Only necessary software is x11vnc and samba. Else I log into it just for updates/restarts.

 

I have tried upgrading my armbian several times. Unfortunately it does not work and I have to restore my sd card backup and start over.

 

I update it making sure all apt updates are done, and then freezing the kernel and running do-release-upgrade.

Here is the data when logging in via terminal for you to see the context, pre-upgrade:

Quote

  ___  ____  _
 / _ \|  _ \(_)  _
| | | | |_) | |_| |_
| |_| |  __/| |_   _|
 \___/|_|   |_| |_|

Welcome to Armbian 23.02.2 Focal with Linux 6.1.40-sunxi

No end-user support: untested automated build & unsupported (focal) userspace!

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/pro
System load:   116%             Up time:       2 min
Memory usage:  27% of 997M      IP:            192.168.2.106
CPU temp:      50°C             Usage of /:    12% of 56G
storage/:      74% of 916G      storage temp:  32°C

Tip of the day: Check what is new in Armbian v24.2 https://www.armbian.com/newsflash/armbian-24-2-kereru/

[ General system configuration (beta): armbian-config ]

 

Here goes lsb_release and etc/armbian-release info, pre-upgrade:

Quote

orangepiplus:~:%  lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:        20.04
Codename:       focal

 

orangepiplus:~:% cat /etc/armbian-release
# PLEASE DO NOT EDIT THIS FILE
BOARD=orangepiplus
BOARD_NAME="Orange Pi+"
BOARDFAMILY=sun8i
BUILD_REPOSITORY_URL=https://github.com/armbian/build
BUILD_REPOSITORY_COMMIT=1a8daf0
VERSION=23.02.2
LINUXFAMILY=sunxi
ARCH=arm
IMAGE_TYPE=nightly
BOARD_TYPE=csc
INITRD_ARCH=arm
KERNEL_IMAGE_TYPE=Image
BRANCH=current

 

And finally, after I brick my board, I get this data via serial interface:

Quote

=>

bootd
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
3964 bytes read in 3 ms (1.3 MiB/s)
## Executing script at 43100000
U-boot loaded from SD
Boot script loaded from mmc
419 bytes read in 2 ms (204.1 KiB/s)
Failed to load '/boot/uInitrd'
Failed to load '/boot/zImage'
Found mainline kernel configuration
Failed to load '/boot/dtb/sun8i-h3-orangepi-plus.dtb'
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
No FDT memory address configured. Please configure
the FDT address via "fdt addr <address>" command.
Aborting!
Failed to load '/boot/dtb/overlay/sun8i-h3-fixup.scr'
SCRIPT FAILED: continuing...
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
3964 bytes read in 3 ms (1.3 MiB/s)
## Executing script at 43100000
U-boot loaded from SD
Boot script loaded from mmc
419 bytes read in 2 ms (204.1 KiB/s)
Failed to load '/boot/uInitrd'
Failed to load '/boot/zImage'
Found mainline kernel configuration
Failed to load '/boot/dtb/sun8i-h3-orangepi-plus.dtb'
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
No FDT memory address configured. Please configure
the FDT address via "fdt addr <address>" command.
Aborting!
Failed to load '/boot/dtb/overlay/sun8i-h3-fixup.scr'
SCRIPT FAILED: continuing...

Device 0: Vendor: ST1000LM Rev: 0016 Prod: 035-1RK172
            Type: Hard Disk
            Capacity: 953869.7 MB = 931.5 GB (1953525164 x 512)
... is now current device
Scanning usb 0:1...
BOOTP broadcast 1
DHCP client bound to address 192.168.2.106 (5 ms)
*** ERROR: `serverip' not set
Cannot autoload with TFTPGET
missing environment variable: pxeuuid
Retrieving file: pxelinux.cfg/01-02-81-67-7c-a4-c3
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/C0A8026A
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/C0A8026
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/C0A802
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/C0A80
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/C0A8
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/C0A
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/C0
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/C
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/default-arm-sunxi-sunxi
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/default-arm-sunxi
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/default-arm
*** ERROR: `serverip' not set
Retrieving file: pxelinux.cfg/default
*** ERROR: `serverip' not set
Config file not found
BOOTP broadcast 1
DHCP client bound to address 192.168.2.106 (5 ms)
*** ERROR: `serverip' not set
Cannot autoload with TFTPGET
BOOTP broadcast 1
DHCP client bound to address 192.168.2.106 (5 ms)
*** ERROR: `serverip' not set
Cannot autoload with TFTPGET
=>

 

 

 

Where do I start?

 

Edited April 11 by Domas
posted before actually finishing the post

[SteeMan]

Armbian does not support upgrades from one userspace to another (i.e

 Focal to jammy in your case).  The recommendation is to reinstall and reapply your changes.

 

Upgrades can sometimes work, but they are not tested nor supported.

[Domas]

I am very well aware they are not officially supported. But as I understand, it does not mean that it won't work.

So generally they more often don't work than they do? And this is bad luck, nothing to be done about that - suck it up and do a fresh install? Or there is some research I can do to avoid this?

[Werner]

You can put the microsd in your computer and check a few things like is /boot populated? does the UUID of the root partition match in armbianEnv.txt and /etc/fstab?

[Domas]

I can do that but this would require to run the update again to brick my pc. I have restored the backup for now

But you mean to do this  when it is bricked after update, right?

[Werner]

1 hour ago, Domas said:

you mean to do this  when it is bricked after update, right?

Correct. Glad you made a backup beforehand.

[Domas]

This will take some time since it takes a few hours to upgrade, so maybe someday this week.

 

I will not leave this thread without the final outcome, even if I will give up and do a fresh install. I remember it took me forever to configure autologin, samba and x11vnc for some reason (can't remember, it was 3  years ago). So my lazy ass just wants to upgrade.

 

But looking at my log from serial interface I can see that /boot is there, not sure what it is inside though, at least it seems to have boot script (boot.scr is found). First one to fail is /boot/uInitrd. this is where I have to look.

Quote

Found U-Boot script /boot/boot.scr
3964 bytes read in 3 ms (1.3 MiB/s)
## Executing script at 43100000
U-boot loaded from SD
Boot script loaded from mmc
419 bytes read in 2 ms (204.1 KiB/s)
Failed to load '/boot/uInitrd'

 

And also what Werner said too.

[Domas]

So, upgraded and bricked again

armbianEnv:
 

Quote

verbosity=1
bootlogo=false
console=both
disp_mode=1920x1080p60
overlay_prefix=sun8i-h3
rootdev=UUID=cfe08448-2eb2-4bcb-a95a-4f720f8b52e4
rootfstype=ext4
usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u0x2537:0x1066:u,0x2537:0x1068:u0x2537:0x1066:u,0x2537:0x1068:u0x2537:0x1066:u,0x2537:0x1068:u0x2537:0x1066:u,0x2537:0x1068:u0x2537:0x1066:u,0x2537:0x1068:u0x2537:0x1066:u,0x2537:0x1068:u0x2537:0x1066:u,0x2537:0x1068:u

 

fstab

Quote

UUID=cfe08448-2eb2-4bcb-a95a-4f720f8b52e4 / ext4 defaults,noatime,commit=600,errors=remount-ro 0 1
tmpfs /tmp tmpfs defaults,nosuid 0 0
/dev/disk/by-id/usb-ST1000LM_035-1RK172_000000000033-0:0-part1 /mnt/bup-1tb auto nosuid,nodev,nofail,x-gvfs-show 0 0

 

 

/boot seems to be populated

 

And

 

According to boot script, it seems that /boot/dtb directory is missing and this is causing the whole havoc?

[Domas]

This is how healthy starting system looks like

 

And the serial output when starting this older backup:

Quote

U-Boot SPL 2020.10-armbian (Aug 08 2021 - 16:17:28 +0200)
DRAM: 1024 MiB
Trying to boot from MMC1

U-Boot 2020.10-armbian (Aug 08 2021 - 16:17:28 +0200) Allwinner Technology

CPU:   Allwinner H3 (SUN8I 1680)
Model: Xunlong Orange Pi Plus / Plus 2
DRAM:  1 GiB
MMC:   mmc@1c0f000: 0, mmc@1c10000: 2, mmc@1c11000: 1
Loading Environment from FAT... Unable to use mmc 1:1... In:    serial@1c28000
Out:   serial@1c28000
Err:   serial@1c28000
Net:   phy interface7
eth0: ethernet@1c30000
starting USB...
Bus usb@1c1b000: USB EHCI 1.00
Bus usb@1c1d000: USB EHCI 1.00
scanning bus usb@1c1b000 for devices... 2 USB Device(s) found
scanning bus usb@1c1d000 for devices... 2 USB Device(s) found
       scanning usb for storage devices... 1 Storage Device(s) found
Autoboot in 1 seconds
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
3964 bytes read in 3 ms (1.3 MiB/s)
## Executing script at 43100000
U-boot loaded from SD
Boot script loaded from mmc
419 bytes read in 2 ms (204.1 KiB/s)
12300217 bytes read in 592 ms (19.8 MiB/s)
8239608 bytes read in 405 ms (19.4 MiB/s)
Found mainline kernel configuration
36384 bytes read in 10 ms (3.5 MiB/s)
4185 bytes read in 9 ms (454.1 KiB/s)
Applying kernel provided DT fixup script (sun8i-h3-fixup.scr)
## Executing script at 45000000
## Loading init Ramdisk from Legacy Image at 43300000 ...
   Image Name:   uInitrd
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    12300153 Bytes = 11.7 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 43000000
   Booting using the fdt blob at 0x43000000
   Loading Ramdisk to 49445000, end 49ffff79 ... OK
   Loading Device Tree to 493d3000, end 49444fff ... OK

Starting kernel ...

 

 

Shall I now just give up and do a clean install?

 

This time I made a backup of failed installation too.

 

[SteeMan]

It looks to me like the upgrade is uninstalling the kernel, dtb and likely other armbian packages.  Likely because it thinks they are extra for some reason.

During the upgrade are you seeing an option to remove old packages?  If so look at that list and see if there are any Armbian packages in that list (there shouldn't be) (specifically anything with Armbian in the name as well as linux-image-* and linux-dtb-*)

I think successful upgrades disable the Armbian apt repository which should leave all the armbian packages untouched.  Then after the upgrade, you manually enable the armbian apt repository for the correct release and then do an apt update/upgrade of the armbian packages.

[Domas]

Yes, in the very end I was asked to remove obsolete packages. I said yes all the times.

I don't remember what was in the list, since it was about 76~ packages

 

I may try with no option to keep them

Quote

I think successful upgrades disable the Armbian apt repository which should leave all the armbian packages untouched. 

Is there any way to enforce this manually pre-upgrade?

 

Quote

Then after the upgrade, you manually enable the armbian apt repository for the correct release and then do an apt update/upgrade of the armbian packages.

I only freeze the kernel, i suppose it is something different?

 

 

Or maybe I can try to upgrade without freezing kernel? Or is that a guaranteed fail?

 

 

[SteeMan]

43 minutes ago, Domas said:

I only freeze the kernel, i suppose it is something different?

 

 

Or maybe I can try to upgrade without freezing kernel? Or is that a guaranteed fail?

Ive never frozen the kernel when doing upgrades.  So that shouldn't be necessary.

 

I would recommend redoing the upgrade, and at the end looking at the list of obsolete packages and not removing them.  Finish the upgrade (without removing obsolete packages) then go see what the status of your /etc/apt/sources.d/armbian.list file is and make that correct, then use apt to update/upgrade your armbian packages.  Then look at your obsolete packages and make sure the list is sane (that is it's not removing any armbian needed packages, *armbian*, linux-image-* or linux-dtb-*.)  Only then would I remove obsolete packages.

 

[Domas]

Ok, upgraded successfully. I said no to removing obsolete stuff in the end and it seemed to want to delete armbian stuff.

 

I did not freeze the kernel this time

 

one thing is a bit weird

 

Now it says Armbian 23.02.2 Jammy

shouldn't the version be 24. something?

 

 

I am nearly done, let me know if I can safely run apt update/upgrade?

I'd really hate to start over.

 

My  /etc/apt/sources.list.d/armbian.list file contents - the directory was a bit different

Quote

# deb http://apt.armbian.com jammy main focal-utils focal-desktop # disabled on upgrade to jammy

Thank you all so far!

Edited April 18 by Domas
added info about freezing kernel

[SteeMan]

So you don't want to remove:

armbian-config

armbian-firmware

armbian-zsh

linux-dtb-current-sunxi

linux-image-current-sunxi

(those last two are your linux kernel and dtb files - which was the reason you couldn't boot previously)

 

1 hour ago, Domas said:

# deb http://apt.armbian.com jammy main focal-utils focal-desktop # disabled on upgrade to jammy

So you can see here the upgrade disabled the armbian apt repository.  So it can't install the updated armbian packages.

 

So should be:

deb http://apt.armbian.com jammy main jammy-utils jammy-desktop

 

(So uncomment and change all references from focal to jammy)

 

1 hour ago, Domas said:

Now it says Armbian 23.02.2 Jammy

shouldn't the version be 24. something?

That is coming from one of the armbian* files located in /etc.  I don't remember which one.  If after everything is upgraded, if it still isn't showing the correct value, you can edit.  I think this should get updated by the installation of the correct armbian-bsp-* package.

[Domas]

Will attempt tonight.

 

 

[Domas]

So it was a success.

 

Version information updated too now

 

 

But it did want to both uninstall and update armbian-bsp-cli-orangepiplus after updating armbian.list file and running apt update/upgrade

 

 

Do you think it is safe to autoremove them if the system is up and running?

 

I am so happy I managed to fix my junk from 2015.... It is probably a $30 board from china. But there is just something about making old hardware work for as long as possible.

 

Thank you all!

[SteeMan]

Glad to hear you got it working.  I've gone through the same issues you had in the past when trying upgrades, so the steps were in the back of my mind. It should be fine to now remove the obsolete packages.