Not booting anymore

[Tomate]

Hello,

I'm identifying myself has a newbie concerning linux.
After some time I reboot my rock5c and it is not booting anymore.

 

Here is what I've done initially: I bought a rock 5c and a penta sata hat.
-I installed Armbian_24.5.1_Rock-5c_bookworm_edge_6.8.10_minimal.
-I installed OMV (configure some samba shared folder)
-I installed ZFS (for that I needed linux headers that I found online because there were not included)
-I create a ZFS pull with two SSD.

-I installed immich and setup a cloudflare tunel to use it.
-I installed plex media server.

That's pretty much all I have setup on my SBC.
I've done apt upgrade multiple time & update some immich / Plex.
Every time I did update I rebooted the SBC.
And then yesterday I did an immich & plex update, apt upgrade and lastly rebooted.
And my SBC never did come back online.

I try to install a fresh armbian install to see if it still boot and it did boot.

I'm not sure where to start to get it boot again with keeping has much stuff has I can.
I wanted to try this:
https://docs.armbian.com/User-Guide_Recovery/#replacing-boot

But I'm not sure what Root file system, Kernel, Firmware & Device Tree Binary (DTB) version I should take.

Or maybe someone could point me to an other path to fix my install?

[Werner]

2 hours ago, Tomate said:

edge

Don't use bleeding edge if you expect stability. Use vendor until mainline is more matured.

[Tomate]

Thanks for the advice.
I will do next time but now what should I do?
Can I rollback the version?

[Werner]

I'd try with linux-image-vendor-rk35xx and linux-dtb-vendor-rk35xx and leave cli bsp package for now as it is.

That being said it would be more useful to get an idea about WHAT is actually failing an attempting to fix before messing with the kernels. Hook up a serial console and set verbosity to 7. https://debug.armbian.de

[Tomate]

Thanks,
I'll try it.
Just to be sure this is the two packages you recommend:
https://uk.mirrors.naho.moe/armbian/pool/main/l/linux-6.1.43/linux-image-vendor-rk35xx_24.5.1_arm64__6.1.43-Sd06d-D440d-Pb426-Ccc84H5c21-HK01ba-Vc222-Bf15a-R448a.deb

https://uk.mirrors.naho.moe/armbian/pool/main/l/linux-dtb-vendor-rk35xx/linux-dtb-vendor-rk35xx_24.5.1_arm64__6.1.43-Sd06d-D440d-Pb426-Ccc84H5c21-HK01ba-Vc222-Bf15a-R448a.deb

After reading the recovery guide I'm not sure of the steps I should do.
This one will work only changing the mnt path:
 

~/tmp/recovery # for f in *.deb; do dpkg -x $f /mnt/sdcard; done

 

But this one I'm not sure but maybe after dpkg the deb files It will be obvious.
 

~/tmp/recovery # cd /mnt/sdcard/boot
/mnt/sdcard/boot # ln -s vmlinuz-5.8.16-sunxi64 zImage
/mnt/sdcard/boot # ln -s uInitrd-5.8.16-sunxi64 uInitrd
/mnt/sdcard/boot # ln -s dtb-5.8.16 dtb

In the mean time I order an usb uart it will certainly proved helpful in my journey.

[Tomate]

I got my uart and connect to the rock5c console and I got this:
 

���������������>DDR 9fffbe1e78 cym 24/02/04-10:09:20,fwver: v1.16
LPDDR4X, 2112MHz
channel[0] BW=16 Col=10 Bk=8 CS0 Row=17 CS=1 Die BW=16 Size=2048MB
channel[1] BW=16 Col=10 Bk=8 CS0 Row=17 CS=1 Die BW=16 Size=2048MB
channel[2] BW=16 Col=10 Bk=8 CS0 Row=17 CS=1 Die BW=16 Size=2048MB
channel[3] BW=16 Col=10 Bk=8 CS0 Row=17 CS=1 Die BW=16 Size=2048MB
Manufacturer ID:0xff
CH0 RX Vref:26.7%, TX Vref:22.8%,0.0%
CH1 RX Vref:28.3%, TX Vref:20.8%,0.0%
CH2 RX Vref:28.5%, TX Vref:18.8%,0.0%
CH3 RX Vref:28.5%, TX Vref:20.8%,0.0%
change to F1: 528MHz
change to F2: 1068MHz
change to F3: 1560MHz
change to F0: 2112MHz
out
U-Boot SPL board init
U-Boot SPL 2017.09-armbian (Jun 30 2024 - 21:45:07)
unrecognized JEDEC id bytes: 00, 00, 00
unknown raw ID 0 0 0
Trying to boot from MMC2
spl: partition error
Trying fit image at 0x4000 sector
## Verified-boot: 0
## Checking atf-1 0x00040000 ... sha256(a7d1d8d191...) + OK
## Checking uboot 0x00200000 ... sha256(5ecfc82c23...) + OK
## Checking fdt 0x0031e2e0 ... sha256(049e983a22...) + OK
## Checking atf-2 0xff100000 ... sha256(4b2065349b...) + OK
## Checking atf-3 0x000f0000 ... sha256(aa71013e72...) + OK
Jumping to U-Boot(0x00200000) via ARM Trusted Firmware(0x00040000)
Total: 200.400/400.957 ms

INFO:    Preloader serial: 2
NOTICE:  BL31: v2.3():v2.3-682-g4ca8a8422:derrick.huang, fwver: v1.45
NOTICE:  BL31: Built : 10:11:21, Dec 27 2023
INFO:    spec: 0x13
INFO:    code: 0x88
INFO:    ext 32k is not valid
INFO:    ddr: stride-en 4CH
INFO:    GICv3 without legacy support detected.
INFO:    ARM GICv3 driver initialized in EL3
INFO:    valid_cpu_msk=0xff bcore0_rst = 0x0, bcore1_rst = 0x0
INFO:    l3 cache partition cfg-0
INFO:    system boots from cpu-hwid-0
INFO:    disable memory repair
INFO:    idle_st=0x21fff, pd_st=0x11fff9, repair_st=0xfff70001
INFO:    dfs DDR fsp_params[0].freq_mhz= 2112MHz
INFO:    dfs DDR fsp_params[1].freq_mhz= 528MHz
INFO:    dfs DDR fsp_params[2].freq_mhz= 1068MHz
INFO:    dfs DDR fsp_params[3].freq_mhz= 1560MHz
INFO:    BL31: Initialising Exception Handling Framework
INFO:    BL31: Initializing runtime services
WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
ERROR:   Error initializing runtime service opteed_fast
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x200000
INFO:    SPSR = 0x3c9


U-Boot 2017.09-armbian (Jun 30 2024 - 21:45:07 +0000)

Model: Radxa ROCK 5C
MPIDR: 0x81000000
PreSerial: 2, raw, 0xfeb50000
DRAM:  8 GiB
Sysmem: init
Relocation Offset: eda3f000
Relocation fdt: eb9f8fc0 - eb9fece0
CR: M/C/I
Using default environment

DM: v2
no mmc device at slot 1
mmc@fe2c0000: 1 (SD), mmc@fe2e0000: 0
Bootdev(atags): mmc 1
MMC1: Legacy, 52Mhz
PartType: EFI
No misc partition
boot mode: None
FIT: No boot partition
Failed to load DTB, ret=-2
No valid DTB, ret=-22
Failed to get kernel dtb, ret=-22
Model: Radxa ROCK 5C
MPIDR: 0x81000000
CLK: (sync kernel. arm: enter 1008000 KHz, init 1008000 KHz, kernel 0N/A)
  b0pll 24000 KHz
  b1pll 24000 KHz
  lpll 24000 KHz
  v0pll 24000 KHz
  aupll 24000 KHz
  cpll 1500000 KHz
  gpll 1188000 KHz
  npll 24000 KHz
  ppll 1100000 KHz
  aclk_center_root 702000 KHz
  pclk_center_root 100000 KHz
  hclk_center_root 396000 KHz
  aclk_center_low_root 500000 KHz
  aclk_top_root 750000 KHz
  pclk_top_root 100000 KHz
  aclk_low_top_root 396000 KHz
No misc partition
Net:   No ethernet found.
Hit key to stop autoboot('CTRL+C'):  0 
starting USB...
Bus usb@fc800000: USB EHCI 1.00
Bus usb@fc840000: USB OHCI 1.0
Bus usb@fc880000: USB EHCI 1.00
Bus usb@fc8c0000: USB OHCI 1.0
scanning bus usb@fc800000 for devices... 2 USB Device(s) found
scanning bus usb@fc840000 for devices... 1 USB Device(s) found
scanning bus usb@fc880000 for devices... 1 USB Device(s) found
scanning bus usb@fc8c0000 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found

Device 0: unknown device
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found U-Boot script /boot/boot.scr
3920 bytes read in 11 ms (347.7 KiB/s)
## Executing script at 00500000
Boot script loaded from mmc 1:1
245 bytes read in 10 ms (23.4 KiB/s)
39687743 bytes read in 3202 ms (11.8 MiB/s)
32371200 bytes read in 2611 ms (11.8 MiB/s)
147231 bytes read in 41 ms (3.4 MiB/s)
259 bytes read in 13 ms (18.6 KiB/s)
Applying user provided DT overlay pwm-fan.dtbo
failed on fdt_overlay_apply(): FDT_ERR_NOTFOUND
Error applying DT overlays, restoring original DT
147231 bytes read in 41 ms (3.4 MiB/s)
Trying kaslrseed command... Info: Unknown command can be safely ignored since kaslrseed does not apply to all boards.
Unknown command 'kaslrseed' - try 'help'
Fdt Ramdisk skip relocation
No misc partition
## Loading init Ramdisk from Legacy Image at 0a200000 ...
   Image Name:   uInitrd
   Image Type:   AArch64 Linux RAMDisk Image (gzip compressed)
   Data Size:    39687679 Bytes = 37.8 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 0x08300000
   Booting using the fdt blob at 0x08300000
   Using Device Tree in place at 0000000008300000, end 0000000008326f1e
can't found rockchip,drm-logo, use rockchip,fb-logo
WARNING: could not set reg FDT_ERR_BADOFFSET.
failed to reserve fb-loader-logo memory
Adding bank: 0x00200000 - 0xf0000000 (size: 0xefe00000)
Adding bank: 0x100000000 - 0x200000000 (size: 0x100000000)
Adding bank: 0x2f0000000 - 0x300000000 (size: 0x10000000)
Total: 12172.921/12630.205 ms

Starting kernel ...

And it kept hanging there, no more line after "Starting kernel ..."
I will search a bit with this debug log and come back if I found something new.

[Werner]

edit /boot/armbianEnv.txt and set verbosity to 7 to get more info after Starting kernel.

[Tomate]

Thanks,

I did not try that but I get it working again. I just copy the boot folder from a working install of the original release I installed.
I did had to move /lib/modules/6.8.11-edge-rockchip-rk3588 to 6.8.10-edge-rockchip-rk3588 to get docker running again.

 

Now a big thanks for your help.

Just a couple questions.
It seams I did start a kernel upgrade from 6.8.10 to 6.8.11 and to fix my issue I rolled back to 6.8.10 again.
Can I prevent kernel upgrade from happening while doing apt upgrade?
Can I and should I migrate to a more stable kernel branch like 6.1.43? And if yes how so?