Regression in Armbian 24.2.1 make Orange Pi 4 LTS unusable, downgrade required

[bJiC5xbZdy]

Hi all, reporting a regression in new Armbian 24.2.1 kernel version.

New kernel is causing Orange Pi 4 LPS not to boot most of the time, when it boots, then there is no network. Kernel hangs with oops errors on the screen, and all debugging stuff. Sorry that I did not caught any of them on my phone, but it would have been incomplete anyway.

How to reproduce:

Install any 24.2.1 based system on Orange Pi 4 LTS

Upon first boot, you will have issues. No Ethernet network.

Second boot will most likely not succeed no matter what you do and system will brick permanently, by not booting at all (no video output), or booting and then showing kernel crash.

 

I've tested:

Armbian_24.2.1_Orangepi4-lts_jammy_current_6.6.16_xfce_desktop.img.xz (system booted into desktop only once, never managed to do it again. No network in first boot)

Armbian_24.2.1_Orangepi4-lts_jammy_current_6.6.16.img.xz (kernel crash in terminal on second boot, no network on first boot)

Armbian_23.8.1_Orangepi4-lts_bookworm_current_6.1.50.img.xz - here I installed older image, all was good, network good, reboots good, so I upgraded all packages, upon restart everything was broken again)

Armbian_23.8.1_Orangepi4-lts_bookworm_current_6.1.50.img.xz - formatted the card again, got it working again, I went to armbian config tool and marked Armbian kernel packages to freeze, not going to upgrade them unless regression is confirmed to be gone.

 

Notes: I've tested this with nothing connected to the board except for HDMI cable and some GPIO pins. Nothing on USB etc. New kernel 6.6.16. on Armbian 24.2.1 is experiencing this regression.

 

Kind regards

Edited April 2 by bJiC5xbZdy

[bJiC5xbZdy]

Today I rebooted the device, and it went back online right now. Success. 🙂

 

I have held back the following packages using Armbian config tool (freeze kernel):

$ apt-mark showhold
armbian-firmware
linux-dtb-current-rockchip64
linux-image-current-rockchip64
linux-u-boot-orangepi4-lts-current

 

If I upgrade them, Orange Pi will brick again.

All other packages are updated and in newest versions.

 

$ hostnamectl  
Static hostname: orangepi4-lts
      Icon name: computer
     Machine ID: 0265aaf029e640c0b696c11228627e99
        Boot ID: 038c59a9c28d487d8b2b9b957497a52a
Operating System: Armbian 24.2.1 bookworm          
         Kernel: Linux 6.1.50-current-rockchip64
   Architecture: arm64

 

[jock]

I did not experience all the issues you had, and it is strange that so many issues appeared just with updated kernel revisions.

[bJiC5xbZdy]

Thanks for your reply! Upgrade from kernel 6.1.50 to 6.6.16 is a major thing, I would have thought?

 

It could be related to GPIO pins I have plugged in (and not using). Orange Pi board is connected to an ASIC board, by company Futurebit.io, it's called Apollo and it's a Bitcoin miner. I only get power from it, but I don't use any pins connected by serial connector, because I use this device through USB instead.

If I disconnect Orange from it, I will have to figure out an extra cable to power the Orange board, which I didn't need to do in my years of using this device.

 

This device is creating some error and debug entries in dmesg (that's my interpretation of what I see in dmesg), but never caused any instability. But now, something does.

Edited April 4 by bJiC5xbZdy

[bJiC5xbZdy]

Attaching full dmesg from working 6.1.50 system. As you can see, I have things in red (errors). I have ASIC board connected via GPIO (not used) and USB (fully operational).

Crashes with 6.6.16 system happened with no USB connected at all, only GPIO (because that's how I power Orange board). So, something must be happening around that, if you can't reproduce it. Previous versions always worked just fine, since I started using this Apollo device 2-3 years ago. I always used USB connection though, I only have GPIO connected (and power) because they are in one physical enclosure.

dmesg.txt

Edited April 4 by bJiC5xbZdy

[jock]

You can take a look to this thread also:

it does not contain any solution, but there I posted an experimental image with kernel 6.3 and some patches cut out from libreelec patches set that solved the problem.

The problem is that it is a wide number of patches to import to solve the issue, and since I don't have a setup with HDMI issues I can't do tests to reduce the number of patches to an essential set.