Zyxx Posted Wednesday at 11:47 AM Posted Wednesday at 11:47 AM Hi @ all, long time reader, but never managed to register. Now it is time... as I'm not able to solve this myself, kindly asking for help. I'm stuck, my Orangepi 5 plus stops boot at this: Loading, please wait... Starting systemd-udevd version 252.36-1~deb12u1 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... (initramfs) What happened before? Last year I bought the Orangepi 5 plus (8GB) and went ahead, testing everything out - having incredible fun with this little box. But with a way to old/slow sd card which finally failed two or three months ago. I went ahead and wrote the Armbian based Cinnamon (Kernel 6.1 incl. MESA/VPU) to a new SD Card. Everything was working flawless, but still ... SD Card. Therefore I bought a 1TB nvme, installed it and used armbian-install to install the bootloader on MTD Flash. Afterwards I got everything working from nvme. Amazing results, it just worked after armbian-install was finished. I shut down the system, removed SD card and used the nvme install from now on for weeks on a daily basis. Several apt-get dist-upgrades no problem at all. The last one done Sunday, if I remember correctly. Until today. I had some spare time and decided to apt-get update and apt-get dist-upgrade again. The commands finished without any error and new updates were done. But now -I wanted to reboot the device- reboot stalled. I see a black screen with above mentioned white text. So, something must have gone wrong I think. The Orangepi boots fine when I insert the SD Card again, also the nvme is there and can be mounted. Seemingly all data is still there. But how to proceed? I'm at a loss. Can anything be done on my side to solve this? Or is a reinstall to be done? The apt history logfile on the nvme (/var/logs/apt/history.log) contains both upgrades I mentioned and reads: Start-Date: 2025-04-01 16:15:10 Commandline: apt-get dist-upgrade Requested-By: edited (1000) Upgrade: armbian-config:arm64 (25.5.0-trunk.286.0324.065110, 25.5.0-trunk.325.0401.114753), tzdata:arm64 (2025a-0+deb12u1, 2025b-0+deb12u1), firefox-nightly:arm64 (138.0a1~20250324091950, 139.0a1~20250401094015), ghostscript-x:arm64 (10.0.0~dfsg-11+deb12u6, 10.0.0~dfsg-11+deb12u7), libgs10-common:arm64 (10.0.0~dfsg-11+deb12u6, 10.0.0~dfsg-11+deb12u7), libgs10:arm64 (10.0.0~dfsg-11+deb12u6, 10.0.0~dfsg-11+deb12u7), ghostscript:arm64 (10.0.0~dfsg-11+deb12u6, 10.0.0~dfsg-11+deb12u7), linux-headers-vendor-rk35xx:arm64 (25.2.3, 25.2.3), libgs-common:arm64 (10.0.0~dfsg-11+deb12u6, 10.0.0~dfsg-11+deb12u7) End-Date: 2025-04-01 16:15:46 Start-Date: 2025-04-02 09:42:43 Commandline: apt-get dist-upgrade Requested-By: edited (1000) Upgrade: linux-image-vendor-rk35xx:arm64 (25.2.3, 25.2.3), firefox-nightly:arm64 (139.0a1~20250401094015, 139.0a1~20250401212749), linux-headers-vendor-rk35xx:arm64 (25.2.3, 25.2.3), armbian-bsp-cli-orangepi5-plus-vendor:arm64 (25.2.2, 25.2.3) End-Date: 2025-04-02 09:44:34 term.log in this directory shows nothing unusual, too. As far as my limited knowledge reaches ofc: Log started: 2025-04-01 16:15:10 (Reading database ... 218528 files and directories currently installed.) Preparing to unpack .../0-tzdata_2025b-0+deb12u1_all.deb ... Unpacking tzdata (2025b-0+deb12u1) over (2025a-0+deb12u1) ... Preparing to unpack .../1-armbian-config_25.5.0-trunk.325.0401.114753_all.deb ... Unpacking armbian-config (25.5.0-trunk.325.0401.114753) over (25.5.0-trunk.286.0324.065110) ... Preparing to unpack .../2-firefox-nightly_139.0a1~20250401094015_arm64.deb ... Unpacking firefox-nightly (139.0a1~20250401094015) over (138.0a1~20250324091950) ... Preparing to unpack .../3-ghostscript-x_10.0.0~dfsg-11+deb12u7_arm64.deb ... Unpacking ghostscript-x:arm64 (10.0.0~dfsg-11+deb12u7) over (10.0.0~dfsg-11+deb12u6) ... Preparing to unpack .../4-ghostscript_10.0.0~dfsg-11+deb12u7_arm64.deb ... Unpacking ghostscript (10.0.0~dfsg-11+deb12u7) over (10.0.0~dfsg-11+deb12u6) ... Preparing to unpack .../5-libgs10_10.0.0~dfsg-11+deb12u7_arm64.deb ... Unpacking libgs10:arm64 (10.0.0~dfsg-11+deb12u7) over (10.0.0~dfsg-11+deb12u6) ... Preparing to unpack .../6-libgs10-common_10.0.0~dfsg-11+deb12u7_all.deb ... Unpacking libgs10-common (10.0.0~dfsg-11+deb12u7) over (10.0.0~dfsg-11+deb12u6) ... Preparing to unpack .../7-libgs-common_10.0.0~dfsg-11+deb12u7_all.deb ... Unpacking libgs-common (10.0.0~dfsg-11+deb12u7) over (10.0.0~dfsg-11+deb12u6) ... Preparing to unpack .../8-linux-headers-vendor-rk35xx_25.2.3_arm64.deb ... Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'prerm' starting. Cleaning directory /usr/src/linux-headers-6.1.99-vendor-rk35xx ... Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'prerm' finishing. Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'preinst' starting. Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'preinst' finishing. Unpacking linux-headers-vendor-rk35xx (25.2.3) over (25.2.3) ... Setting up libgs-common (10.0.0~dfsg-11+deb12u7) ... Setting up linux-headers-vendor-rk35xx (25.2.3) ... Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'postinst' starting. Configuring kernel-headers (6.1.99-vendor-rk35xx) - please wait ... HOSTCC scripts/basic/fixdep HOSTCC scripts/kconfig/conf.o HOSTCC scripts/kconfig/confdata.o HOSTCC scripts/kconfig/expr.o LEX scripts/kconfig/lexer.lex.c YACC scripts/kconfig/parser.tab.[ch] HOSTCC scripts/kconfig/lexer.lex.o HOSTCC scripts/kconfig/menu.o HOSTCC scripts/kconfig/parser.tab.o HOSTCC scripts/kconfig/preprocess.o HOSTCC scripts/kconfig/symbol.o HOSTCC scripts/kconfig/util.o HOSTLD scripts/kconfig/conf # # configuration written to .config # Compiling kernel-headers scripts (6.1.99-vendor-rk35xx) using 8 CPUs - please wait ... SYNC include/config/auto.conf.cmd HOSTCC scripts/dtc/flattree.o HOSTCC scripts/dtc/dtc.o HOSTCC scripts/dtc/fstree.o HOSTCC scripts/dtc/data.o HOSTCC scripts/dtc/livetree.o HOSTCC scripts/dtc/treesource.o HOSTCC scripts/dtc/srcpos.o HOSTCC scripts/dtc/checks.o HOSTCC scripts/dtc/util.o LEX scripts/dtc/dtc-lexer.lex.c YACC scripts/dtc/dtc-parser.tab.[ch] HOSTCC scripts/dtc/libfdt/fdt.o HOSTCC scripts/dtc/libfdt/fdt_ro.o HOSTCC scripts/dtc/libfdt/fdt_wip.o HOSTCC scripts/dtc/libfdt/fdt_sw.o HOSTCC scripts/dtc/libfdt/fdt_rw.o HOSTCC scripts/dtc/libfdt/fdt_strerror.o HOSTCC scripts/dtc/libfdt/fdt_empty_tree.o HOSTCC scripts/dtc/libfdt/fdt_addresses.o HOSTCC scripts/dtc/libfdt/fdt_overlay.o HOSTCC scripts/dtc/fdtoverlay.o HOSTCC scripts/dtc/dtc-lexer.lex.o HOSTCC scripts/dtc/dtc-parser.tab.o HOSTLD scripts/dtc/fdtoverlay HOSTLD scripts/dtc/dtc HOSTCC scripts/kallsyms HOSTCC scripts/sorttable HOSTCC scripts/resource_tool HOSTCC scripts/asn1_compiler HOSTCC scripts/genksyms/genksyms.o YACC scripts/genksyms/parse.tab.[ch] HOSTCC scripts/selinux/mdp/mdp HOSTCC scripts/selinux/genheaders/genheaders LEX scripts/genksyms/lex.lex.c HOSTCC scripts/genksyms/parse.tab.o HOSTCC scripts/genksyms/lex.lex.o HOSTLD scripts/genksyms/genksyms Compiling kernel-headers scripts/mod (6.1.99-vendor-rk35xx) using 8 CPUs - please wait ... CC scripts/mod/empty.o HOSTCC scripts/mod/mk_elfconfig CC scripts/mod/devicetable-offsets.s UPD scripts/mod/devicetable-offsets.h MKELF scripts/mod/elfconfig.h HOSTCC scripts/mod/modpost.o HOSTCC scripts/mod/file2alias.o HOSTCC scripts/mod/sumversion.o HOSTLD scripts/mod/modpost MODPOST scripts/mod/Module.symvers Done compiling kernel-headers (6.1.99-vendor-rk35xx). Done compiling kernel-headers tools (6.1.99-vendor-rk35xx). Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'postinst' finishing. Setting up tzdata (2025b-0+deb12u1) ... Current default time zone: 'Europe/Berlin' Local time is now: Tue Apr 1 16:15:44 CEST 2025. Universal Time is now: Tue Apr 1 14:15:44 UTC 2025. Run 'dpkg-reconfigure tzdata' if you wish to change it. Setting up armbian-config (25.5.0-trunk.325.0401.114753) ... Setting up firefox-nightly (139.0a1~20250401094015) ... Setting up libgs10-common (10.0.0~dfsg-11+deb12u7) ... Setting up libgs10:arm64 (10.0.0~dfsg-11+deb12u7) ... Setting up ghostscript (10.0.0~dfsg-11+deb12u7) ... Setting up ghostscript-x:arm64 (10.0.0~dfsg-11+deb12u7) ... Processing triggers for mailcap (3.70+nmu1) ... Processing triggers for desktop-file-utils (0.26-1) ... Processing triggers for hicolor-icon-theme (0.17-2) ... Processing triggers for libc-bin (2.36-9+deb12u10) ... Processing triggers for man-db (2.11.2-2) ... Log ended: 2025-04-01 16:15:46 Log started: 2025-04-02 09:42:43 (Reading database ... 218527 files and directories currently installed.) Preparing to unpack .../armbian-bsp-cli-orangepi5-plus-vendor_25.2.3_arm64.deb ... Armbian 'armbian-bsp-cli-orangepi5-plus-vendor' for '1-PCe1b9-V4064-H2670-B1e5d-R756d': 'preinst' starting. Armbian 'armbian-bsp-cli-orangepi5-plus-vendor' for '1-PCe1b9-V4064-H2670-B1e5d-R756d': 'preinst' finishing. Unpacking armbian-bsp-cli-orangepi5-plus-vendor (25.2.3) over (25.2.2) ... Armbian 'armbian-bsp-cli-orangepi5-plus-vendor' for '1-PC7d0b-Vcff1-H2670-B6e44-R89c9': 'postrm' starting. Armbian 'armbian-bsp-cli-orangepi5-plus-vendor' for '1-PC7d0b-Vcff1-H2670-B6e44-R89c9': 'postrm' finishing. Preparing to unpack .../firefox-nightly_139.0a1~20250401212749_arm64.deb ... Unpacking firefox-nightly (139.0a1~20250401212749) over (139.0a1~20250401094015) ... Preparing to unpack .../linux-headers-vendor-rk35xx_25.2.3_arm64.deb ... Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'prerm' starting. Cleaning directory /usr/src/linux-headers-6.1.99-vendor-rk35xx ... Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'prerm' finishing. Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'preinst' starting. Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'preinst' finishing. Unpacking linux-headers-vendor-rk35xx (25.2.3) over (25.2.3) ... Preparing to unpack .../linux-image-vendor-rk35xx_25.2.3_arm64.deb ... Armbian 'linux-image-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'prerm' starting. Armbian 'linux-image-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'prerm' finishing. Armbian 'linux-image-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'preinst' starting. Armbian 'linux-image-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'preinst' finishing. Unpacking linux-image-vendor-rk35xx (25.2.3) over (25.2.3) ... Armbian 'linux-image-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'postrm' starting. Armbian 'linux-image-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'postrm' finishing. Setting up linux-image-vendor-rk35xx (25.2.3) ... Armbian 'linux-image-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'postinst' starting. dkms: running auto installation service for kernel 6.1.99-vendor-rk35xx. dkms: autoinstall for kernel: 6.1.99-vendor-rk35xx. update-initramfs: Generating /boot/initrd.img-6.1.99-vendor-rk35xx update-initramfs: Armbian: Converting to u-boot format: /boot/uInitrd-6.1.99-vendor-rk35xx Image Name: uInitrd Created: Wed Apr 2 09:43:11 2025 Image Type: AArch64 Linux RAMDisk Image (gzip compressed) Data Size: 17096044 Bytes = 16695.36 KiB = 16.30 MiB Load Address: 00000000 Entry Point: 00000000 update-initramfs: Armbian: Symlinking /boot/uInitrd-6.1.99-vendor-rk35xx to /boot/uInitrd '/boot/uInitrd' -> 'uInitrd-6.1.99-vendor-rk35xx' update-initramfs: Armbian: done. Armbian: update last-installed kernel symlink to 'Image'... '/boot/Image' -> 'vmlinuz-6.1.99-vendor-rk35xx' Armbian: Debian compat: linux-update-symlinks install 6.1.99-vendor-rk35xx boot/vmlinuz-6.1.99-vendor-rk35xx Armbian 'linux-image-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'postinst' finishing. Setting up linux-headers-vendor-rk35xx (25.2.3) ... Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'postinst' starting. Configuring kernel-headers (6.1.99-vendor-rk35xx) - please wait ... HOSTCC scripts/basic/fixdep HOSTCC scripts/kconfig/conf.o HOSTCC scripts/kconfig/confdata.o HOSTCC scripts/kconfig/expr.o LEX scripts/kconfig/lexer.lex.c YACC scripts/kconfig/parser.tab.[ch] HOSTCC scripts/kconfig/lexer.lex.o HOSTCC scripts/kconfig/menu.o HOSTCC scripts/kconfig/parser.tab.o HOSTCC scripts/kconfig/preprocess.o HOSTCC scripts/kconfig/symbol.o HOSTCC scripts/kconfig/util.o HOSTLD scripts/kconfig/conf # # configuration written to .config # Compiling kernel-headers scripts (6.1.99-vendor-rk35xx) using 8 CPUs - please wait ... SYNC include/config/auto.conf.cmd HOSTCC scripts/dtc/dtc.o HOSTCC scripts/dtc/flattree.o HOSTCC scripts/dtc/fstree.o HOSTCC scripts/dtc/data.o HOSTCC scripts/dtc/livetree.o HOSTCC scripts/dtc/srcpos.o HOSTCC scripts/dtc/treesource.o HOSTCC scripts/dtc/checks.o HOSTCC scripts/dtc/util.o LEX scripts/dtc/dtc-lexer.lex.c YACC scripts/dtc/dtc-parser.tab.[ch] HOSTCC scripts/dtc/libfdt/fdt.o HOSTCC scripts/dtc/libfdt/fdt_ro.o HOSTCC scripts/dtc/libfdt/fdt_wip.o HOSTCC scripts/dtc/libfdt/fdt_sw.o HOSTCC scripts/dtc/libfdt/fdt_rw.o HOSTCC scripts/dtc/libfdt/fdt_strerror.o HOSTCC scripts/dtc/libfdt/fdt_empty_tree.o HOSTCC scripts/dtc/libfdt/fdt_addresses.o HOSTCC scripts/dtc/libfdt/fdt_overlay.o HOSTCC scripts/dtc/fdtoverlay.o HOSTCC scripts/dtc/dtc-lexer.lex.o HOSTCC scripts/dtc/dtc-parser.tab.o HOSTLD scripts/dtc/fdtoverlay HOSTLD scripts/dtc/dtc HOSTCC scripts/kallsyms HOSTCC scripts/resource_tool HOSTCC scripts/sorttable HOSTCC scripts/asn1_compiler HOSTCC scripts/genksyms/genksyms.o YACC scripts/genksyms/parse.tab.[ch] HOSTCC scripts/selinux/genheaders/genheaders HOSTCC scripts/selinux/mdp/mdp LEX scripts/genksyms/lex.lex.c HOSTCC scripts/genksyms/parse.tab.o HOSTCC scripts/genksyms/lex.lex.o HOSTLD scripts/genksyms/genksyms Compiling kernel-headers scripts/mod (6.1.99-vendor-rk35xx) using 8 CPUs - please wait ... CC scripts/mod/empty.o HOSTCC scripts/mod/mk_elfconfig CC scripts/mod/devicetable-offsets.s UPD scripts/mod/devicetable-offsets.h MKELF scripts/mod/elfconfig.h HOSTCC scripts/mod/modpost.o HOSTCC scripts/mod/sumversion.o HOSTCC scripts/mod/file2alias.o HOSTLD scripts/mod/modpost MODPOST scripts/mod/Module.symvers Done compiling kernel-headers (6.1.99-vendor-rk35xx). Done compiling kernel-headers tools (6.1.99-vendor-rk35xx). Armbian 'linux-headers-vendor-rk35xx' for '6.1.99-vendor-rk35xx': 'postinst' finishing. Setting up armbian-bsp-cli-orangepi5-plus-vendor (25.2.3) ... Armbian 'armbian-bsp-cli-orangepi5-plus-vendor' for '1-PCe1b9-V4064-H2670-B1e5d-R756d': 'postinst' starting. Armbian 'armbian-bsp-cli-orangepi5-plus-vendor' for '1-PCe1b9-V4064-H2670-B1e5d-R756d': 'postinst' finishing. Setting up firefox-nightly (139.0a1~20250401212749) ... Processing triggers for initramfs-tools (0.142+deb12u1) ... update-initramfs: Generating /boot/initrd.img-6.12.12+bpo-arm64-16k W: Possible missing firmware /lib/firmware/rtl_nic/rtl8126a-3.fw for module r8169 W: Possible missing firmware /lib/firmware/rtl_nic/rtl8126a-2.fw for module r8169 update-initramfs: Armbian: Converting to u-boot format: /boot/uInitrd-6.12.12+bpo-arm64-16k Image Name: uInitrd Created: Wed Apr 2 09:44:31 2025 Image Type: AArch64 Linux RAMDisk Image (gzip compressed) Data Size: 57076607 Bytes = 55738.87 KiB = 54.43 MiB Load Address: 00000000 Entry Point: 00000000 update-initramfs: Armbian: Symlinking /boot/uInitrd-6.12.12+bpo-arm64-16k to /boot/uInitrd '/boot/uInitrd' -> 'uInitrd-6.12.12+bpo-arm64-16k' update-initramfs: Armbian: done. Processing triggers for hicolor-icon-theme (0.17-2) ... Processing triggers for man-db (2.11.2-2) ... Processing triggers for mailcap (3.70+nmu1) ... Processing triggers for desktop-file-utils (0.26-1) ... Log ended: 2025-04-02 09:44:34 Any tips how to resolve this and boot from the nvme again? I know initramfs exists and I can wrap my mind arround it what it does, but in the last years never had to solve anything regarding it. I do not know where to look or start debugging 0 Quote
Solution Werner Posted Wednesday at 12:48 PM Solution Posted Wednesday at 12:48 PM Check how and why this kernel is installed: /boot/uInitrd-6.12.12+bpo-arm64-16k Seems like Debian backports kernel which WILL not work with your board. Make sure the vendor kernel is the only kernel installed. Write a fresh image to microsd and boot from it. chroot into your nvme and remove the wrong kernel. Make sure the symlinks to initramfs and kernel image are correct again. Should be sufficient. 0 Quote
Zyxx Posted Wednesday at 03:17 PM Author Posted Wednesday at 03:17 PM (edited) Ty @Werner, I did as you recommended. First I flashed armbian cinnamon to an unused old SD Card. Booted it, chrooted into my nvme and checked apt-mark showmanual. Nothing to be found, no hint of anything related to *6.12.12*. BUT apt-mark showauto did show "linux-image-6.12.12+bpo-arm64-16k" and "linux-image-arm64-16k". Strange... I "apt-get autoremove linux-image-6.12.12+bpo-arm64-16k --purge" and it removed both packets and also "wireguard*". I use wireguard on a daily basis and know it just works in modern kernels... but maybe back in the past, late at night I messed up and confused SSH connection and konsole by mistake? No idea. I thought to be fine, restarted from nvme and same error as in inital thread *sigh*. So back to sd, rebooted into the freshly made SD card, chrooted into the nvme and had a look into /boot on the nvme. There was still a file located uInitrd-6.12.12... I removed it and decided to also regenerate initramfs for good. checked uname -r in chroot konsole and issued this afterwards: "update-initramfs -c -k 6.1.99-vendor-rk35xx" The command proceeded generated the entry in /boot and also updated uInitrd-6.1.99-vendor-rk35xx after this I exited chroot, shutdown the system, removed SD card aaaaand: Went to the forum to say a big THANK YOU 😃 It took hours to feel at home in this installation and I wasn't really happy about the thought of having to rebuild it. Keep up the good work! Edited Wednesday at 03:18 PM by Zyxx typos a lot of typos... 0 Quote
Werner Posted Wednesday at 03:23 PM Posted Wednesday at 03:23 PM 5 minutes ago, Zyxx said: wireguard You don't need the wireguard or wireguard-dkms package since all Armbian images ship the wireguard kernel module by default. All you need is the wireguard-tools package which you can install with the --no-install-recommends switch for apt which will prevent installing useless dependencies like kernels which wont work 0 Quote
Zyxx Posted Wednesday at 05:39 PM Author Posted Wednesday at 05:39 PM Yes, I know. I'd even swear I never installed it, but here I'am. Only user on this PC, looks like I fucked this up Thank you for your guidance! 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.