eselarm

I have currently this on SD-card: root@rock3a:~# ls -1 /boot/vmlinuz-* /boot/vmlinuz-5.10.160-39-rk356x /boot/vmlinuz-6.12.12-current-rockchip64 /boot/vmlinuz-6.13.1-edge-rockchip64 /boot/vmlinuz-6.1.99-vendor-rk35xx Only with the 5.10 kernel from Radxa, I get both NVMe and SATA when I use the overlay 'rock-3a-sata.dtbo'. With 6.12 and 6.13, there is only SATA, no NVMe. 6.1 lets the kernel/board crash, I have not further looked into that. So I can reliably use the SATA HDD, also hd-idle spindown works, I monitor the powerconsumption per day on 12V level and it is great. So as a slow clone/backup NAS it works and is acceptable. But ultimate plan is to make it a fast/main NAS, using the NVMe as cache and also for OS, so only SPI+NVMe+SATA. 5.10 kernel lacks various Btrfs features, so that is a no-go. I need at least 6.1, although I know weird virtualization tricks to get it to work on a functional level, but then also, I might hit issues with KVM on the Radxa rk356x kernel. I got the kernel from: https://radxa-repo.github.io/bookworm/files.list Regarding power, I now use 12V as base (common GND), feed 12V into the Rock3A USB-C connector and 3.5inch 12V pin on HDD and tap 5V from Rock3A GPIO and that feeds the 5V pin of the HDD. As said, that works very nice and stable. So it still looks like the 'rock-3a-sata.dtbo' overlay 'hides/disables all PCIe', where it should not do anything with the PCIe3x2 where the NVMe is on. I saw this: https://patchwork.kernel.org/project/linux-rockchip/patch/20250106100001.1344418-2-amadeus@jmu.edu.cn/ but not sure if it is related.

On the RK3568 (Rock3A) there are: pcie3x2 and pcie2x1. On the RK3566 (Rock3C) only pcie2x1. pcie3x2 seems to be fixed function PCI-E 3.0 2 lanes and I see that related to 'rockchip-dw-pcie 3c0800000.pcie: host bridge /pcie@fe280000' on my Rock3A and that is also where the NMVe is. I am not sure what 'rockchip-naneng-combphy' does or should do; For running the Rock3C rootfs from NVMe on the pcie2x1 multiPHY, it should be PCI-E function at least, not SATA or USB Quick search leads me to: https://lwn.net/Articles/880042/ It will need more study as I at least do not understand the details. At least the U-Boot version gets it right, the kernel does not is my simple conclusion from the post of @north1

Using Radxa U-Boot version "2023.07.02-3-b1eb2bde-gb1eb2bde" makes no difference.

Unfortunately, something random is happening as after a reboot, the NVMe is again not found if SATA overlay is used. The only PCI-e related text in dmesg is then: root@rock3a:~# dmesg | grep -e pcie -e nvme [ 0.174540] /pcie@fe280000: Fixed dependency cycle(s) with /pcie@fe280000/legacy-interrupt-controller [ 4.702382] rockchip-dw-pcie 3c0800000.pcie: probe with driver rockchip-dw-pcie failed with error -110 At least I know now that the used U-Boot does not find my NVME, so that is maybe already the (random) showstopper. So I have to see if maybe a Radxa U-Boot makes it work.

W.r.t. U-boot, I see the following: I have this in SPI (that is used to boot the Rock3A): U-Boot SPL 2024.10-armbian-2024.10-Sf919-Pe8a7-H29de-Vf307-Bb703-R448a (Jan 06 2025 - 02:00:45 +0000) When I stop autoboot and scan: => bootflow scan Card did not respond to voltage select! : -110 Card did not respond to voltage select! : -110 No EFI system partition No EFI system partition Failed to persist EFI variables No EFI system partition Failed to persist EFI variables No EFI system partition Failed to persist EFI variables Card did not respond to voltage select! : -110 pcie_dw_rockchip pcie@fe260000: PCIe-0 Link Fail pcie_dw_rockchip pcie@fe260000: PCIe-0 Link Fail scanning bus for devices... Bus usb@fd000000: Register 2000140 NbrPorts 2 Starting the controller USB XHCI 1.10 Bus usb@fd800000: USB EHCI 1.00 Bus usb@fd880000: USB EHCI 1.00 Bus usb@fd8c0000: USB OHCI 1.0 scanning bus usb@fd000000 for devices... 1 USB Device(s) found scanning bus usb@fd800000 for devices... 3 USB Device(s) found scanning bus usb@fd880000 for devices... 1 USB Device(s) found scanning bus usb@fd8c0000 for devices... 1 USB Device(s) found pcie_dw_rockchip pcie@fe260000: PCIe-0 Link Fail ethernet@fe010000 Waiting for PHY auto negotiation to complete........ done BOOTP broadcast 1 The line with PCIe-0 Link Fail made me think something is wrong, but this is actually correct as there is no PCiE device on the pcie2x1, only a Radxa SATA breakout board. A bootscan list shows: => bootflow list Showing all bootflows Seq Method State Uclass Part Name Filename --- ----------- ------ -------- ---- ------------------------ ---------------- 0 efi_mgr ready (none) 0 <NULL> 1 extlinux ready mmc 1 mmc@fe2b0000.bootdev.part /extlinux/extlinux.conf 2 script ready mmc 1 mmc@fe2b0000.bootdev.part /boot.scr 3 efi ready nvme 1 nvme#0.blk#1.bootdev.part /EFI/BOOT/BOOTAA64.EFI 4 pxe ready ethernet 0 ethernet@fe010000.bootdev extlinux/extlinux.conf --- ----------- ------ -------- ---- ------------------------ ---------------- (5 bootflows, 5 valid) So check via serial console what is happening I would say.

To my surprise this works now again; I boot + root from SD-card, and both NVMe and SATA are there. The change is that I run Armbian Bookworm now instead of Armbian Noble and that I just did update+upgrade; 6.13.0-edge-rockchip64 got newly reinstalled (higher beta trunk number). The Armbian Bookworm is cloned from my NanoPi-R6C (changed some packages from nanopi-r6c -> rock-3a, but for the rest I have no clue why it works now and not a few days ago. The userspace should not matter. Anyway, FYI: root@rock3a:~# strings /dev/mtdblock0 | grep "U-Boot" | head -n1 U-Boot SPL 2024.10-armbian-2024.10-Sf919-Pe8a7-H29de-Vf307-Bb703-R448a (Jan 06 2025 - 02:00:45 +0000) root@rock3a:~# uname -a Linux rock3a 6.13.0-edge-rockchip64 #1 SMP PREEMPT Sun Jan 19 23:51:45 UTC 2025 aarch64 GNU/Linux root@rock3a:~# lspci 0002:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01) 0002:01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 root@rock3a:~# dmesg | grep -e nvme [ 5.949382] nvme nvme0: pci function 0002:01:00.0 [ 5.949860] nvme 0002:01:00.0: enabling device (0000 -> 0002) [ 5.953920] nvme nvme0: missing or invalid SUBNQN field. [ 5.955055] nvme nvme0: D3 entry latency set to 8 seconds [ 5.977503] nvme nvme0: 4/0/0 default/read/poll queues [ 6.007639] nvme0n1: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 [ 17.746225] nvme nvme0: using unchecked data buffer root@rock3a:~# dmesg | grep -e ata1 -e ahci [ 2.904834] ahci-dwc fc800000.sata: supply ahci not found, using dummy regulator [ 2.905739] ahci-dwc fc800000.sata: supply phy not found, using dummy regulator [ 2.906596] ahci-dwc fc800000.sata: supply target not found, using dummy regulator [ 2.907407] ahci-dwc fc800000.sata: PMPn is limited up to 5 ports [ 2.908187] ahci-dwc fc800000.sata: forcing port_map 0x0 -> 0x1 [ 2.908772] ahci-dwc fc800000.sata: AHCI vers 0001.0300, 32 command slots, 6 Gbps, platform mode [ 2.909558] ahci-dwc fc800000.sata: 1/1 ports implemented (port mask 0x1) [ 2.910163] ahci-dwc fc800000.sata: flags: ncq sntf pm led clo only pmp fbs pio slum part ccc apst [ 2.910993] ahci-dwc fc800000.sata: port 0 is not capable of FBS [ 2.919874] scsi host0: ahci-dwc [ 2.920501] ata1: SATA max UDMA/133 mmio [mem 0xfc800000-0xfc800fff] port 0x100 irq 30 lpm-pol 0 [ 3.387739] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 3.388945] ata1.00: ATA-9: WDC WD40EFRX-68WT0N0, 80.00A80, max UDMA/133 [ 3.389578] ata1.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 32) [ 3.390867] ata1.00: configured for UDMA/133 Note that this is using a copied 'rock-3a-sata.dtbo' from the vendor 6.1.84 kernel as I did earlier/initially. I was planning to see if I somehow maybe could modify/reuse 'rockchip-rk3566-sata2.dtbo', but I do not really understand exactly what its de-compiled .dtbo code does.

Interesting, do you have explanation? I have not looked into various systemd-networkd files, but I think it might have to do with the fact that my DNS and router are different; It should not be a problem, but that is what I think. I am not sure anymore what I did, but I got NM installed and a set of .nmconnection files copied from NAS, then got NM running (done via serial console) and did not look back.

@grixm I know the issue you describe from the past 5-10 years (various distros) and made several workarounds depending on distro and release. cron is not default anymore in some other .rpm distro, so I would suggest you debug further and look into dependencies in systemd services and write an own systemd service file with proper dependencies. A problem might be that NetworkManager-wait-online.service gets already passed when still no valid IP+DNS. I don't know the latest status. I see it is disabled on my NanoPi-R6C Armbian Bookworm, but I might have done that myself as I primarily need a bridge there and MAC addresses vary so I set a cloned mac address (on the bridge device) so that DHCP can still work. But a bridge setup can take time (see STP etc), depends on what you do. If no bridges, remember it is not only your SBC but an unknown homelan/router (for us , other forum members) as well, so you better make sure you figure out every trace of NM and networking yourself. I still have a sort of ping-alive-check for 1 SBC to see it it can reach my NAS. That was the only thing I could think of to make it reliable. There is also a method in NM somewhere that you can use to show 'internet works', similar to what Android uses. It is used in openSUSE as extra non-default setting in some .conf file I remember, Armbian I don't know. Ramlog does an rsync, I usually disable or remove it for several reasons. @dnwhoop02 I haven't really looked at that linuxmint forum, it is not Armbian Ubuntu(/Noble?) so better look in more detail at your own system. I recently discovered that netplan generator, it did cost me a lot of time, luckily I use the same .nmconnection files on Armbian Bookworm (VLANs etc), so I got my stuff back, but I did not manage to remove netplan.io. It seems Canonical made a hard dependency between netplan and NM. I might go back to Bookworm (from Noble) or switch to preTrixie/Testing due to that. I know netplan is good for Canonical, but not for me, feels like the new snapd. But in short, I have seen the .yaml files being generated, but that was only once when netplan.io got package update or so. It should not delay network-setup-time, but you need to check in more detail what the relation between netplan and/or netplan.io is I think. The other option, systemd-networkd, does not work out-of-the-box in my LAN (in the minimal images), but maybe it does work for you.

NVMe is back with kernels 6.12.10-current-rockchip64 and 6.13.0-edge-rockchip64. I just keep the same rootfs (ubuntu/noble based, on SD-card currently until issues are sorted out) There was a problem with repositories yesterday. 'Normally', I change the single line in the armbian.list, now I have 2 lines as the u-boot edge package was orphan. So: root@rock3a:/tmp# cat /etc/apt/sources.list.d/armbian.list deb [signed-by=/usr/share/keyrings/armbian.gpg] http://apt.armbian.com noble main noble-utils noble-desktop deb [signed-by=/usr/share/keyrings/armbian.gpg] http://beta.armbian.com noble main noble-utils noble-desktop This allows to install legacy, current, edge (of kernel/DTB, U-Boot). I am not sure about U-Boot versions. My goal is to have/use mainline/Armbian, but It might be that I need to hack an older 2017 or Radxa 2023 version into SD-card boot area (and disable SPI for the time being) as I can't get both NVMe and SATA working at the moment, but that is another (but maybe related) issue. This is specific Rock3A, I haven't looked at details of Rock3C yet.

After kernel version 6.12.6, the overlay to enable SATA on the PCie is not included in the build anymore. So https://github.com/armbian/linux-rockchip/pull/237/commits/5bdf4a4243e6c149290cdd2c8f5a7d1a1946670e is not effective anymore. The existing .dts can be compiled and works with kernel 6.12.10-current-rockchip64 and 6.13.0-edge-rockchip64, but then NVMe is not enabled. From Rock3A schematics and RK3568 datasheet I see that there is no reason that it could not work as the NVMe is on independent PCIe 3x2 lanes in the M.2 M-key connector. PCIe 2x1 lane meant for WiFi/SATA is on E-key connector. It looks like 'entity PCIe is mixed up' somehow. Does anyone have a hint where and how to fix this? Does this also happen on Radxa RK3588 based boards (not the 3588S variant I think)? Workaround now is to use a 2-port SATA JMB58x breakout board. Additional info: root@rock3a:/tmp# strings /dev/mtdblock0 | grep "U-Boot" U-Boot SPL 2024.10-armbian-2024.10-Sf919-Pe8a7-H29de-Vf307-Bb703-R448a (Jan 06 2025 - 02:00:45 +0000) root@rock3a:/tmp# lspci 0000:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01) 0000:01:00.0 SATA controller: JMicron Technology Corp. JMB58x AHCI SATA controller 0002:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01) 0002:01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983

That is not possible as I modify originally downloaded images heavily already at the point when a newly ordered computer/board arrives by mail. So about 2024-12-10 I see I downloaded Armbian_24.11.1_Rock-3a_noble_current_6.6.62_kde-neon-kisak_desktop.img.xz and got that working on my new Rock3A on a 12G extra partition on a 32G SD-card. I converted the rootfs from Ext4 to Btrfs in that step and might have used a 6.12.x edge/beta kernel. At least it worked very nice although I use Rock3A now as headless server. Btrfs allows streams to send from one to the other computer/filesystem, that I did, so now the same rootfs tree is running on my NanoPi-R6C (RK3588S, 8GB RAM). AND I copied/merged the edge/beta 6.13.0 kernel into the root tree. I use a dedicated extra FAT partition to boot (not Armbian compliant). I just upgraded, now it runs KDE6.3beta I see, still great GUI acceleration as before upgrade ( was KDE6.2.x). But I won't use it much as I am more fan of openSUSE Tumbleweed for desktop usage, that also works great (on 6.13.0 rockchip64 Armbian kernel). So what image doesn't really matter, you need to run normal apt upgrades and make sure you have the right OS components. An image is mostly some older pre-installation. As I indicated, this all is quite impossible if you do not have/use a proper serial console. I did the OS cloning all remote via serial console. I just saw later that the KDE login screen was as expected when I went to the room where the NanoPi-R6C is. Monitor is an old 24 inch LG 1080p60, a bit flaky HDMI connector but it works if I don't pull on cables.

You should have and use a USB serial console cable first. I hope you have heard of that before, it costs about 2 Euros. Shipping costs might be 10 Euros or so. Maybe also try to get some non-4K HDMI monitor. I don't know specifics about OP5+, but I have no issues with RK3588S and RK3568 to do 1080p60 on a non-4K monitor. Worked since kernel 6.8 or so. The 3 points you want don't require build because it is available. I have it running on my NanoPi-R6C and kernel 6.13 you can get by switching to beta repo.

Usually, the way to achieve this is a video=... kernel cmdline statement. You can add it in armbianEnv.txt. Look in general kernel commandline docs for video= what exactly you need to specify for your particular board.

@Dr Dread I can only comment on the last screenshot: Apparently the error encountered cannot be fixed, so startup won't continue. Make sure the SD-card is a gooed one and that the filesystem on it is correct (not damages or fatally corrupted. Yo might need to start/try with another new SD-card where you did write a latest Armbian image for the N2+. Yo can also fix/check the SD-card in a Linux PC. Mixing up USB and SD-card (and TFTP) and not knowing what bootloader is used makes me think that you need to take some time for a more structured approach.

With U-Boot in SPI: U-Boot SPL 2024.10-armbian-2024.10-Sf919-Pe8a7-H29de-Vf307-Bb703-R448a (Jan 06 2025 - 02:00:45 +0000): - I can confirm that there is no NVMe when kernel 6.12.9-current-rockchip64 (I did take out a Samsung 970 EVO 500G from another computer temporarily) - it works with 6.12.6-edge-rockchip64, 6.6.67-current-rockchip64, 6.13.0-rc5-edge-rockchip64 Another issue is that when overlay rock-3a-sata.dtbo is used, the SATA HDD is available, but the NVMe is not there then. It looks like mixing up of PCIe entities (in general), that is my guess.

It looks like some more generic issue with PCIe/NVMe as Rock4A is Rk3399, not RK35xx. On my RK3588S NVMe works OK with 6.12.9 and 6.13.0-rc5 (2024.10 U-Boot loader in eMMC). I hope you have backups, so then there is always tricks to copy 6.6.x kernel+DTB+modules from there and get it to boot again. You will need spare/temporary SD-card I think. I do rather crazy things with SBC Linux installs, so I have my methods to flip to an older/other kernel (Btrfs snapshots). SBC/embedded Linux does not have GRUB bootmanager by default, otherwise that would be the way to select older kernel. You can always download and older image (from archives) and use that (for manually extracting older kernel). Same as my earlier comment for Rock3C, I think newer U-Boot is needed. It also might be a power stability issue. I know with my Rock3A I needed even to look in schematics before I got some trust in the board. The 6.12.9 kernel might initialize things in a different way/sequence, but all pure guessing. I do not know the power architecture of the Rock3C nor Rock4A. For my Rock3A, I soldered a 12V wire pair to a USB-C DIY connector, so the board uses its own 5V 8A DC/DC convertor. That works fine.

I almost never use armbian-config, so don't know if it has options to limit partitions sizes, I think there are, but you will have to look in the script how things are done. But you can boot from SD-card with another independent Armbian installation and from there run a partitions program on the eMMC. Gparted I would say, it should allow you to shrink/resize existing partition and filesystem. An add an extra 1 or more. I usually use gdisk or fdisk/sfdisk via serial console (CLI) and Btrfs as filesystem (not Ext4), so cannot really tell how parted works, but what you want is possible, up to 128 partitions if you want.

I think 6.12.9 is not compatible with the U-Boot versions you used so far. I would try with a 2024.10 mainline U-Boot. I did disable teh SPI flash, see Radxa docs/wiki ( you can also erase it) and then wrote a 2024.10 mainline U-Boot in the boot area of the SD-card. It was from the edge/beta repo a month ago. Currently: root@rock3a:~# apt list| grep rock-3c armbian-bsp-cli-rock-3c-current/noble 25.2.0-trunk.340 arm64 armbian-bsp-cli-rock-3c-edge/noble 25.2.0-trunk.340 arm64 armbian-bsp-cli-rock-3c/noble 25.2.0-trunk.340 arm64 linux-u-boot-rock-3c-current/noble 25.2.0-trunk.340 arm64 linux-u-boot-rock-3c-edge/noble 25.2.0-trunk.340 arm64

What U-Boot version does it run then? I have used also a Radxa version 2023.10 or so pulled as blob from the latest Radxa Rock3A image. Rock3C might be different as especially PCIe is different for RK3568 and RK3566.

You could try: - mainline U-Boot - run rootfs from SD-card and then see if NVMe / PCIe is there and try to mount a filesystem on it. I run OS (edge) from SD-card, so can't tell if NVMe works with >= 6.12.9 I would need to order extra NVMe first.

@asayed Check the usual things like free space etc. And there is a full trace log, I think you have to look into those files. Or just clean/wipe everything and start again, maybe you got some hints/tips/better understanding in the meantime.

Same here for x86-64, but since half a year or so same for aarch64. So I have 2 Ubuntu22 VMs: 1 x86-64 and 1 aarch64. But it turned out that building directly on both RPiOS Bookworm and Armbian Bookworm also works fine. It is mostly compiling Arm kernel and U-Boot, both 32-bit and 64-bit. 32-bit I did via systemd-nspawn of some rootfs-tree clone of a NanoPi-NEO.

A quick look at the shell code makes me think that it is a different purpose compared to how I use virtualization. I use only HW accelerated based virtualization, so -enable-kvm flag is missing, but might be added by virt-install. Also mostly Arm on Arm although I also use x86 on x86 as well. I do not use qcow2, but just flat images or a physical SD-card or NBD or LVM. Before I got my Rock3A, I downloaded the Armbian Noble image for it (was 6.6.62) but put a 6.12 *rk3588 kernel on it as well. With the default Armbian U-Boot for QEMU I could then prepare everything and already 'run' my Rock3A as a VM on NanoPi-R6C like this: taskset --cpu-list 0-3 qemu-system-aarch64 \ -M virt -cpu host -enable-kvm -m 2048 -smp 4 \ -bios u-boot.bin \ -drive if=none,file=rock3a.img,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -netdev bridge,id=hn1 -device virtio-net,netdev=hn1,mac=xx:xx:xx:xx:xx:xx \ -nographic The taskset is needed if host runs 6.1.x vendor kernel and also makes it more realistic w.r.t. speed as then the VM gets 4x Cortex-A55, same as real Rock3A. The 6.12/rk3588 kernel (inside the VM) was needed as that one has all included w.r.t. machine virt (same as vanilla Debian Bookworm kernel). A note is that I later on added Btrfs support to a custom build u-boot for QEMU and converted Ext4 in rock3a.img into Btrfs, but principle is the same. Btrfs offers online shrink and differential snapshots that I use for remote backups. And with a MAC-address equal to the real Rock3A, it is easy flipping between real and VM. Also note is that the following file needs to be created on the host, assuming it uses br0 as base for network I/O: # cat /etc/qemu/bridge.conf allow br0 To use full graphics in VM and for multi-year running VM on my RPi4B, I also use virsh and UEFI. But usually only CLI is needed.

I had to look it up: https://docs.radxa.com/img/rock5itx/rock5itx-system-block-diagram.webp as I was not sure if it was on-chip (RK35xx) or extra. The latter is the case. On a Rock3A I also had success with m.2 e-key port with JMB582, but not used now, instead I use on-chip SATA via overlay. That makes make think that maybe for the Rock5ITX the ASM1164 kernel code is not compiled as module but in-kernel and so maybe there are timing issues in initialization at boot-up. There should not be, but that is what I would look at first as this is PCIe and we know there have been issues with it. If you do not need vendor kernel specifically, you might try 6.12 current/mainline. Also see/check what U-Boot the board boots with.

You could run: sudo apt list | grep linux-image Besides the vanilla Debian/Ubuntu ones, there are Armbian ones: vendor, current, edge In your case your installed U-Boot might be incompatible with kernel 6.1.84, I had that for my Rock3A, I installed mainline U-Boot. If you want all kernels, use http://beta.armbian.com in /etc/apt/sources.list.d/armbian.list Or build yourself from source. The Armbian build runs fine on my NanoPi-R6C Armbian Bookworm