eselarm

Not even building, just adding/installing the edge kernel to a working/running Rock3A image.

While updating/upgrading my NanoPi-R6C, I see there is now: /usr/lib/linux-image-6.13.2-edge-rockchip64/rockchip/rk3568-rock-3b.dtb I do not know if the earlier mentioned patch is included, but at least that DTB file is different from the rk3568-rock-3a.dtb one. So if you add or switch to the beta repo, you can simply install the edge kernel. And as already mentioned in other post, check U-Boot version. If you want generic headless ARM64 computing, use mainline U-Boot and mainline kernel. You can't use a few things like HEVC, H264 (trans)coding, but overall it works fine. I ran even full desktop KDE neon on my Rock3A, it is good Wayland GUI speed. I got my Rock3A 1st week of december 2024, so that was in transition from legacy U-Boot to mainline U-Boot and also 6.6.x to 6.12+ kernel. 6.12+ (the rk3588 or now rockchip64) has several great features enabled/included by default like was already always default in vanilla Debian AMD64/ARM64 kernels, so easy to get things working like for PC/AMD64.

Various things can be wrong; Yours might have an other cause then the one from @ff255. In the posted U-Boot log I also see JMicron; that brand is notorious for failing UAS and trim, although one should see errors (from U-Boot). Also the U-Boot shown is older and not Armbian originated I see. Also compression type for initramfs might be the failure reason. For my RK3688s and RK3568 SBCs, it really makes a differences what U-Boot I use. For mainline U-Boot, general Linux might/can work better, but some HW support might go wrong. So look what your objective is, what is the use-case of the SBC. You can make sure newer U-Boot is used, I don't know if it is available for rock64 family. Ultimately, those 6.12+ mainline kernels support EFI and virt virtual machines, so the whole image content except the bootloader should work in an aarch64 virtual machine; there you could see if kernel + generated initramfs is the problem, or just rock64 DTB, U-Boot on your board. Or as said the storage chain. I have seen endless issues with RaspberryPi(4) that are not a Linux issue, but just some HW/firmware issues originating from RPI platform. Can be similar for other brands boards.

I got 302.87 MB/sec with an older SATA SSD on the "01:00.0 SATA controller: JMicron Technology Corp. JMB58x AHCI SATA controller". That was with a 4T HDD on the second port of that controller. Have not used SATA SSD with native/on-SoC SATA controller. The fear is that when using this dual-port JMB for SSD+HDD and later 2.5Gbps ethernet, that the pcie2x1 will be the bottleneck (when Rock5B or so). An SSD caching a HDD involves more traffic on the busses than just a JBOD setup. I have had such a bottleneck in the past (for Intel motherboard, 6x SATA3). I'll have to do detailed calculations if this is the case now for Rockchip SBC. With NVMe, pcie3x2 one can see/assume there will be no bottleneck and future is NVMe, I likely won't buy new HDD's. Advantage for native SATA is likely not real for many people. But I like as much as possible on the same SoC, for various reasons. I know many people will buy an RPi5 or so and some USB3 enclosure for HDD, but I have had enough of such a setup (issues with USB3). Count the amount of extra entities you will need drivers etc for. As I indicated earlier, it is comparing Intel solution with something Arm (Rockchip in this case), not adding extra foreign interface chips. For JMicron, always the question if features like trim work. Current quick test (2017.09-armbian (legacy package) U-Boot, armbian vendor kernel 6.1.99😞 root@rock3a:/local/s0/tmp# dd if=/dev/nvme0n1p12 iflag=direct bs=4M of=/dev/null count=10k status=progress 42253418496 bytes (42 GB, 39 GiB) copied, 31 s, 1.4 GB/s root@rock3a:/local/s0/tmp# time dd if=/dev/zero of=dummyfile oflag=direct bs=4M count=1k 4294967296 bytes (4.3 GB, 4.0 GiB) copied, 7.71642 s, 557 MB/s root@rock3a:/local/s0/tmp# hdparm -t --direct /dev/nvme0n1p12 /dev/nvme0n1p12: Timing O_DIRECT disk reads: 3082 MB in 3.00 seconds = 1026.81 MB/sec root@rock3a:/local/s0/tmp# time dd of=/dev/null if=dummyfile iflag=direct bs=4M 4294967296 bytes (4.3 GB, 4.0 GiB) copied, 4.09936 s, 1.0 GB/s root@rock3a:/local/s0/tmp# hdparm -t --direct /dev/sda /dev/sda: Timing O_DIRECT disk reads: 580 MB in 3.01 seconds = 192.95 MB/sec Another note now is that system idle powerconsumption is 3.74 Watt. I use another HDD now, but I think it is the NVMe that is not going in powersave, aim is about 2W idle. So more research, measurements, etc is needed. But I still have not done/enabled the actual caching setup. That is a critical thing as any disturbance in the uptime and relation between NVMe and HDD will ruin the 8T filesystem badly. So first will see if that works for a month or so, when rsync, scrub, differential send|receive, power cut, etc.

I have copied/merged the latest Radxa rk356x kernel from the official latest Radxa Bullseye CLI image into Armbian Bookworm that runs on my Rock3A and I see there is the file: /usr/lib/linux-image-5.10.160-39-rk356x/rockchip/rk3568-rock-3b.dtb Same .DTB file is not in Armbian vendor (6.1.99) or mainline/current (6.12.13), which means no Armbian support for Rock3b. But if you use a current Rock3A image as base and copy that old 5.10 kernel into the image and also put the 'rk3568-rock-3b.dtb' in the armbianEnv.txt, I guess it will boot and recognize all Rock3B hardware. Rock3A images use an extra FAT boot partition, so make sure you understand which files and under which names you need to copy. I mount the bootFAT in a own custom folder /boot/uboot, so I keep multiple kernels in /boot (which is rootfs) and copy a particular one to the bootFAT.

That is weird; 'ssd-sata' is not nvme. And it is a rk3588s, so which multiPHY is then put in SATA mode? Not the one operating (connected to) the NVMe device one would assume. I do not know HW architecture of OPi5, maybe you lost USB3 function now? And where did you get that overlay from? I don't see it: /usr/lib/linux-image-6.1.99-vendor-rk35xx# find . -name "*sata*" ./rockchip/overlay/mixtile-blade3-sata2.dtbo ./rockchip/overlay/orangepi-5-plus-sata2.dtbo ./rockchip/overlay/orangepi-5-sata.dtbo ./rockchip/overlay/radxa-cm5-io-sata.dtbo ./rockchip/overlay/rk3566-roc-pc-sata2.dtbo ./rockchip/overlay/rk3588s-roc-pc-sata0.dtbo ./rockchip/overlay/rock-3a-sata.dtbo ./rockchip/overlay/rock-5a-sata.dtbo ./rockchip/overlay/rock-5b-sata.dtbo ./rockchip/overlay/turing-rk1-sata2.dtbo ./rockchip/overlay/yy3568-sata2.dtbo Or is it renamed 'orangepi-5-sata' ? Are you sure it is actually loaded?

Another trial: I put the legacy rock-3a u-boot in the SPI, which is: U-Boot SPL 2017.09-armbian (May 20 2024 - 00:46:51) Then, with kernel 6.1.99-vendor-rk35xx, both NVMe and SATA are available when using the 'rock-3a-sata.dtbo' overlay. It failed to see/boot from SD-card, but TFTP and NVMe work. The newer/mainline kernels still only SATA if overlay used. If no overlay used, only NVMe. MAC address is changed and several other issues, but with this SW combination, I can test the HW/system performance w.r.t. 'caching' NAS I think.

you need to do: sudo apt install virt-manager Then it should also have installed qemu-system-arm package which contains the core virtualization program for 32-bit and 64-bit Virtual machines that you can create and run with/via virt-manager GUI. Note tha you run vendor 6.1.x kernel on big-LITTLE rk3588 SoC, it will fail/crash any VM if more than 1 core allocated or not fixed the CPU types in the VM (xml code). So best to just use 1 core. Otherwhise switch to mainline/current 6.12+ kernel, there the bug/problem is fixed.

That I consider a workaround. There are 3 SATA capable SERDES ports available on RK3568, similar for other Rockchips. What I see is that idle powerconsumpution went up by 2 Watt when using an extra JMB PCIe-SATA chip, I have not look in detail why that is. I am aiming for battery powered as well, so when I see that Rock3A itself is roughly 2W idle (with mainline kernel), doubling that would make an other platform/SoC more or less a better solution. SoC powerconsumption does not really matter if the 3.5 inch HDD spins all the time, but that is not the case/plan. Also a SATA SSD is an option for connecting. I do not have a RK3588 (but have a RK3588S), I indeed guessed that SPI+NVMe+SATA will work on Rock5B when using 'rock-5b-sata.dtbo'. Or do you use a different overlay with edge kernel? Kernel 6.1 has a problem that it does not allow seamless moving of virtual machines (between A76 and A55). Mainline (6.8+ etc) does not have that problem. There must be some (longterm) benefit for using Arm platform instead of continuing Intel (like N100, that is the alternative I keep in mind). So maybe Rock5B 16GB I should buy, that would be a good successor of my current Intel based NAS (and VM host as well every now and then).

I have currently this on SD-card: root@rock3a:~# ls -1 /boot/vmlinuz-* /boot/vmlinuz-5.10.160-39-rk356x /boot/vmlinuz-6.12.12-current-rockchip64 /boot/vmlinuz-6.13.1-edge-rockchip64 /boot/vmlinuz-6.1.99-vendor-rk35xx Only with the 5.10 kernel from Radxa, I get both NVMe and SATA when I use the overlay 'rock-3a-sata.dtbo'. With 6.12 and 6.13, there is only SATA, no NVMe. 6.1 lets the kernel/board crash, I have not further looked into that. So I can reliably use the SATA HDD, also hd-idle spindown works, I monitor the powerconsumption per day on 12V level and it is great. So as a slow clone/backup NAS it works and is acceptable. But ultimate plan is to make it a fast/main NAS, using the NVMe as cache and also for OS, so only SPI+NVMe+SATA. 5.10 kernel lacks various Btrfs features, so that is a no-go. I need at least 6.1, although I know weird virtualization tricks to get it to work on a functional level, but then also, I might hit issues with KVM on the Radxa rk356x kernel. I got the kernel from: https://radxa-repo.github.io/bookworm/files.list Regarding power, I now use 12V as base (common GND), feed 12V into the Rock3A USB-C connector and 3.5inch 12V pin on HDD and tap 5V from Rock3A GPIO and that feeds the 5V pin of the HDD. As said, that works very nice and stable. So it still looks like the 'rock-3a-sata.dtbo' overlay 'hides/disables all PCIe', where it should not do anything with the PCIe3x2 where the NVMe is on. I saw this: https://patchwork.kernel.org/project/linux-rockchip/patch/20250106100001.1344418-2-amadeus@jmu.edu.cn/ but not sure if it is related.

On the RK3568 (Rock3A) there are: pcie3x2 and pcie2x1. On the RK3566 (Rock3C) only pcie2x1. pcie3x2 seems to be fixed function PCI-E 3.0 2 lanes and I see that related to 'rockchip-dw-pcie 3c0800000.pcie: host bridge /pcie@fe280000' on my Rock3A and that is also where the NMVe is. I am not sure what 'rockchip-naneng-combphy' does or should do; For running the Rock3C rootfs from NVMe on the pcie2x1 multiPHY, it should be PCI-E function at least, not SATA or USB Quick search leads me to: https://lwn.net/Articles/880042/ It will need more study as I at least do not understand the details. At least the U-Boot version gets it right, the kernel does not is my simple conclusion from the post of @north1

Using Radxa U-Boot version "2023.07.02-3-b1eb2bde-gb1eb2bde" makes no difference.

Unfortunately, something random is happening as after a reboot, the NVMe is again not found if SATA overlay is used. The only PCI-e related text in dmesg is then: root@rock3a:~# dmesg | grep -e pcie -e nvme [ 0.174540] /pcie@fe280000: Fixed dependency cycle(s) with /pcie@fe280000/legacy-interrupt-controller [ 4.702382] rockchip-dw-pcie 3c0800000.pcie: probe with driver rockchip-dw-pcie failed with error -110 At least I know now that the used U-Boot does not find my NVME, so that is maybe already the (random) showstopper. So I have to see if maybe a Radxa U-Boot makes it work.

W.r.t. U-boot, I see the following: I have this in SPI (that is used to boot the Rock3A): U-Boot SPL 2024.10-armbian-2024.10-Sf919-Pe8a7-H29de-Vf307-Bb703-R448a (Jan 06 2025 - 02:00:45 +0000) When I stop autoboot and scan: => bootflow scan Card did not respond to voltage select! : -110 Card did not respond to voltage select! : -110 No EFI system partition No EFI system partition Failed to persist EFI variables No EFI system partition Failed to persist EFI variables No EFI system partition Failed to persist EFI variables Card did not respond to voltage select! : -110 pcie_dw_rockchip pcie@fe260000: PCIe-0 Link Fail pcie_dw_rockchip pcie@fe260000: PCIe-0 Link Fail scanning bus for devices... Bus usb@fd000000: Register 2000140 NbrPorts 2 Starting the controller USB XHCI 1.10 Bus usb@fd800000: USB EHCI 1.00 Bus usb@fd880000: USB EHCI 1.00 Bus usb@fd8c0000: USB OHCI 1.0 scanning bus usb@fd000000 for devices... 1 USB Device(s) found scanning bus usb@fd800000 for devices... 3 USB Device(s) found scanning bus usb@fd880000 for devices... 1 USB Device(s) found scanning bus usb@fd8c0000 for devices... 1 USB Device(s) found pcie_dw_rockchip pcie@fe260000: PCIe-0 Link Fail ethernet@fe010000 Waiting for PHY auto negotiation to complete........ done BOOTP broadcast 1 The line with PCIe-0 Link Fail made me think something is wrong, but this is actually correct as there is no PCiE device on the pcie2x1, only a Radxa SATA breakout board. A bootscan list shows: => bootflow list Showing all bootflows Seq Method State Uclass Part Name Filename --- ----------- ------ -------- ---- ------------------------ ---------------- 0 efi_mgr ready (none) 0 <NULL> 1 extlinux ready mmc 1 mmc@fe2b0000.bootdev.part /extlinux/extlinux.conf 2 script ready mmc 1 mmc@fe2b0000.bootdev.part /boot.scr 3 efi ready nvme 1 nvme#0.blk#1.bootdev.part /EFI/BOOT/BOOTAA64.EFI 4 pxe ready ethernet 0 ethernet@fe010000.bootdev extlinux/extlinux.conf --- ----------- ------ -------- ---- ------------------------ ---------------- (5 bootflows, 5 valid) So check via serial console what is happening I would say.

To my surprise this works now again; I boot + root from SD-card, and both NVMe and SATA are there. The change is that I run Armbian Bookworm now instead of Armbian Noble and that I just did update+upgrade; 6.13.0-edge-rockchip64 got newly reinstalled (higher beta trunk number). The Armbian Bookworm is cloned from my NanoPi-R6C (changed some packages from nanopi-r6c -> rock-3a, but for the rest I have no clue why it works now and not a few days ago. The userspace should not matter. Anyway, FYI: root@rock3a:~# strings /dev/mtdblock0 | grep "U-Boot" | head -n1 U-Boot SPL 2024.10-armbian-2024.10-Sf919-Pe8a7-H29de-Vf307-Bb703-R448a (Jan 06 2025 - 02:00:45 +0000) root@rock3a:~# uname -a Linux rock3a 6.13.0-edge-rockchip64 #1 SMP PREEMPT Sun Jan 19 23:51:45 UTC 2025 aarch64 GNU/Linux root@rock3a:~# lspci 0002:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01) 0002:01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 root@rock3a:~# dmesg | grep -e nvme [ 5.949382] nvme nvme0: pci function 0002:01:00.0 [ 5.949860] nvme 0002:01:00.0: enabling device (0000 -> 0002) [ 5.953920] nvme nvme0: missing or invalid SUBNQN field. [ 5.955055] nvme nvme0: D3 entry latency set to 8 seconds [ 5.977503] nvme nvme0: 4/0/0 default/read/poll queues [ 6.007639] nvme0n1: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 [ 17.746225] nvme nvme0: using unchecked data buffer root@rock3a:~# dmesg | grep -e ata1 -e ahci [ 2.904834] ahci-dwc fc800000.sata: supply ahci not found, using dummy regulator [ 2.905739] ahci-dwc fc800000.sata: supply phy not found, using dummy regulator [ 2.906596] ahci-dwc fc800000.sata: supply target not found, using dummy regulator [ 2.907407] ahci-dwc fc800000.sata: PMPn is limited up to 5 ports [ 2.908187] ahci-dwc fc800000.sata: forcing port_map 0x0 -> 0x1 [ 2.908772] ahci-dwc fc800000.sata: AHCI vers 0001.0300, 32 command slots, 6 Gbps, platform mode [ 2.909558] ahci-dwc fc800000.sata: 1/1 ports implemented (port mask 0x1) [ 2.910163] ahci-dwc fc800000.sata: flags: ncq sntf pm led clo only pmp fbs pio slum part ccc apst [ 2.910993] ahci-dwc fc800000.sata: port 0 is not capable of FBS [ 2.919874] scsi host0: ahci-dwc [ 2.920501] ata1: SATA max UDMA/133 mmio [mem 0xfc800000-0xfc800fff] port 0x100 irq 30 lpm-pol 0 [ 3.387739] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 3.388945] ata1.00: ATA-9: WDC WD40EFRX-68WT0N0, 80.00A80, max UDMA/133 [ 3.389578] ata1.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 32) [ 3.390867] ata1.00: configured for UDMA/133 Note that this is using a copied 'rock-3a-sata.dtbo' from the vendor 6.1.84 kernel as I did earlier/initially. I was planning to see if I somehow maybe could modify/reuse 'rockchip-rk3566-sata2.dtbo', but I do not really understand exactly what its de-compiled .dtbo code does.