ScottN Posted December 31, 2024 Posted December 31, 2024 Hello all. Trying to get PCIe/NVMe working (even to show on microSD boot, let alone boot NVMe) on a custom compile on my Orange Pi 5 SBC version 1.3.2, because the official images aren't working either for me. No idea where to go from here and need some assistance as NVMe is required for my purpose. Here is what I did during compile.sh > Show kernel config > Board: orangepi5 > Kernel: current > Target: jammy > Image with desktop > Gnome desktop > Base configuration > Tick: Browsers > Device Drivers > PCI support > (default values) > Device Drivers > NVME Support > (default values) I used "SHARE_LOG=yes" but got below error at the end so I'm guessing it did not upload. I'm attaching it instead. SHARE_LOG=yes, uploading log [ uploading logs ] Log uploaded, share URL: [ <!DOCTYPE html> <html lang=en> <head> <meta charset=utf-8> <title>Error</title> </head> <body> <pre>Cannot POST /log</pre> </body> </html> ] Repeat build options Repeat Build Options [ ./compile.sh build BOARD=orangepi5 BRANCH=current BUILD_DESKTOP=yes BUILD_MINIMAL=no DESKTOP_APPGROUPS_SELECTED=browsers DESKTOP_ENVIRONMENT=gnome DESKTOP_ENVIRONMENT_CONFIG_NAME=config_base KERNEL_CONFIGURE=yes RELEASE=jammy SHARE_LOG=yes ] Side notes: I've tested with Joshua-Riek's ubuntu-rockchip builds here https://github.com/Joshua-Riek/ubuntu-rockchip/releases After writing the 22.04 version 3 to the microSD card, then boot to the microSD card on the SBC, doing lsblk lists the NVMe drive just fine. I use the tools this provides to copy the current configuration to the NVMe, update the bootloader, shutdown and only attempt to boot from NVMe and it does not work. So that is working a bit more than Armbian, but still not dedicated NVMe boot. I found this issue that talked about some Armbian changes to the DTS files for the arm64 boot on linux-rockchip, that seems to be the NVMe is not enabled? Here is that link: https://github.com/Joshua-Riek/ubuntu-rockchip/issues/709 One person was able to re-compile the DTS to DTB and manually update and dmesg output seems to work. This is where I'm lost as I've never compiled the device tree before. Assistance is greatly appreciated! Thanks! log-build-7d071dfd-1b54-45cf-8488-2bc8b5e62079.zip 0 Quote
Werner Posted December 31, 2024 Posted December 31, 2024 3 hours ago, ScottN said: Kernel: current mainline support for rk3588 is far from feature-complete and still under heavy development. If you need things just to work for now use vendor branch. 0 Quote
ScottN Posted December 31, 2024 Author Posted December 31, 2024 Hello, @Werner thank you for your response. I was confused with what current meant and made my own assumptions that would be the most feature rich and stable, and the others were less reliable. OK, so I did the script again and now the output filename is "Armbian-unofficial_25.02.0-trunk_Orangepi5_jammy_vendor_6.1.84_gnome_desktop.img" which I assume it's using kernel version 6.1 I'm still experiencing the problem with no NVMe drive detected when booting that image. The NVMe drive is connected and like I mentioned above when using the microSD from Josua-Riek's ubuntu-rockchip the drive is detected, although the u-boot/bootloader does not know what to do with it on power on. I have a UART connection and will attempt to get some more information on this and also Armbian image when booting microSD. Here is the "lsblk" and "lspci" results of the booted image _ _ _ __ __ _ _ _ /_\ _ _ _ __ | |__(_)__ _ _ _ ___ _ _ _ _ ___ / _|/ _(_)__(_)__ _| | / _ \| '_| ' \| '_ \ / _` | ' \___| || | ' \/ _ \ _| _| / _| / _` | | /_/ \_\_| |_|_|_|_.__/_\__,_|_||_| \_,_|_||_\___/_| |_| |_\__|_\__,_|_| v25.02 rolling for Orange Pi 5 running Armbian Linux 6.1.84-vendor-rk35xx Packages: Ubuntu stable (jammy), possible distro upgrade (noble) Updates: Kernel upgrade enabled and 1 package available for upgrade Support: DIY (custom image) IP addresses: (LAN) IPv4: 192.168.1.117 IPv6: fe80::64ef:39d1:7886:e27b (WAN) 47.156.182.141 Performance: Load: 6% Up time: 2 min Local users: 2 Memory usage: 13% of 3.64G CPU temp: 35°C Usage of /: 14% of 29G Commands: Configuration : armbian-config Upgrade : armbian-upgrade Monitoring : htop Last login: Tue Dec 31 05:44:35 PST 2024 on ttyFIQ0 root@orangepi5:~# lspci root@orangepi5:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS mtdblock0 31:0 0 16M 0 disk mmcblk1 179:0 0 29.7G 0 disk \u2514\u2500mmcblk1p1 179:1 0 29.4G 0 part /var/log.hdd / zram0 253:0 0 1.8G 0 disk [SWAP] zram1 253:1 0 50M 0 disk /var/log zram2 253:2 0 0B 0 disk root@orangepi5:~# Here is the dmesg output of pcie root@orangepi5:~# dmesg | grep -i pcie [ 16.110866] reg-fixed-voltage vcc3v3-pcie2x1l2: Looking up vin-supply from device tree [ 16.110871] vcc3v3_pcie2x1l2: supplied by vcc5v0_sys [ 16.169012] vcc3v3_pcie2x1l2: 1800 mV, enabled [ 16.169080] reg-fixed-voltage vcc3v3-pcie2x1l2: vcc3v3_pcie2x1l2 supplying 1800000uV [ 16.231679] dw-pcie fe190000.pcie: invalid resource [ 16.231683] dw-pcie fe190000.pcie: Failed to initialize host [ 16.231686] dw-pcie: probe of fe190000.pcie failed with error -22 [ 16.232169] rk-pcie fe190000.pcie: invalid prsnt-gpios property in node [ 16.232215] rk-pcie fe190000.pcie: Looking up vpcie3v3-supply from device tree [ 16.232671] rk-pcie fe190000.pcie: host bridge /pcie@fe190000 ranges: [ 16.232686] rk-pcie fe190000.pcie: IO 0x00f4100000..0x00f41fffff -> 0x00f4100000 [ 16.232694] rk-pcie fe190000.pcie: MEM 0x00f4200000..0x00f4ffffff -> 0x00f4200000 [ 16.232700] rk-pcie fe190000.pcie: MEM 0x0a00000000..0x0a3fffffff -> 0x0a00000000 [ 16.232740] rk-pcie fe190000.pcie: iATU unroll: enabled [ 16.232743] rk-pcie fe190000.pcie: iATU regions: 8 ob, 8 ib, align 64K, limit 8G [ 16.434896] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 16.455911] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 16.476923] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 16.497935] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 16.518946] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 16.539957] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 16.560969] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 16.581981] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 16.602992] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 16.624003] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 17.111478] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up pcie-supply from device tree [ 17.111528] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up pcie-supply property in node /power-management@fd8d8000/power-controller failed [ 18.532374] rk-pcie fe190000.pcie: PCIe Link Fail, LTSSM is 0x3, hw_retries=0 [ 21.449527] rk_pcie_establish_link: 172 callbacks suppressed [ 21.449539] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.470573] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.491663] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.511769] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.532795] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.553875] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.574953] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.596031] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.617109] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.638188] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 21.827150] rk-pcie fe190000.pcie: PCIe Link Fail, LTSSM is 0x3, hw_retries=1 [ 22.845095] rk-pcie fe190000.pcie: failed to initialize host root@orangepi5:~# 0 Quote
ScottN Posted December 31, 2024 Author Posted December 31, 2024 Here is the UART output of booting the opi5 on the unofficial 6.1 image I made if this is of value [tio 05:58:43] tio v1.32 [tio 05:58:43] Press ctrl-t q to quit [tio 05:58:43] Connected DDR Version V1.08 20220617 LPDDR4X, 2112MHz channel[0] BW=16 Col=10 Bk=8 CS0 Row=16 CS=1 Die BW=16 Size=1024MB channel[1] BW=16 Col=10 Bk=8 CS0 Row=16 CS=1 Die BW=16 Size=1024MB channel[2] BW=16 Col=10 Bk=8 CS0 Row=16 CS=1 Die BW=16 Size=1024MB channel[3] BW=16 Col=10 Bk=8 CS0 Row=16 CS=1 Die BW=16 Size=1024MB Manufacturer ID:0xff CH0 RX Vref:28.7%, TX Vref:22.8%,0.0% CH1 RX Vref:31.7%, TX Vref:23.8%,0.0% CH2 RX Vref:31.7%, TX Vref:20.8%,0.0% CH3 RX Vref:30.7%, TX Vref:22.8%,0.0% change to F1: 528MHz change to F2: 1068MHz change to F3: 1560MHz change to F0: 2112MHz out U-Boot SPL board init U-Boot SPL 2017.09-orangepi-rk3588 (Jun 27 2023 - 23:16:07) Trying to boot from MMC1 Trying fit image at 0x4000 sector ## Verified-boot: 0 ## Checking atf-1 0x00040000 ... sha256(2e8446f969...) + OK ## Checking uboot 0x00200000 ... sha256(71d16432e9...) + OK ## Checking fdt 0x0034f598 ... sha256(906c87aa7c...) + OK ## Checking atf-2 0xff100000 ... sha256(9f75e6ec37...) + OK ## Checking atf-3 0x000f0000 ... sha256(c80587de50...) + OK Jumping to U-Boot(0x00200000) via ARM Trusted Firmware(0x00040000) Total: 750.347 ms INFO: Preloader serial: 2 NOTICE: BL31: v2.3():v2.3-662-g4acbe711b-dirty:finley.xiao, fwver: v1.44 NOTICE: BL31: Built : 16:44:24, Nov 7 2023 INFO: spec: 0x13 INFO: code: 0x88 INFO: ext 32k is valid INFO: ddr: stride-en 4CH INFO: GICv3 without legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: valid_cpu_msk=0xff bcore0_rst = 0x0, bcore1_rst = 0x0 INFO: l3 cache partition cfg-0 INFO: system boots from cpu-hwid-0 INFO: idle_st=0x21fff, pd_st=0x11fff9, repair_st=0xfff70001 ERROR: dfs get fsp_params[0] error, 0xfead0001 != 0xfead0004 ERROR: dfs get fsp_params[1] error, 0xa54 != 0xfead0004 ERROR: dfs get fsp_params[2] error, 0xbc != 0xfead0004 ERROR: dfs get fsp_params[3] error, 0x3f0033 != 0xfead0004 ERROR: loader&trust unmatch!!! Please update loader if need enable dmc ERROR: current trust bl31 need match with loader ddr bin V1.13 or newer ERROR: current loader need match with trust bl31 V1.07-V1.37 INFO: BL31: Initialising Exception Handling Framework INFO: BL31: Initializing runtime services WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK ERROR: Error initializing runtime service opteed_fast INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x200000 INFO: SPSR = 0x3c9 U-Boot 2017.09 (Dec 27 2024 - 03:11:19 +0000) Model: Orange Pi 5 MPIDR: 0x81000000 PreSerial: 2, raw, 0xfeb50000 DRAM: 3.7 GiB Sysmem: init Relocation Offset: eda0e000 Relocation fdt: eb9f9250 - eb9fecb8 CR: M/C/I Using default environment DM: v2 mmc@fe2c0000: 1, mmc@fe2e0000: 0 Card did not respond to voltage select! mmc_init: -95, time 13 switch to partitions #0, OK mmc1 is current device Boot from SDcard Bootdev(scan): mmc 1 MMC1: Legacy, 52Mhz PartType: EFI No misc partition boot mode: None FIT: No boot partition Failed to load DTB, ret=-2 No valid DTB, ret=-22 Failed to get kernel dtb, ret=-22 Model: Orange Pi 5 MPIDR: 0x81000000 CLK: (sync kernel. arm: enter 1008000 KHz, init 1008000 KHz, kernel 0N/A) b0pll 24000 KHz b1pll 24000 KHz lpll 24000 KHz v0pll 24000 KHz aupll 24000 KHz cpll 1500000 KHz gpll 1188000 KHz npll 24000 KHz ppll 1100000 KHz aclk_center_root 702000 KHz pclk_center_root 100000 KHz hclk_center_root 396000 KHz aclk_center_low_root 500000 KHz aclk_top_root 750000 KHz pclk_top_root 100000 KHz aclk_low_top_root 396000 KHz No misc partition Net: No ethernet found. Hit key to stop autoboot('CTRL+C'): 0 starting USB... Bus usb@fc000000: usb maximum-speed not found Register 2000140 NbrPorts 2 Starting the controller USB XHCI 1.10 Bus usb@fc800000: USB EHCI 1.00 Bus usb@fc840000: USB OHCI 1.0 Bus usb@fc880000: USB EHCI 1.00 Bus usb@fc8c0000: USB OHCI 1.0 Bus usb@fcd00000: usb maximum-speed not found Register 2000140 NbrPorts 2 Starting the controller USB XHCI 1.10 scanning bus usb@fc000000 for devices... 1 USB Device(s) found scanning bus usb@fc800000 for devices... 1 USB Device(s) found scanning bus usb@fc840000 for devices... 1 USB Device(s) found scanning bus usb@fc880000 for devices... 1 USB Device(s) found scanning bus usb@fc8c0000 for devices... 1 USB Device(s) found scanning bus usb@fcd00000 for devices... 1 USB Device(s) found scanning usb for storage devices... 0 Storage Device(s) found Device 0: unknown device switch to partitions #0, OK mmc1 is current device Scanning mmc 1:1... Found U-Boot script /boot/boot.scr 4149 bytes read in 25 ms (161.1 KiB/s) ## Executing script at 00500000 Boot script loaded from mmc 1:1 244 bytes read in 23 ms (9.8 KiB/s) 16877874 bytes read in 1393 ms (11.6 MiB/s) 39770624 bytes read in 3230 ms (11.7 MiB/s) 249244 bytes read in 298 ms (816.4 KiB/s) Trying kaslrseed command... Info: Unknown command can be safely ignored since kaslrseed does not apply to all boards. Unknown command 'kaslrseed' - try 'help' Fdt Ramdisk skip relocation No misc partition ## Loading init Ramdisk from Legacy Image at 0a200000 ... Image Name: uInitrd Image Type: AArch64 Linux RAMDisk Image (gzip compressed) Data Size: 16877810 Bytes = 16.1 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Flattened Device Tree blob at 0x08300000 Booting using the fdt blob at 0x08300000 reserving fdt memory region: addr=8300000 size=a3000 Using Device Tree in place at 0000000008300000, end 00000000083a5fff WARNING: could not set reg FDT_ERR_BADOFFSET. ## reserved-memory: cma: addr=10000000 size=10000000 ramoops@110000: addr=110000 size=e0000 Adding bank: 0x00200000 - 0xf0000000 (size: 0xefe00000) Total: 13710.462/13739.216 ms Starting kernel ... Armbian-unofficial 25.02.0-trunk Jammy ttyFIQ0 orangepi5 login: 0 Quote
ScottN Posted December 31, 2024 Author Posted December 31, 2024 I have moved back to testing with Joshua-Riek's ubuntu-rockchip builds. I'm using the "ubuntu-22.04.3-preinstalled-desktop-arm64-orangepi-5.img" image downloaded from https://joshua-riek.github.io/ubuntu-rockchip-download/boards/orangepi-5.html That version of 22.04.3 is exhibiting the same thing as Armbian where PCIe/NVMe controller is not powered on it seems. Below is the dmesg output desktop:~$ dmesg | grep -i pcie [ 8.122563] reg-fixed-voltage vcc3v3-pcie2x1l2: Looking up vin-supply from device tree [ 8.122566] vcc3v3_pcie2x1l2: supplied by vcc5v0_sys [ 8.177639] vcc3v3_pcie2x1l2: 1800 mV, enabled [ 8.177702] reg-fixed-voltage vcc3v3-pcie2x1l2: vcc3v3_pcie2x1l2 supplying 1800000uV [ 9.001385] rk-pcie fe190000.pcie: invalid prsnt-gpios property in node [ 9.001395] rk-pcie fe190000.pcie: Looking up vpcie3v3-supply from device tree [ 9.002077] rk-pcie fe190000.pcie: IRQ msi not found [ 9.002112] rk-pcie fe190000.pcie: use outband MSI support [ 9.002119] rk-pcie fe190000.pcie: Missing *config* reg space [ 9.002151] rk-pcie fe190000.pcie: host bridge /pcie@fe190000 ranges: [ 9.002181] rk-pcie fe190000.pcie: err 0x00f4000000..0x00f40fffff -> 0x00f4000000 [ 9.002199] rk-pcie fe190000.pcie: IO 0x00f4100000..0x00f41fffff -> 0x00f4100000 [ 9.002216] rk-pcie fe190000.pcie: MEM 0x00f4200000..0x00f4ffffff -> 0x00f4200000 [ 9.002267] rk-pcie fe190000.pcie: MEM 0x0a00000000..0x0a3fffffff -> 0x0a00000000 [ 9.002299] rk-pcie fe190000.pcie: Missing *config* reg space [ 9.002342] rk-pcie fe190000.pcie: invalid resource [ 9.210057] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 9.235591] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 9.262255] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 9.288914] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 9.315597] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 9.342261] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 9.368923] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 9.395590] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 9.422255] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 9.448922] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 10.226537] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up pcie-supply from device tree [ 10.226557] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up pcie-supply property in node /power-management@fd8d8000/power-controller failed [ 11.912625] rk-pcie fe190000.pcie: PCIe Link Fail [ 11.912779] rk-pcie fe190000.pcie: failed to initialize host I'm going to now move onto more recent versions of "ubuntu-rockchip" like 24.04 and 24.10 and report back 0 Quote
ozacas Posted January 2 Posted January 2 this doesnt look good in your logs - means the device tree information is possibly missing/corrupted (and likely devices are missing as a result) - Failed to load DTB, ret=-2 No valid DTB, ret=-22 Failed to get kernel dtb, ret=-22 i would check that any configuration related to DTB's is valid and pointing to the correct files. Some clues to look at include /boot/dtb* and /boot/armbianEnv.txt eg (this is a different rk3588 board, but results should be similar): $ more /boot/armbianEnv.txt verbosity=1 bootlogo=false console=both overlay_prefix=rockchip-rk3588 fdtfile=rockchip/rk3588-radxa-rock-5b+.dtb rootdev=UUID=7050d993-d3c0-457c-baf0-282bfb0afb63 rootfstype=ext4 usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u $ ls /boot/dtb* /boot/dtb: rockchip /boot/dtb-6.1.84-vendor-rk35xx: rockchip $ ls -l /boot/dtb-6.1.84-vendor-rk35xx/rockchip/rk3588-radxa-rock-5b+.dtb -rwxr-xr-x 1 root root 272592 Dec 15 10:37 /boot/dtb-6.1.84-vendor-rk35xx/rockchip/rk3588-radxa-rock-5b+.dtb 0 Quote
ScottN Posted January 4 Author Posted January 4 Thank you for your replies. Right now it seems that any new image won't reliably power on the PCIe controller. I've emailed efectn who maintains the Orange Pi 5 Armbian image and he is going to look into if there are some reset gpio changes. I'm not dead in the water on my end. My older Armbian Jammy 5.1 kernel still works and boots. I just need to flash the MTD on the board with the new image Orange Pi official released in July of 2024. Also any official OS image from Orange Pi will boot and show PCIe devices connected, so I can prep my storage device that way prior to booting my Armbian image. I've gone back in time pretty far with older Armbian images for OPi5 board and it seems this issue has existed for a while because they all seem to have issues activating the PCIe. Honestly don't know if that's an issue with reset gpio that efectn is looking into or something else. That's pretty low level for me to know for sure. 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.