Hi @alexc, I'd be happy to test your changes! I've now achieved full functionality with AI; it did a very extensive and time-consuming refactoring of not only the kernel but also the driver. Its conclusions about what it did were as follows; I don't know if this will help! I've now achieved functionality with the old kernel (taking into account Claude's edits). I'm attaching its conclusions. CLAUDE CONCLUSION: The stock BSP/mainline implementation fails at two distinct architectural levels: 1. Hardware/SoC Layer (Inbound ATU) The SUNXI PCIe Root Complex (sun60iw2) does not have its inbound ATU (Address Translation Unit) entries properly configured for the DDR memory range. The Consequence: When the Hailo-8 endpoint attempts to write an MSI packet to the physical DDR address (pp->msi_data), the Root Complex returns an Unsupported Request (UR) TLP response. The Hailo firmware immediately panics on the unexpected UR, dropping the PCIe link entirely. From this point onward, the entire BAR register space returns 0xFF. 2. Interrupt Path Layer (DTS Routing) Even when falling back to legacy INTx interrupts to avoid the MSI-driven crash, the INTA# signal forwarded from the Hailo-8 through the ASM1182e switch never reaches the CPU. The IRQ counter in /proc/interrupts stays completely stagnant. This strongly points to an incorrect PCIe interrupt-map configuration in the device tree (DTS) for the sun60iw2. ComponentResponsibility / Issue Kernel (pci-sun6i.c / RC driver)Bug: Lacks inbound ATU window configuration for DDR space. MSI target addresses are dropped at the RC level. Driver (hailo-pci)Limitation: Expects a fully functional hardware interrupt path. Requires a software fallback mechanism to handle the broken RC. The Proper Solution (Kernel-Level Fix) The cleanest fix belongs in the SUNXI PCIe RC driver (e.g., drivers/pci/controller/dwc/pci-sun6i.c or its equivalent for the A527). Right after the PCIe link is established, an inbound window covering the system's DDR space must be explicitly mapped: /* After setting up the PCIe link, program the inbound ATU window for system DDR */ dw_pcie_prog_inbound_atu(pci, 0, PCIE_ATU_TYPE_MEM, phys_ddr_base, /* target address */ phys_ddr_base, /* base address */ ddr_size); /* size (Map entire DDR pool) */ Implementing this would allow: Inbound MSI writes to successfully target the GIC. Inbound DMA transfers to pass cleanly without getting filtered out by the RC. Note: The DTS interrupt-map for sun60iw2 also needs a close look to ensure legacy INTx lines map correctly to the GIC. The Temporary Workaround (3 Driver-Level Patches) Since changing the upstream kernel wasn't immediately viable, we implemented a robust workaround directly inside the hailo-pci kernel driver split across 3 patches: Patch 1: Enforce Legacy INTx over MSI (pcie.c) We forced the driver to avoid MSI and use legacy INTx. Because legacy interrupts rely on dedicated sideband signaling rather than inbound write TLPs, it prevents the inbound ATU failure from triggering an immediate firmware panic and a hard PCIe link drop. Patch 2: The Core Fix — Kernel Polling Thread (fops.c + pcie.c) Since regular interrupts are completely deaf on this platform, we introduced a dedicated kernel polling thread that fires alongside enable_interrupts(): // Kernel thread mimicking the interrupt handler via register polling int hailo_poll_thread_fn(void *data) { struct hailo_pcie_board *board = (struct hailo_pcie_board *)data; u32 irq_source; while (!kthread_should_stop()) { usleep_range(900, 1100); // ~1ms interval balances latency and CPU overhead if (!hailo_pcie_read_interrupt(&board->pcie_resources, &irq_source)) continue; // Manually dispatch: firmware control SW IRQ + vDMA channel completions nnc_irq_handler(board, &irq_source); if (irq_source.vdma_channels_bitmap) { hailo_vdma_irq_handler(&board->vdma, 0, irq_source.vdma_channels_bitmap); } } return 0; } Why this works: The Hailo firmware writes completion bits directly to the BCS_ISTATUS_HOST register (BAR0 offset 0x018C) regardless of whether the actual MSI/INTx block successfully alerts the CPU. By reading this register in a 1ms loop and manually routing the completions, processing functions seamlessly. Patch 3: VMA Locking Update for Modern Kernels (memory.c) For compatibility with modern kernels (Linux $\ge$ 6.18), we wrapped find_vma() with the mandatory read lock to prevent kernel warnings and memory subsystem instability: mmap_read_lock(mm); vma = find_vma(mm, address); mmap_read_unlock(mm); Real-World Performance & Results We tested this setup under heavy real-time Object Detection in Frigate NVR: Model: yolov8s (running entirely on Hailo-8 via the patched driver) Inference Latency: ~25–30 ms Throughput: ~24 FPS split across 9 simultaneous camera streams Stability: 0 crashes over long-term testing (previously crashed within exactly 1 second of starting vDMA operations). Conclusion A polling thread with a 1ms interval proves to be more than fast enough to handle high-throughput AI inference workloads. The performance overhead is negligible, pinning one CPU thread into a low-impact state since usleep_range leaves the CPU resting most of the time. While a kernel-level inbound ATU and DTS fix is the ideal target for Armbian/Radxa mainlines, this driver-level patch provides a completely stable workaround for anyone trying to build edge AI platforms using Allwinner chips right now. Let me know if anyone needs the raw patch files or wants to help upstream the correct ATU fixes to pci-sun6i.c!

attach serial console to check uboot log if the overlay is actually loaded

Orange pi 5 (i dont find way hot to make it in orange pi 5 section) English is not my native, i hope you understand and sorry for mistakes I spend like 3 days on this problem and i think i am stuck I tested offisial image and Josua and there i can use my wifi normally Setup orange pi 5 v1.3.2 clened spi (i got problem like here) armbian 26.5.1 vendor 6.1.115 (from armbian imager on windows) sandisk extreme 64gb sd card after boot i runs sudo apt update sudo apt upgrade -y armbian-config (choose orange pi 5 ap6275p overlay) /boot/armbianEnv.txt verbosity=1 bootlogo=false console=both extraargs=cma=256M overlay_prefix=rockchip-rk3588 fdtfile=rockchip/rk3588s-orangepi-5.dtb rootdev=UUID=... rootfstype=ext4 overlays=orangepi-5-ap6275p usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u But after reboot wifi module do not works as expected ip a lspci do not show any device As i think now it is some dtbo files problems and i think key is somthing around line: [ 121.023832] rk-pcie fe190000.pcie: PCIe Link Fail, LTSSM is 0x3, hw_retries=1 there is a guy figures same error but the lines he added already included for me wifi and pcie dmesg greps (by the way armbian takes too long to boot and i fond sombody figures same problem and he syas that time depends on size of sd card) root@orangepi5:~# dmesg | grep wifi [ 116.704296] [WLAN_RFKILL]: wlan_platdata_parse_dt: wifi_chip_type = ap6275p [ 116.704300] [WLAN_RFKILL]: wlan_platdata_parse_dt: enable wifi power control. [ 116.704303] [WLAN_RFKILL]: wlan_platdata_parse_dt: wifi power controled by gpio. [ 116.704340] [WLAN_RFKILL]: wlan_platdata_parse_dt: The ref_wifi_clk not found ! [ 116.704348] [WLAN_RFKILL]: rfkill_set_wifi_bt_power: 1 root@orangepi5:~# dmesg | grep pcie [ 19.019616] reg-fixed-voltage vcc3v3-pcie2x1l2: Looking up vin-supply from device tree [ 19.020209] vcc3v3_pcie2x1l2: supplied by vcc5v0_sys [ 19.079124] vcc3v3_pcie2x1l2: 1800 mV, enabled [ 19.085641] reg-fixed-voltage vcc3v3-pcie2x1l2: vcc3v3_pcie2x1l2 supplying 1800000uV [ 115.377859] dw-pcie fe190000.pcie: invalid resource [ 115.378688] dw-pcie fe190000.pcie: Failed to initialize host [ 115.378975] dw-pcie: probe of fe190000.pcie failed with error -22 [ 115.405440] rk-pcie fe190000.pcie: invalid prsnt-gpios property in node [ 115.405658] rk-pcie fe190000.pcie: Looking up vpcie3v3-supply from device tree [ 115.407223] rk-pcie fe190000.pcie: can't get current limit. [ 115.409940] rk-pcie fe190000.pcie: host bridge /pcie@fe190000 ranges: [ 115.410471] rk-pcie fe190000.pcie: IO 0x00f4100000..0x00f41fffff -> 0x00f4100000 [ 115.410659] rk-pcie fe190000.pcie: MEM 0x00f4200000..0x00f4ffffff -> 0x00f4200000 [ 115.410757] rk-pcie fe190000.pcie: MEM 0x0a00000000..0x0a3fffffff -> 0x0a00000000 [ 115.410961] rk-pcie fe190000.pcie: iATU unroll: enabled [ 115.410983] rk-pcie fe190000.pcie: iATU regions: 8 ob, 8 ib, align 64K, limit 8G [ 115.613333] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 115.634444] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 115.655522] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 115.676607] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 115.697683] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 115.718758] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 115.739838] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 115.760912] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 115.781986] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 115.803060] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 116.912753] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up pcie-supply from device tree [ 116.912767] rockchip-pm-domain fd8d8000.power-management:power-controller: Looking up pcie-supply property in node /power-management@fd8d8000/power-controller failed [ 117.713882] rk-pcie fe190000.pcie: PCIe Link Fail, LTSSM is 0x3, hw_retries=0 [ 120.627876] rk_pcie_establish_link: 171 callbacks suppressed [ 120.627888] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 120.648138] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 120.668247] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 120.689287] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 120.710332] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 120.731354] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 120.751458] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 120.772474] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 120.793551] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 120.814586] rk-pcie fe190000.pcie: PCIe Linking... LTSSM is 0x3 [ 121.023832] rk-pcie fe190000.pcie: PCIe Link Fail, LTSSM is 0x3, hw_retries=1 [ 122.034993] rk-pcie fe190000.pcie: failed to initialize host Context about long boot: On 19 sec [ 19.315369] NetLabel: Initializing [ 19.315574] NetLabel: domain hash size = 128 [ 19.315819] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 19.318283] NetLabel: unlabeled traffic allowed by default [ 19.328104] vgaarb: loaded [ 19.365221] clocksource: Switched to clocksource arch_sys_counter [ 69.713069] VFS: Disk quotas dquot_6.6.0 [ 69.715555] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 69.734240] AppArmor: AppArmor Filesystem Enabled And on 73 sec [ 73.253310] NET: Registered PF_ALG protocol family [ 73.253416] Key type asymmetric registered [ 73.253451] Asymmetric key parser 'x509' registered [ 113.848019] Freeing initrd memory: 15860K [ 114.505595] alg: self-tests for CTR-KDF (hmac(sha256)) passed [ 114.508695] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 241) [ 114.512435] io scheduler mq-deadline registered So i do not know reason and where i need to look. Maybe it is broken dtbo or i need to load spesial spi because i am not spesialist at this point and i cand find sombody figured same problem. I also got full dmesg for orange pi os (where wifi works) and armbian boot with and without wifi configured if any body needs it

Hi @Sand_Death, Thanks again for identifying the MSI issue. I've just updated my branch with some MSI-related changes. I tried to align the implementation as closely as possible with the Raspberry Pi 5 PCIe driver. Could you give it a try and let me know whether it resolves the issue on your end? I'm now seeing additional MSI interrupts being generated by my NVMe drives. https://github.com/alexcaoys/allwinner-bsp/commit/a9eaa51dace582a15b1a3a7380f588cd31e06a54#diff-63112d28b443d435556e546cdc59423df248d8f963a80b15680a52ae67f1cf1c That said, I still believe there are some significant issues in the PCIe driver. For example, I think it causes my NVMe SSD to overheat when used with a board also based on the ASM1182e. Please keep an eye on whether you observe similar behavior with your Hailo-8 as well. If the HAILO_DRIVER_TIMEOUT issue still persists after testing this change, could you also attach your dmesg output? Thanks!

@Nick A Thanks that's interesting to see! I don't have a Cubie board to test with but I will look at those scripts for signal of what else can be shared as common between the boards. I think the mainline work you are doing (along with linux-sunxi's work) are definitely the long term solution.