KingJ

To keep the forum updated: I closed my PR to downgrade all NanoPC-T6 boards to use HS200 mode, since there are other folks who do have the NanoPC-T6 and can successfully use HS400 mode. @SuperKali opened up a PR to downgrade all NanoPC-T6 boards to use HS400 mode without enhanced strobe as "This should maintains the same ~400MB/s throughput while eliminating I/O errors under load.". I tried applying the patch on my board, but unfortunately still hit I/O errors. @SuperKali had suggested on my original PR to try reducing the eMMC frequency from the default 200000000 to 150000000. I created an overlay for this... and it worked! I'm going to prepare a PR tomorrow to add that overlay as a kernel overlay, so that others who are experiencing this issue can opt-in to it without needing to find this thread and copy the overlay file.

@SuperKali I have the original NanoPC-T6 - purchased around 2 years ago. While it does negotiate HS400 mode, it's not stable - medium or heavy I/O activity like performing an apt upgrade or even booting will cause I/O errors to be logged by the kernel. I've just tried switching to 6.16.4-edge-rockchip64 via armbian-config. It took a few attempts to get it booted - it hung several times at "begin: running /scripts/init-bottom ... done". On the successful boot, I could see quite a few I/O errors during boot. Here's the kernel logs showing HS400 mode was negotiated along with the I/O errors: root@nanopct6:~# dmesg | grep mmc [ 1.776595] mmc0: SDHCI controller on fe2e0000.mmc [fe2e0000.mmc] using ADMA [ 1.875696] mmc0: new HS400 Enhanced strobe MMC card at address 0001 [ 1.876758] mmcblk0: mmc0:0001 A3A444 230 GiB [ 1.880737] mmcblk0: p1 [ 1.881457] mmcblk0boot0: mmc0:0001 A3A444 4.00 MiB [ 1.883682] mmcblk0boot1: mmc0:0001 A3A444 4.00 MiB [ 1.885887] mmcblk0rpmb: mmc0:0001 A3A444 4.00 MiB, chardev (243:0) [ 2.191107] dwmmc_rockchip fe2c0000.mmc: IDMAC supports 32-bit address mode. [ 2.191125] dwmmc_rockchip fe2c0000.mmc: Using internal DMA controller. [ 2.191132] dwmmc_rockchip fe2c0000.mmc: Version ID is 270a [ 2.191158] dwmmc_rockchip fe2c0000.mmc: DW MMC controller at irq 122,32 bit host data width,256 deep fifo [ 2.191894] dwmmc_rockchip fe2c0000.mmc: Got CD GPIO [ 2.204956] mmc_host mmc1: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0) [ 4.370210] EXT4-fs (mmcblk0p1): mounted filesystem a4f48be8-f667-4cac-a9a7-61ce8f9035d1 ro with ordered data mode. Quota mode: none. [ 5.796770] I/O error, dev mmcblk0, sector 261857 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2 [ 5.819713] I/O error, dev mmcblk0, sector 261857 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 [ 5.848297] I/O error, dev mmcblk0, sector 261857 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 [ 7.379796] I/O error, dev mmcblk0, sector 221648 op 0x0:(READ) flags 0x80700 phys_seg 2 prio class 2 [ 7.538332] I/O error, dev mmcblk0, sector 183451920 op 0x0:(READ) flags 0x80700 phys_seg 5 prio class 2 [ 7.727265] I/O error, dev mmcblk0, sector 182816073 op 0x0:(READ) flags 0x80700 phys_seg 4 prio class 2 [ 7.889544] I/O error, dev mmcblk0, sector 182816073 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 [ 8.181746] I/O error, dev mmcblk0, sector 182816073 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 [ 8.325932] I/O error, dev mmcblk0, sector 182816073 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 [ 8.865460] I/O error, dev mmcblk0, sector 189262696 op 0x0:(READ) flags 0x80700 phys_seg 5 prio class 2 [ 9.097362] EXT4-fs (mmcblk0p1): re-mounted a4f48be8-f667-4cac-a9a7-61ce8f9035d1 r/w. [ 11.883700] I/O error, dev mmcblk0, sector 189160184 op 0x0:(READ) flags 0x80700 phys_seg 3 prio class 2 [ 13.327491] I/O error, dev mmcblk0, sector 189233040 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2 [ 13.534473] I/O error, dev mmcblk0, sector 189247499 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2 [ 14.342633] I/O error, dev mmcblk0, sector 214158 op 0x0:(READ) flags 0x80700 phys_seg 2 prio class 2 [ 16.773905] I/O error, dev mmcblk0, sector 189145145 op 0x0:(READ) flags 0x80700 phys_seg 2 prio class 2 [ 16.943465] I/O error, dev mmcblk0, sector 189142964 op 0x0:(READ) flags 0x80700 phys_seg 2 prio class 2 [ 16.946051] I/O error, dev mmcblk0, sector 189142966 op 0x0:(READ) flags 0x80700 phys_seg 2 prio class 2 [ 17.923621] I/O error, dev mmcblk0, sector 182816073 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 [ 17.943695] I/O error, dev mmcblk0, sector 182816073 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2 Once booted it's pretty easy to trigger on-demand with a lightweight fio test: fio --filename=fio.bin --size=1GB --direct=1 --rw=randrw --bs=64k --ioengine=libaio --iodepth=1 --runtime=120 --numjobs=1 --time_based --group_reporting --name=rw --eta-newline=1 rw: (g=0): rw=randrw, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T) 64.0KiB-64.0KiB, ioengine=libaio, iodepth=1 fio-3.36 Starting 1 process fio: io_u error on file fio.bin: Input/output error: read offset=705167360, buflen=65536 fio: pid=2237, err=5/file:io_u.c:1896, func=io_u error, error=Input/output error (with similar I/O errors logged in to the kernel logs at the same time). When running in HS200 mode using the DTB provided by @usual user, I can't trigger any I/O errors - no matter how hard I push it via fio. I then tried switching to the vendor kernel ("linux-image-vendor-rk35xx=25.8.1 v6.1.115"). That did result in a bunch of I/O errors being logged (including on the apt output), and when rebooting in to the new kernel there was no output whatsoever (I assume it got corrupted and couldn't boot). So instead, I booted back off the SD card and switched that to the same vendor kernel - which did work. And in better news, I was able to mount the MMC partition from there and run the same fio test without issue. fio --filename=/mnt/mmc/root/fio.bin --size=1GB --direct=1 --rw=randrw --bs=64k --ioengine=libaio --iodepth=1 --runtime=120 --numjobs=1 --time_based --group_reporting --name=rw --eta-newline=1 rw: (g=0): rw=randrw, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T) 64.0KiB-64.0KiB, ioengine=libaio, iodepth=1 fio-3.36 Starting 1 process Jobs: 1 (f=1): [m(1)][2.5%][r=40.8MiB/s,w=42.6MiB/s][r=653,w=682 IOPS][eta 01m:57s] Jobs: 1 (f=1): [m(1)][4.2%][r=43.1MiB/s,w=42.8MiB/s][r=690,w=685 IOPS][eta 01m:55s] <snip - it continues like this> The kernel log output is slightly different on this vendor kernel. While it also negotiated in HS400 mode, there's a few extra things - e.g. CQHCI, a second Bus speed line etc. dmesg | grep mmc [ 8.561754] sdhci-dwcmshc fe2e0000.mmc: Looking up vmmc-supply from device tree [ 8.561774] sdhci-dwcmshc fe2e0000.mmc: Looking up vmmc-supply property in node /mmc@fe2e0000 failed [ 8.565806] sdhci-dwcmshc fe2e0000.mmc: Looking up vqmmc-supply from device tree [ 8.565834] sdhci-dwcmshc fe2e0000.mmc: Looking up vqmmc-supply property in node /mmc@fe2e0000 failed [ 8.566361] mmc0: CQHCI version 5.10 [ 8.597464] mmc0: SDHCI controller on fe2e0000.mmc [fe2e0000.mmc] using ADMA [ 8.729576] mmc0: Command Queue Engine enabled [ 8.729603] mmc0: new HS400 Enhanced strobe MMC card at address 0001 [ 8.731147] mmcblk0: mmc0:0001 A3A444 230 GiB [ 8.739493] mmcblk0: p1 [ 8.740546] mmcblk0boot0: mmc0:0001 A3A444 4.00 MiB [ 8.742822] mmcblk0boot1: mmc0:0001 A3A444 4.00 MiB [ 8.744487] mmcblk0rpmb: mmc0:0001 A3A444 4.00 MiB, chardev (234:0) [ 8.946192] dwmmc_rockchip fe2c0000.mmc: No normal pinctrl state [ 8.946214] dwmmc_rockchip fe2c0000.mmc: No idle pinctrl state [ 8.946285] dwmmc_rockchip fe2c0000.mmc: IDMAC supports 32-bit address mode. [ 8.946331] dwmmc_rockchip fe2c0000.mmc: Using internal DMA controller. [ 8.946345] dwmmc_rockchip fe2c0000.mmc: Version ID is 270a [ 8.946397] dwmmc_rockchip fe2c0000.mmc: DW MMC controller at irq 154,32 bit host data width,256 deep fifo [ 8.946494] dwmmc_rockchip fe2c0000.mmc: Looking up vmmc-supply from device tree [ 8.946761] dwmmc_rockchip fe2c0000.mmc: Looking up vqmmc-supply from device tree [ 8.960553] mmc_host mmc1: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0) [ 9.063046] mmc_host mmc1: Bus speed (slot 0) = 198000000Hz (slot req 200000000Hz, actual 198000000HZ div = 0) [ 9.176670] dwmmc_rockchip fe2c0000.mmc: Successfully tuned phase to 257 [ 9.176684] mmc1: new ultra high speed SDR104 SDXC card at address aaaa [ 9.177135] mmcblk1: mmc1:aaaa SD256 238 GiB [ 9.180796] mmcblk1: p1 [ 10.118815] EXT4-fs (mmcblk1p1): mounted filesystem with writeback data mode. Quota mode: none. [ 11.268903] EXT4-fs (mmcblk1p1): re-mounted. Quota mode: none. [ 110.819476] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data mode. Quota mode: none. ------ @usual user Thank you for the information about how you compiled the DTB with the overlay built in. It's also good to hear that the board isn't too difficult to remove. I tried to apply a little bit of pressure around the M.2 slot and the board just fell out! Now I need to find where I put my UART adapter...

Thanks for compiling that overlay in to a DTB - i've just tried putting that in to /boot/dtb/rockchip/rk3588-nanopc-t6.dtb, rebooted, and it successfully booted and used HS200 mode for the MMC! [ +0.015056] mmc0: SDHCI controller on fe2e0000.emmc [fe2e0000.emmc] using ADMA [ +0.006109] dwmmc_rockchip fe2c0000.mmc: IDMAC supports 32-bit address mode. [ +0.000031] dwmmc_rockchip fe2c0000.mmc: Using internal DMA controller. [ +0.000008] dwmmc_rockchip fe2c0000.mmc: Version ID is 270a [ +0.000023] dwmmc_rockchip fe2c0000.mmc: DW MMC controller at irq 95,32 bit host data width,256 deep fifo [ +0.000004] mmc0: new HS200 MMC card at address 0001 [ +0.000045] dwmmc_rockchip fe2c0000.mmc: Got CD GPIO [ +0.000236] mmcblk0: mmc0:0001 A3A444 230 GiB [ +0.002488] mmcblk0: p1 [ +0.000267] mmcblk0boot0: mmc0:0001 A3A444 4.00 MiB [ +0.000759] mmcblk0boot1: mmc0:0001 A3A444 4.00 MiB [ +0.000672] mmcblk0rpmb: mmc0:0001 A3A444 4.00 MiB, chardev (243:0) I'm puzzled as to why that worked, but the overlay didn't work. I would have grabbed some logs if I could, but there was no display output so I think the only output would have been on the UART? Unfortunately, I bought the version that comes in a case and the UART is on the top side of the board that I can't access. Searching around, it doesn't look like it is easy to pop it out of the case either. Just in case putting it in to user overlays was wrong, I tried reverting to the original DTB, putting the overlay in /boot/dtb-6.12.44-current-rockchip64/rockchip/overlay/rockchip-rk3588-nanopc-t6-emmc.dtbo, activating it via armbian-config, and rebooting. Unfortunately I hit the same issue as before where it wouldn't boot. Is there a way I can debug why the overlay isn't working without getting access to the UART header on the top of the board?

Right, that's the downside - anyone with a different eMMC chip will see a drop in performance. I've just given that overlay a try - thank you for suggesting it. I put it in to /boot/overlay-user, then edited /boot/armbianEnv.txt to include the line user_overlays=rk3588-nanopc-t6-emmc . Unfortunately, it then failed to boot. I had to put in an SD card, boot from that, mount the eMMC partition, and then remove the overload from armbianEnv.txt to get it to boot successfully from the eMMC again.

Raised up a PR with a patch. 🤞

I think i've managed to work around this, and set HS200 mode. I did this by decompiling /boot/dtb/rockchip/rk3588-nanopc-t6.dtb, modifying the mmc@fe2e0000 section to remove the mmc-hs400-1_8v; and mmc-hs400-enhanced-strobe; lines, adding a new mmc-hs200-1_8v; line and then recompiling it again. dtc -I dtb -O dts -f /boot/dtb/rockchip/rk3588-nanopc-t6.dtb -o rk3588-nanopc-t6.dts # Modify rk3588-nanopc-t6.dts here dtc -I dts -O dtb rk3588-nanopc-t6.dts -o /boot/dtb/rockchip/rk3588-nanopc-t6.dtb After rebooting, I confirmed that it was now running in HS200 mode: [ +0.010753] mmc0: SDHCI controller on fe2e0000.mmc [fe2e0000.mmc] using ADMA [ +0.034734] mmc0: new HS200 MMC card at address 0001 [ +0.000571] mmcblk0: mmc0:0001 A3A444 230 GiB I ran a heavy random read/write test, and observed no errors. However, I think this will be lost during a future update - so i'll see if I can figure out how to submit a patch!

I've recently switched my FriendlyElec NanoPC-T6 to use the Armbian Linux v6.12 server image, built on the 24th of May 2025. Booting of SD works fine, but when installing to the eMMC chip some I/O errors can be found in the kernel logs: [ 151.814773] I/O error, dev mmcblk0, sector 176 op 0x0:(READ) flags 0x80700 phys_seg 10 prio class 2 This happens when under heavy I/O load - e.g. performing an apt upgrade. I ran badblocks over the entire eMMC chip without issue - but that puts a much lower strain on the eMMC. Therefore, i'm convinced that the eMMC chip itself is fine. Poking around a bit, this seems to be because the A3A444 eMMC chip which some NanoPC-T6 SBCs shipped with do not support HS400 mode properly. There is an OpenWRT bug report about this, which fixed the issue by patching the dtsi to force HS200 mode. My kernel logs confirm that I have the A3A444 eMMC chip, and that it is currently running in HS400 mode: sudo dmesg -e | grep -i mmc [ +0.015143] mmc0: SDHCI controller on fe2e0000.mmc [fe2e0000.mmc] using ADMA [ +0.051640] dwmmc_rockchip fe2c0000.mmc: IDMAC supports 32-bit address mode. [ +0.000020] dwmmc_rockchip fe2c0000.mmc: Using internal DMA controller. [ +0.000008] dwmmc_rockchip fe2c0000.mmc: Version ID is 270a [ +0.000025] dwmmc_rockchip fe2c0000.mmc: DW MMC controller at irq 91,32 bit host data width,256 deep fifo [ +0.000210] dwmmc_rockchip fe2c0000.mmc: Got CD GPIO [ +0.012806] mmc_host mmc1: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0) [ +0.007795] mmc0: new HS400 Enhanced strobe MMC card at address 0001 [ +0.000906] mmcblk0: mmc0:0001 A3A444 230 GiB [ +0.004124] mmcblk0: p1 [ +0.000720] mmcblk0boot0: mmc0:0001 A3A444 4.00 MiB [ +0.001831] mmcblk0boot1: mmc0:0001 A3A444 4.00 MiB [ +0.001742] mmcblk0rpmb: mmc0:0001 A3A444 4.00 MiB, chardev (243:0) [ +0.227497] EXT4-fs (mmcblk0p1): mounted filesystem a4f48be8-f667-4cac-a9a7-61ce8f9035d1 ro with ordered data mode. Quota mode: none. [ +0.021022] EXT4-fs (mmcblk0p1): re-mounted a4f48be8-f667-4cac-a9a7-61ce8f9035d1 r/w. I've tried to use a user device overlay to use mmc-hs200-1_8v, but this doesn't appear to work. I think this is because Device Overlays can only replace elements or add to the tree? i.e. I cannot use an overlay to remove the existing mmc-hs400-1_8v; and mmc-hs400-enhanced-strobe; entries. Is there a way to do this with a user device overlay, or will I need to try to add a similar patch to OpenWRT's one in the kernel? This is the first time i've used Armbian so i'm a little unsure about how i'd go about doing the latter. I've found a similar report of this issue from a couple of years ago by @SuperKali, albeit with no resolution. However, I can see they're now listed as one of the community maintainers for this board so i'm hoping it's OK to mention them in this thread to see if they know how best to force HS200 mode for the eMMC!