NanoPC-T6 - eMMC I/O errors under heavy load due to HS400 mode

[KingJ]

I've recently switched my FriendlyElec NanoPC-T6 to use the Armbian Linux v6.12 server image, built on the 24th of May 2025. Booting of SD works fine, but when installing to the eMMC chip some I/O errors can be found in the kernel logs:

 

[  151.814773] I/O error, dev mmcblk0, sector 176 op 0x0:(READ) flags 0x80700 phys_seg 10 prio class 2

 

This happens when under heavy I/O load - e.g. performing an apt upgrade. I ran badblocks over the entire eMMC chip without issue - but that puts a much lower strain on the eMMC. Therefore, i'm convinced that the eMMC chip itself is fine.

 

Poking around a bit, this seems to be because the A3A444 eMMC chip which some NanoPC-T6 SBCs shipped with do not support HS400 mode properly. There is an OpenWRT bug report about this, which fixed the issue by patching the dtsi to force HS200 mode.

 

My kernel logs confirm that I have the A3A444 eMMC chip, and that it is currently running in HS400 mode:

 

sudo dmesg -e | grep -i mmc
[  +0.015143] mmc0: SDHCI controller on fe2e0000.mmc [fe2e0000.mmc] using ADMA
[  +0.051640] dwmmc_rockchip fe2c0000.mmc: IDMAC supports 32-bit address mode.
[  +0.000020] dwmmc_rockchip fe2c0000.mmc: Using internal DMA controller.
[  +0.000008] dwmmc_rockchip fe2c0000.mmc: Version ID is 270a
[  +0.000025] dwmmc_rockchip fe2c0000.mmc: DW MMC controller at irq 91,32 bit host data width,256 deep fifo
[  +0.000210] dwmmc_rockchip fe2c0000.mmc: Got CD GPIO
[  +0.012806] mmc_host mmc1: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
[  +0.007795] mmc0: new HS400 Enhanced strobe MMC card at address 0001
[  +0.000906] mmcblk0: mmc0:0001 A3A444 230 GiB
[  +0.004124]  mmcblk0: p1
[  +0.000720] mmcblk0boot0: mmc0:0001 A3A444 4.00 MiB
[  +0.001831] mmcblk0boot1: mmc0:0001 A3A444 4.00 MiB
[  +0.001742] mmcblk0rpmb: mmc0:0001 A3A444 4.00 MiB, chardev (243:0)
[  +0.227497] EXT4-fs (mmcblk0p1): mounted filesystem a4f48be8-f667-4cac-a9a7-61ce8f9035d1 ro with ordered data mode. Quota mode: none.
[  +0.021022] EXT4-fs (mmcblk0p1): re-mounted a4f48be8-f667-4cac-a9a7-61ce8f9035d1 r/w.

 

I've tried to use a user device overlay to use mmc-hs200-1_8v, but this doesn't appear to work. I think this is because Device Overlays can only replace elements or add to the tree? i.e. I cannot use an overlay to remove the existing mmc-hs400-1_8v; and mmc-hs400-enhanced-strobe; entries.

 

Is there a way to do this with a user device overlay, or will I need to try to add a similar patch to OpenWRT's one in the kernel?  This is the first time i've used Armbian so i'm a little unsure about how i'd go about doing the latter.

 

I've found a similar report of this issue from a couple of years ago by @SuperKali, albeit with no resolution. However, I can see they're now listed as one of the community maintainers for this board so i'm hoping it's OK to mention them in this thread to see if they know how best to force HS200 mode for the eMMC!

 

 

[KingJ]

I think i've managed to work around this, and set HS200 mode. I did this by decompiling /boot/dtb/rockchip/rk3588-nanopc-t6.dtb, modifying the mmc@fe2e0000 section to remove the mmc-hs400-1_8v; and mmc-hs400-enhanced-strobe; lines, adding a new mmc-hs200-1_8v; line and then recompiling it again.

 

 

dtc -I dtb -O dts -f /boot/dtb/rockchip/rk3588-nanopc-t6.dtb -o rk3588-nanopc-t6.dts
# Modify rk3588-nanopc-t6.dts here
dtc -I dts -O dtb rk3588-nanopc-t6.dts -o /boot/dtb/rockchip/rk3588-nanopc-t6.dtb

 

After rebooting, I confirmed that it was now running in HS200 mode:

 

[  +0.010753] mmc0: SDHCI controller on fe2e0000.mmc [fe2e0000.mmc] using ADMA
[  +0.034734] mmc0: new HS200 MMC card at address 0001
[  +0.000571] mmcblk0: mmc0:0001 A3A444 230 GiB

 

I ran a heavy random read/write test, and observed no errors.

 

However, I think this will be lost during a future update - so i'll see if I can figure out how to submit a patch!

[KingJ]

Raised up a PR with a patch. 🤞

[usual user]

1 hour ago, KingJ said:

Raised up a PR with a patch. 🤞

You will certainly gain many grateful users whose devices are not equipped with an affected eMMC, but who still have to suffer from the slowdown nonetheless.
Does the attached overlay work for you?rk3588-nanopc-t6-emmc.dtbo

[KingJ]

Right, that's the downside - anyone with a different eMMC chip will see a drop in performance.

 

I've just given that overlay a try - thank you for suggesting it. I put it in to /boot/overlay-user, then edited /boot/armbianEnv.txt to include the line user_overlays=rk3588-nanopc-t6-emmc . Unfortunately, it then failed to boot. I had to put in an SD card, boot from that, mount the eMMC partition, and then remove the overload from armbianEnv.txt to get it to boot successfully from the eMMC again. 

[usual user]

18 hours ago, KingJ said:

Unfortunately, it then failed to boot.

Admittedly, I had not tested the DTBO at runtime so far, but only applied it statically to the base DTB and checked whether all desired changes were made as intended.
Now I am running my device with the applied overlay and I get the following:

[    0.940862] mmc0: new HS200 MMC card at address 0001
[    0.941665] mmcblk0: mmc0:0001 A3A561 57.6 GiB
[    0.943039]  mmcblk0: p1 p2
[    0.943767] mmcblk0boot0: mmc0:0001 A3A561 4.00 MiB
[    0.945199] mmcblk0boot1: mmc0:0001 A3A561 4.00 MiB
[    0.946661] mmcblk0rpmb: mmc0:0001 A3A561 16.0 MiB, chardev (506:0)

So everything is as expected, and yes, I have ensured that my system is running with the applied overlay. In the past, I have at least once noticed far too late that my system was running in a fallback, and a feature to be tested was not applied at all. That's just the disadvantage of a fail-safe system that only leaves a non-functional system in an extreme exceptional situation.
Since you haven't provided meaningful logs, I can't say what is going wrong on your end.
To rule out an error when applying the overlay, you could start your system with this DTB (rk3588-nanopc-t6.dtb) , which already contains the overlay applied
 statically.