Problem with 1G WAN port when receiving large file transfers

[ct100]

I have the newest Armbian 28.08.0-trunk image running on a NanoPi R6C with a 4TB NVMe SSD. The NanoPi is only connected to power and 1G ethernet via the 1G WAN port.

 

When using the machine as NAS with Samba and a Windows client connected there is a strange problem when receiving large files. The files are stored on the NVMe SSD with ~450MB/s read/write via PCIe 2.1x1.

 

 

When transmitting files from the NanoPi to a SMB client the 1G ethernet is maxed out but when receiving files the connection drops down to ~60 MB/s every few seconds.

 

This instantly gets fixed when using the 2.5G LAN port instead of the 1G WAN port. On that port the RX looks exactly the same as the TX. Both with 112MB/s maxed out speed.

I noticed that the 1G WAN port is a combination of GMAC in RK3588s SoC and a Realtek RTL8211F Transceiver while the 2.5G LAN port is a dedicated Realtek RTL8125BG.

 

So i suspect there is something wrong with that combination of GMAC and Realtek Transceiver on the 1G WAN port. I already tried changing the rgmii rx settings with an overlay, but that didn't have any effect. Driver problem?!

[Dantes]

I hate to sound like a broken record, but that 3rd party NanoPI OpenWRT image I suggested in the other thread comes with SMB4 preinstalled. Its real easy to setup (5 mins). If the problem is the same, it could be a driver, dtb-file or something else both have in common. If not, then its something to do with Armbian itself.

[ct100]

@Dantes Sorry but that image doesn't really help. Yes it boots but there is a lot of stuff not working with that edge kernel.

 

 

There is definitely something wrong with the hardware itself or the driver of the GMAC and Realtek Transceiver combo on the 1G WAN.

 

Its possible to increase the RX performance by deactivating energy efficient ethernet with "ethtool --set-eee eth0 eee off" but the performance is still worse than the Realtek RTL8125BG NIC on 2.5G LAN port. Also deactivating eee leads to ~0,25W increased idle power consumption.

 

Running iperf3 tests everything looks ok but when there is the slightest I/O at the same time esp. on the NVME drive the NIC stalls.

 

When reducing the Kernel write cache from 20% to 1% with "echo 1 > /proc/sys/vm/dirty_ratio" its very easy to see the difference:

 

 

Edited August 28, 2023 by ct100