NanoPi R4S rk_gmac-dwmac WAN on 1Gbps/Full network packet loss problem

[Pedro Lucas]

Hello everyone,

 

I have a problem with my NanoPi R4S. The WAN (eth0) port does not work well when it is configured to work on 1Gbps/Full. When this setting is active there is about 15% packet loss, which is quite annoying.

 

In my tests I have already been able to identify that if I configure the port to work at 100Mbps/Full everything works normally. So I found the forum-post below, that explain that some u-Boot version wasn't setting the correct clock configuration for the netwok device at boot time, but now I have installed the linux-u-boot-nanopi-r4s-edge  22.08.6 and linux-image-edge-rockchip64 22.08.8 but the problem still there. 

 

I've read millions of forums for the solution, and so far I haven't made much progress. Would anyone be able to help me?

 

in dmesg it looks like:

[    1.853954] rk_gmac-dwmac fe300000.ethernet: IRQ eth_wake_irq not found
[    1.853967] rk_gmac-dwmac fe300000.ethernet: IRQ eth_lpi not found
[    1.854071] rk_gmac-dwmac fe300000.ethernet: PTP uses main clock
[    1.854234] rk_gmac-dwmac fe300000.ethernet: clock input or output? (input).
[    1.854244] rk_gmac-dwmac fe300000.ethernet: TX delay(0x28).
[    1.854253] rk_gmac-dwmac fe300000.ethernet: RX delay(0x11).
[    1.854267] rk_gmac-dwmac fe300000.ethernet: integrated PHY? (no).
[    1.854320] rk_gmac-dwmac fe300000.ethernet: cannot get clock clk_mac_speed
[    1.854326] rk_gmac-dwmac fe300000.ethernet: clock input from PHY
[    1.859341] rk_gmac-dwmac fe300000.ethernet: init for RGMII
[    1.860175] rk_gmac-dwmac fe300000.ethernet: User ID: 0x10, Synopsys ID: 0x35
[    1.860191] rk_gmac-dwmac fe300000.ethernet:         DWMAC1000
[    1.860199] rk_gmac-dwmac fe300000.ethernet: DMA HW capability register supported
[    1.860206] rk_gmac-dwmac fe300000.ethernet: RX Checksum Offload Engine supported
[    1.860212] rk_gmac-dwmac fe300000.ethernet: COE Type 2
[    1.860218] rk_gmac-dwmac fe300000.ethernet: TX Checksum insertion supported
[    1.860224] rk_gmac-dwmac fe300000.ethernet: Wake-Up On Lan supported
[    1.860283] rk_gmac-dwmac fe300000.ethernet: Normal descriptors
[    1.860291] rk_gmac-dwmac fe300000.ethernet: Ring mode enabled
[    1.860297] rk_gmac-dwmac fe300000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[   11.139092] rk_gmac-dwmac fe300000.ethernet eth0: Register MEM_TYPE_PAGE_POOL RxQ-0
[   11.178717] rk_gmac-dwmac fe300000.ethernet eth0: PHY [stmmac-0:01] driver [RTL8211E Gigabit Ethernet] (irq=75)
[   11.189571] rk_gmac-dwmac fe300000.ethernet eth0: No Safety Features support found
[   11.189610] rk_gmac-dwmac fe300000.ethernet eth0: PTP not supported by HW
[   11.190145] rk_gmac-dwmac fe300000.ethernet eth0: configuring for phy/rgmii link mode
[   14.868809] rk_gmac-dwmac fe300000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx

 

I was thinking that the line below was the problem, but I'm confused now:

 

Quote

rk_gmac-dwmac fe300000.ethernet: cannot get clock clk_mac_speed

 

armbianmonitor -u: https://paste.armbian.com/joperiqoso

Edited November 16, 2022 by Pedro Lucas

[Werner]

Providing logs with 

armbianmonitor -u

 helps with troubleshooting and significantly raises chances that issue gets addressed.

[Pedro Lucas]

The command below seems to be the solution for this problem: 

 

ethtool -K enp1s0 tso off

 

I have found this post, where the command was.

 

Edited November 22, 2022 by Pedro Lucas

[whitefox]

I have a similar problem with packet loss on orange pi  rk3399 when connected on 1Gbps/Full

Armbianmonitor: http://ix.io/4ia5

[   10.377653] rk_gmac-dwmac fe300000.ethernet eth0: PHY [stmmac-0:01] driver [RTL8211E Gigabit Ethernet] (irq=84)
[   10.386477] rk_gmac-dwmac fe300000.ethernet eth0: Register MEM_TYPE_PAGE_POOL RxQ-0
[   10.396512] rk_gmac-dwmac fe300000.ethernet eth0: No Safety Features support found
[   10.396545] rk_gmac-dwmac fe300000.ethernet eth0: PTP not supported by HW
[   10.397193] rk_gmac-dwmac fe300000.ethernet eth0: configuring for phy/rgmii link mode
[   10.557743] aufs 5.15.41-20220620
[   11.825681] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[   11.831210] Bridge firewalling registered
[   12.194629] Initializing XFRM netlink socket
[   12.666036] rk_gmac-dwmac fe300000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[   12.666114] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready


root@orangepi-rk3399:~# speedtest 

   Speedtest by Ookla

      Server: DOM.RU - Yekaterinburg (id: 5844)
         ISP: Dom.ru
Idle Latency:     2.01 ms   (jitter: 0.09ms, low: 1.96ms, high: 2.13ms)
    Download:   629.96 Mbps (data used: 888.3 MB)                                                   
                 66.92 ms   (jitter: 50.00ms, low: 1.11ms, high: 875.41ms)
      Upload:     1.36 Mbps (data used: 2.5 MB)                                                   
                  1.58 ms   (jitter: 5.65ms, low: 1.22ms, high: 210.62ms)
 Packet Loss:     8.3%
  Result URL: https://www.speedtest.net/result/c/70477fc4-89f0-4b2a-b9a7-4248e25af9d7

 

 

After switching to 100 Mbps packet loss disappears:

ethtool -s eth0 autoneg on speed 100 duplex full

[  614.706367] rk_gmac-dwmac fe300000.ethernet eth0: Link is Down
[  616.829581] rk_gmac-dwmac fe300000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx

root@orangepi-rk3399:~# speedtest 

   Speedtest by Ookla

      Server: DOM.RU - Yekaterinburg (id: 5844)
         ISP: Dom.ru
Idle Latency:     2.03 ms   (jitter: 0.12ms, low: 1.86ms, high: 2.15ms)
    Download:    93.89 Mbps (data used: 42.3 MB)                                                   
                  2.90 ms   (jitter: 0.32ms, low: 2.03ms, high: 4.10ms)
      Upload:    93.80 Mbps (data used: 43.5 MB)                                                   
                  3.89 ms   (jitter: 0.60ms, low: 2.46ms, high: 8.08ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/03847e3f-6400-457f-bd6b-067b59b9c243

 

Edited December 9, 2022 by whitefox

[Pedro Lucas]

@whitefox, have you tested disabling the tcp-segmentation-offload and generic-segmentation-offload? If not, could you try? With the Nanopi-R4s it was the "Solution". It ins't performant, but work far best now. The packet loss is gone.

 

Here are the lines to run:

 

ethtool -K eth0 tso off
ethtool -K eth0 gso off

 

To verify: 

 

ethtool -k eth0

 

I hope it helps.

[whitefox]

@Pedro LucasYes, I tried this "solution" did not work for me.

 

root@orangepi-rk3399:~# ethtool -s eth0 autoneg on speed 1000 duplex full
root@orangepi-rk3399:~# ethtool -K eth0 tso off
root@orangepi-rk3399:~# ethtool -K eth0 gso off

root@orangepi-rk3399:~# ethtool -k eth0 | grep -e [tg]so
tx-gso-robust: off [fixed]
tx-gso-partial: off [fixed]
tx-gso-list: off [fixed]
root@orangepi-rk3399:~# speedtest 

   Speedtest by Ookla

      Server: DOM.RU - Yekaterinburg (id: 5844)
         ISP: Dom.ru
Idle Latency:     1.96 ms   (jitter: 0.16ms, low: 1.77ms, high: 2.14ms)
    Download:   625.19 Mbps (data used: 654.8 MB)                                                   
                 25.84 ms   (jitter: 36.44ms, low: 1.08ms, high: 457.34ms)
      Upload:     1.69 Mbps (data used: 2.7 MB)                                                   
                  1.54 ms   (jitter: 2.90ms, low: 1.15ms, high: 417.26ms)
 Packet Loss:     5.4%
  Result URL: https://www.speedtest.net/result/c/c12b33b6-1196-42d7-8d77-75180d8e1aac

 

[Pedro Lucas]

From the link below: 

https://www.stupid-projects.com/posts/nanopi-r4s-benchmarks-with-networking-optimizations/

 

The MTU size seems to be important too. 

With the commands: 

ip link set dev end0 mtu 512
ip link set dev enp1s0 mtu 512

 

The packet loss dropped from 15%-30% to 0%.