Hey fellas

 

I have FOUR different armbian devices (one official armbian, three running ophub's armbian) which all seem to struggle with network throughput at high latency connections. It is not a cpu issue and this doesn't occur on x86 debian. 

 

Here's the odd thing.

 

If I connect an ethernet usb dongle, like rtl8152b I get the full throughput even at high latency. What??? So it's a nic issue?

 

I thought maybe the onboard ethernet isn't well supported by the kernel so I got a nanopi neo4 with onboard realtek gigabit running official armbian 6.12.3x ... AND it's slow too. Any onboard nic with armbian is slow in my tests.

 

With slow I mean that a single connection gets 5-10Mbit. With a dongle I can get up to 100Mbit on a SINGLE connection from the same device.

 

Why is a usb realtek dongle better supported than onboard realtek gigabit? I don't get it. Does anybody have an idea I could try?

 

What's really difficult is that the problem only shows up at high latency. If I set up a new device at home (low latency) I get full throughput, but when it's in another country 200-300ms latency it just slows down to 5-10Mbit. The workaround with a dongle was just a fluke accident that I happen to notice. 

 

I truly believe armbian has a bright future but right now I'm tempted to go back to x86 for my needs and I'd rather not because arm SHOULD be better than x86 for IoT (my use case). 

 

Thanks for helping me figure it out. Love your work! 

4 hours ago, Mike H said:

three running ophub's armbian

For those we cannot and will not help. This is a fork which uses the name Armbian without permission and does not contribute to the core development process. Ask at their place for help.

 

For the leftover: armbianmonitor -u would be a good start.

Hi and thanks for the reply

 

I have the device running official armbian at my location (already brought it back) and so I'm trying to figure out a way to "simulate" high latency. Any ideas?

 

Otherwise we'll have to wait a few months so I can place it at it's intended location again. As you can imagine, this is something that's taken a LONG time. Many many months and progress is slow.

Edited July 29, 20251 yr by Mike H

Seriously?

 

https://www.lmgt.org/?q=simulate+high+latency

https://duckduckgo.com/?q=simulate+high+latency

 

You're welcome.  "Many many months" of what?

Of placing the device back with a new kernel/distro or even doing testing on it. I don't travel much and my friends who do don't have any experience nor equipment to change things.

If you look at it from my point of view there's a bug in armbian. At the very least it's now been reported even if I'm not clever enough to figure it out. You aren't showing much sympathy to the regular users trying to help out by such condescending language. 

Pointing out that you are not even willing to put your own words into a search engine is "condescending"?  That takes 5 seconds or less when you claim to have put months and months into it.  You are expecting others to do the work for you.  I find that quite presumptuous.

 

And no, we do not know there is a bug in Armbian.  You haven't done enough research to know enough to claim this for certain.  I will put the chances of that being the case at fity-fifty at best.  You also have presented no logs whatsoever (even after being asked) to even give others the moderate chance to do this work for you (for free).

 

If you came here to vent, OK, you have now vented.  If you came here for a solution, then sorry to say, but you are not even doing the very minimum to get that done.

@Mike H

 

This is a community forum, and your message is for everyone. Armbian maintainers are part of this community, but please keep in mind that support here is provided on a best‑effort basis by volunteers and maintainers who give their time when they can.

 

In open‑source projects, especially those maintained by communities, work is prioritized based on available time, skills, and resources. Sometimes an issue might be addressed quickly; other times it may take months — or it may not be addressed at all if the effort required is too great compared to the benefit. This is a normal reality in community‑driven software development. The concept of sunk cost often plays a role in deciding whether to invest effort in fixing certain issues.

 

It’s worth noting that even in large, well‑funded projects, some bugs remain unresolved for years. In our case, resources are far more limited — for example, Raspberry Pi OS has a budget exceeding €1M annually for one platform, while Armbian supports many platforms with a fraction of that funding, outside few boards we have some small maintaining budget, there is only a beer money, and far fewer people. Best-effort is the only promise that can be provided under such circumstances.

 

We appreciate bug reports, even they have nothing to do with our work as we understand users often can't judge that, but we ask that no pressure be placed on contributors to resolve them on demand. In some cases, solving an issue can take weeks or even months of work, which is not something users can reasonably expect from volunteers. And there are thousands of bugs already in the to do list.

 

Finally, please understand that part of the challenge we face is that some projects already monetize or re‑brand our work while presenting it as inexpensive and easy to maintain — when in reality, it’s not. This can create misunderstandings about the real cost and effort behind keeping a complex software stack running.

 

Your report is welcome, but we ask for patience, understanding, and respect for the time and resources of those trying to help. Pressuring volunteers to prioritize some issue, or expecting them to personally fund the work, is neither reasonable nor effective way to fix bugs.

I faced a similar Ethernet throughput issue on Armbian under high latency. One thing that helped was using a USB Ethernet dongle instead of the onboard NIC. To simulate latency, you can use the tc command:
sudo tc qdisc add dev eth0 root netem delay 200ms
sudo tc qdisc del dev eth0 root
For more solutions, check out this.

 

I'm not sure it's related to your issue, but on some pi's I've seen Ethernet bandwidth issues accumulating over uptime.

It was related to only to a part of specific hardware models, so suspect might be related to either hardware or kernel/driver version/implementation.

 

After some tests I've found that the issue affects only per TCP connection downlink bandwidth - UDP does not have limits, as well as uplink TCP, multiple downlink TCP connections bandwidth stacks linearly.

The worst bw case I've seen is ~2Mbps per TCP conn.

 

Reboot fixes the issue for a random time (briefly from a few hours to a few months). Sometimes(?) the issue disappears on its own.

So far I've not found a fault-proof fix except a reboot, but changing tcp congestion algo is worth to try:

# sysctl net.ipv4.tcp_available_congestion_control
net.ipv4.tcp_available_congestion_control = reno bic cubic westwood highspeed hybla htcp vegas
nv veno scalable lp yeah illinois bbr
# sysctl net.ipv4.tcp_congestion_control     
net.ipv4.tcp_congestion_control = cubic
# sysctl net.ipv4.tcp_congestion_control=bbr   
net.ipv4.tcp_congestion_control = bbr

 

For tests you may use iperf3 varying -C/-P/-R/..., e.g.:
 

# iperf3 -4 -f m -P 1 -i 0 -t 5 -c IPERF3_SRV_IP --bind LAN_IP -R -b 100M -u  # Single connection DL UDP up to 100Mbps for 5 sec
# iperf3 -4 -f m -P 1 -i 0 -c IPERF3_SRV_IP --bind LAN_IP -R -C bbr  # Single TCP conn DL using BBR as a congestion algo
...
# iperf3 -4 -f m -P 50 -i 0 -c IPERF3_SRV_IP --bind LAN_IP -R  # 50 TCP conns DL

 

Edited October 17, 20251 yr by vmspike

On 7/29/2025 at 11:50 AM, Werner said:

For the leftover: armbianmonitor -u would be a good start.

Hi again Werner. Please excuse the long waiting time. It's taken me a long time to place back the device at its intended location. Here's the requested output.

 

https://paste.armbian.com/alefaladep

 

At the moment I've got it running on a realtek usb-ethernet dongle at full speed (confirmed throughput up to about 300Mbit). Last year when I created the thread we were using the Nanopi Neo4's onboard realtek ethernet port. No matter what we did it wouldn't run at speeds over 10Mbit. And no, the port isn't faulty. It's happening on ALL armbian devices (official and unofficial) on current distributions still to this day. Bookworm/Trixie/Noble/Jammy, with kernels 6.12, 6.6, 6.1 and 5.15 tested. On x86 debian it works right out of the box at full speed and we're talking much weaker hardware (Intel z8350).

 

Thanks for your work. Armbian is definitely the future IMO.

Edited October 19, 20251 yr by Mike H

Just wanted to give an update that years later this issue is still there. Eth0 doesn't respond (well) to congestion control algorithms. There is a difference between cubic/bbr but I only get a proper speed with a usb dongle. (onboard ~7mbit, dongle ~250mbit)

 

The dongle is realtek and so is the "problematic" onboard ethernet. So it's not a driver issue. It seems to be an issue with the eth0 config on all official and unoffocial armbian releases. The same issue exists on non realtek, amlogic devices. Plug usb and the problem goes away.

 

Wouldn't it be cool to try and rename eth0 to eth1? Ha!

On 10/20/2025 at 1:18 AM, Mike H said:

On x86 debian it works right out of the box at full speed and we're talking much weaker hardware (Intel z8350).

This it makes little sense, it is just totally different HW and I am quite sure the problem in this topic is related to HW / drivers. I see in you 1 year old armbianmonitorlog:

[ 3.039362] rockchip-pcie f8000000.pcie: PCIe link training gen1 timeout!
[ 3.039448] rockchip-pcie f8000000.pcie: probe with driver rockchip-pcie failed with error -110

 

BUT it takes me too long to determine what actual HW SBC this is about, yes Rockchip based, but what brand, series, HW revision, etc, etc. Also what U-Boot, maybe you still have some old illegal one as the first topic line suggests.

 

1 hour ago, Mike H said:

Wouldn't it be cool to try and rename eth0 to eth1? Ha!

This is really a no-brainer compared to the topic issue and also depends on which HW/SBC it is. Some need extra udev rules, some might handle it in systemd-networkd. But easy to change yourself, I have done it several times as it fits my preferences better, depending on what computer it is. You need to do that yourself if you plug in extra RJ45 devices for sure.

 

So can you provide:

- U-Boot log via serial console UART TTL cable?

- an URL to the vendor HW and a version number of your specific HW revision ?

 

1 hour ago, Mike H said:

There is a difference between cubic/bbr but I only get a proper speed with a usb dongle. (onboard ~7mbit, dongle ~250mbit)

What does this mean? And it seems USB2, so half duplex down to the lowest level of the chain. Or not?

 

I can add more confusion, for example why don't you build/compile a latest kernel (7.3 based) yourself and also latest U-Boot (so select edge or bleedingedge branch). I have several of my Armbian Trixie devices with 7.3 or 7.2 rockchip64 or sunxi kernels. Or you run just standard Debian Unstable/Sid kernel version and compare. Maybe there are buffer sizes somewhere that matter, but is up to you to report, only then maybe others can suggest what to do or change. 

Quote

The dongle is realtek and so is the "problematic" onboard ethernet. So it's not a driver issue

 

It is highly likely that a realtek onboard device will use a different driver to a USB attached one. I have a UGreen 2.5Gb USB adapter that uses the r8152 driver and my Radxa Rock-5b has an RTL8125 pcie device that uses the r8169 driver. Sure there might be code reuse between them but they are a different driver.

Edited October 20Oct 20 by TrevorH

First of all I have to thank you gentlemen for reading and writing. Cheers!

 

I'll be honest. I haven't tried the newer armbian builds. I just remember that about a year ago I tried 6.1, 6.6 and 6.12 and they behaved the same.

 

On x86 the onboard (realtek) ethernet works ok under any version of debian tried.

 

On 10/20/2026 at 4:17 PM, eselarm said:

What does this mean? And it seems USB2, so half duplex down to the lowest level of the chain. Or not?

On armbian on the onboard ethernet (realtek OR amlogic) I get with BBR congestion control about 7mbit throughput (high latency). Without configuring congestion control I get maybe 1mbit.

 

On armbian on a realtek dongle (be it rockchip or amlogic soc, usb2 or usb3) I get with BBR enabled 250mbit. With an 8152 dongle I get its max 100Mbit. 

 

Because the issue appears on different socs with different nics, the only constant is the primary nic eth0. That's always the affected nic.

 

To make things complicated, I don't have hands on access because they're spread around the world. I'd need to create a local test setup with simulated high latency to test things. It's a bit over my ability. I thought I'd just report the bug and hope someone else more knowledgeable would have encountered it and reported their findings back. Seems only I use armbian for high latency applications. Hmm...

 

For the lack of people reporting this, you guys deserve credit for your interest. Thx!

Edited October 22Oct 22 by Mike H

4 hours ago, Mike H said:

To make things complicated, I don't have hands on access because they're spread around the world. I'd need to create a local test setup with simulated high latency to test things. It's a bit over my ability. I thought I'd just report the bug and hope someone else more knowledgeable would have encountered it and reported their findings back. Seems only I use armbian for high latency applications.

'Spread around the world' means to me costs of plane tickets and hours lost in travel time etc, so for me it was/is an easy calculation: You can buy a handful of computers/boards for that money and also spend time to think about a dual-setup, e.g. a working link with x86 and devices under test. In the past for just 1000km distance (by car) I just changed ISP's/contracts at the 2 locations such that I got Fritz!Boxes (DSL, Cable modems build in), those have build-in (old fashion) VPN, so I could rather easily create my own OpenVPN and later WireGuard links as well. That was already a few hundred Euros, besides lots a time to figure out IPv6 issues etc. The latency was no real issue, but is just an example how to approach. And really is all DIY, as Linux computers, no support form computer shop what so ever. 

 

Nowadays, just at home I use USB serial UART/TTL console/debug cables and maybe a wall plug with some ESP/Tasmota so I can powercycle things. z8350 I have as well, they are low-power, same as RPI4, much better actually as everything with sleep/standby also works and LiPo battery included etc.

So one known working computer is 'HID' via the serial cable on header pins 6 8 10 or dedicated extra generic USB-C cable even like for my NanoPi-R6C. So I can watch everything already at power-on, most important is U-Boot/bootloader. For your unknown SBC's the kernel-loading might just takeover buffer sizes w.r.t. on-board ethernet as that is usually initialized before kernel as U-Boot might want to do networkboot. That is just local, might be simple UDP so nothing complex with TCP windowing you will have to deal with at high latency. But is all guessing, you need at least a serial debug cable to start with, else you waste your time also with guessing. 

 

Then it is important to consider the HW I/O path what you already figured out already: USB is an extra layer, for sure has extra other/more buffers; you can calculate how big they will be or grow based on latency and speed. And not known by U-Boot, so likely own/new buffer sizes. Same can be the case for PCI-E connected RJ45, as mentioned ROCK5B has its 2.5GbE RTL chip like that, same as my NanoPi-R6C 'LAN' port. I use EDK2 UEFI with netboot disabled as I just have NVME inside, so I guess other buffersizes. Armbian on those high-end but low-power devices uses the same as on x86, I can use Armbian edge rockchip64 kernel or Debian sid generic arm64, makes no real difference, at least not for networking. Even more for userpace, it is both the same Debian Trixie arm64 packages and configs.

 

However, if on-SoC ethernet, various the driver levels are different, and even per SoC. That is where you need to look at I think. Might be very complex and at kernel/uboot development level. Maybe something in DeviceTree, but not likely I think. You might also try 'the OS that came with the SBC', although I guess those are hopelessly outdated, but I guess same problem with high latency IP traffic.

Are all the armbian machines that are affected using a mixture of different cpu cores/speeds? The Neo4 uses a mixed big:LITTLE set up, do the others also do so?

I've got four Amlogic S905L3A boxes and an RK3399. 

 

Why would the cpu only affect the onboard nic?

 

Thanks eselarm and TrevorH. If I find out anyhing

interesting I'll let you know.

Google says the S905L3A has cpu cores that are all the same so it is not the cause of the problem.