I've been having the same issue of eth1 just not working right on a 2.5Gbps link on kernel 5.9.x. The same applies for a self-built 5.10.1 kernel and also when going back to r8152 2.13.0 instead of the default 2.14.0 (by adapting the revision in build/lib/compilation-prepare.sh). With a 1Gbps connection, everything is perfectly fine. This connection speed was "forced" by running "ethtool -s enp6s0 advertise 0x3f" on the desktop side, taking 2.5Gbps out of the advertised speeds for autonegotiation. I have a direct 5m Cat6 connection from my Helios64 to my desktop machine. The latter has a 2.5Gbps RTL8125 on the mainboard, using the vanilla kernel's r8169 module. MTUs were kept on 1500 for all tests.
Running the Helios64 on kernel 4.4.213 (via Armbian_20.11.4_Helios64_buster_legacy_4.4.213.img.xz), I was able to transfer 4TiB back and forth in parallel without any issue: Helios64 -> desktop was running with ~1.7Gbps; desktop -> Helios64 was running at ~2.1Gbps. 1.7Gbps aren't great, but ok enough with tx offload disabled - certainly better than "just" 1Gbps (~933, realistically).
One surefire way to kill the Helios64's eth1 is to just start a simple iperf3 run with it sending data to my desktop: "iperf -c <desktop-ip> -t 600". After a few seconds, the speed goes down to 0, the kernel watchdog resets the USB device (as per r8152.c's rtl8152_tx_timeout), it recovers for a few more seconds, and then eth1 is absolutely dead until I reboot the entire NAS. No ping, no more connection, nothing.
Here's what I can see via journalctl -f from a parallel connection when such a crash happens:
If there are any more logs you'd need or software changes to try and apply, I'd be eager to dig deeper.
I think so, I did a fresh install on the sd card from this file - Armbian_20.11.4_Helios64_buster_current_5.9.14.img.xz
Here is the dmesg output from after reboot. Can't access the previous boot records though. This one shows a lot of eth1 errors and warnings, but it hasn't crashed yet.
I tried to put the output in both spoiler and code. Hope it works.