2 2
Spemerchina

Espressobin - etherchannel?

Recommended Posts

Hi everyone, 

 

especially armbian maintainers!

 

I'd like to make use of great performance of this board regarding SATA performance which is crippled by 1Gbps so my question is, would etherchannel work on this hardware? I understand that the board has only 1Gbps from the switch (dual LAN ports) so I guess the only possible way to exceed 1Gbps output would be to bundle WAN && [LAN01|LAN02]. Anyone tried this or believe it should work? I don't have the board to try.

 

Thanks.

Share this post


Link to post
Share on other sites

Here is the configuration of

 

Cisco switch side:

 

interface Port-channel2
 description LACP Channel for mk2
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 1,2
 switchport mode trunk
 spanning-tree portfast trunk
!
interface GigabitEthernet1/0/23
 description mk2 eth0
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 2 mode active
!
interface GigabitEthernet1/0/24
 description mk2 eth1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 2 mode active
!

 

armbian module config:

# /etc/modules: kernel modules to load at boot time.
 bonding mode=4 miimon=100 lacp_rate=1

 

armbian networking:

#/etc/network/interfaces 
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
 
auto eth0
    iface eth0 inet manual
    bond-master bond0
 
auto eth1
     iface eth1 inet manual
     bond-master bond0
 
auto bond0
     iface bond0 inet static
     address 10.0.0.80
     gateway 10.0.0.1
     netmask 255.255.255.0
 
 
bond-mode 802.3ad
bond-miimon 100
bond-lacp-rate 4
bond-slaves none

thanks

Share this post


Link to post
Share on other sites

It was my understanding that the whole Topaz switch has only the 1 Gbps connection to the SOC, and that all three ethernet ports were from that switch.  The labels for wan, lan0, and lan1 are just labels, on switch ports 1, 2, and 3.

Share this post


Link to post
Share on other sites
On 2/7/2019 at 3:20 PM, ManoftheSea said:

It was my understanding that the whole Topaz switch has only the 1 Gbps connection to the SOC, and that all three ethernet ports were from that switch.  The labels for wan, lan0, and lan1 are just labels, on switch ports 1, 2, and 3.

 

unfortunately you are right, Topaz is connected to SoC via RGMII (1Gbps) so no chance for port channel. 

Share this post


Link to post
Share on other sites

How about with a mPCIE network card?

 

Currently there is Syba Gigabit Ethernet Mini PCI Express card on Amazon for about $17. That particular card only gets you one additional port, but I have seen options with two ports (although I haven't found any 2-port models anywhere near this price point).

 

Would it be possible to use a gigabit port on the PCI expansion slot in an etherchannel with one of the ports of the built-in switch to achieve a 2gbps link? I haven't figured out how to configure those built-in switch ports to anything other than the 3-port bridge yet, but I wonder how limited our options are with that Topaz switch in the middle.

 

If the 2-port gigabit expansion card wasn't so cost-prohibitive, I think the mPCIE slot would have the bandwidth to do 2gbps by itself. Judging by tkaiser's benchmarks of the mPCI SATA expansion board, it looks like he's hitting between 250,000 and nearly 300,000 kiloBytes/sec when using a single drive on the expansion board. Of course that's dependent on the drive and several other factors, but that gives us an upper limit of at least 1.9 to 2.3 gigabits/sec.

 

And of course this all assumes you're not already using the mPCIE slot for more SATA ports. :-\

Share this post


Link to post
Share on other sites
12 hours ago, Jbobspants said:

How about with a mPCIE network card?

 

Currently there is Syba Gigabit Ethernet Mini PCI Express card on Amazon for about $17. That particular card only gets you one additional port, but I have seen options with two ports (although I haven't found any 2-port models anywhere near this price point).

 

Would it be possible to use a gigabit port on the PCI expansion slot in an etherchannel with one of the ports of the built-in switch to achieve a 2gbps link? I haven't figured out how to configure those built-in switch ports to anything other than the 3-port bridge yet, but I wonder how limited our options are with that Topaz switch in the middle.

 

If the 2-port gigabit expansion card wasn't so cost-prohibitive, I think the mPCIE slot would have the bandwidth to do 2gbps by itself. Judging by tkaiser's benchmarks of the mPCI SATA expansion board, it looks like he's hitting between 250,000 and nearly 300,000 kiloBytes/sec when using a single drive on the expansion board. Of course that's dependent on the drive and several other factors, but that gives us an upper limit of at least 1.9 to 2.3 gigabits/sec.

 

And of course this all assumes you're not already using the mPCIE slot for more SATA ports. :-\

 

not a bad idea :) By looking at http://espressobin.net/wp-content/uploads/2017/01/ESPRESSObin-V3-Hardware-Block-diagram-v3-1.pdf , one could imagine that we might get much higher than 2Gbps if we go 10Gbps PCI-E adapter route.. SATA has it's own direct lane to SoC..

Share this post


Link to post
Share on other sites
On 2/15/2019 at 5:33 AM, Spemerchina said:

 

not a bad idea :) By looking at http://espressobin.net/wp-content/uploads/2017/01/ESPRESSObin-V3-Hardware-Block-diagram-v3-1.pdf , one could imagine that we might get much higher than 2Gbps if we go 10Gbps PCI-E adapter route.. SATA has it's own direct lane to SoC..

 

True, the onboard SATA has it's own direct path, and that's what some of tkaiser's benchmarks were testing...the onboard SATA port vs ports on his mPCIE SATA expansion board. It seems the onboard port is significantly faster, while the tests on the mPCIE SATA ports were limited to roughly 2Gbps. I'm not sure if the bottleneck was the cheap 88SE9215 SATA card, or the mPCIE path. I looked for specs on the mPCIE port, but was not able to determine how many PCIE lanes or what PCIE revision it uses (this might be common knowledge to the community, but I wasn't able to dig it up in a quick search).

 

A 10Gbps card would be very cool, but in the same test, he came to the conclusion that overall speed using both onboard and mPCIE SATA simultaneously (in RAID 0/striping mode) was limited by the CPU. I know SATA w/software RAID and network benchmarks are very different, but in a real-world scenario, I doubt we'd be able to get anywhere near 10Gbps.

Share this post


Link to post
Share on other sites
10 hours ago, Jbobspants said:

 

True, the onboard SATA has it's own direct path, and that's what some of tkaiser's benchmarks were testing...the onboard SATA port vs ports on his mPCIE SATA expansion board. It seems the onboard port is significantly faster, while the tests on the mPCIE SATA ports were limited to roughly 2Gbps. I'm not sure if the bottleneck was the cheap 88SE9215 SATA card, or the mPCIE path. I looked for specs on the mPCIE port, but was not able to determine how many PCIE lanes or what PCIE revision it uses (this might be common knowledge to the community, but I wasn't able to dig it up in a quick search).

 

A 10Gbps card would be very cool, but in the same test, he came to the conclusion that overall speed using both onboard and mPCIE SATA simultaneously (in RAID 0/striping mode) was limited by the CPU. I know SATA w/software RAID and network benchmarks are very different, but in a real-world scenario, I doubt we'd be able to get anywhere near 10Gbps.

image.png.6666a885e13bef33f292d08720f105a7.png

 

It looks like we have PCIe 2.0 which supports 500/500MB simultaneous read/write bandwidth. There are some cheap Asus 10Gbe adapters that could possible use all the bandwidth available (read from SATA, write to PCIe and in the opposite direction), so we can see close to 400/400 out of 10Gbe port without RAID overhead. I'm not interested in RAID, just a JBOD performance :)

 

Edit: Not sure about the exact wiring in Espressobin and capabilities of shared SERDES lanes among PCIe2.0 and SATA, some research says around 7.5Gbps total throughput, so I'll lower expectations to 3Gbe full duplex speeds with separte 10Gbe card in mPCI slot..

 

I believe @tkaiser could say a word or two on the matter :)

 

 

 

Share this post


Link to post
Share on other sites
12 hours ago, Spemerchina said:

Edit: Not sure about the exact wiring in Espressobin and capabilities of shared SERDES lanes among PCIe2.0 and SATA, some research says around 7.5Gbps total throughput, so I'll lower expectations to 3Gbe full duplex speeds with separte 10Gbe card in mPCI slot..

 

I believe @tkaiser could say a word or two on the matter :)

 

 

 

You have to make a choice with the shared SERDES lanes.

Espressobin has already made that for you with it's PCB layout - one lane goes to SATA, one to USB3 and one to PCIe.

There is only one lane on the PCIe slot, which is also apparently limited to 2.5Gb in early boards due issues running at PCI 2 speeds.

The switch is connected via a 1Gb link to the 3270.

 

Your physical limitations will be 2.5* or 5Gb on the PCIe slot and 3Gb on the SATA 2 connection. The 3Gb link uses 8b/10b encoding, so it's limited to 300MB/s.

 

 

* OpenWRT has reduce PCIe speed to 1.0 

https://git.openwrt.org/?p=openwrt/openwrt.git;a=commitdiff;h=772258044b48036699302840abf96cd34c4e5078

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
2 2