Jump to content

BPi-R1 with new B53 switch driver (DSA)


Tido

Recommended Posts

Help us fund some #badass gear for our continuous integration build framework

Now I made a test with your config. I copied a 6GB file via nfs and it worked without problems. Pings are okay too.

 

I made two minor changes to your script:

  • added "bridge vlan del ... master" commands to remove all rules with VID 1
  • commented "ip link set dev br0 type bridge stp_state 0" out because it resulted in an error
  • assigned a IP address and route
set -xue


ip link set eth0 down
ip addr flush eth0
ip link set eth0 up

ip link add link eth0 name eth0.101 type vlan id 101
ip link set eth0.101 up
ip link add link eth0 name eth0.102 type vlan id 102
ip link set eth0.102 up
ip link add link eth0 name eth0.103 type vlan id 103
ip link set eth0.103 up
ip link add link eth0 name eth0.104 type vlan id 104
ip link set eth0.104 up
ip link add link eth0 name eth0.105 type vlan id 105
ip link set eth0.105 up

 

ip link add br0 type bridge
#ip link set dev br0 type bridge stp_state 0

# BCM53125 ports 4, 0, 1, 2, 3
ip link set lan1 master br0
ip link set lan2 master br0
ip link set lan3 master br0
ip link set lan4 master br0
ip link set wan master br0


bridge vlan add vid 101 dev lan1 pvid untagged
bridge vlan del dev lan1 vid 1 self
bridge vlan del dev lan1 vid 1 master
bridge vlan add vid 102 dev lan2 pvid untagged
bridge vlan del dev lan2 vid 1 self
bridge vlan del dev lan2 vid 1 master
bridge vlan add vid 103 dev lan3 pvid untagged
bridge vlan del dev lan3 vid 1 self
bridge vlan del dev lan3 vid 1 master
bridge vlan add vid 104 dev lan4 pvid untagged
bridge vlan del dev lan4 vid 1 self
bridge vlan del dev lan4 vid 1 master
bridge vlan add vid 105 dev wan pvid untagged
bridge vlan del dev wan vid 1 self
bridge vlan del dev wan vid 1 master

ip link set eth0.101 master br0
ip link set eth0.102 master br0
ip link set eth0.103 master br0
ip link set eth0.104 master br0
ip link set eth0.105 master br0

 
ip addr add 192.168.178.2/24 broadcast 192.168.178.255 dev br0
ip route add default via 192.168.178.1

ip link set br0 up
ip link set lan1 up
ip link set lan4 up
ip link set lan2 up
ip link set lan3 up
ip link set wan up

 

Congratulation - I think you found a working configuration!

Link to comment
Share on other sites

On 4/6/2017 at 5:49 PM, dumischbaenger said:

Now I made a test with your config. I copied a 6GB file via nfs and it worked without problems. Pings are okay too.

 

I made two minor changes to your script:

  • added "bridge vlan del ... master" commands to remove all rules with VID 1
  • commented "ip link set dev br0 type bridge stp_state 0" out because it resulted in an error
  • assigned a IP address and route

set -xue


ip link set eth0 down
ip addr flush eth0
ip link set eth0 up

ip link add link eth0 name eth0.101 type vlan id 101
ip link set eth0.101 up
ip link add link eth0 name eth0.102 type vlan id 102
ip link set eth0.102 up
ip link add link eth0 name eth0.103 type vlan id 103
ip link set eth0.103 up
ip link add link eth0 name eth0.104 type vlan id 104
ip link set eth0.104 up
ip link add link eth0 name eth0.105 type vlan id 105
ip link set eth0.105 up

 

ip link add br0 type bridge
#ip link set dev br0 type bridge stp_state 0

# BCM53125 ports 4, 0, 1, 2, 3
ip link set lan1 master br0
ip link set lan2 master br0
ip link set lan3 master br0
ip link set lan4 master br0
ip link set wan master br0


bridge vlan add vid 101 dev lan1 pvid untagged
bridge vlan del dev lan1 vid 1 self
bridge vlan del dev lan1 vid 1 master
bridge vlan add vid 102 dev lan2 pvid untagged
bridge vlan del dev lan2 vid 1 self
bridge vlan del dev lan2 vid 1 master
bridge vlan add vid 103 dev lan3 pvid untagged
bridge vlan del dev lan3 vid 1 self
bridge vlan del dev lan3 vid 1 master
bridge vlan add vid 104 dev lan4 pvid untagged
bridge vlan del dev lan4 vid 1 self
bridge vlan del dev lan4 vid 1 master
bridge vlan add vid 105 dev wan pvid untagged
bridge vlan del dev wan vid 1 self
bridge vlan del dev wan vid 1 master

ip link set eth0.101 master br0
ip link set eth0.102 master br0
ip link set eth0.103 master br0
ip link set eth0.104 master br0
ip link set eth0.105 master br0

 
ip addr add 192.168.178.2/24 broadcast 192.168.178.255 dev br0
ip route add default via 192.168.178.1

ip link set br0 up
ip link set lan1 up
ip link set lan4 up
ip link set lan2 up
ip link set lan3 up
ip link set wan up

 

Congratulation - I think you found a working configuration!

 

I have to give this a try.  I've been using the R1 for past few months but I skipped setting up the LAN ports.  I'm on the latest build and latest driver.  I have my R1 plugged into a WRT router (so my DSL modem is managed by the WRT, and my R1 compliments my main network).  I was disappointed a bit with the wifi.  Signal strength is dismal even in the same room.  I do have an interesting bit of a setup (I must document it to share it) where it really doesn't matter too much about the wifi.  I have a USB-hub-powered dlink USB wifi dongle that I plug in on the main floor and plugs back into the R1.  Signal strength issue is resolved there, but now I have 3 networks broadcastings.

 

Anyways.  First time I tried to setup the LAN ports, I got it working but I had packet loss (ping were sometimes failing).  Like your setup, my goal was not to bridge in the WLAN.  Only difference though is the WAN port I'm using to connect to the other network (my R1 needs to lease a IP from my WRT).

 

I'll try to adapt your setup and see if I have success.  What did you put into the interface file?

Link to comment
Share on other sites

I wasn't sure how to connect to the second network without using a bridge (I wanted to use wan tag instead of br1), but this works:

set -xue


ip link set eth0 down
ip addr flush eth0
ip link set eth0 up

ip link add link eth0 name eth0.101 type vlan id 101
ip link set eth0.101 up
ip link add link eth0 name eth0.102 type vlan id 102
ip link set eth0.102 up
ip link add link eth0 name eth0.103 type vlan id 103
ip link set eth0.103 up
ip link add link eth0 name eth0.104 type vlan id 104
ip link set eth0.104 up

#WAN-router hooked up to WAN
ip link add link eth0 name eth0.105 type vlan id 105
ip link set eth0.105 up

 

ip link add br0 type bridge

#WAN-router hooked up to WAN

ip link add br1 type bridge


#ip link set dev br0 type bridge stp_state 0

# BCM53125 ports 4, 0, 1, 2, 3
ip link set lan1 master br0
ip link set lan2 master br0
ip link set lan3 master br0
ip link set lan4 master br0

 

#WAN-router hooked up to WAN
ip link set wan master br1

 

bridge vlan add vid 101 dev lan1 pvid untagged
bridge vlan del dev lan1 vid 1 self
bridge vlan del dev lan1 vid 1 master
bridge vlan add vid 102 dev lan2 pvid untagged
bridge vlan del dev lan2 vid 1 self
bridge vlan del dev lan2 vid 1 master
bridge vlan add vid 103 dev lan3 pvid untagged
bridge vlan del dev lan3 vid 1 self
bridge vlan del dev lan3 vid 1 master
bridge vlan add vid 104 dev lan4 pvid untagged
bridge vlan del dev lan4 vid 1 self
bridge vlan del dev lan4 vid 1 master

#WAN-router hooked up to WAN

bridge vlan add vid 105 dev wan pvid untagged
bridge vlan del dev wan vid 1 self
bridge vlan del dev wan vid 1 master

 

ip link set eth0.101 master br0
ip link set eth0.102 master br0
ip link set eth0.103 master br0
ip link set eth0.104 master br0

 

#WAN-router hooked up to WAN
ip link set eth0.105 master br1


ip addr add 192.168.4.2/24 broadcast 192.168.4.255 dev br0
 

ip link set br0 up
ip link set lan1 up
ip link set lan4 up
ip link set lan2 up
ip link set lan3 up
ip link set wan up

 

#allow routing back to WAN router
iptables -t nat -A POSTROUTING -s 192.168.4.0/24 ! -d 192.168.4.0/24  -j MASQUERADE 

 

Edited by Tido
put the code into a code-box
Link to comment
Share on other sites

Possible there is misssing "ip link set br1 up" ?

...

ip link set br1 up

ip link set wan up

...

lan1-4 and wan only need to address and set PVID and VLAN table entries.

Have you checked correct work with wireshark at eth0.105? There should be never seen frames in and out from other ports.

 

But another interesting point is the startup of R1. Have you deactivated switch forwarding in u-boot?

 

Link to comment
Share on other sites

Odd, it is starting up br1 without it, but will add it for properness.

 

I haven't tried wireshark yet, but I did't monitor through nmon.  I'm seeing traffic only out through eth0.105, but I do see outsize of 60 periodically through wan.

 

I didn't modify the startup routines on the R1 (using the default).  Do you have any info about deactiviating the switch forwarding?

 

A side note, I did some testing with and without running that above LAN setup.  What I noticed is on a laptop plugged into the LAN switch, the laptop will acquire a connection and IP from DHCP from my router connected over the WAN port on the R1.  I can access that laptop using that IP address from either the R1 or my WRT router.  The only side affect is that the laptop doesn't have internet access in this situation.  

 

Link to comment
Share on other sites

I dont know, how to deactivate eth0 or forwarding in u-boot. Settings possible available on USB (OTG) I remember. My Linux know how is low.

 

I have problems with lldpd and lldpad, maybe it will make effects in other programs too:

- lldpad can not save own port parameter if port have a dot in name. It is a feature from used parameter function. So I have renamed eth0.101 in eth0x101, ..., its a workaround.

- lldpd and lldpad think eth0.101 ... is true VLAN. Information get by ioctl(). But it is not so and both servers fails with later and skip this ports.  I have deactivated in lldpd/lldpad source files. But I think, it is not the correct linux way.

Link to comment
Share on other sites

On 14.4.2017 at 9:44 PM, BrUser said:

how to deactivate eth0

Hardware fix to disable frame forwarding by the switch in the unconfigured/booted state !
Just for the curious. There is a pin strapping option for the BCM53125 via LED[9] which Lamobo exported to us as a resistor soldering option.

If you like to disable frame forwarding by the switch in the unconfigured/booted state, you can solder a 2k2 ohm resister on unpopulated R1308 soldering pads on the board (bottom side, between unpopulated R110 and R132, next to populated C195).

Ref: BCM53125U datasheet (53125U-DS02-R.pdf, e.g. from here, Table 25 p.117, LED[9] description) and BPI-R1 MP Schematic-SD_V3-20140922.pdf from here p. 11). You can solder the resistor in series with a dip switch or alike so you can enable/disable this feature without repetitive soldering.

According to the docs, this pin strapping option affects the Port Control Register (Page 00h: Addresses 00h-05h STP_STATE[2:0]) and Switch Mode Register (Page 00h: Address 0Bh, SW_FWDG_EN and SW_FWDG_MODE) which might need additional care by the driver.

Note that I have not tried that myself and I haven’t looked at the driver sources whether it can handle the unconfigured switch properly. Any modification to the board is done at your own risk.

LED[9] is used for the strap pin for HW_FWDG_EN. Forwarding enable.
Active high.
• When this pin is pulled high (default) at power-up, traffic is forwarded without any register settings, based on default register settings.
• When this pin is pulled low at power-up, frame forwarding is disabled. Traffic forwarding is enabled through Software Register bit set.
https://github.com/igorpecovnik/lib/issues/511#issuecomment-258647387

Link to comment
Share on other sites

On 13.4.2017 at 2:38 PM, ddurdle said:

What did you put into the interface file?

 

Sorry for my late answer, but I didn't realized your question.

 

At the moment I use the standard interface file that just starts up eth0 with dhcp.  After boot I start my script manually.

 

In the last days I tried to contact the driver developer in order to get more information on the switch configuration, perhaps working examples, some explanations, ... but he was not interested to support me.

 

Link to comment
Share on other sites

I have seen, at startup my script /etc/network/if-pre-up.d/dsa  is called 5 times from different tasks. 4 times with execution overlapp. This makes trouble. What can be the reason and how can I prevent the script from execution, more than one time?

Link to comment
Share on other sites

Hello everyone,

 

Quote

The R1 should work without VLAN in switch mode, but I dont know, how to do this over DSA or other way.

 

I have installed a new mainline armbian-server on my BPiR1 and i want to configure the switch in exact this behavior. (LAN and WAN-Ports should be interconnected with each other and together with the internal Ethernet-adapter of the BananaPi). I thought that therefore no configuration is needed. But when i start the freshly installed armbian 5.25-mainline-image, then the external Ethernet-ports are not interconnected. So i can connect from all ports to the BPiR1 but not between two devices attached to the BPiR1.

On startup i got this:

[    4.200897] libphy: stmmac: probed
[    4.200930] mdio_bus stmmac-0:1e: mdio_device_register
[    4.201677] b53_common: found switch: BCM53125, rev 4
[    4.201755] bcm53xx stmmac-0:1e: failed to register switch: -517

 

The configuration-scipt by dumischbaenger (postet on April 6th) does work for me, but as i understand it correctly this isn't a true hardware-based unmanaged switch-mode, isn't it? As i understand it correctly the script is creating 5 VLANs, one for each Ethernet-Port, and then bridge them together.... Does someone know a way to setup the switch in fully unmanaged mode?

 

Best regards,

flashbanger

System:

Hello everyone,

Edited by Igor
doubled text
Link to comment
Share on other sites

Hallo Flashbanger, R1 works with this VLAN-DSA examples more like a hardware based unmanaged switch. You can check this using filetransfer between two independ ethernet devices for example at lan3 and lan4. The system load at R1 ( top ) should be unchanged. I have seen this at green led (activity led) too, i have changed led-settings in driver B53_common.c.

In dsa - driver, sorry i dont know, how to set back in unmanaged mode after initialization.

Link to comment
Share on other sites

On 7.5.2017 at 11:57 PM, flashbanger said:

The configuration-scipt by dumischbaenger (postet on April 6th) does work for me, but as i understand it correctly this isn't a true hardware-based unmanaged switch-mode, isn't it? As i understand it correctly the script is creating 5 VLANs, one for each Ethernet-Port, and then bridge them together.... Does someone know a way to setup the switch in fully unmanaged mode?

 

First, the configuration was originally from BrUser. Honor goes to him.

 

Second, I tried at the beginning exactly what you want - a very basic simple switch configuration, but I had no luck. Every time I tried it, network packages disappeared somewhere in the network stack. If you have such a kind of configuration please report here. I think BrUser tried it too. Please read the thread. Unfortunately there is no "manual" for the br53 driver. Documentation from other producer like Melanox (see below) do not always apply. That means documented  ip and link configs do not work on our switchboard.

 

Third, the DSA driver for br53 tries to offload the work to the hardware. I find this paper useful to understand the driver hardware relation:

https://people.netfilter.org/pablo/netdev0.1/slides/Hardware-accelerating-Linux-network-functions.pdf

 

Further IMHO interesting documents:

https://www.kernel.org/doc/Documentation/networking/dsa/dsa.txt

https://www.kernel.org/doc/Documentation/networking/switchdev.txt

https://github.com/Mellanox/mlxsw/wiki/Switch-Port-Configuration

https://github.com/Mellanox/mlxsw/wiki/Bridge

http://events.linuxfoundation.org/sites/events/files/slides/LinuxConJapan2014_makita_0.pdf

Link to comment
Share on other sites

Ethernet does not working for legacy kernel + desktop, generated by Igors build tool:  ARMBIAN 5.27 stable Ubuntu 16.04.2 LTS 3.4.113-sun7i.

Same effect with Debian. Wifi works. In running wireshark on lamobo R1 I can see all network broadcasts (dst-MAC ff:ff:ff:ff:ff:ff) but no unicast. For example DHCP request never receive response. Not any frames from R1 seen in network. Is this behavior known, or is there a solution?

Link to comment
Share on other sites

45 minutes ago, BrUser said:

Ethernet does not working for legacy kernel + desktop, generated by Igors build tool:  ARMBIAN 5.27 stable Ubuntu 16.04.2 LTS 3.4.113-sun7i.


It looks like in last u-boot some parameters were messed up for some Allwinner boards. Workaround:

https://forum.armbian.com/index.php?/topic/4374-compiling-kernel-for-banana-pi/

 

Link to comment
Share on other sites

Hi Igor, many, many thanks, now legacy Debian works.

Possible there is a new thermal issue or wrong detection bug. Only after reboot to desktop, 5 seconds later linux shoutdown, power off. If I wait 2 minutes then desktop starts normal.

This effect I have not with older legacy images with hardware acceleration. Power is connected at battary connector, consumtion is the same at startup.

Link to comment
Share on other sites

13 minutes ago, BrUser said:

older legacy images with hardware acceleration

 

Where did you spot those (at least here I don't find anything with 'hardware acceleration': https://dl.armbian.com/lamobo-r1/archive/)?

 

BTW: Aren't underpowering issues caused by a crappy board design and video stuff with an outdated legacy kernel (and the whole idea to run a graphics stack on a device used as a firewall or router!) really on topic in a thread dealing with mainline kernel and DSA?

Link to comment
Share on other sites

Two month before i have tested same device with same SATA, DVBT-Stick, power-supply, energymeter. Today is very hot. Maybe it was all time at limit.

Mainline is OK. I have used R1 for 5-port managed switch and wifi-bridge. Device is running 24h/day and most time it does nothing (except WIFI and switch), only take power.

Unfortunly wireshark is very slow compared with legacy kernel and if I want to use quiccly webbrowser I must start my PC. Thats why I want to have hardware acceleration.

In switch driver module B53_common.c I hope to activate local link layer like mainline. Then R1 for me is near perfect for this. OK, audio goes not over HDMI and second USB does not work.

 

Link to comment
Share on other sites

8 minutes ago, BrUser said:

if I want to use quiccly webbrowser I must start my PC. Thats why I want to have hardware acceleration.

 

Wow. And you think the sort of hardware acceleration possible with legacy kernel will speed up web browsing? It will not. Crapboard R1 is just a perfect example for numerous design flaws paired with surreal high expectations at the user side. And maybe most hours of life wasted on workarounding the many issues by so many people. I still think Armbian should ASAP phase out support for this failed board.

 

Please let's stop now since this is really off-topic here. It would be great if you could post a mini tutorial in a new thread showing how this device can be operated as a router (solder tutorial and pictures of R1308 with resistor soldered)  and provide some evidence that this works now and WAN and LAN ports are truly separated at all time if not configured differently otherwise.

Link to comment
Share on other sites

Zador postet on github this:

  1. there are still problems with DSA configuration, so it is not provided for the mainline based images
  2. b53 driver probing was fixed and ARM: dts: sun7i: lamobo-r1: Fix CPU port RGMII settings patch was merged upstream


For my tests I used: Armbian_5.25_Lamobo-r1_Debian_jessie_4.9  and I didn't get DSA working as stated above as a Router.

I just looked into https://dl.armbian.com/lamobo-r1/ there are images for 3.4.113 legacy and should work with SWCONFIG,  4.9 still come with SWCONFIG binary but I haven't tried and 4.11 nightlies.

 

Which will carry the fix for CPU port RGMII ?   and what shall I test ?

 

Link to comment
Share on other sites

9 minutes ago, Tido said:

Which will carry the fix for CPU port RGMII ?

This fix was previously included in Armbian (don't want to dig the git history to find when it was added) but now it is present in mainline since kernel 4.11. If I remember correctly it fixes packet loss issues, nothing more.

 

Regarding DSA configuration - I have to post this again: I don't see anybody who is willing to maintain the old swconfig compatible interface for the mainline kernel, so we can't provide support for the onboard b53 switch and are relying on upstream mainline DSA interface.

Link to comment
Share on other sites

To bad it only fixes this.

 

On 20.4.2017 at 11:20 PM, dumischbaenger said:

In the last days I tried to contact the driver developer in order to get more information on the switch configuration, perhaps working examples, some explanations, ... but he was not interested to support me.

I don't know on which channel and how he contacted him though..

Link to comment
Share on other sites

Also regadring DSA on mainline - this is a good source of information regarding b53 related fixes that will be included in next mainline releases: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/log/drivers/net/dsa/b53

A lot of changes/fixes were included in 4.11, for example:

Quote
net: dsa: b53: Include IMP/CPU port in dumb forwarding mode
 
Since Broadcom tags are not enabled in b53 (DSA_PROTO_TAG_NONE), we need to make sure that the IMP/CPU port is included in the forwarding decision. Without this change, switching between non-management ports would work, but not between management ports and non-management ports thus breaking the default state in which DSA switch are brought up.

 

Link to comment
Share on other sites

Thank you, good news. Funny to see that 4 days after dumischbaenger's message some new code was submitted :)

Wow, so the code from April 24 just made it into 4.11 which was released 1. May. So I will give 4.11 another shot and see if it starts to work

Link to comment
Share on other sites

On 29.5.2017 at 7:55 PM, Tido said:

I don't know on which channel and how he contacted him though..

 

I wrote an email to Florian Fainelli.  He commented on the "Drop support for Lamobo R1" thread at Github. I asked him if he would mind to answer some questions about R1 switch configuration but he pointed me to the Github discussion.

Link to comment
Share on other sites

  • Igor unpinned this topic
Guest
This topic is now closed to further replies.
×
×
  • Create New...