lmdc45 Posted July 7, 2020 Posted July 7, 2020 (edited) HI, I am using armbian config to set up an hotspot on orangepipcplus I have eth0 and eth1, two different interface connected to the internet when i set up the hotspot, wlan0 will access internet through eth0 I would like to change this on the fly then so that wlan0 can access internet through eth1. I tried a lot of things but not getting anywhere... I looked at the source and tried to redo clear iptables/ apply new iptables nat config but it is not workinghttps://github.com/armbian/config/blob/master/debian-config-jobs#L617 I just changed $DEFAULT_ADAPTER to eth1, and keep $WIRELESS_ADAPTER to wlan0 to run: # Clear iptables iptables-save | awk '/^[*]/ { print $1 } /^:[A-Z]+ [^-]/ { print $1 " ACCEPT" ; } /COMMIT/ { print $0; }' | iptables-restore # - Apply iptables iptables -t nat -A POSTROUTING -o $DEFAULT_ADAPTER -j MASQUERADE iptables -A FORWARD -i $DEFAULT_ADAPTER -o $WIRELESS_ADAPTER -m state --state RELATED,ESTABLISHED -j ACCEPT iptables -A FORWARD -i $WIRELESS_ADAPTER -o $DEFAULT_ADAPTER -j ACCEPT # - Save IP tables, applied during ifup in /etc/network/interfaces. iptables-save > /etc/iptables.ipv4.nat But then this is not working, no internet access. (Noting I verified internet access through curl --interface eth1 and working perfectly) If I revert back to eth0, working again properly. What do I miss to route traffic through eth1 properly? Thanks Louis Edited July 7, 2020 by lmdc45
lmdc45 Posted July 8, 2020 Author Posted July 8, 2020 No... I understand the gateway for eth1 is link to this interface eth1... Now seems a good point so I would need to change gateway on wlan0 to match gateway of eth1. I am a bit confused quite new to linux and looking for a good book/pdf to get basics infos, any idea what I could get?
Technicavolous Posted July 9, 2020 Posted July 9, 2020 try ip route | grep default ip -4 addr show scope global cat /etc/network/interfaces the default gateway is where all internet traffic will be sent. Try the changes you made originally, then see how it affects your default gateway. I think you will need to affect a change to the dgw in addition to your other changes. you seem advanced with iptables but maybe these might give some references https://www.unixmen.com/how-to-find-default-gateway-in-linux/ https://www.systutorials.com/setting-up-gateway-using-iptables-and-route-on-linux/ https://www.cyberciti.biz/faq/ip-route-add-network-command-for-linux-explained/ https://serverfault.com/questions/567487/iptables-packet-forwarding-to-one-of-two-gateways-depending-on-origin Thanks for prompting me to look this up. I've never really dove into iptables, and I found these https://www.linode.com/docs/security/firewalls/control-network-traffic-with-iptables/ is nftables in armbian yet? I didn't know about nftables until I read a comment from one of the above threads...
lmdc45 Posted July 10, 2020 Author Posted July 10, 2020 I am just ok to copy and paste the code for the iptables and change as required but have not much knowledge yet. I got javscript/nodejs background so quite ok to understand the code if I see it but cannot make it. nftables should be in armbian as latest version is based on Debian 10 buster, so shoud go though nftables probably. ip route gives me: root@orangepipcplus:~# ip route default via 192.168.8.1 dev eth0 proto dhcp metric 50 default via 172.20.10.1 dev eth1 proto dhcp metric 78 169.254.0.0/16 dev eth0 scope link metric 1000 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 172.20.10.0/28 dev eth1 proto kernel scope link src 172.20.10.5 metric 78 172.24.1.0/24 dev wlan0 proto kernel scope link src 172.24.1.1 192.168.8.0/24 dev eth0 proto kernel scope link src 192.168.8.104 metric 50 ip -4 addr show scope global : root@orangepipcplus:~# ip -4 addr show scope global 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 inet 192.168.8.104/24 brd 192.168.8.255 scope global dynamic noprefixroute eth0 valid_lft 78604sec preferred_lft 78604sec 3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 inet 172.24.1.1/24 brd 172.24.1.255 scope global wlan0 valid_lft forever preferred_lft forever 5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 valid_lft forever preferred_lft forever 45: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 inet 172.20.10.5/28 brd 172.20.10.15 scope global dynamic noprefixroute eth1 valid_lft 85245sec preferred_lft 85245sec and cat /etc/network/interfaces: root@orangepipcplus:~# cat /etc/network/interfaces source /etc/network/interfaces.d/* # Network is managed by Network manager auto lo iface lo inet loopback and in /etc/network/interfaces.d there is only one file armbian.ap.nat with: root@orangepipcplus:/etc/network/interfaces.d# cat armbian.ap.nat # armbian NAT hostapd allow-hotplug wlan0 iface wlan0 inet static address 172.24.1.1 netmask 255.255.255.0 network 172.24.1.0 broadcast 172.24.1.255 Noting I understand amrbian-config use nmcli so here is the output as well: nmcli device show : GENERAL.DEVICE: eth0 GENERAL.TYPE: ethernet GENERAL.HWADDR: 02:81:13:E9:A3:CA GENERAL.MTU: 1500 GENERAL.STATE: 100 (connected) GENERAL.CONNECTION: Wired connection 1 GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/1 WIRED-PROPERTIES.CARRIER: on IP4.ADDRESS[1]: 192.168.8.104/24 IP4.GATEWAY: 192.168.8.1 IP4.ROUTE[1]: dst = 0.0.0.0/0, nh = 192.168.8.1, mt = 50 IP4.ROUTE[2]: dst = 169.254.0.0/16, nh = 0.0.0.0, mt = 1000 IP4.ROUTE[3]: dst = 192.168.8.0/24, nh = 0.0.0.0, mt = 50 IP4.DNS[1]: 192.168.8.1 IP6.ADDRESS[1]: fe80::4670:cd96:29fa:c462/64 IP6.GATEWAY: fe80::e2a3:acff:fe58:f6f3 IP6.ROUTE[1]: dst = ::/0, nh = fe80::e2a3:acff:fe58:f6f3, mt = 100 IP6.ROUTE[2]: dst = fe80::/64, nh = ::, mt = 100 IP6.ROUTE[3]: dst = ff00::/8, nh = ::, mt = 256, table=255 IP6.DNS[1]: fe80::e2a3:acff:fe58:f6f3 GENERAL.DEVICE: eth1 GENERAL.TYPE: ethernet GENERAL.HWADDR: 62:8B:0E:AD:DB:49 GENERAL.MTU: 1500 GENERAL.STATE: 100 (connected) GENERAL.CONNECTION: Wired connection 2 GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/12 WIRED-PROPERTIES.CARRIER: on IP4.ADDRESS[1]: 172.20.10.5/28 IP4.GATEWAY: 172.20.10.1 IP4.ROUTE[1]: dst = 0.0.0.0/0, nh = 172.20.10.1, mt = 78 IP4.ROUTE[2]: dst = 172.20.10.0/28, nh = 0.0.0.0, mt = 78 IP4.DNS[1]: 172.20.10.1 IP6.ADDRESS[1]: fe80::7a8d:2c94:4a6a:c639/64 IP6.GATEWAY: -- IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 101 IP6.ROUTE[2]: dst = ff00::/8, nh = ::, mt = 256, table=255 GENERAL.DEVICE: docker0 GENERAL.TYPE: bridge GENERAL.HWADDR: 02:42:5E:DD:46:90 GENERAL.MTU: 1500 GENERAL.STATE: 10 (unmanaged) GENERAL.CONNECTION: -- GENERAL.CON-PATH: -- IP4.ADDRESS[1]: 172.17.0.1/16 IP4.GATEWAY: -- IP4.ROUTE[1]: dst = 172.17.0.0/16, nh = 0.0.0.0, mt = 0 IP6.GATEWAY: -- GENERAL.DEVICE: lo GENERAL.TYPE: loopback GENERAL.HWADDR: 00:00:00:00:00:00 GENERAL.MTU: 65536 GENERAL.STATE: 10 (unmanaged) GENERAL.CONNECTION: -- GENERAL.CON-PATH: -- IP4.ADDRESS[1]: 127.0.0.1/8 IP4.GATEWAY: -- IP6.ADDRESS[1]: ::1/128 IP6.GATEWAY: -- IP6.ROUTE[1]: dst = ::1/128, nh = ::, mt = 256 GENERAL.DEVICE: wlan0 GENERAL.TYPE: wifi GENERAL.HWADDR: 12:81:13:E9:A3:CA GENERAL.MTU: 1500 GENERAL.STATE: 10 (unmanaged) GENERAL.CONNECTION: -- GENERAL.CON-PATH: -- IP4.ADDRESS[1]: 172.24.1.1/24 IP4.GATEWAY: -- IP4.ROUTE[1]: dst = 172.24.1.0/24, nh = 0.0.0.0, mt = 0 IP6.ADDRESS[1]: fe80::1081:13ff:fee9:a3ca/64 IP6.GATEWAY: -- IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 256 IP6.ROUTE[2]: dst = ff00::/8, nh = ::, mt = 256, table=255 I also tried to set up a bond conenction before I understand a bond is only for two cables connected to the same gateway... Took ne too long to get that unfortunately. Idea would be to have a new connection 'con0' that is linked to eth0 and eth1, and then I could bring eth0 or eth1 up or down, the wifi will be connected to con0 and would not notice. A bit like the bond system 'active-backup', but with two differents IP addresses and gateway on eth0 and eth1. I am sure there is a proper linux way to do this but could not yet find it. Also root@orangepipcplus:/etc/network/interfaces.d# iptables -nvL Chain INPUT (policy ACCEPT 50287 packets, 14M bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 3137 packets, 283K bytes) pkts bytes target prot opt in out source destination 422K 451M ACCEPT all -- eth0 wlan0 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED 307K 36M ACCEPT all -- wlan0 eth0 0.0.0.0/0 0.0.0.0/0 Chain OUTPUT (policy ACCEPT 51731 packets, 12M bytes) pkts bytes target prot opt in out source destination root@orangepipcplus:~# iptables --t nat -nvL Chain PREROUTING (policy ACCEPT 22926 packets, 2106K bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 9720 packets, 672K bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 6247 packets, 392K bytes) pkts bytes target prot opt in out source destination 21594 1783K MASQUERADE all -- * eth0 0.0.0.0/0 0.0.0.0/0 Chain OUTPUT (policy ACCEPT 20645 packets, 1600K bytes) pkts bytes target prot opt in out source destination From what I see the only 'bridge' that connect wlan0 and eth0 is on iptables rules, and I don't understand why the connection do not go trhough when I change these rules from eth0 to eth1.
Technicavolous Posted July 10, 2020 Posted July 10, 2020 iptables taxes my ability already lol but I read a few places that iptables and network manager don't always play well together? In Official Ubuntu iptables documentation it says "WARNING: Iptables and NetworkManager can conflict." Perhaps explore if this is happening to you. https://askubuntu.com/questions/904048/network-manager-is-adding-unwanted-rules-to-iptables https://feeding.cloud.geek.nz/posts/using-iptables-with-network-manager/ note the pre-up statements ... and dumb question ... are we sure that your second adapter is called eth1 and not something like wlp3s0? I think this is as far as I can help you :\ https://wiki.nftables.org/wiki-nftables/index.php/Main_Page
lmdc45 Posted July 11, 2020 Author Posted July 11, 2020 Thank you for your help I though that any ip packet directed to eth0 would use eth0 gateway, and any packet using eth1 would use eth1 gateway. But its not the case and any packet directed to addresses in the outside wold would take the same gateway, the one with lowest metric. I also had to change the metric of the ip route so that if wlan0 is directed to eth1, eth1 gateway will be used. Seems that packet were directed to eth1 but using eth0 gateway and then never hearing back from them. had to change the metric with nmcli con mod 'eth1 connection' connection.route-metric 45 (or any lowest number) and then nmcli con up 'eth1 connection' to activate, and internet is working fine then through eth1. Only issue is that active socket are disconnected, I would like to find a way to keep socket active but it is not a big issue for me at this stage. then I monitor network with nmcli monitor and change to eth1 as required is eth1 is connected, and swith back to eth0 when eth1 is disconnected... Seeems like there is a way to do this a bit more easily with network configuration file but would mean not using nmcli anymore and I guess armbian-config will not be ok with that as it is managing connection through nmcli at least the above is working, seems like there is not confilct between nmcli and iptables, working ok together
Technicavolous Posted July 12, 2020 Posted July 12, 2020 Awesome glad you are able to get it going. On 7/10/2020 at 11:57 PM, lmdc45 said: I would like to find a way to keep socket active That sounds like a challenge as some connections might see that as a security risk. Maybe you write up a little tutorial now that you have it figured out? ;]
lmdc45 Posted July 15, 2020 Author Posted July 15, 2020 On 7/13/2020 at 8:50 AM, Technicavolous said: Awesome glad you are able to get it going. That sounds like a challenge as some connections might see that as a security risk. Maybe you write up a little tutorial now that you have it figured out? ;] I understand maybe a bridge could do it? Would create a bridge between eth0 and wlan0, then on connection of eth1 could disconnect eth0 from the bridge and connect eth1. I am a very beginner in linux so not sure. At this stage I use a copy of the armbian-config script to switch the interface, with a monitor to trigger the switch. #!/bin/bash current_interface='eth0' nmcli dev monitor | while read event do echo $event case $event in "eth1: device removed") if [ "$current_interface" != "eth0" ]; then bash /root/lscript/switch.sh "eth0" current_interface="eth0" echo -e "\t switched to eth0" fi ;; "eth1: connected") if [ "$current_interface" != "eth1" ]; then bash /root/lscript/switch.sh "eth1" current_interface="eth1" echo -e "\t switched to eth1" fi ;; *) echo -e "\t unmanaged event " esac done #and in switch.sh : #!/bin/bash # DEFAULT_ADAPTER=$1 WIRELESS_ADAPTER="wlan0" function switch_int () { # Clear iptables /sbin/iptables-save | awk '/^[*]/ { print $1 } /^:[A-Z]+ [^-]/ { print $1 " ACCEPT" ; } /COMMIT/ { print $0; }' | /sbin/iptables-restore # - Apply iptables /sbin/iptables -t nat -A POSTROUTING -o $DEFAULT_ADAPTER -j MASQUERADE /sbin/iptables -A FORWARD -i $DEFAULT_ADAPTER -o $WIRELESS_ADAPTER -m state --state RELATED,ESTABLISHED -j ACCEPT /sbin/iptables -A FORWARD -i $WIRELESS_ADAPTER -o $DEFAULT_ADAPTER -j ACCEPT # - Save IP tables, applied during ifup in /etc/network/interfaces. /sbin/iptables-save > /etc/iptables.ipv4.nat sed -i 's/^bridge=.*/#&/' /etc/hostapd.conf #sed -e 's/exit 0//g' -i /etc/rc.local # workaround if hostapd is too slow #echo "service dnsmasq start" >> /etc/rc.local #echo "iptables-restore < /etc/iptables.ipv4.nat" >> /etc/rc.local #echo "exit 0" >> /etc/rc.local systemctl stop armbian-restore-iptables.service systemctl disable armbian-restore-iptables.service cat <<-EOF > /etc/systemd/system/armbian-restore-iptables.service [Unit] Description="Restore IP tables" [Timer] OnBootSec=20Sec [Service] Type=oneshot ExecStart=/sbin/iptables-restore /etc/iptables.ipv4.nat [Install] WantedBy=sysinit.target EOF systemctl enable armbian-restore-iptables.service } function select_default_interface () { # set highest metric to default adaptor HIGHEST_METRIC=$(nmcli -t -f UUID,TYPE,DEVICE connection show --active | grep $DEFAULT_ADAPTER | sed 's/:.*$//') # set metric to 50 nmcli connection modify $HIGHEST_METRIC ipv4.route-metric 50 2> /dev/null #nmcli con down $HIGHEST_METRIC > /dev/null nmcli con up $HIGHEST_METRIC > /dev/null METRIC=77 # set others wired REMAINING=( `nmcli -t -f UUID,TYPE,DEVICE connection show --active | grep ethernet | grep -v $DEFAULT_ADAPTER | sed 's/:.*$//'` ) if [[ ${#REMAINING[@]} -ge 1 ]]; then for i in "${REMAINING[@]}" do METRIC=$(( $METRIC + 1 )) nmcli connection modify ${i} ipv4.route-metric $METRIC #nmcli con down ${i} > /dev/null nmcli con up ${i} > /dev/null done fi # set other wireless METRIC=88 REMAINING=( `nmcli -t -f UUID,TYPE,DEVICE connection show --active | grep wireless | grep -v $DEFAULT_ADAPTER | sed 's/:.*$//'` ) if [[ ${#REMAINING[@]} -ge 1 ]]; then for i in "${REMAINING[@]}" do METRIC=$(( $METRIC + 1 )) nmcli connection modify ${i} ipv4.route-metric $METRIC done fi # create default metrics file cat <<-EOF > /etc/NetworkManager/conf.d/armbian-default-metric.conf [connection-ethernet-gateway] match-device=interface-name:$DEFAULT_ADAPTER ipv4.route-metric=50 [connection-wifi-other] match-device=type:wifi ipv4.route-metric=88 [connection-ethernet-other] match-device=type:ethernet ipv4.route-metric=77 EOF } switch_int select_default_interface
lmdc45 Posted July 16, 2020 Author Posted July 16, 2020 (edited) Noting it is possible to change the switch interface function to work with nftables The main advantage seems to be the inet service which means rules will apply for both ipv4 and ipv6, only issue is that it does not allow to use armbian-config for the hostpot anymore as this would use iptables #!/bin/bash DEFAULT_ADAPTER=$1 WIRELESS_ADAPTER="wlan0" function switch_int () { #clear nf cat << EOF > /etc/nftables.rules #!/usr/sbin/nft -f flush ruleset add table inet nat #add chain inet nat PREROUTING { type nat hook prerouting priority -100; policy accept; } #add chain inet nat INPUT { type nat hook input priority 100; policy accept; } add chain inet nat POSTROUTING { type nat hook postrouting priority 100; policy accept; } #add chain nat OUTPUT { type nat hook output priority -100; policy accept; } add table inet filter #add chain inet filter INPUT { type filter hook input priority 0; policy accept; } add chain inet filter FORWARD { type filter hook forward priority 0; policy accept; } #add chain inet filter OUTPUT { type filter hook output priority 0; policy accept; } EOF chmod u+x /etc/nftables.rules /etc/nftables.rules nft add rule inet nat POSTROUTING oifname $DEFAULT_ADAPTER counter masquerade nft add rule inet filter FORWARD iifname $DEFAULT_ADAPTER oifname $WIRELESS_ADAPTER ct state related,established counter accept nft add rule inet filter FORWARD iifname $WIRELESS_ADAPTER oifname $DEFAULT_ADAPTER counter accept sed -i 's/^bridge=.*/#&/' /etc/hostapd.conf #save nft nft list ruleset > /etc/nftables.ruleset systemctl stop armbian-restore-nftables.service systemctl disable armbian-restore-nftables.service cat <<-EOF > /etc/systemd/system/armbian-restore-nftables.service [Unit] Description="Restore NF tables" [Timer] OnBootSec=20Sec [Service] Type=oneshot ExecStart=nft -f /etc/nftables.ruleset [Install] WantedBy=sysinit.target EOF systemctl enable armbian-restore-nftables.service } switch_int Edited July 16, 2020 by lmdc45
Recommended Posts