Hello,

 

I would like to replace the internet box of my provider with an armbian card.

 

So, I need a card with an ethernet plug for the WAN, which will be connected to the RJ45/fibre transceiver,

and a wifi/other ethernet plugs for the LAN, for usual device connections (laptop, smartphones, ...).

 

I used to have a clearfogpro which was perfect for this job. But it is no longer maintained.

 

Also, I would like a card with the possibility to add some disk storage either via standard M2

sata/nvme (preferred) or otherwise usb3.

 

Regards

 

deb2016

 

PS : it was required to add a tag, but do not know why, the sole choice I had to publish this post is to tag it "solved"

 

 

Edited October 17Oct 17 by deb2016

moved.

I assume card = sbc?

5 hours ago, deb2016 said:

I used to have a clearfogpro which was perfect for this job. But it is no longer maintained.

There are recent Armbian images and Armbian standard support, see https://armbian.com/boards/clearfogpro

 

How to make a 'router instance' is something generic Linux, almost anything is possible.

yes, card=sbc

 

Quote

There are recent Armbian images and Armbian standard support, see https://armbian.com/boards/clearfogpro

 

I was not aware of this, armbian support for the clearfogpro used to be deprecated (very old images).

There are some distributions like pfsense or OpenWRT for that particular use case. You may consider to run a virtualized instance of them on top of proxmox. As well as you might want to run a NAS instance in a different container instead of mixing them up in  single installation. Armbian has added proxmox support recently (https://docs.armbian.com/software/proxmox-ve/) 

 

A usual setup may contain:

• OpenWRT or Pfsense
• AdguardHome or PiHole
• WiFi-controller depending on your wifi hardware, e.g. unifi
• OwnCloud, NextCloud, OpenCloud, OpenMediaVault, ...
• DoveCot

If your SBC has only one nic, you can still build a router with the help of a VLAN capable switch.  

I now see the clearfogpro is 2x Cortex-A9 based, not the newer Marvell SoCs with Cortex-A72. Cortex-A9 has no HW virtualization, so the board is rather dedicated router only then. Despite having lots of networking HW, I would prefer some other 64-bit capable SBC. I personally have separated 'WiFi' and 'routing' as for WiFi I want the best location(s) in/around the house. I am using the standard Linux virt-manager which is available by default in all 64-bit distros, runs even OK on RPI3 (4x Cortex-A53), although SD-cards can be bottleneck.

OpenWRT has images that are tailored for RPI 64-boards, and the funny thing is that it is basically 2 partitions, 1 boot and 1 root. I had to look twice at the size numbers, but the root is really only 128MB diskspace while the boot is same or bigger but mostly empty (that is due to what is common how RPI boots). For a generic EFI variant boot is or can be much smaller. If U-Boot is used, it would/is not much more than that 128MB. I used it as test in a KVM (just CLI virsh tooling) on RPI4 to figure out issues with an MT7921 series USB WiFi stick that I use as extra AP. In the end at least Armbian rk35xx vendor 6.1.115 showed issues where a mainline based did not. Now with 6.1.172-vendor-rk35xx build few days old no issues anymore.

 

So my experience is, with some VLAN capable switches and fiber2ethernet box supplied by ISP, router can be anywhere, assuming that you know how to handle VLANs and bridges in Linux. And make sure mainline kernel and generic/standards as much as possible.

A lot depends on what speeds you want. RPI3 is '~50Mbps half-duplex single stream'. But ARMv8.2 like RK3588 boards should have no issue with almost full 940Mbps full-duplex even with Wireguard etc all in a KVM. I haven't tested it as I try to keep lowest price for (fiber) internet so don't have the full 1Gbps speed. Also low power as much as possible.

Last week I did some measurements at USB-C connector level with my NanoPi-R6C: with 2024.01_armbian U-Boot and 6.1.172-vendor-rk35xx kernel+dtb I got it down to 1.15Watt idle. That is with Armbian Trixie running from Samsung 970 NVME. And temporary both ethernet ports set off-line. With the 1GbE (labeled 'WAN)' on again full-duplex 1000Mbit/s it is about 1.4Watt idle. Half-duplex 10 Mbit/s was 1.23Watt idle or so. If I run an Armbian Build on it, it is of course max at about 10.5Watt or so.

Also interesting is that if only base 5V input, you don't loose about 20% in the onboard DC/DC conversion (when USB-C PD input is 20V or 15V). Of course those numbers only matter if longer period of battery operation. To compare: I had an old APC UPS that takes a constant 7W by itself. My old BananaPi-M1 (2x Cortex-A9, SATA on-board, GbE) is 1.7W idle AFAIR. I soldered a LiPo to it, so it is its own UPS as well. SATA is down however when running from that LiPo battery. I have looked at Radxa Dragon6 as this can also be powered by very wide range of input voltages, but it cannot charge a LiPo when you solder it to it, so although faster than RK3588 boards, also due to high RAM/storage prices, I did not buy one.

I'd start with a lxc container image, e.g. from here: https://images.linuxcontainers.org/ , instead of a VM.

18 hours ago, umiddelb said:

lxc container image

Does this suggest to use same kernel for host/HW as the router instance?

Also, how do you make sure 'LAN' and 'WAN' are separated, to the same level as when using something with 2 physical ethernet ports?

 

I use nmtui mostly to create a basic set of ethernet devices on the host and then in virt-manager I allocate 2 (known) MAC addresses to the VM that is then used by for example OpenWRT VM or Opnsense VM or something else the same way as a real HW/ISP router. If not same MAC address for the WAN side it takes an undefined amount of long time before DHCP on the WAN side succeeds. There is a lot more to deal with w.r.t. my ISP, but that is too much off-topic detail.  Or could be even plain qemu-system-aarch64 with a lot of options, but only used that for just 1 ethernet port, e.g. like testing Armbian image boot with U-Boot for HW/SBC I don't have myself.

 

For containers, so 'shared kernel', I usually use a btrfs subvolume snapshot of the Armbian rootfs itself, so I have all settings/preferences in a blink of an eye and use systemd-nspawn to start it. If plain/vanilla systemd-networkd on both host and guest, there is  /usr/lib/systemd/network/80-container-host0.network file that, if put in /etc/systemd/network/ and changed with:

[Link]
MACAddress=<MAC-address>

presents the correct/needed MAC address, but it is the same broadcast or PHY link so not usable for 'router'.

 

Armbian datacenter gateway runs on Nanopi R6S, more then enough, I would say even overkill. If you want to do virtualization, I think 32Gb version exists. Should do the job. We don't run much stuff on it, so memory usage: 11% of 7.75G

11 hours ago, eselarm said:

Also, how do you make sure 'LAN' and 'WAN' are separated, to the same level as when using something with 2 physical ethernet ports?

More or less. Use a tagged VLAN for the WAN port (e.g. 7) and an untagged VLAN for the LAN port. Define an untagged switch-port with the same VLAN-ID (7) and connect your WAN-uplink to this port. Some ISPs require a tagged switch-port in order to communicate with their modems, e.g. Deutsche Telekom requires VLAN-ID 7.

ProxmoxVE offers a nice UI for virtual networking. 

11 hours ago, umiddelb said:

More or less.

I assume then shared kernel. So might be interesting for OP as Proxmox/KVM does not work on Cortex-A9 but containers should (I think, never used container host 32-bit ARM).

But I see no option to run OpenWRT (EFI boot) as KVM with Armbian/Debian/Tumbleweed/Ubuntu/RasberryPiOS as host.

 

As indicated, OpenWRT is very small, not only on disk size. I see I have allocated the VM 1 vCPU and 512MiB. Less memory should also work, but not sure how low it can be. It is mostly the GUI that needs RAM I think as the kernel+routing is very small, should be somewhere 50MiB to 100MiB I think. At least if only speeds up to 100Mbps or so and low latency.  Also which CPU type (Cortex-A55 or Cortex-A76) is up to libvirtd/QEMU/KVM as this was with mainline based kernel on RK3588, that does not need CPU pinning.

For 32-bit guests, I use RPi4 (Cortex-A72) or ROCK3A (only Cortex-A55) as 32-bit does not work at kernel level on Cortex-A76.

 

Related to that, a NanoPi-R76 (RK3576 based) might also be an option for OP as that is bigLITTLE Cortex-A72 + Cortex-A53, still ARMv8.0 but great low-power. An option is then that the whole storage/image of the clearfogpro is put as a KVM on the RK3576, although with different bootloader and kernel (EDK2 and some Debian armhf kernel). Is for people who have some well working 32-bit and fear the transition to 64-bit (so redo whole userspace tuned for many years).

11 hours ago, umiddelb said:

Use a tagged VLAN for the WAN port (e.g. 7) and an untagged VLAN for the LAN port. Define an untagged switch-port with the same VLAN-ID (7) and connect your WAN-uplink to this port. Some ISPs require a tagged switch-port in order to communicate with their modems, e.g. Deutsche Telekom requires VLAN-ID 7.

Indeed, various possibilities; added to that is that I also have used 4G as DSL was in kilobit speed range and 4G is just IP packets, although I put those on VLANs, so to have smooth transition between DSL 4B fiber (so multi-WAN in fact).

12 hours ago, umiddelb said:

ProxmoxVE offers a nice UI for virtual networking. 

I'll will need a test install then to see what it is. I know the GUI's from some managed switches, was difficult to figure out what is meant with what, but after that and having done the same in Opensuse Tumbleweed years ago on RPi4 with 'wickedd' networking and also Ubuntu 'netplan.io', it was a relief to just use 'sudo nmtui' to create a bridge with enslaved VLAN. And that can be done via a serial console cable, so I prefer that as no HDMI monitor+cables needed then, just a laptop, good for panic/trouble situations to keep things as simple as possible IMO. Even Rj45 can be unconnected initially, only power needed.

 

More complex but maybe also interesting is if IOMMU works on the host computer. Is no problem on e.g. N100 boards, but for RK3588, I could not get it to work (yet). Also makes little sense as for example the NanoPi-R6C has only 2 PCI-E devices that could be mapped from host to guest: NVME and 2.5GbE. But NVME is needed for high speed I/O for Armbian Build and VM image storage and I have no 2.5G switches (yet) and also network virtualization works very well if Linux as virtio network driver or a virtual intel 1000e (so then FreeBSD is also no issue). 

Quote

Armbian datacenter gateway runs on Nanopi R6S, more then enough, I would say even overkill. If you want to do virtualization, I think 32Gb version exists. Should do the job. We don't run much stuff on it, so memory usage: 11% of 7.75G

 

Thanks

 

I have seen the banapi r2 which has wifi and sata ports on board, however bananapi seems to have a low reputation and there are some criticism about the on board graphic and security. But besides this, it is nearly what I am looking for.

Do you know some similar boards ?

 

Regards

Edited October 20Oct 20 by deb2016

9 hours ago, deb2016 said:

Do you know some similar boards ?

No, maybe others know, I just had a look in Armbian space and I see something I did not know: https://armbian.com/vendors/linkease

1 is RK3568 based which I know from my ROCK3A, other I hadn't noticed before. Maybe use AI based search to get some hints. At least be aware that many ARM computer boards need proprietary drivers/firmware, so for something like an internet router where ISP's themselves consider 10 years operation/maintenance, it is quite a challenge to get that open-source based I think. That is why I focused on virtualization and less integrated HW solution.