ssh missfire on bpi-m5 noble

[gene1934]

I had to install (they were missing) sshfs, then fuse and finally ssh on a bpi-m5, using a 128Gb u-sd as boot media. iso was noble from a week ago's download, and updated at 2nd boot after establishing my /etc/hosts file based network.  I can ssh from it /to/ the main box w/o any problem. But any attempt to reverse into it /from/ another machine /to/ it is "connection reset by peer". I haven't touched /etc/ssh/*config's from default values. Is there something I need to set/reset in those files to restore incoming logins?  And no, I do not enable root logins. Also, systemd is still a puzzle to me if its involved.

 

Thanks for any advice.

[Werner]

what?

 

armbianmonitor -u

[gene1934]

<https://paste.armbian.com/bebayihagi>

 

I did look at it, but nothing yelled at me in a quick scan. Not sure what I s/b looking at.

 

Thank you Werner.

 

 

[eselarm]

31 minutes ago, gene1934 said:

https://paste.armbian.com/bebayihagi

Network is down:

2: end0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000

 

All instances I could find.

 

I think your own networking methods conflict with what is common nowadays and also might be reason ssh server is not even running or depending on a pseudo random situation. Same as for other topic where you could not reach armbian repo server. 

[gene1934]

You might not have been looking at the newest dmesg, I had a hard time convincing NetworkManager to stop calling for dhcp and just use the static settings I gave it.  Everytime I looked it was re-enabled. Once I had convinced it to skip the dhcp BS, network is fine, apt works, ping -c1 yahoo.com works and FF can browse the whole planet.  Everything works except /incoming ssh/.

 

I can goto that printer and log into this machine, but I can't ssh into that machine.  I can't "ssh-copy-id gene@e5p" which aliases in the hosts file to 192.168.71.122, it's refusing the incoming connection that worked with 6 other machines.  Is there some other tool to trace this?  htop says it is ATM. 

 

Thank you.

[gene1934]

To clarify that last line htop say's ssh_config is in the cmd listing. 

put a tail on /var/log/syslog since it has the newest "access time" of any file in e5p's /var/log

Further: "ssh gene@coyote" successfully logged into coyote, so assuming that ssh is running

set e5p's /etc/apparmor verbose on with nano.

from that login "ssh gene@e5p" gets "connection reset by peer", and NO update to syslog. The rejection is not even making it to ANY of the logs.  So what's next?  Switch back to a 64G u-sd? Is a 128G u-sd too big? I am out of ideas. And a $3000 3d printer is Dead In The Water, no way to restore its klipper /home/gene/printer_data directory after a forced upgrade from jammy to noble.

 

Thanks.

 

[gene1934]

Now I have restarted the install with a 64G u-card, and it is totally different. it only took 1 pass at networkmanager to make all the local network work in that I can ssh gene@any-local-machine and bavk again.  But now I cannot get thru the router for its dnsmasq server. All addresses in the network setup are checked and correct. The hostname has been reset to its normal, e5p, has been used for 3 or 4 years but the Network is unreachable because the router isn't functioning as a dns lookup.  Its even been rebooted a couple times.  New faster router about 4 months back with dd-wrt in it.

 

ip a:

   end0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 2a:b6:6c:1a:3a:1f brd ff:ff:ff:ff:ff:ff
    inet 192.168.71.122/24 brd 192.168.71.255 scope global noprefixroute end0
       valid_lft forever preferred_lft forever
    inet6 fe80::28b6:6cff:fe1a:3a1f/64 scope link 
       valid_lft forever preferred_lft forever

ip r:

192.168.71.0/24 dev end0 proto kernel scope link src 192.168.71.122 metric 100 
192.168.71.0/24 via 192.168.71.1 dev end0 proto static metric 200 

/etc/netplan/armbian.yaml:

root@e5p:/etc/netplan# cat armbian.yaml
network:
  version: 2
  renderer: NetworkManager
  ethernets:
    end0:
      addresses:
      - "192.168.71.122/24"
      nameservers:
        addresses:
        - 192.168.71.1
      dhcp4: false
      dhcp6: false
      macaddress: "2a:b6:6c:1a:3a:1f"
      routes:
      - metric: 200
        to: "192.168.71.1/24"
        via: "192.168.71.1"
 

Can anyone see what is wrong with that?

 

Thanks.

[eselarm]

3 hours ago, gene1934 said:

root@e5p:/etc/netplan# cat armbian.yaml

I stopped using netplan and its extra yaml layer on top of NetworkManager or systemd-networkd. I have a handful of VLANs (and several managed switches and cables in- and outside the house and also bridges 'on top' of those VLANs and USB/4G/WiFi as well.  Several ISP's also use VLANs, so a must to have that working properly. It all works fine based on only NetworkManager essentially, but if needed I would do direct setup with ip tool. I have also Opensuse, that does not use netplan, but works simply by exchanging .nmconnection files with Debian. No new study or testing for days/weeks/months. Not Ubuntu because of that netplan and it generates .nmconnection files, so overwriting your own.

 

The problem is, NetworkManager package in Ubuntu has a dependency on netplan.io package. So if you purge netplan.io , also NetworkManager is removed. This is not the case in Debian, hence Ubuntu again after 5 years when they also introduced could-init, turns out to be a PITA and simply waste of time for me. So for example for my ROCK3A (similar to your BPI-M5), I just threw away the Armbian Noble install and cloned Armbian Bookworm from NanoPi-R6C and later upgraded in-place to Trixie. Same networking, very little effort, almost unattended. As a sort of sick joke (for experienced home Linux users at least), also RPL put netplan and cloud-init as default in their images. There are several hacks/tweaks/workarounds on that forum. Even more because also many RPi users still use ifupdown and interfaces file (and dhcpcd based although not needed for static). Maybe have a look to see what could be best for you. I basically removed/blocked additional package sources lists, so I get the same on ARM as on x86-64. But note that that is for servers based, pure client computing should work out of the box if you have standard/average/common router, like ISP's 'give you'. If not, lot's own work to do as you see, but not Armbian specific, just home networking and router maintenance.

 

So maybe think about hostnames and IP addressing in your home. I have own router (own software, Linux based) for more than 2 decades basically, so easy to keep a list of MACaddresses/computers, although I still have a simple spreadsheet as a sort of design philosophy/overview. You can reserve/fix MACaddress+IPaddress in dd-wrt, so then all client computer DHCP based tools should be no issues. Else you get a mess as you see.

 

I have setup some dev/test environment based on a physical PCIe with RTL chipset ethernet port for systemd-networkd in combination with managed ethernet switches, but first trial locked up a certain VLAN (something in the switch I think). That was 3 monhts ago and not sure if I continue with it. It means more reading the long systemd-networkd docs, it seems not worth the effort when I compare with NM and its nmtui tool.

 

You might also have silently installed a firewall package. firewalld defaults to public zone, that blocks incoming ssh. I experienced that some time ago on 1 computer with rolling release Linux distro, so that overwrites things every now and then when a fundamental upgrade of a certain part, but noted/warned in changelogs. 

[gene1934]

The puzzle seems to have resolved itself. I gave up and shut the printer off about 04:00 this morning and went to bed.  Got back up, powered it on about 08:30, and its all working, updating 304 pkgs right now.  It was rebooted at least 15 times as I worked last night w/o any affect.  What the hell is the diff between a reboot and a full powerdown of 4+ hours???  Did dd-wrt flush its cache of MAC addresses?  IDK. But w/o a global net, I was trapped, I could not DL the tools needed to troubleshoot. A very frustrating state of affairs.

 

I do not see anyplace where I could mark this as solved.

[bedna]

I have experienced something similar in the past and it made me very puzzled until I figured out what was wrong in that situation.

Make sure the clock on both systems are synchronized to an NTP server.

IIRC SSH just threw a fit exactly like this without logging why, and it turned out to be a security feature where the ssh server suspected the communication to be intercepted by man-in-the-middle due to clocks not being synchronized.

And if logging in physically to the system, apt refused to work as-well until clock was synced.

 

Could be worth to check next time...

 

As for dhcp and systemd. Check if systemd-resolved.service is messing something up for you, maybe try to disable it?

Edited 7 hours ago by bedna

[gene1934]

How best to do that?

 

I've been treating systemd like a rattlesnake which I've seen in record breaking size, 7 feet long and the size of my calf. I was about 6, my stepfather was farming, had come in for lunch, opened the door to go back out and plow, slammed the door, reached over it for a cheap busted shotgun and a shell. reopened the door and took the head off it.  85 years ago.  I haven't had a whole lot of use for a rattlesnake since although I've seen hundreds.  Blacksnakes OTOH are goodfellows. They keep the rodents away by eating them.