Jump to content

[N2] resolvconf-pull-resolved bug -> Connection timed out

Z11ntal33r

By Z11ntal33r
in Odroid N2/N2+

 Share

Recommended Posts

Z11ntal33r

Z11ntal33r

Posted
Posted
Armbianmonitor:
https://pastebin.com/MeJdigT8


Odroid-N2 using Armbian_20.08.1_Odroidn2_focal_current_5.8.5

I've recently got a "Connection timed out" issue. For instance, I can open files with nano, use htop, armbian-config, update and upgrade packages, but I can't reboot (everything over ssh). I'm having a feeling that the issue started when I tried to figure out if it's possible to get HDMI to work with my 4K TV when the HDMI cable is not plugged in at boot. In that case, I booted several times and chaged boot.ini with setenv display_autodetect "false", setenv hdmimode "1080p60hz" and  setenv disablehpd "true" without success. armbianmonitor -u information is attached at the end in my post where this line might be of interest: "[    0.000000] [Firmware Bug]: Kernel image misaligned at boot, please fix your bootloader!".

Conection timed out message:

Spoiler 

$ sudo systemctl reboot
Failed to set wall message, ignoring: Connection timed out
Failed to reboot system via logind: Connection timed out
Failed to start reboot.target: Connection timed out
See system logs and 'systemctl status reboot.target' for details.
It is possible to perform action directly, see discussion of --force --force in man:systemctl(1).
$ sudo systemctl reboot --force
Failed to execute operation: Connection timed out
It is possible to perform action directly, see discussion of --force --force in man:systemctl(1).
$ sudo systemctl status reboot.target
Failed to get properties: Connection timed out

 


Running  armbianmonitor -v:

Spoiler 

$ sudo armbianmonitor -v
Starting package integrity check. This might take some time. Be patient please...

It appears you may have corrupt packages.

This is usually a symptom of filesystem corruption caused by SD cards or eMMC
dying or burning the OS image to the installation media went wrong.

The following changes from packaged state files were detected:

/usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf

 

Does armbianmonitor -v give any useful information? It detects corrupted pacakges, but which? The 10-globally-managed-devices.conf does not seem to exist. At least I can't read it with cat or nano.

sysctl --system

Spoiler 

$ sudo sysctl --system
* Applying /etc/sysctl.d/10-console-messages.conf ...
kernel.printk = 4 4 1 7
* Applying /etc/sysctl.d/10-ipv6-privacy.conf ...
net.ipv6.conf.all.use_tempaddr = 2
net.ipv6.conf.default.use_tempaddr = 2
* Applying /etc/sysctl.d/10-kernel-hardening.conf ...
kernel.kptr_restrict = 1
* Applying /etc/sysctl.d/10-link-restrictions.conf ...
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
* Applying /etc/sysctl.d/10-magic-sysrq.conf ...
kernel.sysrq = 176
* Applying /etc/sysctl.d/10-network-security.conf ...
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.all.rp_filter = 2
* Applying /etc/sysctl.d/10-ptrace.conf ...
kernel.yama.ptrace_scope = 1
* Applying /etc/sysctl.d/10-zeropage.conf ...
vm.mmap_min_addr = 32768
* Applying /usr/lib/sysctl.d/50-default.conf ...
net.ipv4.conf.default.promote_secondaries = 1
sysctl: setting key "net.ipv4.conf.all.promote_secondaries": Invalid argument
net.ipv4.ping_group_range = 0 2147483647
net.core.default_qdisc = fq_codel
fs.protected_regular = 1
fs.protected_fifos = 1
* Applying /usr/lib/sysctl.d/50-pid-max.conf ...
kernel.pid_max = 4194304
* Applying /etc/sysctl.d/99-sysctl.conf ...
kernel.printk = 3 4 1 3
vm.swappiness = 30
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
kernel.sysrq = 0
net.ipv4.tcp_sack = 0
net.ipv4.tcp_timestamps = 0
net.ipv4.icmp_echo_ignore_all = 1
net.ipv6.icmp.echo_ignore_all = 1
kernel.randomize_va_space = 2
fs.suid_dumpable = 0
kernel.dmesg_restrict = 1
kernel.yama.ptrace_scope = 1
kernel.unprivileged_bpf_disabled = 1
net.core.bpf_jit_enable = 1
net.core.bpf_jit_harden = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_syn_retries = 2
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.ip_forward = 0
net.ipv4.conf.all.forwarding = 0
net.ipv4.conf.default.forwarding = 0
net.ipv6.conf.all.forwarding = 0
net.ipv6.conf.default.forwarding = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.default.accept_source_route = 0
net.ipv4.conf.all.bootp_relay = 0
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.tcp_rfc1337 = 1
net.ipv4.conf.default.log_martians = 0
net.ipv4.conf.all.log_martians = 0
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.route.flush = 1
net.ipv6.route.flush = 1
* Applying /usr/lib/sysctl.d/protect-links.conf ...
fs.protected_fifos = 1
fs.protected_hardlinks = 1
fs.protected_regular = 2
fs.protected_symlinks = 1
* Applying /etc/sysctl.conf ...
kernel.printk = 3 4 1 3
vm.swappiness = 30
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
kernel.sysrq = 0
net.ipv4.tcp_sack = 0
net.ipv4.tcp_timestamps = 0
net.ipv4.icmp_echo_ignore_all = 1
net.ipv6.icmp.echo_ignore_all = 1
kernel.randomize_va_space = 2
fs.suid_dumpable = 0
kernel.dmesg_restrict = 1
kernel.yama.ptrace_scope = 1
kernel.unprivileged_bpf_disabled = 1
net.core.bpf_jit_enable = 1
net.core.bpf_jit_harden = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_syn_retries = 2
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.ip_forward = 0
net.ipv4.conf.all.forwarding = 0
net.ipv4.conf.default.forwarding = 0
net.ipv6.conf.all.forwarding = 0
net.ipv6.conf.default.forwarding = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.default.accept_source_route = 0
net.ipv4.conf.all.bootp_relay = 0
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.tcp_rfc1337 = 1
net.ipv4.conf.default.log_martians = 0
net.ipv4.conf.all.log_martians = 0
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.route.flush = 1
net.ipv6.route.flush = 1

 


armbianmonitor -u

Spoiler

Full log: https://pastebin.com/MeJdigT8
Snippets: 


[    0.000000] [Firmware Bug]: Kernel image misaligned at boot, please fix your bootloader!
...
[    1.977141] thermal thermal_zone0: binding zone cpu-thermal with cdev thermal-cpufreq-0 failed:-17
[    1.977817] cpufreq: cpufreq_online: CPU2: Running at unlisted freq: 999999 KHz
[    1.977878] cpufreq: cpufreq_online: CPU2: Unlisted initial frequency changed to: 1000000 KHz
[    1.978854] thermal thermal_zone0: binding zone cpu-thermal with cdev thermal-cpufreq-1 failed:-17
...

 


I guess the only way to reboot is to pull the plug? Or could it be done without pulling the plug if I connect to the N2 with UART? I'm thinking about doing a new fresh install, but before that, is it worth trying to use the legacy kernel (4.9)? Thanks for the awesome work with Armbian! It has truly been a blessing since I first discovered it!

Igor

Igor

Posted
Posted
2 hours ago, Z11ntal33r said:

is it worth trying to use the legacy kernel


Yes. 5.8.y seems to have many issues. Overall. Not just here.

Z11ntal33r

Z11ntal33r

Posted
  • Author
Posted

Thanks for letting me know! I'll do a new clean install using Armbian_20.08.1_Odroidn2_focal_current_5.8.5 and change kernel to use legacy in armbian-config forwards. I'm looking for a much more stable experience running a home server than what I've encountered the last few months running mainline. 

Igor

Igor

Posted
Posted

Even 5.7.y would be better ... switch to that and Freeze kernel upgrading within armbian-config.

Z11ntal33r

Z11ntal33r

Posted
  • Author
Posted

So if you were going to setup N2 as a home NAS, you would rather use 5.7 for some time before jumping to a higher stable kernel (>= 5.9) rather than setup 4.9 and staying there?

Igor

Igor

Posted
Posted
13 minutes ago, Z11ntal33r said:

So if you were going to setup N2 as a home NAS, you would rather use 5.7 for some time before jumping to a higher stable kernel (>= 5.9) rather than setup 4.9 and staying there?


Long story short. 4.9.y is legacy kernel, a kernel where initial support for this hardware was developed. Its old lack modern features, but board specific functions work well / better. In modern kernel basic functions also works well, but 5.8.y is ATM in general fragile ... we had decent stability with 5.6.y / 5.7.y.

5.8.y is next LTS kernel and things will get better soon. If you need a stable operations now, use my recommendation.


And. This board is not recommended for a NAS. Expect same problems as on Raspberry Pi. Regardless of kernel.

Z11ntal33r

Z11ntal33r

Posted
  • Author
Posted

Thanks again for a great reply! I'll stay put to mainline and use 5.7 until 5.8 has matured. 

I do not have high expectations or hope for a extremely stable experience with N2, but if it can somewhat give me a decent experience that will be more than enough. I bet I've changed my arm device in a year or two. Out of couriosity, do you have any similar boards you would recommend for NAS usage in the $100-150 price range?

Z11ntal33r

Z11ntal33r

Posted
  • Author
Posted

I think I found the root of the issue I had. It seems to be related to resolvconf and particularly resolvconf-pull-resolved where I had systemd-resolved installed and handling resolv.conf with a symlink. Removing the resolvconf package did the trick. I'll let the server run for some days, yet it seems fine some far. I'm going to make sure that I remove resolvconf in my installation script to prevent this mess from happening again... This is something to be aware of as others might be a victim of the issue as well. It's a quite serious one too.

 

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968015

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=967906

 

Guest
This topic is now closed to further replies.
 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept