NanoPi Neo after update: Now restarting services hangs on systemd-tty-ask-password-agent

[willie-m]

I have a NanoPi Neo that's been up about 4 months and I am using it for DHCP and DNS caching (using dnsmasq). This has been working fine over SSH until after the update. Previously, I could log in, edit the dnsmasq settings, and restart the service without issue.

 

Now, when I attempt to restart a service (not just dnsmasq, I tested a couple others), it hangs and looks like it's waiting for a password entry on the TTY. (I'm logged in over SSH and I do not have a TTY connected).

 

... in first SSH session, this just hangs forever until I ctrl-C:

root@nanopineo:/etc# systemctl restart dnsmasq

... in second SSH session, PS shows:

root      1825   783  0 11:18 pts/0    00:00:00 systemctl restart dnsmasq
root      1826  1825  0 11:18 pts/0    00:00:00 /bin/systemd-tty-ask-password-agent --watch
root      1827  1624  0 11:18 pts/1    00:00:00 ps -ef

Much reading online and I still cannot figure out if this is intentional, nor can I find how I can disable this behavior. Why would it ask for a prompt when running as root user?

 

I think perhaps this might be a bug - you cannot remotely manage a server if it's going to hang waiting for a password every time you restart a service! That's pretty bad. Possibly this one which does not seem resolved.

 

https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1456789

 

Any ideas? Has anyone seen this?

 

Thanks

 

[willie-m]

Further debugging shows (I think) that I am "stuck" waiting for graphical target to complete. ?? Not familiar with this.

 

root@nanopineo:/etc# systemctl list-jobs
JOB UNIT                                 TYPE    STATE  
  1 graphical.target                     start   waiting
  2 multi-user.target                    start   waiting
 68 systemd-update-utmp-runlevel.service start   waiting
 74 sysfsutils.service                   start   waiting
 76 asterisk.service                     start   waiting
 79 cpufrequtils.service                 start   waiting
 97 getty.target                         start   waiting
 98 serial-getty@ttyS0.service           start   waiting
101 getty@tty1.service                   start   waiting
131 NetworkManager.service               restart waiting
197 dbus.service                         start   waiting
263 systemd-logind.service               restart waiting
330 ssh.service                          restart waiting
331 lirc.service                         restart waiting
397 systemd-tmpfiles-clean.service       start   waiting
884 dnsmasq.service                      restart waiting
952 nss-lookup.target                    restart waiting

 

[willie-m]

Confirmed was in Graphical mode and then switched to "run level 3" equivalent:

root@nanopineo:~# systemctl get-default
graphical.target

root@nanopineo:~# systemctl set-default multi-user.target
Created symlink from /etc/systemd/system/default.target to /lib/systemd/system/multi-user.target.

root@nanopineo:~# reboot

 

Now after reboot, it's in multi user mode, but still waiting for something...

root@nanopineo:~# systemctl list-jobs
JOB UNIT                                 TYPE    STATE  
  1 multi-user.target                    start   waiting
 74 nss-lookup.target                    start   waiting
 84 systemd-update-utmp-runlevel.service start   waiting
 86 getty.target                         start   waiting
 87 serial-getty@ttyS0.service           start   waiting
 90 getty@tty1.service                   start   waiting
 98 sysfsutils.service                   start   waiting
100 asterisk.service                     start   waiting
103 cpufrequtils.service                 start   waiting
130 systemd-logind.service               restart waiting
197 NetworkManager.service               restart waiting
263 dbus.service                         start   waiting

 

 

 

[willie-m]

Appears that startup is having issues due to a dependency problem caused by dbus. It's crashing with a Segmentation violation when it is run. Do I need dbus? The system is up and serving requests so arguably the answer is "no".
 

root@nanopineo:~# systemctl list-jobs
JOB UNIT                                 TYPE    STATE  
  1 multi-user.target                    start   waiting
 74 nss-lookup.target                    start   waiting
 84 systemd-update-utmp-runlevel.service start   waiting
 86 getty.target                         start   waiting
 87 serial-getty@ttyS0.service           start   waiting
 90 getty@tty1.service                   start   waiting
 98 sysfsutils.service                   start   waiting
102 cpufrequtils.service                 start   waiting
129 NetworkManager.service               restart waiting
195 systemd-logind.service               restart waiting
262 dbus.service                         start   waiting

11 jobs listed.
root@nanopineo:~# systemctl status dbus
● dbus.service - D-Bus System Message Bus
   Loaded: loaded (/lib/systemd/system/dbus.service; static)
   Active: failed (Result: signal) since Mon 2017-08-07 12:51:08 CDT; 2min 15s ago
     Docs: man:dbus-daemon(1)
  Process: 548 ExecStart=/usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation (code=killed, signal=SEGV)
 Main PID: 548 (code=killed, signal=SEGV)

Aug 07 12:49:46 nanopineo systemd[1]: Started D-Bus System Message Bus.
Aug 07 12:51:08 nanopineo systemd[1]: dbus.service: main process exited, code=killed, status=11/SEGV
Aug 07 12:51:08 nanopineo systemd[1]: Unit dbus.service entered failed state.
root@nanopineo:~#

 

[willie-m]

I will not claim solved, but I have found a workaround by removing dbus from the dependency list:

 

systemctl mask dbus.service

 

Will symlink it to /dev/null.

 

I then removed the network-manager dependency by uninstalling the package, removed from all rc.d directories, and similarly masked in systemctl.

 

The initial issue is no longer happening - systemctl does not block when I attempt to restart the services.