Jump to content

cannot start haveged.service and sysstat.service failed

g00d

By g00d
in Beginners

 Share

Recommended Posts

g00d

g00d

Posted
Posted

Hello everybody,


I am using Armbian Bullseye image on an ODROID-HC2 and I moved the root filesystem to an encrypted SATA drive. During boot in UART / USB serial console I see that two services fail to start: Resets System Activity Logs and Haveged

  

[...]

[  OK  ] Reached target Sockets.
         Starting Armbian hardware monitoring...
         Starting Armbian hardware optimization...
[  OK  ] Started RPC bind portmap service.
[  OK  ] Finished Armbian hardware monitoring.
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.
[  OK  ] Reached target RPC Port Mapper.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Finished Armbian hardware optimization.
[  OK  ] Reached target Basic System.
         Starting Save/Restore Sound Card State...
[  OK  ] Started Regular background program processing daemon.
[  OK  ] Started D-Bus System Message Bus.
         Starting Network Manager...
         Starting Remove Stale Onli…t4 Metadata Check Snapshots...
         Starting LSB: Load kernel …d to enable cpufreq scaling...
         Starting Dispatcher daemon for systemd-networkd...
         Starting System Logging Service...
         Starting Self Monitoring a…g Technology (SMART) Daemon...
         Starting Resets System Activity Logs...
         Starting User Login Management...
         Starting WPA supplicant...
[  OK  ] Started System Logging Service.
[  OK  ] Finished Save/Restore Sound Card State.
[  OK  ] Reached target Sound Card.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started WPA supplicant.
[FAILED] Failed to start Resets System Activity Logs.
See 'systemctl status sysstat.service' for details.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Self Monitoring an…ing Technology (SMART) Daemon.
[  OK  ] Started User Login Management.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Finished Remove Stale Onli…ext4 Metadata Check Snapshots.
[  OK  ] Started LSB: Load kernel m…ded to enable cpufreq scaling.
         Starting LSB: set CPUFreq kernel parameters...
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[FAILED] Failed to start Entropy Da…based on the HAVEGE algorithm.
See 'systemctl status haveged.service' for details.
[  OK  ] Started LSB: set CPUFreq kernel parameters.
         Starting LSB: Set sysfs variables from /etc/sysfs.conf...
[  OK  ] Started LSB: Set sysfs variables from /etc/sysfs.conf.
[  OK  ] Started Network Manager.
[  OK  ] Reached target Network.
[  OK  ] Reached target Network is Online.
         Starting chrony, an NTP client/server...

[...]

 

the logs saying...

Zitat

 


root@myhost~# systemctl status sysstat.service
● sysstat.service - Resets System Activity Logs
     Loaded: loaded (/lib/systemd/system/sysstat.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2021-10-27 18:09:02 CEST; 8min ago
       Docs: man:sa1(8)
             man:sadc(8)
             man:sar(1)
    Process: 992 ExecStart=/usr/lib/sysstat/sa1 --boot (code=exited, status=2)
   Main PID: 992 (code=exited, status=2)
        CPU: 20ms

Okt 27 18:09:02 myhost systemd[1]: Starting Resets System Activity Logs...
Okt 27 18:09:02 myhost sa1[992]: Daten können nicht in die Systemaktivitätendatei geschrieben werden: Auf dem Gerät ist kein Sp>
Okt 27 18:09:02 myhost systemd[1]: sysstat.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Okt 27 18:09:02 myhost systemd[1]: sysstat.service: Failed with result 'exit-code'.
Okt 27 18:09:02 myhost systemd[1]: Failed to start Resets System Activity Logs.

 

 

The line "Okt 27 18:09:02 myhost sa1[992]: Daten können nicht in die Systemaktivitätendatei geschrieben werden: Auf dem Gerät ist kein Sp>" is truncated but it says, that there is no space available (german language). I have no idea which process and service (sa1??) that is and where it tries to output any data. Possibly that this is causing the error.

 

It seems I can manually start sysstat.service successfully ... 

systemctl start haveged.service && systemctl status haveged.service
Zitat

 


● sysstat.service - Resets System Activity Logs
     Loaded: loaded (/lib/systemd/system/sysstat.service; enabled; vendor preset: enabled)
     Active: active (exited) since Wed 2021-10-27 18:19:10 CEST; 1s ago
       Docs: man:sa1(8)
             man:sadc(8)
             man:sar(1)
    Process: 2096 ExecStart=/usr/lib/sysstat/sa1 --boot (code=exited, status=0/SUCCESS)
   Main PID: 2096 (code=exited, status=0/SUCCESS)
        CPU: 47ms

Okt 27 18:19:10 myhost systemd[1]: Starting Resets System Activity Logs...
Okt 27 18:19:10 myhost systemd[1]: Finished Resets System Activity Logs.
 

 

 

haveged.service cannot be restarted manually, it still fails ...

 

Zitat

 


root@myhost:~# systemctl status haveged.service
● haveged.service - Entropy Daemon based on the HAVEGE algorithm
     Loaded: loaded (/lib/systemd/system/haveged.service; enabled; vendor preset: enabled)
     Active: failed (Result: signal) since Wed 2021-10-27 18:09:03 CEST; 12min ago
       Docs: man:haveged(8)
             http://www.issihosts.com/haveged/
    Process: 1114 ExecStart=/usr/sbin/haveged --Foreground --verbose=1 $DAEMON_ARGS (code=killed, signal=SYS)
   Main PID: 1114 (code=killed, signal=SYS)
        CPU: 97ms

Okt 27 18:09:03 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 5.
Okt 27 18:09:03 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:03 myhost systemd[1]: haveged.service: Start request repeated too quickly.
Okt 27 18:09:03 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 27 18:09:03 myhost systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm.

 

 

Zitat

 


root@myhost:~# journalctl -u haveged.service
-- Journal begins at Wed 2021-10-27 17:07:54 CEST, ends at Wed 2021-10-27 18:20:09 CEST. --
Okt 27 18:09:00 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:01 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 27 18:09:01 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 27 18:09:01 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 1.
Okt 27 18:09:01 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:01 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:02 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 27 18:09:02 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 27 18:09:02 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 2.
Okt 27 18:09:02 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:02 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:02 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 27 18:09:02 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 27 18:09:02 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 3.
Okt 27 18:09:02 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:02 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:02 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 27 18:09:02 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 27 18:09:03 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 4.
Okt 27 18:09:03 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:03 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:03 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 27 18:09:03 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 27 18:09:03 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 5.
Okt 27 18:09:03 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 27 18:09:03 myhost systemd[1]: haveged.service: Start request repeated too quickly.
Okt 27 18:09:03 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 27 18:09:03 myhost systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm.
 

 

 

but it seems I can start the haveged process manually, I see no errors here and the process runs until I abort with keypress CTRL+C 


root@myhostname:~# /usr/sbin/haveged --Foreground --verbose=1
haveged: command socket is listening at fd 3
haveged starting up
haveged: ver: 1.9.14; arch: generic; vend: ; build: (gcc 10.2.1 CTV); collect: 128K
haveged: cpu: (VC); data: 16K (D); inst: 16K (D); idx: 12/40; sz: 15006/57790
haveged: tot tests(BA8): A:1/1 B:1/1 continuous tests(B):  last entropy estimate 8.00131
haveged: fills: 0, generated: 0

 

Question 1) Why does the sysstat.service fail in starting at the boot process but works ok when manually initiiated ?

Question 2) Why does haveged.service fail at all in starting ?

 

df -h shows plenty of free space available on all my mounts and partitions.

 

I have absolutely no clue where to begin to dig. Any clues?

0
tparys

tparys

Posted
Posted

For 2, the haveged service is not terribly reliable on ARM architectures. There are other threads on the subject.

 

Not sure about 1. May be a boot race condition, looking for activity on a cryptographic device that hasn't been started yet?

0
g00d

g00d

Posted
  • Author
Posted

Well, I was wrong about the remaining free space in file systems. I did some more research with those services and the logs revealed that they cannot write to /var/log/foo.log

 

df -h shows: 


udev                          925M       0  925M    0% /dev
tmpfs                         200M    8,0M  192M    4% /run
/dev/mapper/rootfs      32G    1,3G   29G    5% /
tmpfs                          996M       0  996M    0% /dev/shm
tmpfs                           5,0M    4,0K  5,0M    1% /run/lock
tmpfs                          996M       0  996M    0% /tmp
/dev/zram1                 49M     48M     0  100% /var/log
tmpfs                         200M       0  200M    0% /run/user/1000

 

what is zram1 and why is it so small (50MB) ? how can I correct this? can you share some information about this strange zram1 filesystem that is mounted to /var/log ? I also see a /var/log.hdd on my system, what's that and what's the difference between those?

0
Werner

Werner

Posted
Posted

Because logs are usually plain text which can be compressed easily. But sometimes daemons or the system create an abnormal amount of logs and then the ramdisk runs out of space. Either fix your logs or check /etc/default for armbian-zramconfig or whatever the file is called and increase/disable.

0
g00d

g00d

Posted
  • Author
Posted

Hi Werner,

 

I realized already that my armbian install has 1GB swap assigned to zram and I was not clear where this originates from :-) After examining /etc/default/armbian-zram-config I understand that this corresponds to the default value of 50% of my existing RAM (2GB on the ODROID-HC2). Thanks for that information and pointing to that direction.

 

A) I have configured 4GB swap space on my own (hard disk), actually I have 5GB in summary. Do I really need zram-config at all or can I disable it in the mentioned configuration file and what the pro/con ?

 

B) I do not understand yet where the 50MB for my /var/log mount comes from? How can I adjust/extend this size, didn't find anything related to /var/log in /etc/default/armbian-zram-config ?

 

thank you in advance.

 

 

0
Werner

Werner

Posted
Posted

A. No you can disable it if you want. Pro: More "native" ram and faster speed because ram compression takes a bit of CPU time. Con: Slower swap speed if system needs it.

I recommend changing vm.swappiness in /etc/sysctl.conf if you use your external drive as swap to reduce its usage. To something like 25 instead of 100

 

B. check armbian-ramlog file

0
g00d

g00d

Posted
  • Author
Posted

A. thanks for the hint. I did set ENABLE=false in /etc/default/armbian-zram-config and vm.swappiness=25 in /etc/sysctl.conf

 

B. I wasn't able to find helpful information under /var/log.hdd/armbian-ramlog.log but after some more research I found the file /etc/default/armbian-ramlog so I'm pretty sure this is the file you wanted to point me at :) I increased the default file size of 50MB to 100MB.

 

Now the initial issue with sysstat.service is solved. But haveged.service still fails

0
g00d

g00d

Posted
  • Author
Posted
vor 27 Minuten schrieb g00d:

Now the initial issue with sysstat.service is solved. But haveged.service still fails

 

/var/log/syslog 

Oct 28 13:28:12 myhost kernel: [ 1380.296918] audit: type=1326 audit(1635420492.803:57): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2575 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6f57e98 code=0x80000000
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 1.
Oct 28 13:28:13 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Oct 28 13:28:13 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Oct 28 13:28:13 myhost kernel: [ 1380.577192] audit: type=1326 audit(1635420493.083:58): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2584 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6f7de98 code=0x80000000
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 2.
Oct 28 13:28:13 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Oct 28 13:28:13 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Oct 28 13:28:13 myhost kernel: [ 1380.829670] audit: type=1326 audit(1635420493.343:59): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2587 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6f78e98 code=0x80000000
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 3.
Oct 28 13:28:13 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Oct 28 13:28:13 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Oct 28 13:28:13 myhost kernel: [ 1381.079713] audit: type=1326 audit(1635420493.593:60): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2590 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6fbce98 code=0x80000000
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 4.
Oct 28 13:28:13 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Oct 28 13:28:13 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Oct 28 13:28:13 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Oct 28 13:28:13 myhost kernel: [ 1381.329773] audit: type=1326 audit(1635420493.843:61): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2593 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6f86e98 code=0x80000000
Oct 28 13:28:14 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 5.
Oct 28 13:28:14 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Oct 28 13:28:14 myhost systemd[1]: haveged.service: Start request repeated too quickly.
Oct 28 13:28:14 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Oct 28 13:28:14 myhost systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm.

 

what is sig=31 and is this related to the kernel ? any clues how to debug the failing haveged.service ? I'm really stuck here :/ 

-- Boot 7f29c5436e05479595fb657fef877bc8 --
Okt 28 13:05:55 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:55 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 28 13:05:55 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 28 13:05:55 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 1.
Okt 28 13:05:56 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:56 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:56 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 28 13:05:56 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 28 13:05:57 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 2.
Okt 28 13:05:57 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:57 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:57 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 28 13:05:57 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 28 13:05:57 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 3.
Okt 28 13:05:57 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:57 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:57 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 28 13:05:57 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 28 13:05:57 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 4.
Okt 28 13:05:57 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:57 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:57 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Okt 28 13:05:57 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 28 13:05:58 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 5.
Okt 28 13:05:58 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Okt 28 13:05:58 myhost systemd[1]: haveged.service: Start request repeated too quickly.
Okt 28 13:05:58 myhost systemd[1]: haveged.service: Failed with result 'signal'.
Okt 28 13:05:58 myhost systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm.
0
This thread is quite old. Please consider starting a new thread rather than reviving this one.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept