g00d Posted October 27, 2021 Share Posted October 27, 2021 Hello everybody, I am using Armbian Bullseye image on an ODROID-HC2 and I moved the root filesystem to an encrypted SATA drive. During boot in UART / USB serial console I see that two services fail to start: Resets System Activity Logs and Haveged [...] [ OK ] Reached target Sockets. Starting Armbian hardware monitoring... Starting Armbian hardware optimization... [ OK ] Started RPC bind portmap service. [ OK ] Finished Armbian hardware monitoring. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped Entropy Daemon based on the HAVEGE algorithm. [ OK ] Started Entropy Daemon based on the HAVEGE algorithm. [ OK ] Finished Armbian hardware optimization. [ OK ] Reached target Basic System. Starting Save/Restore Sound Card State... [ OK ] Started Regular background program processing daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Remove Stale Onli…t4 Metadata Check Snapshots... Starting LSB: Load kernel …d to enable cpufreq scaling... Starting Dispatcher daemon for systemd-networkd... Starting System Logging Service... Starting Self Monitoring a…g Technology (SMART) Daemon... Starting Resets System Activity Logs... Starting User Login Management... Starting WPA supplicant... [ OK ] Started System Logging Service. [ OK ] Finished Save/Restore Sound Card State. [ OK ] Reached target Sound Card. [ OK ] Stopped Entropy Daemon based on the HAVEGE algorithm. [ OK ] Started Entropy Daemon based on the HAVEGE algorithm. [ OK ] Started WPA supplicant. [FAILED] Failed to start Resets System Activity Logs. See 'systemctl status sysstat.service' for details. [ OK ] Stopped Entropy Daemon based on the HAVEGE algorithm. [ OK ] Started Entropy Daemon based on the HAVEGE algorithm. [ OK ] Started Self Monitoring an…ing Technology (SMART) Daemon. [ OK ] Started User Login Management. [ OK ] Stopped Entropy Daemon based on the HAVEGE algorithm. [ OK ] Started Entropy Daemon based on the HAVEGE algorithm. [ OK ] Finished Remove Stale Onli…ext4 Metadata Check Snapshots. [ OK ] Started LSB: Load kernel m…ded to enable cpufreq scaling. Starting LSB: set CPUFreq kernel parameters... [ OK ] Stopped Entropy Daemon based on the HAVEGE algorithm. [FAILED] Failed to start Entropy Da…based on the HAVEGE algorithm. See 'systemctl status haveged.service' for details. [ OK ] Started LSB: set CPUFreq kernel parameters. Starting LSB: Set sysfs variables from /etc/sysfs.conf... [ OK ] Started LSB: Set sysfs variables from /etc/sysfs.conf. [ OK ] Started Network Manager. [ OK ] Reached target Network. [ OK ] Reached target Network is Online. Starting chrony, an NTP client/server... [...] the logs saying... Zitat root@myhost~# systemctl status sysstat.service ● sysstat.service - Resets System Activity Logs Loaded: loaded (/lib/systemd/system/sysstat.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Wed 2021-10-27 18:09:02 CEST; 8min ago Docs: man:sa1(8) man:sadc(8) man:sar(1) Process: 992 ExecStart=/usr/lib/sysstat/sa1 --boot (code=exited, status=2) Main PID: 992 (code=exited, status=2) CPU: 20ms Okt 27 18:09:02 myhost systemd[1]: Starting Resets System Activity Logs... Okt 27 18:09:02 myhost sa1[992]: Daten können nicht in die Systemaktivitätendatei geschrieben werden: Auf dem Gerät ist kein Sp> Okt 27 18:09:02 myhost systemd[1]: sysstat.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Okt 27 18:09:02 myhost systemd[1]: sysstat.service: Failed with result 'exit-code'. Okt 27 18:09:02 myhost systemd[1]: Failed to start Resets System Activity Logs. The line "Okt 27 18:09:02 myhost sa1[992]: Daten können nicht in die Systemaktivitätendatei geschrieben werden: Auf dem Gerät ist kein Sp>" is truncated but it says, that there is no space available (german language). I have no idea which process and service (sa1??) that is and where it tries to output any data. Possibly that this is causing the error. It seems I can manually start sysstat.service successfully ... systemctl start haveged.service && systemctl status haveged.service Zitat ● sysstat.service - Resets System Activity Logs Loaded: loaded (/lib/systemd/system/sysstat.service; enabled; vendor preset: enabled) Active: active (exited) since Wed 2021-10-27 18:19:10 CEST; 1s ago Docs: man:sa1(8) man:sadc(8) man:sar(1) Process: 2096 ExecStart=/usr/lib/sysstat/sa1 --boot (code=exited, status=0/SUCCESS) Main PID: 2096 (code=exited, status=0/SUCCESS) CPU: 47ms Okt 27 18:19:10 myhost systemd[1]: Starting Resets System Activity Logs... Okt 27 18:19:10 myhost systemd[1]: Finished Resets System Activity Logs. haveged.service cannot be restarted manually, it still fails ... Zitat root@myhost:~# systemctl status haveged.service ● haveged.service - Entropy Daemon based on the HAVEGE algorithm Loaded: loaded (/lib/systemd/system/haveged.service; enabled; vendor preset: enabled) Active: failed (Result: signal) since Wed 2021-10-27 18:09:03 CEST; 12min ago Docs: man:haveged(8) http://www.issihosts.com/haveged/ Process: 1114 ExecStart=/usr/sbin/haveged --Foreground --verbose=1 $DAEMON_ARGS (code=killed, signal=SYS) Main PID: 1114 (code=killed, signal=SYS) CPU: 97ms Okt 27 18:09:03 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 5. Okt 27 18:09:03 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:03 myhost systemd[1]: haveged.service: Start request repeated too quickly. Okt 27 18:09:03 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 27 18:09:03 myhost systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm. Zitat root@myhost:~# journalctl -u haveged.service -- Journal begins at Wed 2021-10-27 17:07:54 CEST, ends at Wed 2021-10-27 18:20:09 CEST. -- Okt 27 18:09:00 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:01 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 27 18:09:01 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 27 18:09:01 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 1. Okt 27 18:09:01 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:01 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:02 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 27 18:09:02 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 27 18:09:02 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 2. Okt 27 18:09:02 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:02 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:02 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 27 18:09:02 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 27 18:09:02 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 3. Okt 27 18:09:02 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:02 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:02 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 27 18:09:02 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 27 18:09:03 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 4. Okt 27 18:09:03 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:03 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:03 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 27 18:09:03 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 27 18:09:03 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 5. Okt 27 18:09:03 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 27 18:09:03 myhost systemd[1]: haveged.service: Start request repeated too quickly. Okt 27 18:09:03 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 27 18:09:03 myhost systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm. but it seems I can start the haveged process manually, I see no errors here and the process runs until I abort with keypress CTRL+C root@myhostname:~# /usr/sbin/haveged --Foreground --verbose=1 haveged: command socket is listening at fd 3 haveged starting up haveged: ver: 1.9.14; arch: generic; vend: ; build: (gcc 10.2.1 CTV); collect: 128K haveged: cpu: (VC); data: 16K (D); inst: 16K (D); idx: 12/40; sz: 15006/57790 haveged: tot tests(BA8): A:1/1 B:1/1 continuous tests(B): last entropy estimate 8.00131 haveged: fills: 0, generated: 0 Question 1) Why does the sysstat.service fail in starting at the boot process but works ok when manually initiiated ? Question 2) Why does haveged.service fail at all in starting ? df -h shows plenty of free space available on all my mounts and partitions. I have absolutely no clue where to begin to dig. Any clues? 0 Quote Link to comment Share on other sites More sharing options...
tparys Posted October 28, 2021 Share Posted October 28, 2021 For 2, the haveged service is not terribly reliable on ARM architectures. There are other threads on the subject. Not sure about 1. May be a boot race condition, looking for activity on a cryptographic device that hasn't been started yet? 0 Quote Link to comment Share on other sites More sharing options...
g00d Posted October 28, 2021 Author Share Posted October 28, 2021 Well, I was wrong about the remaining free space in file systems. I did some more research with those services and the logs revealed that they cannot write to /var/log/foo.log df -h shows: udev 925M 0 925M 0% /dev tmpfs 200M 8,0M 192M 4% /run /dev/mapper/rootfs 32G 1,3G 29G 5% / tmpfs 996M 0 996M 0% /dev/shm tmpfs 5,0M 4,0K 5,0M 1% /run/lock tmpfs 996M 0 996M 0% /tmp /dev/zram1 49M 48M 0 100% /var/log tmpfs 200M 0 200M 0% /run/user/1000 what is zram1 and why is it so small (50MB) ? how can I correct this? can you share some information about this strange zram1 filesystem that is mounted to /var/log ? I also see a /var/log.hdd on my system, what's that and what's the difference between those? 0 Quote Link to comment Share on other sites More sharing options...
Werner Posted October 28, 2021 Share Posted October 28, 2021 Because logs are usually plain text which can be compressed easily. But sometimes daemons or the system create an abnormal amount of logs and then the ramdisk runs out of space. Either fix your logs or check /etc/default for armbian-zramconfig or whatever the file is called and increase/disable. 0 Quote Link to comment Share on other sites More sharing options...
g00d Posted October 28, 2021 Author Share Posted October 28, 2021 Hi Werner, I realized already that my armbian install has 1GB swap assigned to zram and I was not clear where this originates from :-) After examining /etc/default/armbian-zram-config I understand that this corresponds to the default value of 50% of my existing RAM (2GB on the ODROID-HC2). Thanks for that information and pointing to that direction. A) I have configured 4GB swap space on my own (hard disk), actually I have 5GB in summary. Do I really need zram-config at all or can I disable it in the mentioned configuration file and what the pro/con ? B) I do not understand yet where the 50MB for my /var/log mount comes from? How can I adjust/extend this size, didn't find anything related to /var/log in /etc/default/armbian-zram-config ? thank you in advance. 0 Quote Link to comment Share on other sites More sharing options...
Werner Posted October 28, 2021 Share Posted October 28, 2021 A. No you can disable it if you want. Pro: More "native" ram and faster speed because ram compression takes a bit of CPU time. Con: Slower swap speed if system needs it. I recommend changing vm.swappiness in /etc/sysctl.conf if you use your external drive as swap to reduce its usage. To something like 25 instead of 100 B. check armbian-ramlog file 0 Quote Link to comment Share on other sites More sharing options...
g00d Posted October 28, 2021 Author Share Posted October 28, 2021 A. thanks for the hint. I did set ENABLE=false in /etc/default/armbian-zram-config and vm.swappiness=25 in /etc/sysctl.conf B. I wasn't able to find helpful information under /var/log.hdd/armbian-ramlog.log but after some more research I found the file /etc/default/armbian-ramlog so I'm pretty sure this is the file you wanted to point me at I increased the default file size of 50MB to 100MB. Now the initial issue with sysstat.service is solved. But haveged.service still fails 0 Quote Link to comment Share on other sites More sharing options...
Werner Posted October 28, 2021 Share Posted October 28, 2021 2 minutes ago, g00d said: /etc/default/armbian-ramlog Yes 0 Quote Link to comment Share on other sites More sharing options...
g00d Posted October 28, 2021 Author Share Posted October 28, 2021 vor 27 Minuten schrieb g00d: Now the initial issue with sysstat.service is solved. But haveged.service still fails /var/log/syslog Oct 28 13:28:12 myhost kernel: [ 1380.296918] audit: type=1326 audit(1635420492.803:57): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2575 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6f57e98 code=0x80000000 Oct 28 13:28:13 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 1. Oct 28 13:28:13 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Oct 28 13:28:13 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Oct 28 13:28:13 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Oct 28 13:28:13 myhost kernel: [ 1380.577192] audit: type=1326 audit(1635420493.083:58): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2584 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6f7de98 code=0x80000000 Oct 28 13:28:13 myhost systemd[1]: haveged.service: Failed with result 'signal'. Oct 28 13:28:13 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 2. Oct 28 13:28:13 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Oct 28 13:28:13 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Oct 28 13:28:13 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Oct 28 13:28:13 myhost systemd[1]: haveged.service: Failed with result 'signal'. Oct 28 13:28:13 myhost kernel: [ 1380.829670] audit: type=1326 audit(1635420493.343:59): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2587 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6f78e98 code=0x80000000 Oct 28 13:28:13 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 3. Oct 28 13:28:13 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Oct 28 13:28:13 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Oct 28 13:28:13 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Oct 28 13:28:13 myhost systemd[1]: haveged.service: Failed with result 'signal'. Oct 28 13:28:13 myhost kernel: [ 1381.079713] audit: type=1326 audit(1635420493.593:60): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2590 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6fbce98 code=0x80000000 Oct 28 13:28:13 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 4. Oct 28 13:28:13 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Oct 28 13:28:13 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Oct 28 13:28:13 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Oct 28 13:28:13 myhost systemd[1]: haveged.service: Failed with result 'signal'. Oct 28 13:28:13 myhost kernel: [ 1381.329773] audit: type=1326 audit(1635420493.843:61): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=2593 comm="haveged" exe="/usr/sbin/haveged" sig=31 arch=40000028 syscall=122 compat=0 ip=0xb6f86e98 code=0x80000000 Oct 28 13:28:14 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 5. Oct 28 13:28:14 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Oct 28 13:28:14 myhost systemd[1]: haveged.service: Start request repeated too quickly. Oct 28 13:28:14 myhost systemd[1]: haveged.service: Failed with result 'signal'. Oct 28 13:28:14 myhost systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm. what is sig=31 and is this related to the kernel ? any clues how to debug the failing haveged.service ? I'm really stuck here :/ -- Boot 7f29c5436e05479595fb657fef877bc8 -- Okt 28 13:05:55 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:55 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 28 13:05:55 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 28 13:05:55 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 1. Okt 28 13:05:56 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:56 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:56 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 28 13:05:56 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 28 13:05:57 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 2. Okt 28 13:05:57 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:57 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:57 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 28 13:05:57 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 28 13:05:57 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 3. Okt 28 13:05:57 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:57 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:57 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 28 13:05:57 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 28 13:05:57 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 4. Okt 28 13:05:57 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:57 myhost systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:57 myhost systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS Okt 28 13:05:57 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 28 13:05:58 myhost systemd[1]: haveged.service: Scheduled restart job, restart counter is at 5. Okt 28 13:05:58 myhost systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm. Okt 28 13:05:58 myhost systemd[1]: haveged.service: Start request repeated too quickly. Okt 28 13:05:58 myhost systemd[1]: haveged.service: Failed with result 'signal'. Okt 28 13:05:58 myhost systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm. 0 Quote Link to comment Share on other sites More sharing options...
Mascaret Posted October 3, 2022 Share Posted October 3, 2022 Hi! I'm probably a little late.. edit haveged.service and comment the 2 lines begining with "SystemCallFilter". See : https://github.com/jirka-h/haveged/issues/63 1 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.