Fault in logrotate functionality?

[eejeel]

I'am running the latest Stretch server version on a PiOne (Linux 4.14.65-sunxi).
I think that something is going wrong with the logrotate. The rotation is done in /var/log.hdd directory (see also the scripts in /etc/logrotate.d/).
But the original file of each is in /var/log directory and is not "cleared" after the rename, because it is not renamed.
So, if Zram copies /var/log directory to the /var/log.hdd directory, the original file in /var/log with old logging and included new logging is written to the /var/log.hdd directory. In the next logrotate this growing file is compressed.
In this situation/in my case the compressed files contains all the loggings and not only the loggings of the last day or week or month, depending which file is compressed. And the file is still growing and growing.

 

Is my conclusion right or is something wrong in my Stretch installation. In the latest case, I could not find it. Please give me a hint.

[dmeey]

I have already adressed this issue with a little demonstrations script in this thread:

 

however, nobody answered yet.

[Markus Rohner-Bührer]

Yes, Same problem here. var/log in ZRAM grows and grows. I see no sense in rotating logs on var/log.hdd

What would be a feasible solution? Change all cron jobs?

[Igor]

2 hours ago, Markus Rohner-Bührer said:

Yes, Same problem here. var/log in ZRAM grows and grows. I see no sense in rotating logs on var/log.hdd

What would be a feasible solution? Change all cron jobs?

It should work this way: https://github.com/armbian/build/blob/master/packages/bsp/common/usr/lib/armbian/armbian-truncate-logs

# write to SD
/usr/lib/armbian/armbian-ramlog write >/dev/null 2>&1

# rotate logs on "disk"
chown root.root -R /var/log.hdd
/usr/sbin/logrotate --force /etc/logrotate.conf

# truncate
/usr/bin/find /var/log -name '*.log' -or -name '*.xz' -or -name 'lastlog' -or -name 'messages' -or -name 'debug' -or -name 'syslog' | xargs truncate --size 0
/usr/bin/find /var/log -name 'btmp' -or -name 'wtmp' -or -name 'faillog' | xargs truncate --size 0

# remove
/usr/bin/find /var/log -name '*.[0-9]' -or -name '*.gz' | xargs rm >/dev/null 2>&1

triggered by cron job:

*/15 * * * * root /usr/lib/armbian/armbian-truncate-logs

Where is the problem?

[DoubleHP]

Previous time I asked where to report bugs:

 

 

New bug:

It's an old bug, met by many people, already reported and fixed by several distros, but, I still had it on a fresh new install:

The problem and the fix:

https://www.raspberrypi.org/forums/viewtopic.php?f=82&t=218609&p=1406567#p1406567

 

known bugs:

https://bugs.launchpad.net/ubuntu/+source/watchdog/+bug/1448924

https://bugzilla.redhat.com/show_bug.cgi?id=1259816

 

# systemctl status watchdog.service
â watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset: enabled)
   Active: inactive (dead)

 

echo "WantedBy=default.target" >> /lib/systemd/system/watchdog.service
systemctl daemon-reload
systemctl enable watchdog
reboot
# wait 2 mn, because it needs to reboot twice.

 

# systemctl status watchdog.service
â watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-12-21 03:02:14 CET; 58s ago
  Process: 1122 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
  Process: 1120 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
 Main PID: 1126 (watchdog)
   CGroup: /system.slice/watchdog.service
           ââ1126 /usr/sbin/watchdog

Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: int=1s realtime=yes sync=no soft=no mla=5 mem=0
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: ping: no machine to check
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: file: /var/log/syslog:0
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: pidfile: no server process to check
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: interface: no interface to check
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: temperature: maximum = 80
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: temperature: /sys/class/thermal/thermal_zone0/temp
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: test=none(0) repair=none(0) alive=/dev/watchdog heartbeat=none to=...ce=no
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: watchdog now set to 10 seconds
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: hardware watchdog identity: sunxi-wdt
Hint: Some lines were ellipsized, use -l to show in full.

 

[Melroy van den Berg]

Quote

 

/usr/sbin/logrotate --force /etc/logrotate.conf

logrotate_script: 2: logrotate_script: /usr/lib/rsyslog.hdd/rsyslog-rotate: not found

error: error running non-shared postrotate script for /var/log.hdd/syslog of '/var/log.hdd/syslog

'

logrotate_script: 2: logrotate_script: /usr/lib/rsyslog.hdd/rsyslog-rotate: not found

error: error running shared postrotate script for '/var/log.hdd/mail.info

/var/log.hdd/mail.warn

/var/log.hdd/mail.err

/var/log.hdd/mail.log

/var/log.hdd/daemon.log

/var/log.hdd/kern.log

/var/log.hdd/auth.log

/var/log.hdd/user.log

/var/log.hdd/lpr.log

/var/log.hdd/cron.log

/var/log.hdd/debug

/var/log.hdd/messages

 

 

[Igor]

You seems to have some old version. Edit /etc/logrotate.d/rsyslog and fix path from /usr/lib/rsyslog.hdd/rsyslog-rotate to /usr/lib/rsyslog/rsyslog-rotate

[Sinan]

After weeks of searching I finally found out why my /var/log folder fills up and armbian log2ram service can not make the expected housekeeping. 

/usr/lib/armbian/armbian-truncate-logs script ONLY truncates files in /var/log folder but not its subfolders. I have samba in my case. I have edited the script and added /var/log/samba as another line to be truncated and problem solved.

 

[Igor]

4 hours ago, Sinan said:

I have solved the problem by manually editing the script. 

Perhaps submit a fix?

https://github.com/armbian/build/tree/master/packages/bsp/common/usr/lib/armbian

[Sinan]

4 hours ago, Igor said:

Perhaps submit a fix?

I am not that much experienced but this is what I added to /usr/lib/armbian/armbian-truncate-logs

 # truncate
    /usr/bin/find /var/log -name '*.log' -or -name '*.xz' -or -name 'lastlog' -or -name 'messages' -or -name 'debug' -or -name 'sysl$
    /usr/bin/find /var/log -name 'btmp' -or -name 'wtmp' -or -name 'faillog' -or -name 'firewalld' | xargs truncate --size 0 >/dev/n$
    /usr/bin/find /var/log -name 'mail.err' -or -name 'mail.info' -or -name 'mail.warning' | xargs truncate --size 0 >/dev/null 2>&1
    # I have added line below
    /usr/bin/find /var/log/samba -name '*.log' -or -name 'log.*' | xargs truncate --size 0 >/dev/null 2>&1
 

[Mark4Arm]

Hi Igor, all,

 

I had issues with the 'remove' line matching on psad's logging of bad IP addresses as folders matching the bad IP address in /var/log/psad

 

The quick fix for me was to add '-type f' to the find commands in /usr/lib/armbian/armbian-truncate-logs to filter to just files.

 

The diff is below:

 

Quote

--- /usr/lib/armbian/armbian-truncate-logs.orig 2021-11-22 22:24:33.697314625 +0000

+++ /usr/lib/armbian/armbian-truncate-logs 2021-11-22 22:21:47.266662399 +0000

@@ -23,11 +23,11 @@

     # rotate logs on "disk"

     /usr/sbin/logrotate --force /etc/logrotate.conf

     # truncate

-    /usr/bin/find /var/log -name '*.log' -or -name '*.xz' -or -name 'lastlog' -or -name 'messages' -or -name 'debug' -or -name 'syslog' | xargs -r truncate --size 0

-    /usr/bin/find /var/log -name 'btmp' -or -name 'wtmp' -or -name 'faillog' -or -name 'firewalld' | xargs -r truncate --size 0

-    /usr/bin/find /var/log -name 'mail.err' -or -name 'mail.info' -or -name 'mail.warning' | xargs -r truncate --size 0

+    /usr/bin/find /var/log -type f -name '*.log' -or -name '*.xz' -or -name 'lastlog' -or -name 'messages' -or -name 'debug' -or -name 'syslog' | xargs -r truncate --size 0

+    /usr/bin/find /var/log -type f -name 'btmp' -or -name 'wtmp' -or -name 'faillog' -or -name 'firewalld' | xargs -r truncate --size 0

+    /usr/bin/find /var/log -type f -name 'mail.err' -or -name 'mail.info' -or -name 'mail.warning' | xargs -r truncate --size 0

     # remove

-    /usr/bin/find /var/log -name '*.[0-9]' -or -name '*.gz' | xargs -r rm -f

+    /usr/bin/find /var/log -type f -name '*.[0-9]' -or -name '*.gz' | xargs -r rm -f

     # vacuum systemd-journald

     [ -d /var/log/journal ] && journalctl --vacuum-size=${JOURNAL_SIZE}

fi

 

The trivial diff is free for anyone to use (if anyone was worried about licensing).

 

Cheers,

 

Mark