Jump to content

Fault in logrotate functionality?


eejeel

Recommended Posts

I'am running the latest Stretch server version on a PiOne (Linux 4.14.65-sunxi).
I think that something is going wrong with the logrotate. The rotation is done in /var/log.hdd directory (see also the scripts in /etc/logrotate.d/).
But the original file of each is in /var/log directory and is not "cleared" after the rename, because it is not renamed.
So, if Zram copies /var/log directory to the /var/log.hdd directory, the original file in /var/log with old logging and included new logging is written to the /var/log.hdd directory. In the next logrotate this growing file is compressed.
In this situation/in my case the compressed files contains all the loggings and not only the loggings of the last day or week or month, depending which file is compressed. And the file is still growing and growing.

 

Is my conclusion right or is something wrong in my Stretch installation. In the latest case, I could not find it. Please give me a hint.

Link to comment
Share on other sites

2 hours ago, Markus Rohner-Bührer said:

Yes, Same problem here. var/log in ZRAM grows and grows. I see no sense in rotating logs on var/log.hdd

What would be a feasible solution? Change all cron jobs?


It should work this way: https://github.com/armbian/build/blob/master/packages/bsp/common/usr/lib/armbian/armbian-truncate-logs

# write to SD
/usr/lib/armbian/armbian-ramlog write >/dev/null 2>&1

# rotate logs on "disk"
chown root.root -R /var/log.hdd
/usr/sbin/logrotate --force /etc/logrotate.conf

# truncate
/usr/bin/find /var/log -name '*.log' -or -name '*.xz' -or -name 'lastlog' -or -name 'messages' -or -name 'debug' -or -name 'syslog' | xargs truncate --size 0
/usr/bin/find /var/log -name 'btmp' -or -name 'wtmp' -or -name 'faillog' | xargs truncate --size 0

# remove
/usr/bin/find /var/log -name '*.[0-9]' -or -name '*.gz' | xargs rm >/dev/null 2>&1

triggered by cron job:

*/15 * * * * root /usr/lib/armbian/armbian-truncate-logs

Where is the problem?

Link to comment
Share on other sites

Previous time I asked where to report bugs:

 

 

New bug:

It's an old bug, met by many people, already reported and fixed by several distros, but, I still had it on a fresh new install:

The problem and the fix:

https://www.raspberrypi.org/forums/viewtopic.php?f=82&t=218609&p=1406567#p1406567

 

known bugs:

https://bugs.launchpad.net/ubuntu/+source/watchdog/+bug/1448924

https://bugzilla.redhat.com/show_bug.cgi?id=1259816

 

# systemctl status watchdog.service
â watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset: enabled)
   Active: inactive (dead)


 

echo "WantedBy=default.target" >> /lib/systemd/system/watchdog.service
systemctl daemon-reload
systemctl enable watchdog
reboot
# wait 2 mn, because it needs to reboot twice.


 

# systemctl status watchdog.service
â watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-12-21 03:02:14 CET; 58s ago
  Process: 1122 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
  Process: 1120 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
 Main PID: 1126 (watchdog)
   CGroup: /system.slice/watchdog.service
           ââ1126 /usr/sbin/watchdog

Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: int=1s realtime=yes sync=no soft=no mla=5 mem=0
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: ping: no machine to check
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: file: /var/log/syslog:0
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: pidfile: no server process to check
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: interface: no interface to check
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: temperature: maximum = 80
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: temperature: /sys/class/thermal/thermal_zone0/temp
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: test=none(0) repair=none(0) alive=/dev/watchdog heartbeat=none to=...ce=no
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: watchdog now set to 10 seconds
Dec 21 03:02:14 opi-06-app-c13 watchdog[1126]: hardware watchdog identity: sunxi-wdt
Hint: Some lines were ellipsized, use -l to show in full.

 

Link to comment
Share on other sites

Quote

 

/usr/sbin/logrotate --force /etc/logrotate.conf

logrotate_script: 2: logrotate_script: /usr/lib/rsyslog.hdd/rsyslog-rotate: not found

error: error running non-shared postrotate script for /var/log.hdd/syslog of '/var/log.hdd/syslog

'

logrotate_script: 2: logrotate_script: /usr/lib/rsyslog.hdd/rsyslog-rotate: not found

error: error running shared postrotate script for '/var/log.hdd/mail.info

/var/log.hdd/mail.warn

/var/log.hdd/mail.err

/var/log.hdd/mail.log

/var/log.hdd/daemon.log

/var/log.hdd/kern.log

/var/log.hdd/auth.log

/var/log.hdd/user.log

/var/log.hdd/lpr.log

/var/log.hdd/cron.log

/var/log.hdd/debug

/var/log.hdd/messages

 

 

Link to comment
Share on other sites

You seems to have some old version. Edit /etc/logrotate.d/rsyslog and fix path from /usr/lib/rsyslog.hdd/rsyslog-rotate to /usr/lib/rsyslog/rsyslog-rotate

Link to comment
Share on other sites

After weeks of searching I finally found out why my /var/log folder fills up and armbian log2ram service can not make the expected housekeeping. 

/usr/lib/armbian/armbian-truncate-logs script ONLY truncates files in /var/log folder but not its subfolders. I have samba in my case. I have edited the script and added /var/log/samba as another line to be truncated and problem solved.

 

Link to comment
Share on other sites

4 hours ago, Igor said:

Perhaps submit a fix?

I am not that much experienced but this is what I added to /usr/lib/armbian/armbian-truncate-logs

 # truncate
    /usr/bin/find /var/log -name '*.log' -or -name '*.xz' -or -name 'lastlog' -or -name 'messages' -or -name 'debug' -or -name 'sysl$
    /usr/bin/find /var/log -name 'btmp' -or -name 'wtmp' -or -name 'faillog' -or -name 'firewalld' | xargs truncate --size 0 >/dev/n$
    /usr/bin/find /var/log -name 'mail.err' -or -name 'mail.info' -or -name 'mail.warning' | xargs truncate --size 0 >/dev/null 2>&1
    # I have added line below
    /usr/bin/find /var/log/samba -name '*.log' -or -name 'log.*' | xargs truncate --size 0 >/dev/null 2>&1
 

Link to comment
Share on other sites

Hi Igor, all,

 

I had issues with the 'remove' line matching on psad's logging of bad IP addresses as folders matching the bad IP address in /var/log/psad

 

The quick fix for me was to add '-type f' to the find commands in /usr/lib/armbian/armbian-truncate-logs to filter to just files.

 

The diff is below:

 

Quote

--- /usr/lib/armbian/armbian-truncate-logs.orig 2021-11-22 22:24:33.697314625 +0000

+++ /usr/lib/armbian/armbian-truncate-logs 2021-11-22 22:21:47.266662399 +0000

@@ -23,11 +23,11 @@

     # rotate logs on "disk"

     /usr/sbin/logrotate --force /etc/logrotate.conf

     # truncate

-    /usr/bin/find /var/log -name '*.log' -or -name '*.xz' -or -name 'lastlog' -or -name 'messages' -or -name 'debug' -or -name 'syslog' | xargs -r truncate --size 0

-    /usr/bin/find /var/log -name 'btmp' -or -name 'wtmp' -or -name 'faillog' -or -name 'firewalld' | xargs -r truncate --size 0

-    /usr/bin/find /var/log -name 'mail.err' -or -name 'mail.info' -or -name 'mail.warning' | xargs -r truncate --size 0

+    /usr/bin/find /var/log -type f -name '*.log' -or -name '*.xz' -or -name 'lastlog' -or -name 'messages' -or -name 'debug' -or -name 'syslog' | xargs -r truncate --size 0

+    /usr/bin/find /var/log -type f -name 'btmp' -or -name 'wtmp' -or -name 'faillog' -or -name 'firewalld' | xargs -r truncate --size 0

+    /usr/bin/find /var/log -type f -name 'mail.err' -or -name 'mail.info' -or -name 'mail.warning' | xargs -r truncate --size 0

     # remove

-    /usr/bin/find /var/log -name '*.[0-9]' -or -name '*.gz' | xargs -r rm -f

+    /usr/bin/find /var/log -type f -name '*.[0-9]' -or -name '*.gz' | xargs -r rm -f

     # vacuum systemd-journald

     [ -d /var/log/journal ] && journalctl --vacuum-size=${JOURNAL_SIZE}

fi

 

The trivial diff is free for anyone to use (if anyone was worried about licensing).

 

Cheers,

 

Mark

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines