logrotate - rotating of /var/log.hdd is problematic

[dmeey]

First time armbian user and poster here. Since this is not the right time and place for an introduction, lets keep it with a simple "Hi all!" and get straight to the topic:

 

for completeness, my system:

BOARD=orangepizeroplus
BOARD_NAME="Orange Pi Zero Plus"
BOARDFAMILY=sun50iw2
VERSION=5.59
LINUXFAMILY=sunxi64
BRANCH=next
ARCH=arm64
IMAGE_TYPE=testing
BOARD_TYPE=conf
INITRD_ARCH=arm64
KERNEL_IMAGE_TYPE=Image

logging, in particular logrotate with ram-logging

As far as I understand:

- Three different cron files are responsible to handle ram-logging and logrotate

-- /etc/cron.daily/logrotate

-- /etc/cron.daily/armbian-ram-logging

-- /etc/cron.d/armbian-truncate-logs

- The configuration files for logrotate follow an agreement, that rotation is performed on the "cold" log files in log.hdd

 

Now while armbian-truncate-logs seems to do the right thing(i.e. truncating the "hot" log files in /var/log after rotation),  I have an impression that the consequences of a regular call to logrotate(as found in /etc/cron.daily/logrotate) have not been thought through. Since the hot log is not truncated in this case, it's contents get synced back by the next call to armbian-ram-logging.

 

I have created a minimal working example to demonstrate my point:

armbian-logrotate-test

#!/bin/bash
LOGROTATE_CONF=armbian-logrotate-test.conf
LOGFILE=armbian-logrotate-test.log

counter=0

# fn
log_logline () {
    echo "logline ${counter}" >> /var/log/${LOGFILE}
    counter=$((counter+1))
}

log_rotate () {
    chown root.root ${LOGROTATE_CONF}
    /usr/sbin/logrotate --force ${LOGROTATE_CONF}
}

log_ramlog () {
    /usr/lib/armbian/armbian-ramlog write >/dev/null 2>&1
}

log_cleanup () {
    rm -f /var/log/${LOGFILE}*
    rm -f /var/log.hdd/${LOGFILE}*
}

log_status () {
    echo "### /var/log.hdd ###"
    find /var/log.hdd -type f -name "${LOGFILE}*" -exec echo {} \; -exec cat {}  \;
    echo "### /var/log ###"
    find /var/log -type f -name "${LOGFILE}*" -exec echo {} \; -exec cat {} \;
}

# main
log_cleanup
# - logging scenario
log_logline
log_logline
log_ramlog
log_rotate
log_logline
log_logline
log_ramlog
log_rotate
log_logline
log_logline
# - output
log_status
log_cleanup

with

armbian-logrotate-test.conf

/var/log.hdd/armbian-logrotate-test.log
{
        rotate 10
        daily
        missingok
        notifempty
}

after calling the script as root the following output is obtained:

### /var/log.hdd ###
/var/log.hdd/armbian-logrotate-test.log.1
logline 0
logline 1
logline 2
logline 3
/var/log.hdd/armbian-logrotate-test.log.2
logline 0
logline 1
### /var/log ###
/var/log/armbian-logrotate-test.log
logline 0
logline 1
logline 2
logline 3
logline 4
logline 5

 

Has this been overlooked or do I miss something out in my test scenario?

Edited September 1, 2018 by dmeey
added system info

[Tido]

On 9/1/2018 at 4:01 PM, dmeey said:

Has this been overlooked or do I miss something out in my test scenario?

in here you find links to threads in which such things were discussed: https://forum.armbian.com/topic/7503-we-need-your-help/

[dmeey]

Thank you for your reply.

 

I followed all(hopefully) links in the thread you provided, but could not find any information regarding the issue which I put on the table here.

 

To me it seems that all development and discussion regarding log rotation has been focused on proper synchronization to/from zram and armbian-truncate-logs - which works great in the way it is implemented now.

 

However, at the same time the proposed solution breaks ordinary logrotate operation, which is hard to spot in the first place, since armbian-truncate-logs will take care of any resulting space issues. However, the archieved log files will be a mess with duplicated entries as demonstrated by my little script and also observed and described by @eejeel in