Don Pedro Posted March 6, 2018 Posted March 6, 2018 Hello! This thread here: https://forum.armbian.com/topic/3728-varlog-varloghdd/ triggered the question in me how I could extend the log2ram. My problem is the following: I have installed syncthing. The default in armbian runs it as root - which is not recommended for syncthing, see here: https://forum.syncthing.net/t/running-syncthing-as-root/6316! So I downgraded the syncthing daemon to a standard user account "syncthing". In order to enable a non-root user to write logfiles it's common practice to create a subdir /var/log/syncthing/ and give it write rights for syncthing. This works, however my logfiles stay completely empty! Nevertheless I can see that something's constantly touching the logfiles, the last modified date is always set to the current time. I'm assuming this is due to the log2ram mechanism (is that assumption correct?), so I'd like to know how to extend that to include my added log files and log subdirectories. THX Don
Don Pedro Posted March 8, 2018 Author Posted March 8, 2018 I found the cause of the problem myself! Syncthing does write a lot of log entries, depending of what it does. So it happens very easily that it floods the small 40M partition of log2ram. Things get totally stuck then, as with a 100% filled partition logrotate fails as well: Quote error: Compressing program wrote following message to stderr when compressing log /var/log/syslog.1: gzip: stdout: No space left on device error: failed to compress log /var/log/syslog.1 It cannot compress logfiles anymore and bails out. You end up with a clogged logging and a failed log rotation, stuck eternally without manual interaction. The solution here is: Manually delete two or three of the oldest or biggest log files Enforce a rotation: logrotate /etc/logrotate.conf -f Edit or create a /etc/logrotate.d/synthing and set it to daily rotate and a maxsize of let's say 10M Then during the next couple of days check how the log size develops, if necessary rotate more often and/or limit the logfiles to smaller sizes. HTH Don
tkaiser Posted March 9, 2018 Posted March 9, 2018 11 hours ago, Don Pedro said: Syncthing does write a lot of log entries, depending of what it does So a good idea is to check whether this software is running with debug/developer options by accident since I can't imagine why a software creates that absurd amount of log data: https://docs.syncthing.net/dev/debugging.html Can you check with which logging/debug switches the software runs by default and report back?
tkaiser Posted March 9, 2018 Posted March 9, 2018 On 6.3.2018 at 11:38 PM, Don Pedro said: I can see that something's constantly touching the logfiles, the last modified date is always set to the current time. I'm assuming this is due to the log2ram mechanism (is that assumption correct?) I don't think so. The log2ram implementation uses /var/log.hdd on disk and syncs the contents to a tmpfs mount /var/log in memory at boot and syncs back at shutdown time (and once every day via cron). There's no activity generated other than 1) sync at boot, 2) sync at shutdown, 3) daily sync from DRAM to 'disk'. Everything else is the result of other daemons/tasks doing what they do (eg. rsyslogd and logrotate). Wrt syncthing running as root... I guess you installed it via armbian-config? If so please file a bug report over at https://github.com/armbian/config/issues
Don Pedro Posted March 9, 2018 Author Posted March 9, 2018 14 hours ago, tkaiser said: So a good idea is to check whether this software is running with debug/developer options by accident since I can't imagine why a software creates that absurd amount of log data: https://docs.syncthing.net/dev/debugging.html Can you check with which logging/debug switches the software runs by default and report back? I can check that, sure: Syncthing has no debug option set and also no verbose switch. So I don't have a problem with its chattiness, it's just possible that it does a lot of things and thus generating a lot of log entries. But as already mailed, I found the source of the problem . THX anyway!
Don Pedro Posted March 9, 2018 Author Posted March 9, 2018 14 hours ago, tkaiser said: Wrt syncthing running as root... I guess you installed it via armbian-config? If so please file a bug report over at https://github.com/armbian/config/issues Yep, I did it via arbian-config, correct. Running it as root comes from the installer there. I'll open an issue on github, sure.
Don Pedro Posted March 9, 2018 Author Posted March 9, 2018 4 minutes ago, Don Pedro said: Yep, I did it via arbian-config, correct. Running it as root comes from the installer there. I'll open an issue on github, sure. Done! https://github.com/armbian/config/issues/18
Recommended Posts