gas_85 Posted October 15, 2018 Posted October 15, 2018 System Information: - Cubietruck with Ubuntu 16.04 - armbian-config 5.62 - armbian-firmware 5.60 - armbian-tools-xenial 5.60 Hallo since last few updated I've noticed that I have no logs... With some update armbian ram logging was enabled and all logs moved to log.hdd folder, but not all of them are filled up after this operation. I did disabled armbian-ram-loggin wia editing /etc/default/armbian-ramlog: ENABLED=false After this services was disabled, but no logs filled in a log folder... Also no log rotation happens. Reveal hidden contents # /var/log/ls -lah total 116K drwxrwxr-x 13 root syslog 4.0K Oct 15 06:30 . drwxr-xr-x 16 root root 4.0K May 4 13:42 .. -rw-r--r-- 1 root root 0 Oct 15 14:45 alternatives.log drwxr-x--- 2 root adm 4.0K Oct 15 06:30 apache2 drwxr-xr-x 2 root root 4.0K Sep 21 20:00 apt -rw-rw-r-- 1 vpn sambashare 0 Oct 15 14:45 aria2.log -rw-r--r-- 1 root root 0 Oct 15 14:45 armbian-hardware-monitor.log -rw-r--r-- 1 root root 0 Oct 15 14:45 armbian-ramlog.log -rw-r--r-- 1 root root 0 Oct 15 14:45 armhwinfo.log -rw-r----- 1 syslog adm 1.3K Oct 15 14:55 auth.log -rw-r--r-- 1 root root 0 Oct 15 14:45 backup_2f695.log -rw-r--r-- 1 root root 0 Oct 15 14:45 battery_checker.log -rw-r--r-- 1 root root 0 Oct 15 14:45 bootstrap.log -rw------- 1 root utmp 0 Oct 15 14:45 btmp drwxr-xr-x 2 www-data www-data 4.0K Oct 14 06:30 cacti drwxr-xr-x 2 root root 4.0K Sep 21 20:00 dbconfig-common -rw-r----- 1 root adm 31 Feb 5 2017 dmesg -rw-r--r-- 1 root root 0 Oct 15 14:45 dpkg.log -rw-r----- 1 root adm 237 Oct 15 14:48 fail2ban.log -rw-r--r-- 1 root root 0 Oct 15 14:45 faillog -rw-r--r-- 1 root root 0 Oct 15 14:45 fontconfig.log drwxr-xr-x 2 root root 4.0K Feb 5 2017 fsck -rw-r----- 1 syslog adm 0 Oct 15 14:45 kern.log -rw-r--r-- 1 root root 13 Oct 15 00:01 last_ip -rw-rw-r-- 1 root utmp 0 Oct 15 14:45 lastlog drwx------ 2 root root 4.0K Oct 15 03:18 letsencrypt -rw-r--r-- 1 root root 0 Oct 15 14:45 log2ram.log -rw-r----- 1 syslog adm 18K Oct 15 14:45 mail.err -rw-r----- 1 syslog adm 1.4K Oct 15 14:45 mail.log -rw-r--r-- 1 root root 24 Feb 27 2018 md5sum -rw-r--r-- 1 minidlna minidlna 0 Oct 15 14:45 minidlna.log drwxr-s--- 2 mysql adm 4.0K Oct 6 06:30 mysql -rw-r--r-- 1 root root 0 Oct 15 14:45 nand-sata-install.log -rw-r--r-- 1 www-data www-data 0 Oct 15 14:45 next-cron.log lrwxrwxrwx 1 root root 35 Mar 5 2018 nextcloud.log -> ../www/nextcloud/data/nextcloud.log -rw-r--r-- 1 root root 0 Oct 15 14:45 noip.log drwxr-xr-x 2 ntp ntp 4.0K Oct 5 2016 ntpstats -rw------- 1 root root 749 Oct 15 14:55 php7.0-fpm.log -rw-r--r-- 1 syslog syslog 3.7K Oct 15 14:55 remote.log drwxr-x--- 3 root adm 4.0K Oct 7 06:30 samba -rw-rw-rw- 1 root root 3.8K Oct 15 14:58 socks.log -rw-r----- 1 syslog adm 2.5K Oct 15 14:55 syslog drwxr-xr-x 2 root root 4.0K May 12 2016 sysstat -rw-r--r-- 1 root root 0 Oct 15 14:45 systembackup.log drwxr-x--- 2 root adm 4.0K Sep 21 20:00 unattended-upgrades -rw-rw-r-- 1 root utmp 0 Oct 15 14:45 wtmp # ls -lah ../log.hdd total 5.0M drwxrwxr-x 13 root root 4.0K Oct 5 06:30 . drwxr-xr-x 16 root root 4.0K May 4 13:42 .. -rw-r--r-- 1 root root 0 Oct 5 09:45 alternatives.log -rw-r--r-- 1 root root 1.3K Aug 31 06:31 alternatives.log.1 -rw-r--r-- 1 root root 198 Jul 31 06:22 alternatives.log.2.gz -rw-r--r-- 1 root root 177 Jun 15 09:13 alternatives.log.3.gz -rw-r--r-- 1 root root 206 May 24 06:42 alternatives.log.4.gz -rw-r--r-- 1 root root 252 Apr 27 08:30 alternatives.log.5.gz -rw-r--r-- 1 root root 533 Mar 20 2018 alternatives.log.6.gz -rw-r--r-- 1 root root 377 Feb 16 2018 alternatives.log.7.gz -rw-r--r-- 1 root root 2.8K Jan 23 2018 alternatives.log.8.gz drwxr-x--- 2 root root 4.0K Oct 5 06:30 apache2 drwxr-xr-x 2 root root 4.0K Sep 21 20:00 apt -rw-rw-r-- 1 root root 0 Oct 5 09:45 aria2.log -rw-rw-r-- 1 root root 283 Oct 1 06:25 aria2.log.1.gz -rw-rw-r-- 1 root root 5.2K Aug 1 06:25 aria2.log.2.gz -rw-rw-r-- 1 root root 28K Jul 1 06:25 aria2.log.3.gz -rw-r--r-- 1 root root 0 Oct 5 09:45 armbian-hardware-monitor.log -rw-r--r-- 1 root root 516K Oct 5 10:00 armbian-ramlog.log -rw-r--r-- 1 root root 0 Oct 5 09:45 armhwinfo.log -rw-r--r-- 1 root root 82K Jul 6 01:23 armhwinfo.log.1.gz -rw-r--r-- 1 root root 73K Jan 27 2018 armhwinfo.log.10.gz -rw-r--r-- 1 root root 28K Jun 12 08:40 armhwinfo.log.2.gz -rw-r--r-- 1 root root 14K May 14 02:17 armhwinfo.log.3.gz -rw-r--r-- 1 root root 44K Apr 27 21:37 armhwinfo.log.4.gz -rw-r--r-- 1 root root 16K Apr 15 2018 armhwinfo.log.5.gz -rw-r--r-- 1 root root 16K Mar 18 2018 armhwinfo.log.6.gz -rw-r--r-- 1 root root 31K Mar 1 2018 armhwinfo.log.7.gz -rw-r--r-- 1 root root 16K Feb 26 2018 armhwinfo.log.8.gz -rw-r--r-- 1 root root 16K Jan 30 2018 armhwinfo.log.9.gz -rw-r----- 1 root root 2.0K Oct 5 10:00 auth.log -rw-r----- 1 root root 19K Oct 1 06:25 auth.log.1 -rw-r----- 1 root root 0 Feb 25 2018 auth.log.1.gz -rw-r----- 1 root root 77K Sep 9 06:25 auth.log.2.gz -rw-r----- 1 root root 66K Sep 2 06:25 auth.log.3.gz -rw-r----- 1 root root 90K Aug 27 06:25 auth.log.4.gz -rw-r--r-- 1 root root 0 Oct 5 09:45 backup_2f695.log -rw-r--r-- 1 root root 0 Oct 5 09:45 battery_checker.log -rw-r--r-- 1 root root 126 Oct 5 06:17 battery_checker.log.1.gz -rw-r--r-- 1 root root 4.3K Aug 1 06:17 battery_checker.log.2.gz -rw-r--r-- 1 root root 1.8K Jul 1 06:17 battery_checker.log.3.gz -rw-r--r-- 1 root root 0 Oct 5 09:45 bootstrap.log -rw------- 1 root root 0 Oct 5 09:45 btmp -rw------- 1 root root 60 Apr 10 2018 btmp.1.gz drwxr-xr-x 2 root root 4.0K Oct 5 06:30 cacti drwxr-xr-x 2 root root 4.0K Sep 21 20:00 dbconfig-common -rw-r----- 1 root root 31 Feb 5 2017 dmesg -rw-r--r-- 1 root root 0 Oct 5 09:45 dpkg.log -rw-r--r-- 1 root root 42K Aug 31 06:31 dpkg.log.1 -rw-r--r-- 1 root root 6.9K Jul 31 06:22 dpkg.log.2.gz -rw-r--r-- 1 root root 4.1K Jun 28 06:36 dpkg.log.3.gz -rw-r--r-- 1 root root 4.2K May 29 08:55 dpkg.log.4.gz -rw-r--r-- 1 root root 5.7K Apr 27 11:24 dpkg.log.5.gz -rw-r--r-- 1 root root 8.4K Mar 29 2018 dpkg.log.6.gz -rw-r--r-- 1 root root 13K Feb 27 2018 dpkg.log.7.gz -rw-r--r-- 1 root root 57K Jan 30 2018 dpkg.log.8.gz -rw-r----- 1 root root 0 Oct 5 09:45 fail2ban.log -rw-r----- 1 root root 110 Oct 5 06:25 fail2ban.log.1 -rw-r----- 1 root root 2.6K Sep 9 00:32 fail2ban.log.2.gz -rw-r----- 1 root root 2.3K Sep 2 00:48 fail2ban.log.3.gz -rw-r----- 1 root root 2.9K Aug 27 00:58 fail2ban.log.4.gz -rw-r--r-- 1 root root 0 Oct 5 09:45 faillog -rw-r--r-- 1 root root 0 Oct 5 09:45 fontconfig.log drwxr-xr-x 2 root root 4.0K Feb 5 2017 fsck -rw-r----- 1 root root 0 Oct 5 09:45 kern.log -rw-r----- 1 root root 5.7K Sep 30 00:06 kern.log.1 -rw-r----- 1 root root 2.6K Sep 9 02:22 kern.log.2.gz -rw-r----- 1 root root 1.8K Sep 2 00:48 kern.log.3.gz -rw-r----- 1 root root 2.4K Aug 27 00:58 kern.log.4.gz -rw-r--r-- 1 root root 13 Oct 5 00:33 last_ip -rw-rw-r-- 1 root root 0 Oct 5 09:45 lastlog drwx------ 2 root root 4.0K Oct 4 20:15 letsencrypt -rw-r--r-- 1 root root 0 Oct 5 09:45 log2ram.log -rw-r----- 1 root root 10K Oct 5 09:45 mail.err -rw-r----- 1 root root 51K Oct 1 04:15 mail.err.1 -rw-r----- 1 root root 0 Feb 25 2018 mail.err.1.gz -rw-r----- 1 root root 607 Sep 9 02:55 mail.err.2.gz -rw-r----- 1 root root 603 Sep 2 02:44 mail.err.3.gz -rw-r----- 1 root root 576 Aug 27 02:56 mail.err.4.gz -rw-r----- 1 root root 1.4K Oct 5 09:45 mail.log -rw-r----- 1 root root 1.4K Oct 1 04:15 mail.log.1 -rw-r----- 1 root root 0 Feb 25 2018 mail.log.1.gz -rw-r----- 1 root root 5.1K Sep 9 02:55 mail.log.2.gz -rw-r----- 1 root root 5.5K Sep 2 02:44 mail.log.3.gz -rw-r----- 1 root root 5.2K Aug 27 02:56 mail.log.4.gz -rw-r--r-- 1 root root 24 Feb 27 2018 md5sum -rw-r--r-- 1 root root 0 Oct 5 09:45 minidlna.log -rw-r--r-- 1 root root 1013 Sep 29 06:27 minidlna.log.1 -rw-r--r-- 1 root root 926 Sep 9 06:25 minidlna.log.2.gz -rw-r--r-- 1 root root 1.1K Sep 2 06:25 minidlna.log.3.gz -rw-r--r-- 1 root root 17K Aug 27 06:25 minidlna.log.4.gz drwxr-s--- 2 root root 4.0K Oct 5 06:30 mysql -rw-r--r-- 1 root root 0 Oct 5 09:45 nand-sata-install.log -rw-r--r-- 1 root root 0 Oct 5 09:45 next-cron.log -rw-r--r-- 1 root root 2.7K Aug 22 12:13 next-cron.log.1.gz -rw-r--r-- 1 root root 3.4K Aug 1 03:56 next-cron.log.2.gz -rw-r--r-- 1 root root 9.0K Jul 1 04:05 next-cron.log.3.gz lrwxrwxrwx 1 root root 35 Mar 5 2018 nextcloud.log -> ../www/nextcloud/data/nextcloud.log -rw-r----- 1 root root 234K Aug 24 06:19 nextcloud.log.1.gz -rw-r----- 1 root root 264K Aug 16 06:15 nextcloud.log.2.gz -rw-r----- 1 root root 186K Aug 8 06:20 nextcloud.log.3.gz -rw-r----- 1 root root 263K Aug 4 06:19 nextcloud.log.4.gz -rw-r----- 1 root root 279K Jul 27 06:22 nextcloud.log.5.gz -rw-r----- 1 root root 270K Jul 18 06:21 nextcloud.log.6.gz -rw-r----- 1 root root 265K Jul 8 05:48 nextcloud.log.7.gz -rw-r----- 1 root root 253K Jun 28 06:20 nextcloud.log.8.gz -rw-r--r-- 1 root root 0 Oct 5 09:45 noip.log -rw-r--r-- 1 root root 105 Oct 2 00:46 noip.log.1.gz -rw-r--r-- 1 root root 303 Aug 1 00:58 noip.log.2.gz -rw-r--r-- 1 root root 1.2K Jul 1 00:32 noip.log.3.gz drwxr-xr-x 2 root root 4.0K Oct 5 2016 ntpstats -rw------- 1 root root 0 Oct 5 09:45 php7.0-fpm.log -rw------- 1 root root 321 Sep 29 20:56 php7.0-fpm.log.1 -rw------- 1 root root 0 Feb 25 2018 php7.0-fpm.log.1.gz -rw------- 1 root root 277 Jul 14 23:35 php7.0-fpm.log.10.gz -rw------- 1 root root 619 Jul 6 20:55 php7.0-fpm.log.11.gz -rw------- 1 root root 364 Jul 1 21:58 php7.0-fpm.log.12.gz -rw------- 1 root root 440 Sep 8 20:47 php7.0-fpm.log.2.gz -rw------- 1 root root 531 Sep 1 10:53 php7.0-fpm.log.3.gz -rw------- 1 root root 912 Aug 26 21:14 php7.0-fpm.log.4.gz -rw------- 1 root root 560 Aug 18 22:12 php7.0-fpm.log.5.gz -rw------- 1 root root 1.1K Aug 12 18:45 php7.0-fpm.log.6.gz -rw------- 1 root root 663 Aug 5 04:02 php7.0-fpm.log.7.gz -rw------- 1 root root 920 Jul 28 21:41 php7.0-fpm.log.8.gz -rw------- 1 root root 653 Jul 22 14:08 php7.0-fpm.log.9.gz -rw-r--r-- 1 root root 6.4K Oct 5 10:00 remote.log -rw-r--r-- 1 root root 48K Oct 1 06:25 remote.log.1 -rw-r--r-- 1 root root 192K Sep 9 06:25 remote.log.2.gz -rw-r--r-- 1 root root 186K Sep 2 06:25 remote.log.3.gz -rw-r--r-- 1 root root 325K Aug 27 06:25 remote.log.4.gz drwxr-x--- 3 root root 4.0K Oct 5 06:30 samba -rw-rw-rw- 1 root root 0 Oct 5 09:45 socks.log -rw-rw-rw- 1 root root 3.0K Oct 5 06:30 socks.log.1 -rw-rw-rw- 1 root root 76K Sep 10 16:40 socks.log.2.gz -rw-rw-rw- 1 root root 53K Sep 3 06:25 socks.log.3.gz -rw-rw-rw- 1 root root 47K Aug 28 06:29 socks.log.4.gz -rw-rw-rw- 1 root root 42K Aug 20 06:25 socks.log.5.gz -rw-r----- 1 root root 4.5K Oct 5 10:00 syslog -rw-r----- 1 root root 4.5K Oct 5 06:25 syslog.1 -rw-r----- 1 root root 0 Feb 26 2018 syslog.1.gz -rw-r----- 1 root root 0 Feb 24 2018 syslog.1.gz-2018022506.backup -rw-r----- 1 root root 0 Feb 25 2018 syslog.1.gz-2018022606.backup -rw-r----- 1 root root 0 Feb 26 2018 syslog.1.gz-2018022706.backup -rw-r----- 1 root root 18K Oct 3 06:25 syslog.2.gz -rw-r----- 1 root root 29K Oct 2 06:25 syslog.3.gz -rw-r----- 1 root root 3.3K Oct 1 06:25 syslog.4.gz -rw-r----- 1 root root 42K Sep 30 06:25 syslog.5.gz -rw-r----- 1 root root 17K Sep 29 06:25 syslog.6.gz -rw-r----- 1 root root 67K Sep 28 06:25 syslog.7.gz drwxr-xr-x 2 root root 4.0K May 12 2016 sysstat -rw-r--r-- 1 root root 0 Oct 5 09:45 systembackup.log -rw-r--r-- 1 root root 67K Sep 1 00:58 systembackup.log.1.gz -rw-r--r-- 1 root root 126 Aug 1 00:00 systembackup.log.2.gz -rw-r--r-- 1 root root 71K Jul 1 00:58 systembackup.log.3.gz drwxr-x--- 2 root root 4.0K Sep 21 20:00 unattended-upgrades -rw-rw-r-- 1 root root 0 Oct 5 09:45 wtmp -rw-rw-r-- 1 root root 538 Aug 26 13:10 wtmp.1.gz
gas_85 Posted October 16, 2018 Author Posted October 16, 2018 (edited) Long story short, to disable arbain-ramlog: 1. Change in /etc/default/armbian-ramlog ENABLED=false 2. Disable in cron root user truncate-logs by editing /etc/cron.d/armbian-truncate-logs (just comment the line) like this: # 0,15,30,45 * * * * root /usr/lib/armbian/armbian-truncate-logs 3. Delete /etc/cron.daily/armbian-ram-logging or comment it like: #!/bin/sh #/usr/lib/armbian/armbian-ramlog write >/dev/null 2>&1 4. Restart Server. Because armbian-zram is connected to ramlog service and will read config of it. It will not create compressed in-memory folder for logs. Without restart it will not be used properly and you will loose your logs. Edited November 2, 2018 by gas_85 Added Step 3, as sxf2000 mentioned.
sfx2000 Posted October 25, 2018 Posted October 25, 2018 On 10/16/2018 at 11:42 AM, gas_85 said: Long story short, Expand Long story short - it's a chained series of events that gets one to here... in systemd... (nice that these are in systemd - I have no skin in the whole systemd discussion other than to recognize that it's there) armbian-ramlog.service armbian-zram-config.service One can disable them easily enough by just doing systemctl... the crontab tip is good... To get zram back - install zram-config, and for the log stuff once the ramlog is disable (along with the crontab entry), look at logrotate... 1
sfx2000 Posted October 25, 2018 Posted October 25, 2018 On 10/16/2018 at 11:42 AM, gas_85 said: Disable in cron root user truncate-logs by editing /etc/cron.d/armbian-truncate-logs (just comment the line) Expand Take a look at /etc/cron.daily/armbian-ram-logging - might want to comment that line as well... In any event - the services and crontab entries for ramlogging at well intentioned to keep pressure off the flash, so step carefully... 1
gas_85 Posted November 2, 2018 Author Posted November 2, 2018 I read zram and log2ram config and source and recognized that zram will evaluate config of log2ram, so if I disable it in config - everything works. Only issue - truncate operation is still in cron and script does not look into config by executing... This bring must be disabled in cron. Yes, I forgot to say about this... daily script was commented # cat /etc/cron.daily/armbian-ram-logging #!/bin/sh #/usr/lib/armbian/armbian-ramlog write >/dev/null 2>&1 Thanks a lot!
Igor Posted November 3, 2018 Posted November 3, 2018 On 11/2/2018 at 8:33 AM, gas_85 said: Only issue - truncate operation is still in cron and script does not look into config by executing... This bring must be disabled in cron. Expand This should fix this issue. 1
montana Posted June 25, 2019 Posted June 25, 2019 Good day. If I understand correctly, will this disable logging in / var / log? I understand correctly ? If yes, then I would like to clarify. I use Orange PI PC2, transferred the entire system to usb hdd, only the boot area remained on the SD card. When log2ram is disabled, logs will be written to my hdd under the / var / log directory? I understand correctly ?
Igor Posted June 25, 2019 Posted June 25, 2019 On 6/25/2019 at 9:25 AM, montana said: I understand correctly ? Expand Yes.
montana Posted June 25, 2019 Posted June 25, 2019 On 6/25/2019 at 12:03 PM, Igor said: Да. Expand Thanks for the answer. I am quite new to this. Forgive one more clarification, with this disconnection will the recording go straight to my hdd or will the recording go through the SD card and then to hdd? I ask this because I wanted to extend the life of the SD card.
Igor Posted June 25, 2019 Posted June 25, 2019 On 6/25/2019 at 12:39 PM, montana said: I wanted to extend the life of the SD card. Expand Armbian does that for you. Don't worry about
montana Posted June 25, 2019 Posted June 25, 2019 On 6/25/2019 at 12:48 PM, Igor said: Armbian does that for you. Don't worry about Expand Thank you very much again!
Dysmas Posted September 26, 2019 Posted September 26, 2019 (edited) September 2019 : despite this discussion, a recent dowload of armbian has still the same default. Logrotate does not work in /var/log.hdd The reason is fairly simple, even if finding this reason was less simple. Armbian added the use or rsync to copy the files. Good idea, but unfortunately bad configuration : rsync deletes all unknown files in /var/log.hdd, which results in deletion of all rotated files. The reason is the rsync command wich includes the "--delete" parameter, in /usr/lib/armbian/armbian-ramlog, line 50 in syncToDisk function. This will delete in /var/log.hdd/ all file not present in /var/log/. So, all rotated files will be deleted. rsync -aXWv --delete --exclude "lost+found" --exclude armbian-ramlog.log --links $HDD_LOG $RAM_LOG Just delete the "--delete" parameter, and your log files will remain. A less important issue will remain, which will be discussed later. Edited September 29, 2019 by Dysmas Give the right solution to the problem
Igor Posted September 26, 2019 Posted September 26, 2019 On 9/26/2019 at 4:18 PM, Dysmas said: despite this discussion Expand Since when discussion fixes bugs?https://www.armbian.com/get-involved/
Dysmas Posted September 27, 2019 Posted September 27, 2019 On 9/26/2019 at 4:55 PM, Igor said: Since when discussion fixes bugs? Expand If I try to file an issue on Github, there is a message indicating : Quote Issues with provided images should go to the Armbian forum. Expand Now that I have a working solution which covers all aspects of the problem, I will file an issue on Github, after one week of testing. ram-log is a good idea, but as is, it is not finished.
Igor Posted September 27, 2019 Posted September 27, 2019 In a current organisation, which is not perfect, we discuss solutions either here on forum or directly at already submitted pull request. But if bug is mentioned on forum, there is no warranty it will be noticed, addressed or fixed. Ratio between bugs and resources is extreme. Extreme because people expect few people can fix Linux kernel or any bug that is found in packages. Well, this bug is accidentally and amazingly a part of our work.
Set3 Posted October 1, 2019 Posted October 1, 2019 Thanks Dysmas, removing one of the 2 --delete solved the problems indeed in /var/log.hdd/syslog, I now see the syslog.1 2.gz etc on a daily basis. Problem I still have , is that the log/syslog is not emptied, but keeps growing. Any ideas on that ?
Dysmas Posted October 1, 2019 Posted October 1, 2019 On 10/1/2019 at 1:00 PM, Set3 said: Problem I still have , is that the log/syslog is not emptied, but keeps growing. Expand Yes, I solved this problem also, I will post the solution quickly on Github, and I will add the link here.
Dysmas Posted October 1, 2019 Posted October 1, 2019 I posted the full solution here : https://github.com/armbian/build/issues/1582 The basic idea is : Once the "--delete" issue is corrected, log.hdd is better, but there is still an issue : the rotated files overlap, because the effect of the rotation is not mirrored to /var/log. I have a log rotated daily, but each rotated file contains 3 or 4 days. This can be solved easily by the following process : ramlog write (to update log.hdd before rotation) logrotate ramlog postrotate (which will copy to /var/log the modified files of /ram/log.hdd). See the code on Github, link above. 1
Set3 Posted October 4, 2019 Posted October 4, 2019 rsync is preferred when it comes to reduce traffic to sd card and that is what this was all about anyway right ? I tried rsync -aXWv /var/log.hdd/* /var/test/ --exclude *.[0-9] --exclude *.[0-9].gz looks good to me, or am I missing something ? is the var/log/syslog updated inbetween so that rsync sees /var/log/syslog as a newer file than the just empty /var/log.hdd/syslog ? But I now also don't understand is that in the middle of the day, I can overwrite syslog with the version of last night, and I cant see updates when I use logger. Hmm strange. Probably needs a log restart. I see discussions here on disabling ramlog, where the idea is very good. That is a shame. The current solution is not easy to understand for the end user. And it is just about something as simple as logging :-) Perhaps a stupid idea while we are might be redesigning anyway ? why don't we - do a standard logrotate in /var/log as designed, described and same as on other ( eg x86) installations - probably do a kind of custom rotate on /var/log.hdd to move .1 to get .2.gz etc - "rsync without --delete" to /var/log.hdd which then holds latest files and previous rotated .1 - remove all rotated files from /var/log to use as little RAM as possible - do a cleanup of old compressed rotated files in /var/log.hdd according to logrorate settings ? or just leave the rotate files on .var.log, makes it more simple, have users decide to change logrotate as per manual Yes there may be complications, but that is for experts to point out. To me looks less error prone, as log-rotate on /var/log then works as designed and described in all manuals. And to avoid confusion, put a small text file with explanation /var/log/Where_are_the_rotated_files.txt for the end user (without being rotated out :-) , explaining on how it works and where we can find the rotated files and why. Including the 75% mechanism explained. As end user I would be happy to see that. Logging is not a sexy subject, but I have a few OPiZero (256MB) with docker apps and memory is a challenge so having logs running up to 50MB is a lot. (although I know about the 75% monitoring which does work well, and I can change that to less % etc etc)
Dysmas Posted October 4, 2019 Posted October 4, 2019 On 10/4/2019 at 1:52 PM, Set3 said: But I now also don't understand is that in the middle of the day, I can overwrite syslog with the version of last night, and I cant see updates when I use logger. Hmm strange. Probably needs a log restart. Expand Sorry, I forgot to say why rync is not good for this operation. The problem is that it creates new files and this is a problem for several programs which probably do not use the filename but uuid or something like that. My experience was the same as yours, after rsync, for most programs, log was no longer working, the file remained empty and restart would not be a good solution. That's why I used cat source > dest, because this process overwrites the content, but do not create a new file. And it works like a charm.
Set3 Posted October 4, 2019 Posted October 4, 2019 By the way, while searching for this problem, I think I read somewhere that there is a command that tells syslog and other apps to restart logs using new logs, cant remember where/which command Is that not postrotate / sharescript?
Set3 Posted October 4, 2019 Posted October 4, 2019 until redesigned I will try your solution on a few OPiZeros, see if it lasts :-) So do we need another request for a possible redesign or a discussion on a more worked out plan here first ?
Set3 Posted October 4, 2019 Posted October 4, 2019 One question Dysmas on implementation, I don't see in your request : "On a working system, it is better to copy this file from /lib/systemd/system to /etc/systemd/system to prevent possible overwriting during an update." So somehow magically /etc/systemd/system overrules /lib/systemd/system ? or do I need to change something else ?
Set3 Posted October 4, 2019 Posted October 4, 2019 In the request you wrote logrotate.service ? I dont have that in ubuntu;18.04.3;lts 5.90 on OPiZ You mean armbian-ramlog.service ? now has [Service] Type=oneshot ExecStart=/usr/lib/armbian/armbian-ramlog start ExecStop=/usr/lib/armbian/armbian-ramlog stop ExecReload=/usr/lib/armbian/armbian-ramlog write RemainAfterExit=yes TimeoutStartSec=30sec
Dysmas Posted October 4, 2019 Posted October 4, 2019 On 10/4/2019 at 2:40 PM, Set3 said: So somehow magically /etc/systemd/system overrules /lib/systemd/system ? or do I need to change something else ? Expand Nothing to do, systemd is designed like that.
Dysmas Posted October 4, 2019 Posted October 4, 2019 On 10/4/2019 at 3:09 PM, Set3 said: In the request you wrote logrotate.service ? I dont have that in ubuntu;18.04.3;lts 5.90 on OPiZ You mean armbian-ramlog.service ? Expand No, I didn't modify armbian-ramlog.service. The only thing I can say is that I have in Armbian (September 2019) /lib/systemd/system/logrotate.service (in the same directory as armbian-ramlog). But I don't have the Ubuntu version, I use the Debian Buster Armbian Linux 4.4.184. Initial content of [service] in this file is only two lines : [Service] Type=oneshot ExecStart=/usr/sbin/logrotate /etc/logrotate.conf
AlexP Posted October 7, 2019 Posted October 7, 2019 This issue really hurts. The thing is that I'm using Orange Pi One Plus with Armbian Bionic (mainline based kernel 5.1.y) with all latest updates. I know it's in Testing mode currently, but anyway. I'm using Orange Pi with PiHole installed to filter ads in my home network. At times it reboots(why it reboots it's a separate topic), it start moving the files into persistant storage and with probability 20-50% fails to start. The flow is: Starting Armbian memory supported logging... Then it tries to move logs to a storage for 4.5 min and fails with: [FAILED] Failed to start Armbian memory supported logging. See 'systemctl status armbian-ramlog.service' for details. And then stuck for 3 more minutes resulting into: [FAILED] Failed to start Flush Journal to Persistant Storage. See 'systemctl status systemd-journal-flush.service' for details. Starting Create Volatile Files and Directories. And at this point it deadly stucks leaving my devices in the network w/o network access. I've tried all the hints provided above by Dysmas except modifying logrotate.service (was not able to find it). TBH, having all these optimizations for the SD life is good, but it's not smth I'd like to have where system dies during its start. I've bought Opi One Plus in honor to change my Opi One that served same functionality of blocking ads via PiHole. The latter one (Opi One) was working for months w/o having any issue from stability stand point. Opi One Plus now stucks each each 2-4 hours disconnecting me from the internet.
Dysmas Posted October 7, 2019 Posted October 7, 2019 Hi, Alex, what I suggested will not help you, it is a different problem. Perhaps the good way for you is to follow the directions of gas_85 and disable this feature, because debugging this without having a Pi One will be difficult. Nevertheless could you follow the recommendation of the message and type the command : sudo systemctl status armbian-ramlog.service and give us the result of this command. It will help us to understand what is happening.
AlexP Posted October 7, 2019 Posted October 7, 2019 Dysmas, thanks for your reply. Not sure if would help... but here is output root@orangepioneplus:~# sudo systemctl status armbian-ramlog.service * armbian-ramlog.service - Armbian memory supported logging Loaded: loaded (/lib/systemd/system/armbian-ramlog.service; enabled; vendor preset: enabled) Active: active (exited) since Mon 2019-10-07 19:23:33 +03; 3h 26min ago Process: 711 ExecStart=/usr/lib/armbian/armbian-ramlog start (code=exited, status=0/SUCCESS) Main PID: 711 (code=exited, status=0/SUCCESS) Oct 07 19:23:32 orangepioneplus armbian-ramlog[711]: lighttpd/access.log Oct 07 19:23:32 orangepioneplus armbian-ramlog[711]: lighttpd/error.log Oct 07 19:23:32 orangepioneplus armbian-ramlog[711]: ntpstats/ Oct 07 19:23:32 orangepioneplus armbian-ramlog[711]: pihole/ Oct 07 19:23:32 orangepioneplus armbian-ramlog[711]: sysstat/ Oct 07 19:23:32 orangepioneplus armbian-ramlog[711]: unattended-upgrades/ Oct 07 19:23:32 orangepioneplus armbian-ramlog[711]: unattended-upgrades/unattended-upgrades-shutdow Oct 07 19:23:32 orangepioneplus armbian-ramlog[711]: sent 2,809,265 bytes received 473 bytes 5,619 Oct 07 19:23:32 orangepioneplus armbian-ramlog[711]: total size is 2,806,659 speedup is 1.00 Oct 07 19:23:33 orangepioneplus systemd[1]: Started Armbian memory supported logging. I can't provide an ouput when it fails to start as the issue happens during device boot, and it's not responsive until I do a power reset.
Recommended Posts