Banana pi pro crashes (ARMBIAN 5.31 stable Ubuntu 16.04.3 LTS 3.4.113-sun7i)

[den416]

Sometimes i can't access to my banana thru ssh and etc. Link led is blinking. It happens everytime in HH:16:00 with random hour.

In syslog i get this messages:

 

Quote

Oct 17 17:15:01 localhost CRON[22766]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 17:17:01 localhost CRON[22917]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Oct 17 18:17:12 localhost rsyslogd: [origin software="rsyslogd" swVersion="8.16.0" x-pid="575" x-info="http://www.rsyslog.com"] start
luxdb/data/telegraf/autogen/17/000000002-000000002.tsm.tmp (#0) engine=tsm1
Oct 17 14:22:46 localhost influxd[13653]: 2017-10-17T11:22:46Z compacted level 1 2 files into 1 files in 7.1
1756119s engine=tsm1
Oct 17 14:25:01 localhost CRON[10257]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 14:35:01 localhost CRON[11014]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 14:36:21 localhost influxd[13653]: 2017-10-17T11:36:21Z retention policy shard deletion check commenc
ing service=retention
Oct 17 14:41:27 localhost ntpd[947]: 93.180.6.3 local addr 192.168.1.200 -> <null>
Oct 17 14:42:32 localhost ntpd[947]: 91.189.89.199 local addr 192.168.1.200 -> <null>
Oct 17 14:43:19 localhost ntpd[947]: 195.3.252.123 local addr 192.168.1.200 -> <null>
Oct 17 14:44:11 localhost ntpd[947]: 178.236.30.81 local addr 192.168.1.200 -> <null>
Oct 17 14:45:01 localhost CRON[11745]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 14:55:01 localhost CRON[12483]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 15:00:14 localhost ntpd[947]: 77.73.232.17 local addr 192.168.1.200 -> <null>
Oct 17 15:05:01 localhost CRON[13208]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 15:06:21 localhost influxd[13653]: 2017-10-17T12:06:21Z retention policy shard deletion check commenc
ing service=retention
Oct 17 15:15:01 localhost CRON[13886]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 15:16:42 localhost ntpd[947]: 91.189.91.157 local addr 192.168.1.200 -> <null>
Oct 17 15:17:01 localhost CRON[14021]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Oct 17 15:25:01 localhost CRON[14670]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 15:35:01 localhost CRON[15377]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 15:36:21 localhost influxd[13653]: 2017-10-17T12:36:21Z retention policy shard deletion check commenc
ing service=retention
Oct 17 15:45:01 localhost CRON[16118]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 15:55:01 localhost CRON[16896]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 16:05:01 localhost CRON[17554]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 16:06:21 localhost influxd[13653]: 2017-10-17T13:06:21Z retention policy shard deletion check commenc
ing service=retention
Oct 17 16:12:58 localhost ntpd[947]: 195.3.254.2 local addr 192.168.1.200 -> <null>
Oct 17 16:15:01 localhost CRON[18302]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 16:17:01 localhost CRON[18439]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Oct 17 16:25:02 localhost CRON[19027]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 16:35:01 localhost CRON[19749]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 16:36:21 localhost influxd[13653]: 2017-10-17T13:36:21Z retention policy shard deletion check commenc
ing service=retention
Oct 17 16:45:01 localhost CRON[20421]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 16:55:01 localhost CRON[21160]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 17:05:01 localhost CRON[21862]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Oct 17 17:06:21 localhost influxd[13653]: 2017-10-17T14:06:21Z retention policy shard deletion check commenc
ing service=retention
Oct 17 18:17:12 localhost rsyslogd-2222: command 'KLogPermitNonKernelFacility' is currently not permitted - did you already set it via a RainerScript command (v6+ config)? [v8.16.0 try http://www.rsyslog.com/e/2222 ]
Oct 17 18:17:12 localhost rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 62: syntax error on token '' [v8.16.0 try http://www.rsyslog.com/e/2207 ]
Oct 17 18:17:12 localhost rsyslogd-2207: CONFIG ERROR: could not interpret master config file '/etc/rsyslog.conf'. [v8.16.0 try http://www.rsyslog.com/e/2207 ]
Oct 17 18:17:12 localhost rsyslogd: rsyslogd's groupid changed to 108
Oct 17 18:17:12 localhost rsyslogd: rsyslogd's userid changed to 104
Oct 17 18:17:12 localhost systemd-modules-load[163]: Inserted module 'hci_uart'
Oct 17 18:17:12 localhost dhclient[444]: Internet Systems Consortium DHCP Client 4.3.3
Oct 17 18:17:12 localhost systemd-modules-load[163]: Module 'gpio_sunxi' is builtin
Oct 17 18:17:12 localhost dhclient[444]: Copyright 2004-2015 Internet Systems Consortium.
Oct 17 18:17:12 localhost systemd-modules-load[163]: Inserted module 'rfcomm'
Oct 17 18:17:12 localhost dhclient[444]: All rights reserved.
Oct 17 18:17:12 localhost loadkeys[158]: Loading /etc/console-setup/cached.kmap.gz
Oct 17 18:17:12 localhost systemd-modules-load[163]: Inserted module 'hidp'
Oct 17 18:17:12 localhost dhclient[444]: For info, please visit https://www.isc.org/software/dhcp/
Oct 17 18:17:12 localhost systemd-modules-load[163]: Inserted module 'sunxi_ir'
Oct 17 18:17:12 localhost dhclient[444]: 
Oct 17 18:17:12 localhost fake-hwclock[162]: Tue Oct 17 15:17:01 UTC 2017
Oct 17 18:17:12 localhost systemd-modules-load[163]: Module 'bonding' is builtin
Oct 17 18:17:12 localhost dhclient[444]: Listening on LPF/eth0/02:03:08:41:2d:f8
Oct 17 18:17:12 localhost systemd-modules-load[163]: Module 'spi_sun7i' is builtin
Oct 17 18:17:12 localhost dhclient[444]: Sending on   LPF/eth0/02:03:08:41:2d:f8
Oct 17 18:17:12 localhost systemd-modules-load[163]: Inserted module '8021q'
Oct 17 18:17:12 localhost dhclient[444]: Sending on   Socket/fallback
Oct 17 18:17:12 localhost systemd-modules-load[163]: Inserted module 'a20_tp'
Oct 17 18:17:12 localhost dhclient[444]: DHCPREQUEST of 192.168.1.200 on eth0 to 255.255.255.255 port 67 (xid=0x43155233)
Oct 17 18:17:12 localhost systemd-modules-load[163]: Inserted module 'ap6211'
Oct 17 18:17:12 localhost systemd-sysctl[190]: Couldn't write '1' to 'fs/protected_symlinks', ignoring: No such file or directory
Oct 17 18:17:12 localhost resolvconf[182]: /etc/resolvconf/update.d/libc: Warning: /etc/resolv.conf is not a symbolic link to /run/resolvconf/resolv.conf
Oct 17 18:17:12 localhost systemd-sysctl[190]: Couldn't write '1' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory
Oct 17 18:17:12 localhost systemd-sysctl[190]: Couldn't write '1' to 'fs/pr

 

i don't understand what is this...

Everytime syslog messages are same

[Igor]

1 hour ago, den416 said:

i don't understand what is this...

 

 

armbianmonitor -u

and a description of what is your system doing. Then we could perhaps get to the bottom of the problem. Since we have a good modern kernel for A20 boards, you might want to try that first.

[den416]

5 hours ago, Igor said:

http://sprunge.us/BWfP

system works like NAS (smb, transmission)

i confused, because problem everytime comes in ~(random)hour : 16 min

modern kernel is https://dl.armbian.com/bananapipro/Ubuntu_xenial_next.7z.torrent ?

5 hours ago, Igor said:

 

armbianmonitor -u

and a description of what is your system doing. Then we could perhaps get to the bottom of the problem. Since we have a good modern kernel for A20 boards, you might want to try that first.

 

[Igor]

7 hours ago, den416 said:

modern kernel is https://dl.armbian.com/bananapipro/Ubuntu_xenial_next.7z.torrent ?

Yes. 

There are no suspicious in system logs. If a crash is initiated xx:16 check cron jobs (which are also not suspicious on first look), logs, ... Something must be running and that something should be checked what it does. Those boards usually run fine, even with this old kernel. I used one A20 machine for years, also for hosting this forum and I had no problems. The same board was also used for torrent downloading. 

[tkaiser]

57 minutes ago, Igor said:

If a crash is initiated

Which crash? According to the provided log the system has been rebooted on 'Thu Sep 28 15:17:14 MSK 2017' and is running fine. If you want to follow this 'issue' I would better try to clarify what the 'issue' could be...

 

Maybe suspicious: Ethernet connection is 100 MBits/sec.

[den416]

I attach grafana screenshot with cpu and disk monitoring.

How we can see, in 16m graphics goes up.

 

[den416]

23 minutes ago, tkaiser said:

Which crash? According to the provided log the system has been rebooted on 'Thu Sep 28 15:17:14 MSK 2017' and is running fine. If you want to follow this 'issue' I would better try to clarify what the 'issue' could be...

 

Maybe suspicious: Ethernet connection is 100 MBits/sec.

system was freezed at ~ 18:17:12
I think, may be rsyslog doing somethink wrong...
Some peoples had same issue when rsyslog try to send log via tcp, but i not configure that.

[tkaiser]

50 minutes ago, den416 said:

system was freezed at ~ 18:17:12

What the heck are talking about? A system that freezes or crashes has to be powercycled afterwards. This obviously is not happening here, isn't it?

 

Do you refer to your Banana not being reactive any more every hour at a specific minute but then shortly after everything is fine again? If so and you use slow storage that can be caused by any heavy IO activity (especially random IO). You have 64.76 IO transactions per second on /dev/sda2 ON AVERAGE SINCE LAST BOOT, something's constantly hammering your disk. Just stop this and you're done (use iostat, iotop or every other tool that is appropriate to nail down such 'problems' on Linux)

[arox]

For a long time, I suspected CRON to crash my Bananapi M1. But the fact is that CRON has a lot improved since I changed my PSU ...

 

As a matter of fact, by me the problem is not even the PSU but the contacts in the connectors. (And not only micro-USB).