Orange Pi Zero loses connectivity - logs erased

[colic]

Armbianmonitor:

http://ix.io/1z4U

I have a new OPIz running ARMBIAN 5.70 stable Debian GNU/Linux 8 (jessie) 3.4.113-sun8i with Pihole version 4.1 (latest). The Opi is connected only with eth0 with a static IP address. Its running a 32G sandisk Ultra (class 10) with a verified image using Etcher.  The CPU is running at around 59c. 

 

It's unusable because the OPi keeps losing connectivity. Usually about every 12-24 hours but more frequently at times (sometimes two or three times an hour).  So no ping, no ssh, no telnet.

 

Ive implemented some scripts on the OPi to try and track down what's happening. But the first thing I notice is syslog gets cleared and/or has large gaps in it.

 

Maybe this is because the log isnt written to disk, and because I usually have to pull the power to recover it, I lose the data? But  if so how can I reliably log messages to get a clue as to what's happening?

 

So for example, after another similar failure, syslog now contains nothing before 12:17. While only yesterday it had entries going back for days. It's been erased. And after every failure part or all of it gets erased. 

 

I can sometimes catch it OPi going offline in syslog, as follows:

 

Jan 24 14:17:22 localhost systemd[1411]: Reached target Sockets.
Jan 24 14:17:22 localhost systemd[1411]: Starting Basic System.
Jan 24 14:17:22 localhost systemd[1411]: Reached target Basic System.
Jan 24 14:17:22 localhost systemd[1411]: Starting Default.
Jan 24 14:17:22 localhost systemd[1411]: Reached target Default.
Jan 24 14:17:22 localhost systemd[1411]: Startup finished in 57ms.
Jan 24 14:17:23 localhost systemd[1]: Started User Manager for UID 1000.
Jan 24 14:17:23 localhost NetworkManager[802]: <info> (wlan0): supplicant inter
face state: disconnected -> inactive
Jan 24 14:17:23 localhost sSMTP[875]: Sent mail for root@PiHole (221 2.0.0 clos
ing connection r130sm29540290pfr.48 - gsmtp) uid=0 username=root outbytes=330
Jan 24 14:17:23 localhost NetworkManager[802]: <info> NetworkManager state is n
ow DISCONNECTED
Jan 24 15:09:52 localhost systemd[1411]: Time has been changed
Jan 24 15:09:52 localhost rsyslogd-2007: action 'action 17' suspended, next ret
ry is Thu Jan 24 15:10:22 2019 [try http://www.rsyslog.com/e/2007 ]
Jan 24 15:09:52 localhost systemd[1250]: Time has been changed
Jan 24 15:09:52 localhost systemd[1]: Time has been changed
Jan 24 15:09:53 localhost CRON[1567]: (root) CMD (  [ -x /usr/lib/php5/sessionc
lean ] && /usr/lib/php5/sessionclean)
Jan 24 15:09:53 localhost CRON[1568]: (root) CMD (/home/colic/scripts/loadMon.s
h)
Jan 24 15:09:53 localhost loadMon: Im alive and I pinnged the Gateway. Temp is 
64 c
Jan 24 15:09:57 localhost dhcpcd[1150]: eth0: no IPv6 Routers available
Jan 24 15:10:01 localhost CRON[1619]: (root) CMD (   PATH="$PATH:/usr/local/bin
/" pihole updatechecker local)
Jan 24 15:10:01 localhost CRON[1620]: (root) CMD (/home/colic/scripts/loadMon.s
h)
Jan 24 15:10:01 localhost loadMon: Im alive and I pinnged the Gateway. Temp is 
58 c

For the above log entries my PC script was able to contact (ping & ssh) OPi until 14:45 (so ping and ssh were working from 14:17 to 14:45) when it reported a ping failure.  At 15:08 I recover by power cycling, and you see the boot sequence beginning. 

 

But every time I recover the Opi, it appears to work perfectly normally except for tampering with its audit trail in syslog!

 

Ive used two different 2A power supplies, and two ethernet cables (since twice I was able to recover by unplugging it and replugging - although usually this does nothing). 

 

So my question is, what is going on with syslog and how do I get it reliably retaining the logger messages I send to it? 

 

 

Edited January 24, 2019 by colic
adding detail

[Igor]

3 hours ago, colic said:

Debian GNU/Linux 8 (jessie)

Set IP address in /etc/network/interfaces and not by using NetworkManager ... which is pretty buggy in Jessie. Or just migrate everything to Debian Stretch/Ubuntu Bionic with kernel from this century.

[colic]

Thanks for the reply Igor.

 

While Ive been using Linux for a while, Im no expert and very new to Armbian. Your second suggestion first, looks like you're saying to buy a new board as the hardware requirements list this? Is that what you mean? If so, that's probably not going to happen just so I can run pihole!!

 

On your first suggestion, everything tells me I should not edit /etc/network/interfaces directly but use nmtui? Do you mean I should simply edit /etc/network/interfaces ?

 

While this might explain the drops in connectivity it doesnt explain the syslog issue I dont think?

 

Is networking that bad in Jessie? Id say its virtually unusable as its dropping out every few hours now, and syslog is empty while its gone AWOL.

[Werner]

Relax. You are just not using proper software for the purpose. I am running PiHole on an OPi Zero in my LAN as well and it runs perfectly fine.

Just use an image which features mainline kernel, not legacy (not 3.4.113 to say as it seems there is no need for advanced graphics output, just server activity) and use Debian Stretch or Ubuntu Bionic as operating system.

 

 

[colic]

OK, cool. Im happy to do whatever it takes (that doesnt cost me buying another board haha).  I thought Id googled the right image to flash for OpiZ - it looks like I stuffed up hahah. Start again....

 

So let me make sure Ive got this right. Im gong to download Bionic 4.19 from here flash that, and try again.

 

Funnily enough I have a second OPIz with similar problems, but its installed on the other side of the world to me so there's not much I can do about it right now!

 

I cant find the links I used to build the OPIz, but that's where I decided (wrongly) that Jessie was what I needed. Hmmm, must improve my googling skills. 

[Igor]

1 hour ago, colic said:

Is networking that bad in Jessie?

Yes, in this particular combination. Classic networking https://wiki.debian.org/NetworkConfiguration will work fine.

[guidol]

5 hours ago, colic said:

I have a new OPIz running ARMBIAN 5.70 stable Debian GNU/Linux 8 (jessie) 3.4.113-sun8i with Pihole version 4.1 (latest).

Since the last year (and around v3.3 of pihole) it isnt recommend to use pihole with debian jessie anymore:
https://pi-hole.net/2018/02/14/pi-hole-v3-3-released-its-extra-special/

So pihole will run much smoother with debian strech (or Ubuntu bionic).

[Werner]

54 minutes ago, colic said:

 

So let me make sure Ive got this right. Im gong to download Bionic 4.19 from here flash that, and try again.

 

Yes. Or Debian with the same kernel. Depends on your taste  

As for myself I prefer Debian as the directly by Debian maintained packages count is way higher than by Ubuntu.

Lots of packages on Ubuntu are Universe or Multiverse which means there is no guarantee for support or (even critical) fixes.

[colic]

Thankyou so much for all your comments and help. I honestly did do my reading, but not what has been pointed out. So I stuffed up. 

 

I rebuilt using bionic before reading the last of those comments, but its up and running so Ill see how it goes.

 

The first thing I notice is the CPU is running at 47c instead of 57-60.c I also changed IP address and it instantly changed it without reboot. Two big differences already.

 

In trawling the logs I did notice some DNS messages. Now they make sense. So it looks like pihole was having problems with Jessie as well, just like you point out. 

 

So all up, although its only been running for an hour or so its already looking better.

 

Thankyou very much for the help. And hopefully I wont be back on this thread !!