Jump to content

pine64: massive date/time clock problem

linda

By linda
in Pine A64

 Share

Recommended Posts

linda

linda

Posted
Posted

Hello,
I have a massive problem as the time/date on my Pine64 keep changing randomly to the year 2113.

In my project, I use several Pine64s and the problem now occurs on many of these Pine64s. Unfortunately I need the correct time for my project.

I am using the following system: ARMBIAN 5.32.170911 nightly Ubuntu 16.04.3 LTS 4.13.0-sun50iw1 (with additional overlays = uart3 and
console = ttyS3)

Could this be due to the error described in the post

and is the bug fixed in kernel version 4.14?

Could I install this kernel version 4.14 via armbian-config (next-kernel)?

 

Thanks a lot for help.

0
Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
9 minutes ago, martinayotte said:

CONFIG_FSL_ERRATUM_A008585=y

According to some discussions and tests this is not enough, A64 timer has worse problems so this was posted on the linux-sunxi mailing list recently. Though I think I've read in IRC logs that even this is not 100% reliable.

And somehow Maxime decided to blame us for not reporting the clock issue, even though it was discussed on IRC several times and we here don't have a perfect understainding of arch timer vs system clock correlation.

0
Link to comment
Share on other sites
martinayotte

martinayotte

Posted
Posted
7 minutes ago, zador.blood.stained said:

A64 timer has worse problems so this was posted on the linux-sunxi mailing list recently

Yes, I've seen that too. Maxime should not blame us, we only faced the issue and in my case, adding this erratum to config worked without any additionnal patches, I've never face the issue again, but that is maybe only luck ...

If newer patch is better, we can probably try to integrate it in Armbian, but it is up to Maxime to ack the patch, not us ;)

0
Link to comment
Share on other sites
linda

linda

Posted
  • Author
Posted

many, many thanks for the quick responses and please excuse my bedelayed answer.

 

In my current version (with kernel 4.13) CONFIG_FSL_ERRATUM_A008585 is not set.
The updated kernel (4.14) results with
CONFIG_FSL_ERRATUM_A008585 = y

Thus, I will upgrade my existing devices to the new kernel (and hope that the issues zador.blood.staine addresses will not occur to me).

 

With my test device I realized this with the tool "armban-config". Is there a way to parameterize the "armban-config" tool to run the update via script or is it possible to do the adjustment to the next kernel via command line?

0
Link to comment
Share on other sites
linda

linda

Posted
  • Author
Posted

Is it correct to install the kernel with the following command:

apt-get -o APT::Status-Fd=4 -o APT::Keep-Fds::=5 -o APT::Keep-Fds::=6 --reinstall -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y -qq --no-install-recommends install linux-image-next-sunxi64 linux-headers-next-sunxi64 linux-u-boot-pine64-next linux-xenial-root-next-pine64 linux-dtb-next-sunxi64

(found in the history.log of the updated device)

 

A test on a device with the kernel 4.13 shows no error and results in the new kernel 4.14.

0
Link to comment
Share on other sites
linda

linda

Posted
  • Author
Posted

I have some pine64 with kernel 4.14 and so far the time-problems did not occur with this kernel.

 

Unfortunately, however the pine64 updated  to kernel 4.17 again show the the problem! Time jumps to the year 2113 (and sometimes even to 2208). I use kernel 4.17.5 and 4.17.6. The flag 'CONFIG_FSL_ERRATUM_A008585' is set (CONFIG_FSL_ERRATUM_A008585=y).

 

Thanks a lot for help.

0
Link to comment
Share on other sites
linda

linda

Posted
  • Author
Posted
19 hours ago, martinayotte said:

Did you built image beofre or after this commit : https://github.com/armbian/build/commit/556b3067086266dd264e3f5e2dcdf61b60468d27

BTW, you can fix your image manually by editing the DT.

I did the upgrades form an older image with 4.13-kernel via

      apt-get  install linux-image-next-sunxi64 linux-headers-next-sunxi64 linux-u-boot-pine64-next linux-xenial-root-next-pine64 linux-dtb-next-sunxi64

on the 13th of july (resulting kernel 4.17.5) and on the 17th of july (resulting kernel 4.17.6).

When was the commit entered into the next-branch?

 

Can I fix the error if I make an upgrade (apt update, upgrade) to the current image?

 

0
Link to comment
Share on other sites
linda

linda

Posted
  • Author
Posted
26 minutes ago, martinayotte said:

My own commit on DEV was on July 13th, but then Igor did a big branch management to switch from old NEXT to new NEXT on July 17th, so maybe the patch got blinded during few days.

 

Thanks a lot for your help.

So if I update the two devices to the current image (apt update/upgrade), then the error should not occur. I will try it.

0
Link to comment
Share on other sites
vlad59

vlad59

Posted
Posted

I know it's an old thread but I just got this problem (date was in 2114) with a recent dev kernel. The symptom was strange -> my DHCP server won't give my Pine64 an IP due to the date difference.

 

I think the patch from Samuel Holland (https://groups.google.com/forum/#!searchin/linux-sunxi/pine64$20timer|sort:date/linux-sunxi/b27QfJJroR0/tZgFFQeCBAAJ

was not applied (or I did not find it)  so I'll try to build a kernel with it.

0
Link to comment
Share on other sites
brian redbeard

brian redbeard

Posted
Posted (edited)

I decided to dig into this since I was seeing the same issue.  

root@string:~# uname -r
4.19.20-sunxi64
root@string:~# dpkg -s  linux-image-next-sunxi64
Package: linux-image-next-sunxi64
Status: install ok installed
Priority: optional
Section: kernel
Installed-Size: 87123
Maintainer: Igor Pecovnik <igor.pecovnik@****l.com>
Architecture: arm64
Source: linux-4.19.20-sunxi64
Version: 5.75
Description: Linux kernel, version 4.19.20-sunxi64
 This package contains the Linux kernel, modules and corresponding other
 files, version: 4.19.20-sunxi64.
Homepage: http://www.kernel.org/
root@string:~# grep CONFIG_FSL_ERRATUM_A008585 /boot/config-4.19.20-sunxi64 
CONFIG_FSL_ERRATUM_A008585=y
root@string:~# date
Sat Apr 28 02:43:25 UTC 2114
root@string:~# timedatectl 
                      Local time: Sat 2114-04-28 02:43:30 UTC
                  Universal time: Sat 2114-04-28 02:43:30 UTC
                        RTC time: Tue 2019-03-05 22:21:31
                       Time zone: Etc/UTC (UTC, +0000)
       System clock synchronized: no
systemd-timesyncd.service active: no
                 RTC in local TZ: no
root@string:~# hwclock -s
hwclock: settimeofday() failed: Invalid argument

 

I decided to do a much more simple test:

Spoiler

  


root@string:~# strace -ff  date 030514032019.10 2>&1 
execve("/bin/date", ["date", "030514032019.10"], 0xffffc9e9e180 /* 18 vars */) = 0
brk(NULL)                               = 0xaaab0d1aa000
faccessat(AT_FDCWD, "/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
faccessat(AT_FDCWD, "/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=32028, ...}) = 0
mmap(NULL, 32028, PROT_READ, MAP_PRIVATE, 3, 0) = 0xffffa84d8000
close(3)                                = 0
faccessat(AT_FDCWD, "/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/aarch64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0 \10\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1341080, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xffffa84d6000
mmap(NULL, 1409880, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xffffa835d000
mprotect(0xffffa849d000, 61440, PROT_NONE) = 0
mmap(0xffffa84ac000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13f000) = 0xffffa84ac000
mmap(0xffffa84b2000, 13144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffffa84b2000
close(3)                                = 0
mprotect(0xffffa84ac000, 16384, PROT_READ) = 0
mprotect(0xaaaacf0fe000, 8192, PROT_READ) = 0
mprotect(0xffffa84e2000, 4096, PROT_READ) = 0
munmap(0xffffa84d8000, 32028)           = 0
brk(NULL)                               = 0xaaab0d1aa000
brk(0xaaab0d1cb000)                     = 0xaaab0d1cb000
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=3004464, ...}) = 0
mmap(NULL, 3004464, PROT_READ, MAP_PRIVATE, 3, 0) = 0xffffa807f000
close(3)                                = 0
openat(AT_FDCWD, "/etc/localtime", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=127, ...}) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=127, ...}) = 0
read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 127
lseek(3, -71, SEEK_CUR)                 = 56
read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 71
close(3)                                = 0
clock_settime(CLOCK_REALTIME, {tv_sec=1551794590, tv_nsec=0}) = -1 EINVAL (Invalid argument)
settimeofday({tv_sec=1551794590, tv_usec=0}, NULL) = -1 EINVAL (Invalid argument)
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
read(3, "# Locale name alias data base.\n#"..., 4096) = 2995
read(3, "", 4096)                       = 0
close(3)                                = 0
openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=578, ...}) = 0
mmap(NULL, 578, PROT_READ, MAP_PRIVATE, 3, 0) = 0xffffa84df000
close(3)                                = 0
write(2, "date: ", 6date: )                   = 6
write(2, "cannot set date", 15cannot set date)         = 15
openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, ": Invalid argument", 18: Invalid argument)      = 18
write(2, "\n", 1
)                       = 1
fstat(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 0), ...}) = 0
write(1, "Tue Mar  5 14:03:10 UTC 2019\n", 29Tue Mar  5 14:03:10 UTC 2019
) = 29
close(1)                                = 0
close(2)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++

I can see that the when date attempts to make the clock_settime syscall, it receives EINVAL back.  I'm wondering then why it sees CLOCK_REALTIME as invalid.  I could potentially see an issue with CLOCK_REALTIME_COARSE which  man 2 CLOCK_GETRES says is "architecture specific", but this one is throwing me of.

Edited by Tido
added spoiler - please add a spoiler nexttime yourself, thx
0
Link to comment
Share on other sites
brian redbeard

brian redbeard

Posted
Posted

Also curious, 1551794590 seems to be the last time at which the system clock may have made a checkpoint via systemd-timesyncd on a previous reboot before going back to the future.  B)

  

$ date --date='@1551794590'
Tue Mar  5 06:03:10 PST 2019

 

And a snippet from kern.log showing the current boot time: 

Mar  5 06:12:58 string kernel: [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
Mar  5 06:12:58 string kernel: [    0.000000] Linux version 4.19.20-sunxi64 (root@armbian.com) (gcc version 7.2.1 20171011 (Linaro GCC 7.2-2017.11)) #5.75 SMP Fri Feb 8 10:29:25 CET 2019

 

0
Link to comment
Share on other sites
KemoNine

KemoNine

Posted
Posted

I'm also seeing this behavior on a sopine that I recently deployed. It's seemingly inconsistent as well.

 

I checked the kernel config and it appears the CONFIG_FSL_ERRATUM_A008585 is enabled

  

cat /boot/config-4.19.38-sunxi64 | grep -i CONFIG_FSL_ERRATUM_
CONFIG_FSL_ERRATUM_A008585=y

 

And it's a very recent Arbian as well. 

4.19.38-sunxi64 #5.83

Short of tracking the date every minute or so in something like Monit to fire off a reboot... does anyone know of a practical work around?

0
Link to comment
Share on other sites
linda

linda

Posted
  • Author
Posted

Hi KemoNine,

unfortunately, the problem still occurs sporadically for me (kernel 4.17).

 

I have not found a solution and would also be very grateful for a practical work around.

[my current work-around: "systemctl restart ntp"]

 

0
Link to comment
Share on other sites
KemoNine

KemoNine

Posted
Posted
4 hours ago, linda said:

Hi KemoNine,

unfortunately, the problem still occurs sporadically for me (kernel 4.17).

 

I have not found a solution and would also be very grateful for a practical work around.

[my current work-around: "systemctl restart ntp"]

 

 

I ended up deploying monit to track and respond to the clock jump. I ended up telling monit to reboot the system whenever the clock jumps ahead too far to keep some of my services I have deployed happy. At the link below you'll find my monit config file for the time jump as well as a small python script that's run by monit to look for any time jumps over 3 months into the future. You can tune the python script to look for larger jumps as well as the monit config to restart ntp instead of performing a full reboot.

 

https://paste.lollipopcloud.solutions/?4e7e74d941e2c77d#ck/F7Tv6UBadpzlU56v+vzalNf1646gM8iHLvA3wd7w=

 

I hope this helps even if it's a rough work around.

0
Link to comment
Share on other sites
Igor

Igor

Posted
Posted
29 minutes ago, pkral78 said:

It seems that upstream kernel (5.2rc4 at the point) has already the fix from Samuel Holland).


Fix should already be a part of Armbian NEXT 4.19.y kernel.

0
Link to comment
Share on other sites
windysea

windysea

Posted
Posted
6 hours ago, pkral78 said:

It seems that upstream kernel (5.2rc4 at the point) has already the fix from Samuel Holland).

 

https://github.com/torvalds/linux/commit/c950ca8c35eeb32224a63adc47e12f9e226da241#diff-27e9d89068f8125140ea388216ae3140

 

 

Yes that fix should already be part of the existing kernels, however there are actually several timer-related issues on the A64 and it is not clear that they are all addressed even upstream.

 

Even with all of the fixes that have been published I still see the issue recur occasionally.  There is a thread in the Development section where this is (or was) being discussed.

0
Link to comment
Share on other sites
markolonius

markolonius

Posted
Posted

Problem still persists with the upstream patches.  Been running 5.2.5-sunxi64 and I still get this issue.  Its somewhat more stable on this kernel however if you stop and disable systemd-timesyncd.  Seems like systemd-timesyncd provokes it more.  But I still have to reboot and use ntpdate to sync the time when it flips to year 2114 usually every few days.  Very unreliable.  Yes I have "CONFIG_SUN50I_ERRATUM_UNKNOWN1=y".

 

@sundbry mentioned he applied this patch from the legacy 3.x kernel and has been stable for 1 month

https://github.com/arctype-co/linux/commit/5fcb4e57eeaa4d670ef4acf5818c6fe16aa0d3d0

 

Not sure if this will ever work its way into official builds at this time.  No one else seems to want to talk about the issue in Dev forms.  (I'm no dev). 

Hope to hear about any new progress in this.

0
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept