maracuja Posted October 11, 2020 Posted October 11, 2020 just to be precise: in my tests I've been using an orange pi zero - board rev 1.5 (the revision number is printed on the circuit board)
5kft Posted October 11, 2020 Posted October 11, 2020 7 hours ago, maracuja said: It is possible to hook a serial console to the board, but what kind of outcome do you expect? If the kernel is trapping/crashing for some reason then this would likely be shown on the console, which could help provide a clue as to what might be going on... 1
maracuja Posted October 12, 2020 Posted October 12, 2020 I think it's a memory issue. I have tested again with kernel 5.4.7 (Armbian 19.11.6) I had a second ssh connection open this time running a "free -h" command every second. While running apt on the first ssh connection it showed me that free swap went down to 0 bytes, used: 59 Mi, total: 59 Mi I had changed zram config of armbian to use 25% instead of 50% memory. But I don't think that's the issue because others have the same problems without this change. And on kernel 4.19.x this memory issue does not seem to arise. Also the fact that users with 512M memory don't have stability issues is a hint that it's a memory issue.
maracuja Posted October 12, 2020 Posted October 12, 2020 Setting CPU governor to performance (like described in this thread) does not seem to be a solution. I had a freeze again while running apt and while CPU governor was set to performance running @ 1.01 GHz. Memory before the test: total used free shared buff/cache available Mem: 238Mi 67Mi 132Mi 0.0Ki 38Mi 165Mi Swap: 59Mi 20Mi 38Mi Memory during the test (during freeze): total used free shared buff/cache available Mem: 238Mi 94Mi 127Mi 0B 16Mi 139Mi Swap: 59Mi 58Mi 0.0Ki
maracuja Posted October 12, 2020 Posted October 12, 2020 Next run - this time with zram = 50% After reboot: total used free shared buff/cache available Mem: 238Mi 72Mi 127Mi 0.0Ki 37Mi 159Mi Swap: 119Mi 9.0Mi 109Mi While "apt-get purge mosquitto" - it got stuck multiple times for a couple of minutes (=system unresponsive) The whole process took waaaaay too long for just purging a package. total used free shared buff/cache available Mem: 238Mi 94Mi 122Mi 0.0Ki 20Mi 137Mi Swap: 119Mi 79Mi 39Mi
maracuja Posted October 12, 2020 Posted October 12, 2020 @5kft You could try to do some apt-get install commands. For example: apt-get install php7.3-fpm, apt-get install mosquitto Then purge again. And look if your swap gets used... Maybe this is only orange pi zero specific. But it is 99.99% NOT a hardware issue. It must be a software issue. Something in the kernel, or a driver is eating up memory or the swap usage isn't configured correctly.
maracuja Posted October 12, 2020 Posted October 12, 2020 Did an "apt-get install mosquitto" and while "Setting up mosquitto" it said "Killed" Dmesg gives me: Oct 12 11:31:32 etj-55555555 systemd[1]: Reloading. Oct 12 11:31:33 etj-55555555 systemd[1]: /lib/systemd/system/nginx.service:8: PIDFile= references path below legacy directory /var/run/, updating /var/run/nginx.pid → /run/nginx.pid; please update the unit file accordingly. Oct 12 11:31:34 etj-55555555 systemd[1]: Reloading. Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711235] systemd-bless-b invoked oom-killer: gfp_mask=0x40cc0(GFP_KERNEL|__GFP_COMP), order=2, oom_score_adj=0 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711260] CPU: 1 PID: 5174 Comm: systemd-bless-b Not tainted 5.4.8-sunxi #19.11.6 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711265] Hardware name: Allwinner sun8i Family Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711308] [<c010da8d>] (unwind_backtrace) from [<c010a0b1>] (show_stack+0x11/0x14) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711326] [<c010a0b1>] (show_stack) from [<c093634f>] (dump_stack+0x6f/0x7c) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711345] [<c093634f>] (dump_stack) from [<c01fc515>] (dump_header+0x41/0x172) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711359] [<c01fc515>] (dump_header) from [<c01fbcfb>] (oom_kill_process+0xfb/0x104) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711369] [<c01fbcfb>] (oom_kill_process) from [<c01fc3f7>] (out_of_memory+0x19f/0x228) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711383] [<c01fc3f7>] (out_of_memory) from [<c022eb0f>] (__alloc_pages_nodemask+0xaff/0xc9c) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711400] [<c022eb0f>] (__alloc_pages_nodemask) from [<c0213e95>] (kmalloc_order+0x19/0x38) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711412] [<c0213e95>] (kmalloc_order) from [<c0213ecd>] (kmalloc_order_trace+0x19/0x94) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711430] [<c0213ecd>] (kmalloc_order_trace) from [<c015d37b>] (devkmsg_open+0x3f/0xc0) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711445] [<c015d37b>] (devkmsg_open) from [<c0256975>] (chrdev_open+0x91/0x11c) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711459] [<c0256975>] (chrdev_open) from [<c0250051>] (do_dentry_open+0xd5/0x2e8) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711477] [<c0250051>] (do_dentry_open) from [<c025e2b5>] (path_openat+0x1ed/0xe14) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711487] [<c025e2b5>] (path_openat) from [<c025fabf>] (do_filp_open+0x4f/0x90) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711500] [<c025fabf>] (do_filp_open) from [<c0251081>] (do_sys_open+0x125/0x194) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711513] [<c0251081>] (do_sys_open) from [<c0101001>] (ret_fast_syscall+0x1/0x62) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711518] Exception stack(0xc0e91fa8 to 0xc0e91ff0) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711529] 1fa0: b6edd000 b6f09968 ffffff9c b6d7385c 000a0101 00000000 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711539] 1fc0: b6edd000 b6f09968 bef46ce4 00000142 00000000 00000000 0044af8c 00000000 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711545] 1fe0: 00000142 bef46b00 b6e7525d b6dfe746 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711550] Mem-Info: Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711572] active_anon:1329 inactive_anon:1532 isolated_anon:85 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711572] active_file:1404 inactive_file:1017 isolated_file:22 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711572] unevictable:4 dirty:0 writeback:0 unstable:0 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711572] slab_reclaimable:4526 slab_unreclaimable:8663 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711572] mapped:1787 shmem:53 pagetables:528 bounce:0 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711572] free:32149 free_pcp:51 free_cma:31941 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711586] Node 0 active_anon:5316kB inactive_anon:6128kB active_file:5616kB inactive_file:4068kB unevictable:16kB isolated(anon):340kB isolated(file):88kB mapped:7148kB dirty:0kB writeback:0kB shmem:212kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711613] Normal free:128596kB min:5388kB low:5708kB high:6028kB active_anon:5288kB inactive_anon:6144kB active_file:5556kB inactive_file:4152kB unevictable:16kB writepending:0kB present:262144kB managed:243968kB mlocked:16kB kernel_stack:1304kB pagetables:2112kB bounce:0kB free_pcp:212kB local_pcp:0kB free_cma:127764kB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711616] lowmem_reserve[]: 0 0 0 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711625] Normal: 149*4kB (UMEC) 74*8kB (UMEC) 6*16kB (C) 5*32kB (C) 6*64kB (C) 4*128kB (C) 2*256kB (C) 2*512kB (C) 2*1024kB (C) 2*2048kB (C) 29*4096kB (C) = 128804kB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711663] 2545 total pagecache pages Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711669] 60 pages in swap cache Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711675] Swap cache stats: add 270671, delete 270604, find 8999/242120 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711678] Free swap = 74584kB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711681] Total swap = 121980kB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711684] 65536 pages RAM Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711687] 0 pages HighMem/MovableOnly Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711690] 4544 pages reserved Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711693] 32768 pages cma reserved Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711699] Unreclaimable slab info: Oct 12 11:31:35 etj-55555555 kernel: [ 1361.711702] Name Used Total Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712107] zs_handle 179KB 204KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712187] RAWv6 47KB 47KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712193] UDPv6 35KB 35KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712216] TCPv6 63KB 63KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712227] sgpool-128 30KB 30KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712236] sgpool-64 31KB 31KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712245] sgpool-32 15KB 15KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712254] sgpool-16 7KB 7KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712259] mqueue_inode_cache 15KB 15KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712281] jfs_mp 3KB 3KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712299] jbd2_inode 15KB 15KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712316] kioctx 7KB 7KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712334] rpc_buffers 32KB 32KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712343] RAW 31KB 31KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712351] UDP 61KB 61KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712359] request_sock_TCP 3KB 3KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712369] TCP 126KB 126KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712383] eventpoll_pwq 15KB 15KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712394] inotify_inode_mark 15KB 15KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712406] request_queue 125KB 125KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712413] blkdev_ioc 16KB 16KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712422] biovec-max 453KB 453KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712432] biovec-128 126KB 126KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712439] biovec-64 63KB 63KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712450] dmaengine-unmap-256 31KB 31KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712457] dmaengine-unmap-128 47KB 47KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712465] audit_buffer 4KB 4KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712473] skbuff_fclone_cache 31KB 31KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712484] skbuff_head_cache 173KB 173KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712492] configfs_dir_cache 7KB 7KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712503] file_lock_cache 16KB 16KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712523] fsnotify_mark_connector 15KB 15KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712545] net_namespace 60KB 60KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712553] task_delay_info 19KB 19KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712564] taskstats 30KB 30KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712574] proc_dir_entry 88KB 88KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712580] pde_opener 15KB 15KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712591] seq_file 15KB 15KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712604] shmem_inode_cache 267KB 267KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712619] kernfs_iattrs_cache 15KB 15KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712635] kernfs_node_cache 2051KB 2051KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712654] filp 409KB 437KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712674] names_cache 180KB 192KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712700] key_jar 141KB 192KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712720] vm_area_struct 154KB 189KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712737] mm_struct 32KB 32KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712745] files_cache 20KB 20KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712754] signal_cache 157KB 157KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712762] sighand_cache 252KB 252KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712769] task_struct 334KB 334KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712777] cred_jar 40KB 40KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712785] anon_vma_chain 220KB 220KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712793] anon_vma 163KB 163KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712798] pid 20KB 20KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712803] trace_event_file 99KB 99KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712812] pool_workqueue 16KB 16KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712833] vmap_area 364KB 436KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712846] kmalloc-8k 984KB 1088KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712851] kmalloc-4k 356KB 356KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712856] kmalloc-2k 480KB 480KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712868] kmalloc-1k 850KB 864KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712881] kmalloc-512 598KB 656KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712886] kmalloc-256 132KB 132KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712891] kmalloc-192 173KB 173KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712896] kmalloc-128 292KB 292KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712903] kmalloc-64 1650KB 1652KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712908] kmem_cache_node 68KB 68KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712913] kmem_cache 251KB 252KB Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712916] Tasks state (memory values in pages): Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712920] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712970] [ 295] 0 295 7021 358 59392 189 0 systemd-journal Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712981] [ 326] 0 326 4060 163 16384 154 -1000 systemd-udevd Oct 12 11:31:35 etj-55555555 kernel: [ 1361.712995] [ 456] 0 456 1812 2 14336 768 0 haveged Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713005] [ 579] 0 579 2657 336 20480 129 0 systemd-logind Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713014] [ 584] 0 584 1830 157 14336 52 0 cron Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713024] [ 592] 104 592 1371 284 16384 74 -900 dbus-daemon Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713046] [ 599] 0 599 6796 0 16384 34 0 rngd Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713063] [ 604] 0 604 2183 17 18432 101 0 wpa_supplicant Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713078] [ 606] 0 606 6197 211 22528 188 0 rsyslogd Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713097] [ 609] 0 609 5583 2 26624 1230 0 networkd-dispat Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713111] [ 615] 0 615 15327 194 45056 387 0 NetworkManager Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713125] [ 808] 0 808 8738 0 36864 1290 0 unattended-upgr Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713145] [ 832] 105 832 2753 133 12288 57 0 chronyd Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713160] [ 838] 105 838 806 0 12288 61 0 chronyd Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713176] [ 862] 0 862 2220 2 16384 143 -1000 sshd Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713194] [ 936] 0 936 8994 53 24576 144 0 polkitd Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713212] [ 943] 0 943 1561 2 14336 212 0 dhclient Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713226] [ 1178] 0 1178 1536 0 16384 29 0 agetty Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713243] [ 1182] 0 1182 959 0 10240 24 0 agetty Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713262] [ 1184] 0 1184 1536 0 14336 28 0 agetty Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713277] [ 1264] 0 1264 2557 184 16384 174 0 sshd Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713292] [ 1267] 0 1267 2987 0 20480 220 0 systemd Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713308] [ 1268] 0 1268 8105 0 30720 410 0 (sd-pam) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713319] [ 1361] 0 1361 1957 210 14336 247 0 bash Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713332] [ 1369] 0 1369 1957 2 16384 233 0 bash Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713345] [ 1452] 0 1452 1688 319 14336 14 0 watch Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713361] [ 1671] 111 1671 1171 78 14336 80 0 avahi-daemon Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713376] [ 1672] 111 1672 1171 0 12288 60 0 avahi-daemon Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713405] [ 5006] 0 5006 47314 298 45056 558 0 php-fpm7.3 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713416] [ 5010] 33 5010 47314 101 38912 571 0 php-fpm7.3 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713425] [ 5011] 33 5011 47314 101 38912 571 0 php-fpm7.3 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713436] [ 5083] 0 5083 14172 798 65536 2688 0 apt-get Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713453] [ 5137] 0 5137 2295 241 14336 373 0 dpkg Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713467] [ 5138] 0 5138 368 241 8192 13 0 mosquitto.posti Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713489] [ 5168] 0 5168 368 262 10240 0 0 invoke-rc.d Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713504] [ 5172] 0 5172 2247 621 18432 0 0 systemctl Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713515] [ 5173] 0 5173 8033 371 28672 340 0 (sd-executor) Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713529] [ 5174] 0 5174 2592 645 18432 0 0 systemd-bless-b Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713541] [ 5175] 0 5175 2595 1058 18432 0 0 systemd-cryptse Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713556] [ 5176] 0 5176 2593 1035 20480 0 0 systemd-debug-g Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713569] [ 5178] 0 5178 2596 1120 20480 0 0 systemd-fstab-g Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713583] [ 5180] 0 5180 2593 662 18432 0 0 systemd-getty-g Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713596] [ 5181] 0 5181 2596 1062 20480 0 0 systemd-gpt-aut Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713606] [ 5182] 0 5182 2593 1064 18432 0 0 systemd-hiberna Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713614] [ 5183] 0 5183 2593 1110 18432 0 0 systemd-rc-loca Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713623] [ 5184] 0 5184 2593 817 18432 0 0 systemd-run-gen Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713631] [ 5186] 0 5186 2593 1081 18432 0 0 systemd-system- Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713640] [ 5187] 0 5187 2596 669 18432 0 0 systemd-sysv-ge Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713662] [ 5188] 0 5188 2593 861 18432 0 0 systemd-veritys Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713675] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-0.slice/session-3.scope,task=apt-get,pid=5083,uid=0 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.713779] Out of memory: Killed process 5083 (apt-get) total-vm:56688kB, anon-rss:2096kB, file-rss:1096kB, shmem-rss:0kB, UID:0 pgtables:65536kB oom_score_adj:0 Oct 12 11:31:35 etj-55555555 kernel: [ 1361.730340] oom_reaper: reaped process 5083 (apt-get), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB Oct 12 11:31:35 etj-55555555 systemd[1]: /lib/systemd/system/nginx.service:8: PIDFile= references path below legacy directory /var/run/, updating /var/run/nginx.pid → /run/nginx.pid; please update the unit file accordingly. Oct 12 11:31:36 etj-55555555 systemd[1]: Starting Mosquitto MQTT v3.1/v3.1.1 Broker... Oct 12 11:31:36 etj-55555555 systemd[1]: Started Mosquitto MQTT v3.1/v3.1.1 Broker. Oct 12 11:31:38 etj-55555555 systemd[1]: Reloading.
heunghingwan Posted October 12, 2020 Posted October 12, 2020 I have the same problem Quote Oct 9 22:37:37 mass-test-34 kernel: RTL871X: rtl8188f_FirmwareDownload(): Shift for fw header! Oct 9 22:37:37 mass-test-34 kernel: RTL871X: rtl8188f_FirmwareDownload by IO write! Oct 9 22:37:37 mass-test-34 kernel: RTL871X: polling_fwdl_chksum: Checksum report OK! (1, 0ms), REG_MCUFWDL:0x00050505 Oct 9 22:37:37 mass-test-34 kernel: RTL871X: _8051Reset8188: Finish Oct 9 22:37:37 mass-test-34 kernel: RTL871X: _FWFreeToGo: Polling FW ready OK! (153, 12ms), REG_MCUFWDL:0x000505c6 Oct 9 22:37:37 mass-test-34 kernel: RTL871X: rtl8188f_FirmwareDownload: DLFW OK ! Oct 9 22:37:37 mass-test-34 kernel: RTL871X: rtl8188f_FirmwareDownload success. write_fw:1, 172ms Oct 9 22:37:37 mass-test-34 kernel: RTL871X: <=== rtl8188f_FirmwareDownload() Oct 9 22:37:37 mass-test-34 kernel: RTL871X: HalDetectPwrDownMode(): PDN=0 Oct 9 22:37:37 mass-test-34 kernel: RTL871X: Set RF Chip ID to RF_6052 and RF type to 3. Oct 9 22:38:07 mass-test-34 kernel: rcu: INFO: rcu_sched self-detected stall on CPU Oct 9 22:38:07 mass-test-34 kernel: rcu: 2-...!: (1 ticks this GP) idle=482/0/0x1 softirq=2690928/2690928 fqs=0 Oct 9 22:38:07 mass-test-34 kernel: (t=15017 jiffies g=12013697 q=16) Oct 9 22:38:07 mass-test-34 kernel: rcu: rcu_sched kthread starved for 15017 jiffies! g12013697 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=2 Oct 9 22:38:07 mass-test-34 kernel: rcu: RCU grace-period kthread stack dump: Oct 9 22:38:07 mass-test-34 kernel: rcu_sched I 0 10 2 0x00000000 Oct 9 22:38:07 mass-test-34 kernel: [<c086fd9f>] (__schedule) from [<c0870199>] (schedule+0x35/0x84) Oct 9 22:38:07 mass-test-34 kernel: [<c0870199>] (schedule) from [<c0872b17>] (schedule_timeout+0xf3/0x218) Oct 9 22:38:07 mass-test-34 kernel: [<c0872b17>] (schedule_timeout) from [<c016ccff>] (rcu_gp_kthread+0x34f/0x790) Oct 9 22:38:07 mass-test-34 kernel: [<c016ccff>] (rcu_gp_kthread) from [<c01360c9>] (kthread+0x109/0x10c) Oct 9 22:38:07 mass-test-34 kernel: [<c01360c9>] (kthread) from [<c01010f9>] (ret_from_fork+0x11/0x38) Oct 9 22:38:07 mass-test-34 kernel: Exception stack(0xef143fb0 to 0xef143ff8) Oct 9 22:38:07 mass-test-34 kernel: 3fa0: 00000000 00000000 00000000 00000000 Oct 9 22:38:07 mass-test-34 kernel: 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 9 22:38:07 mass-test-34 kernel: 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 Oct 9 22:38:07 mass-test-34 kernel: Sending NMI from CPU 2 to CPUs 0: Oct 9 22:38:07 mass-test-34 kernel: Sending NMI from CPU 2 to CPUs 1: Oct 9 22:38:07 mass-test-34 kernel: NMI backtrace for cpu 2 Oct 9 22:38:07 mass-test-34 kernel: CPU: 2 PID: 0 Comm: swapper/2 Tainted: G C E 5.4.45-sunxi #20.05.3 Oct 9 22:38:07 mass-test-34 kernel: Hardware name: Allwinner sun8i Family Oct 9 22:38:07 mass-test-34 kernel: [<c010dc49>] (unwind_backtrace) from [<c010a245>] (show_stack+0x11/0x14) Oct 9 22:38:07 mass-test-34 kernel: [<c010a245>] (show_stack) from [<c085f97b>] (dump_stack+0x6f/0x7c) Oct 9 22:38:07 mass-test-34 kernel: [<c085f97b>] (dump_stack) from [<c0863cdf>] (nmi_cpu_backtrace+0x6b/0x94) Oct 9 22:38:07 mass-test-34 kernel: [<c0863cdf>] (nmi_cpu_backtrace) from [<c0863dd9>] (nmi_trigger_cpumask_backtrace+0xd1/0xdc) Oct 9 22:38:07 mass-test-34 kernel: [<c0863dd9>] (nmi_trigger_cpumask_backtrace) from [<c016f061>] (rcu_dump_cpu_stacks+0x7f/0x9c) Oct 9 22:38:07 mass-test-34 kernel: [<c016f061>] (rcu_dump_cpu_stacks) from [<c016e40d>] (rcu_sched_clock_irq+0x5a5/0x71c) Oct 9 22:38:07 mass-test-34 kernel: [<c016e40d>] (rcu_sched_clock_irq) from [<c0173e99>] (update_process_times+0x29/0x50) Oct 9 22:38:07 mass-test-34 kernel: [<c0173e99>] (update_process_times) from [<c0181a47>] (tick_sched_timer+0x37/0x74) Oct 9 22:38:07 mass-test-34 kernel: [<c0181a47>] (tick_sched_timer) from [<c017486b>] (__hrtimer_run_queues+0xef/0x230) Oct 9 22:38:07 mass-test-34 kernel: [<c017486b>] (__hrtimer_run_queues) from [<c017520d>] (hrtimer_interrupt+0xd1/0x1fc) Oct 9 22:38:07 mass-test-34 kernel: [<c017520d>] (hrtimer_interrupt) from [<c06ff233>] (arch_timer_handler_phys+0x27/0x2c) Oct 9 22:38:07 mass-test-34 kernel: [<c06ff233>] (arch_timer_handler_phys) from [<c0165113>] (handle_percpu_devid_irq+0x57/0x1a0) Oct 9 22:38:07 mass-test-34 kernel: [<c0165113>] (handle_percpu_devid_irq) from [<c0161065>] (generic_handle_irq+0x1d/0x28) Oct 9 22:38:07 mass-test-34 kernel: [<c0161065>] (generic_handle_irq) from [<c016151f>] (__handle_domain_irq+0x43/0x84) Oct 9 22:38:07 mass-test-34 kernel: [<c016151f>] (__handle_domain_irq) from [<c0517211>] (gic_handle_irq+0x39/0x6c) Oct 9 22:38:07 mass-test-34 kernel: [<c0517211>] (gic_handle_irq) from [<c0101ae5>] (__irq_svc+0x65/0x94) Oct 9 22:38:07 mass-test-34 kernel: Exception stack(0xef149f60 to 0xef149fa8) Oct 9 22:38:07 mass-test-34 kernel: 9f60: 00000000 0df67480 ef6c7034 c01164c1 ffffe000 c0e04fa4 c0e04fec 00000004 Oct 9 22:38:07 mass-test-34 kernel: 9f80: 00000000 c0db99f0 c0ec4b25 00000000 c0f0cf98 ef149fb0 c0107c6f c0107c70 Oct 9 22:38:07 mass-test-34 kernel: 9fa0: 400f0033 ffffffff Oct 9 22:38:07 mass-test-34 kernel: [<c0101ae5>] (__irq_svc) from [<c0107c70>] (arch_cpu_idle+0x28/0x2c) Oct 9 22:38:07 mass-test-34 kernel: [<c0107c70>] (arch_cpu_idle) from [<c014126b>] (do_idle+0x143/0x1b0) Oct 9 22:38:07 mass-test-34 kernel: [<c014126b>] (do_idle) from [<c01414b9>] (cpu_startup_entry+0x19/0x20) Oct 9 22:38:07 mass-test-34 kernel: [<c01414b9>] (cpu_startup_entry) from [<40102531>] (0x40102531) Oct 9 22:38:07 mass-test-34 kernel: Sending NMI from CPU 2 to CPUs 3: Oct 9 22:38:07 mass-test-34 kernel: NMI backtrace for cpu 3 Oct 9 22:38:07 mass-test-34 kernel: CPU: 3 PID: 10 Comm: rcu_sched Tainted: G C E 5.4.45-sunxi #20.05.3 Oct 9 22:38:07 mass-test-34 kernel: Hardware name: Allwinner sun8i Family Oct 9 22:38:07 mass-test-34 kernel: PC is at _raw_spin_unlock_irqrestore+0x1c/0x20 Oct 9 22:38:07 mass-test-34 kernel: LR is at force_qs_rnp+0xef/0xf8 Oct 9 22:38:07 mass-test-34 kernel: pc : [<c08734bc>] lr : [<c016c05f>] psr: 00010033 Oct 9 22:38:07 mass-test-34 kernel: sp : ef143f18 ip : 00000004 fp : 00000008 Oct 9 22:38:07 mass-test-34 kernel: r10: 00000008 r9 : c0e05448 r8 : 00000000 Oct 9 22:38:07 mass-test-34 kernel: r7 : 00000001 r6 : c0dbafc0 r5 : c0e04fa8 r4 : c0e10d00 Oct 9 22:38:07 mass-test-34 kernel: r3 : 0000feee r2 : 00000008 r1 : 60010013 r0 : c0e10d00 Oct 9 22:38:07 mass-test-34 kernel: Flags: nzcv IRQs on FIQs on Mode SVC_32 ISA Thumb Segment none Oct 9 22:38:07 mass-test-34 kernel: Control: 50c5387d Table: 6df7006a DAC: 00000051 Oct 9 22:38:07 mass-test-34 kernel: CPU: 3 PID: 10 Comm: rcu_sched Tainted: G C E 5.4.45-sunxi #20.05.3 Oct 9 22:38:07 mass-test-34 kernel: Hardware name: Allwinner sun8i Family Oct 9 22:38:07 mass-test-34 kernel: [<c010dc49>] (unwind_backtrace) from [<c010a245>] (show_stack+0x11/0x14) Oct 9 22:38:07 mass-test-34 kernel: [<c010a245>] (show_stack) from [<c085f97b>] (dump_stack+0x6f/0x7c) Oct 9 22:38:07 mass-test-34 kernel: [<c085f97b>] (dump_stack) from [<c0863d05>] (nmi_cpu_backtrace+0x91/0x94) Oct 9 22:38:07 mass-test-34 kernel: [<c0863d05>] (nmi_cpu_backtrace) from [<c010c885>] (handle_IPI+0x49/0x2bc) Oct 9 22:38:07 mass-test-34 kernel: [<c010c885>] (handle_IPI) from [<c0517241>] (gic_handle_irq+0x69/0x6c) Oct 9 22:38:07 mass-test-34 kernel: [<c0517241>] (gic_handle_irq) from [<c0101ae5>] (__irq_svc+0x65/0x94) Oct 9 22:38:07 mass-test-34 kernel: Exception stack(0xef143ec8 to 0xef143f10) Oct 9 22:38:07 mass-test-34 kernel: 3ec0: c0e10d00 60010013 00000008 0000feee c0e10d00 c0e04fa8 Oct 9 22:38:07 mass-test-34 kernel: 3ee0: c0dbafc0 00000001 00000000 c0e05448 00000008 00000008 00000004 ef143f18 Oct 9 22:38:07 mass-test-34 kernel: 3f00: c016c05f c08734bc 00010033 ffffffff Oct 9 22:38:07 mass-test-34 kernel: [<c0101ae5>] (__irq_svc) from [<c08734bc>] (_raw_spin_unlock_irqrestore+0x1c/0x20) Oct 9 22:38:07 mass-test-34 kernel: [<c08734bc>] (_raw_spin_unlock_irqrestore) from [<c016c05f>] (force_qs_rnp+0xef/0xf8) Oct 9 22:38:07 mass-test-34 kernel: [<c016c05f>] (force_qs_rnp) from [<c016cd65>] (rcu_gp_kthread+0x3b5/0x790) Oct 9 22:38:07 mass-test-34 kernel: [<c016cd65>] (rcu_gp_kthread) from [<c01360c9>] (kthread+0x109/0x10c) Oct 9 22:38:07 mass-test-34 kernel: [<c01360c9>] (kthread) from [<c01010f9>] (ret_from_fork+0x11/0x38) Oct 9 22:38:07 mass-test-34 kernel: Exception stack(0xef143fb0 to 0xef143ff8) Oct 9 22:38:07 mass-test-34 kernel: 3fa0: 00000000 00000000 00000000 00000000 Oct 9 22:38:07 mass-test-34 kernel: 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 9 22:38:07 mass-test-34 kernel: 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 Oct 9 22:38:07 mass-test-34 system-monitor[606]: Frequency: 480MHz LoadAvg: 0.33 CPU Load: 3% Temp: 59.6°C Cooling: 0/8 Oct 9 22:38:12 mass-test-34 system-monitor[606]: Frequency: 1368MHz LoadAvg: 0.75 CPU Load: 0% Temp: 85.7°C Cooling: 0/8 Oct 9 22:38:17 mass-test-34 system-monitor[606]: Frequency: 1368MHz LoadAvg: 0.75 CPU Load: 1% Temp: 79.1°C Cooling: 0/8 Oct 9 22:38:22 mass-test-34 system-monitor[606]: Frequency: 1368MHz LoadAvg: 0.75 CPU Load: 1% Temp: 78.9°C Cooling: 0/8 Oct 9 22:38:27 mass-test-34 system-monitor[606]: Frequency: 1368MHz LoadAvg: 0.75 CPU Load: 1% Temp: 79.6°C Cooling: 0/8 Oct 9 22:38:32 mass-test-34 system-monitor[606]: Frequency: 1368MHz LoadAvg: 0.75 CPU Load: 4% Temp: 80.3°C Cooling: 0/8 Oct 9 22:39:36 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:39:42 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:39:53 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:40:07 mass-test-34 npm[729]: 9 Oct 22:40:07 - [error] [http request:d0d57557.05fd78] no response from server Oct 9 22:40:10 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:40:20 mass-test-34 systemd[1]: systemd-udevd.service: Watchdog timeout (limit 3min)! Oct 9 22:40:20 mass-test-34 systemd[1]: systemd-udevd.service: Killing process 328 (systemd-udevd) with signal SIGABRT. Oct 9 22:40:24 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:40:31 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:40:50 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:40:57 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:41:15 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:41:24 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:41:34 mass-test-34 dhclient[1306]: DHCPREQUEST for 10.50.180.71 on eth0 to 10.48.0.1 port 67 Oct 9 22:41:38 mass-test-34 npm[729]: 9 Oct 22:41:38 - [error] [http request:d0d57557.05fd78] no response from server Oct 9 22:17:14 mass-test-34 syslog-ng[578]: syslog-ng starting up; version='3.19.1' Oct 9 22:17:12 mass-test-34 kernel: Booting Linux on physical CPU 0x0
maracuja Posted October 12, 2020 Posted October 12, 2020 I changed the swappiness from 100 (=aggressive swapping) to 20. sysctl -w vm.swappiness=20 System seemed more responsive first but it got stuck again when installing php7.3 I set up the led to blink when accessing the sd card: echo "mmc0" > /sys/class/leds/orangepi\:red\:status/trigger It's fully on now. System only responds to pings. Guess the system is so busy that it doesn't even switch the LED.
guidol Posted October 12, 2020 Posted October 12, 2020 1 hour ago, maracuja said: Memory before the test: total used free shared buff/cache available Mem: 238Mi 67Mi 132Mi 0.0Ki 38Mi 165Mi Swap: 59Mi 20Mi 38Mi My OPi Zero 512MB with Linux opi-zero 5.9.0-rc7-sunxi #trunk SMP Sat Oct 10 21:41:45 +03 2020 armv7l GNU/Linux after reboot: MiB Mem : 492.1 total, 337.5 free, 56.2 used, 98.4 buff/cache MiB Swap: 246.0 total, 246.0 free, 0.0 used. 419.0 avail Mem governor for all 4 cpus: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 2 3 CPUs which need to have their frequency coordinated by software: 0 1 2 3 maximum transition latency: 5.44 ms. hardware limits: 480 MHz - 1.01 GHz available frequency steps: 480 MHz, 648 MHz, 816 MHz, 960 MHz, 1.01 GHz available cpufreq governors: conservative, ondemand, userspace, powersave, performance, schedutil current policy: frequency should be within 480 MHz and 1.01 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 816 MHz (asserted by call to hardware). cpufreq stats: 480 MHz:99.80%, 648 MHz:0.05%, 816 MHz:0.07%, 960 MHz:0.03%, 1.01 GHz:0.05% (15687) root@opi-zero(192.168.6.99):~# uptime 15:46:22 up 1 day, 15:25, 1 user, load average: 0.00, 0.00, 0.00
maracuja Posted October 12, 2020 Posted October 12, 2020 Next memory exception (page allocation failure) I found while doing rsync... This time I even disabled ram-log (50MB). ZRAM still @ 50%. During this failure the system did not freeze. So even if you have 512M, there might still be something broken with page allocation on kernels after 4.9.x! [ 147.806318] rsync: page allocation failure: order:0, mode:0xa20(GFP_ATOMIC), nodemask=(null),cpuset=/,mems_allowed=0 [ 147.806351] CPU: 0 PID: 1451 Comm: rsync Not tainted 5.4.8-sunxi #19.11.6 [ 147.806355] Hardware name: Allwinner sun8i Family [ 147.806392] [<c010da8d>] (unwind_backtrace) from [<c010a0b1>] (show_stack+0x11/0x14) [ 147.806409] [<c010a0b1>] (show_stack) from [<c093634f>] (dump_stack+0x6f/0x7c) [ 147.806427] [<c093634f>] (dump_stack) from [<c022dfa9>] (warn_alloc+0x99/0x100) [ 147.806440] [<c022dfa9>] (warn_alloc) from [<c022ebf9>] (__alloc_pages_nodemask+0xbe9/0xc9c) [ 147.806451] [<c022ebf9>] (__alloc_pages_nodemask) from [<c022ee1b>] (page_frag_alloc+0xe3/0xec) [ 147.806463] [<c022ee1b>] (page_frag_alloc) from [<c07ec1e1>] (__napi_alloc_skb+0x6d/0xac) [ 147.806481] [<c07ec1e1>] (__napi_alloc_skb) from [<c06e362d>] (stmmac_napi_poll_rx+0x285/0x7d8) [ 147.806498] [<c06e362d>] (stmmac_napi_poll_rx) from [<c07fe703>] (net_rx_action+0xdb/0x2dc) [ 147.806511] [<c07fe703>] (net_rx_action) from [<c01022f7>] (__do_softirq+0xdf/0x288) [ 147.806524] [<c01022f7>] (__do_softirq) from [<c01202c3>] (irq_exit+0x7b/0x90) [ 147.806541] [<c01202c3>] (irq_exit) from [<c01601d3>] (__handle_domain_irq+0x47/0x84) [ 147.806557] [<c01601d3>] (__handle_domain_irq) from [<c05ca51d>] (gic_handle_irq+0x39/0x6c) [ 147.806569] [<c05ca51d>] (gic_handle_irq) from [<c0101ae5>] (__irq_svc+0x65/0x94) [ 147.806574] Exception stack(0xc00bbd78 to 0xc00bbdc0) [ 147.806581] bd60: cd0fde58 c00bbec8 [ 147.806590] bd80: c00bbe10 00002c20 cf0ab3c0 00000002 cd0fde58 cfb3e010 cfb3e037 00000003 [ 147.806599] bda0: 00000027 c00bbe78 5841c1d2 c00bbdc8 c025bf6f c0266be0 a0070033 ffffffff [ 147.806614] [<c0101ae5>] (__irq_svc) from [<c0266be0>] (__d_lookup_rcu+0x50/0x114) [ 147.806630] [<c0266be0>] (__d_lookup_rcu) from [<c025bf6f>] (lookup_fast+0x3b/0x1bc) [ 147.806642] [<c025bf6f>] (lookup_fast) from [<c025e195>] (path_openat+0xcd/0xe14) [ 147.806651] [<c025e195>] (path_openat) from [<c025fabf>] (do_filp_open+0x4f/0x90) [ 147.806665] [<c025fabf>] (do_filp_open) from [<c0251081>] (do_sys_open+0x125/0x194) [ 147.806677] [<c0251081>] (do_sys_open) from [<c0101001>] (ret_fast_syscall+0x1/0x62) [ 147.806681] Exception stack(0xc00bbfa8 to 0xc00bbff0) [ 147.806689] bfa0: b6ee4000 b6f53968 ffffff9c bebada14 00020000 00000000 [ 147.806698] bfc0: b6ee4000 b6f53968 bebada14 00000142 bebada14 00478418 00008000 00466948 [ 147.806705] bfe0: 00000142 bebad8c8 b6e7c25d b6e05746 [ 147.806709] Mem-Info: [ 147.806726] active_anon:99 inactive_anon:249 isolated_anon:0 active_file:447 inactive_file:502 isolated_file:0 unevictable:4 dirty:0 writeback:0 unstable:0 slab_reclaimable:3973 slab_unreclaimable:7604 mapped:592 shmem:80 pagetables:415 bounce:0 free:32645 free_pcp:56 free_cma:32145 [ 147.806739] Node 0 active_anon:396kB inactive_anon:996kB active_file:1788kB inactive_file:2008kB unevictable:16kB isolated(anon):0kB isolated(file):0kB mapped:2368kB dirty:0kB writeback:0kB shmem:320kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [ 147.806756] Normal free:130580kB min:5388kB low:5708kB high:6028kB active_anon:368kB inactive_anon:1004kB active_file:1752kB inactive_file:2052kB unevictable:16kB writepending:0kB present:262144kB managed:243968kB mlocked:16kB kernel_stack:1168kB pagetables:1660kB bounce:0kB free_pcp:224kB local_pcp:32kB free_cma:128580kB [ 147.806758] lowmem_reserve[]: 0 0 0 [ 147.806766] Normal: 3*4kB (EC) 207*8kB (UEC) 158*16kB (UEC) 41*32kB (UC) 12*64kB (C) 13*128kB (C) 7*256kB (C) 2*512kB (C) 1*1024kB (C) 0*2048kB 29*4096kB (C) = 130564kB [ 147.806801] 1050 total pagecache pages [ 147.806808] 6 pages in swap cache [ 147.806813] Swap cache stats: add 18554, delete 18543, find 779/1883 [ 147.806816] Free swap = 52860kB [ 147.806818] Total swap = 121980kB [ 147.806821] 65536 pages RAM [ 147.806824] 0 pages HighMem/MovableOnly [ 147.806826] 4544 pages reserved [ 147.806829] 32768 pages cma reserved
Filipe Carvalhedo Posted December 11, 2020 Posted December 11, 2020 Hello, I have been using many of these cards as gateways for my product for 2 years and I realized that some stop working and do not return to work alone. We have scritps to restart if communicating stop with our server. But still they are in a state that does not respond to ssh and also does not restart the card. I've tried to connect watchdog on the board, but it's as if it isn't locked, but it doesn't respond to ssh (using the zerotier) or the commands we use. I made a test environment here at home with 5 orange pi zero and 2 with the nanopi neo to test the same software running on the boards and try to replicate the problem. I tried to put the same network structure that are used on clients using 4G modems, but I can hardly replicate the problem. I even made freeze it once but since I was unable to connect through the lan on ssh and therefore I didn't get much information. Now I am connecting one of the cards through uart and I am saving the logs to try to verify what is happening. I tested using a forkbomb to try to force the problem and I realized that it goes a long time without responding but after about 15 minutes the plate will respond again but with the processes finished (I don't know if it is normal behavior), but it is not yet the same problem because ssh answers in this case. I would like any suggestions from you. Thanks. Linux LLKGAACAAACAKJKL 4.19.62-sunxi #5.92 SMP Wed Jul 31 22:07:23 CEST 2019 armv7l GNU/Linux
R.Lucassen Posted December 20, 2020 Posted December 20, 2020 Hello guys, I'm back again in this thread. I fixed the CPU freq to 480 MHz or 640MHz (I don't need higher clock speed) and I can report that these boards (now running Armbian 5.9) kernel are stable. Probably the boards get unstable when switching frequency. The approx 10 boards have been running now for at least a few months at a fixed frequency. No problems seen, apart a uSD suffering from bitrot. The board is even stable using bokki wokki USB chargers as power supply. I did not try fixed frequencies at higher speed than 640MHz. R.
Igor Posted December 20, 2020 Posted December 20, 2020 5 minutes ago, R.Lucassen said: le. Probably the boards get unstable when switching frequency. This is common cause of troubles since stabilising CPU frequency switching is a hard job.
R.Lucassen Posted December 20, 2020 Posted December 20, 2020 Apparently yes. I never had issues with governers before and I use them quite a lot. For me it first occurred on Orange-Pi Zero (H2) and Orange-Pi-PC (H3). Anyway, these boards are in use again and fortunately do not need to be trashed :-) R.
Recommended Posts