lomady Posted August 11, 2019 Posted August 11, 2019 I have been experiencing sporadic unresponsiveness from my H3 boards (5 of them). The ping on them still worked but ssh did not. Logs showed that system experienced a time jumping backward somewhere in 1978. Today another such glitch happened but I was able to ssh-in and saw this: Spoiler tesla@desk:~/Desktop$ ssh buster tesla@buster's password: ___ ____ _ ____ _____ / _ \| _ \(_) _ |___ \| ____| | | | | |_) | |_| |_ __) | _| | |_| | __/| |_ _| / __/| |___ \___/|_| |_| |_| |_____|_____| Welcome to Debian Buster with Armbian Linux 4.19.57-sunxi System load: 0.00 0.01 0.07 Up time: 19819 days Memory usage: 5 % of 2013MB Zram usage: 3 % of 1006Mb IP: 192.168.8.190 CPU temp: 63°C Usage of /: 91% of 15G Last login: Wed Jul 24 21:26:32 2019 from fda1:f279:3f90::6fe tesla@buster:~$ date Fri Aug 25 01:36:17 UTC 1978 tesla@buster:~$ dmesg [ 4.820614] FF [ 4.820617] FF [ 4.820619] FF [ 4.820621] FF [ 4.820624] FF [ 4.820626] FF [ 4.820628] FF [ 4.820630] FF [ 4.820633] FF [ 4.820635] FF [ 4.820637] FF [ 4.820639] FF [ 4.820642] FF [ 4.820644] FF [ 4.820646] FF [ 4.820649] FF [ 4.820651] FF [ 4.820653] FF [ 4.820655] FF [ 4.820658] FF [ 4.820660] FF [ 4.820662] FF [ 4.820664] FF [ 4.820667] FF [ 4.820669] FF [ 4.820671] FF [ 4.820674] FF [ 4.820676] FF [ 4.820678] FF [ 4.820681] FF [ 4.820683] FF [ 4.820686] FF [ 4.820688] FF [ 4.820690] FF [ 4.820692] FF [ 4.820695] FF [ 4.820697] FF [ 4.820699] FF [ 4.820701] FF [ 4.820704] FF [ 4.820706] FF [ 4.820708] FF [ 4.820710] FF [ 4.820713] FF [ 4.820715] FF [ 4.820717] FF [ 4.820719] FF [ 4.820721] FF [ 4.820724] FF [ 4.820726] FF [ 4.820728] FF [ 4.820731] FF [ 4.820733] FF [ 4.820735] FF [ 4.820737] FF [ 4.820739] FF [ 4.820742] FF [ 4.820744] FF [ 4.820746] FF [ 4.820749] FF [ 4.820751] FF [ 4.820753] FF [ 4.820755] FF [ 4.820758] FF [ 4.820760] FF [ 4.820762] FF [ 4.820764] FF [ 4.820767] FF [ 4.820769] FF [ 4.820771] FF [ 4.820773] FF [ 4.820775] FF [ 4.820778] FF [ 4.820780] FF [ 4.820782] FF [ 4.820784] FF [ 4.820787] FF [ 4.820789] FF [ 4.820791] FF [ 4.820793] FF [ 4.820796] FF [ 4.820798] FF [ 4.820800] FF [ 4.820803] FF [ 4.820805] FF [ 4.820807] FF [ 4.820809] FF [ 4.820811] FF [ 4.820814] FF [ 4.820816] FF [ 4.820818] FF [ 4.820820] FF [ 4.820822] FF [ 4.820825] FF [ 4.820827] FF [ 4.820829] FF [ 4.820831] FF [ 4.820834] FF [ 4.820836] FF [ 4.820838] FF [ 4.820840] FF [ 5.216808] systemd-journald[236]: Received request to flush runtime journal from PID 1 [ 5.277123] cfg80211: failed to load regulatory.db [ 5.502415] zram: Added device: zram0 [ 5.504449] zram: Added device: zram1 [ 5.506046] zram: Added device: zram2 [ 5.560599] zram1: detected capacity change from 0 to 1055428608 [ 5.642817] cpu cpu0: Linked as a consumer to regulator.6 [ 5.642875] cpu cpu0: Dropping the link to regulator.6 [ 5.643094] cpu cpu0: Linked as a consumer to regulator.6 [ 5.684280] input: r_gpio_keys as /devices/platform/r_gpio_keys/input/input0 [ 5.865526] sun8i_ths 1c25000.thermal-sensor: no memory resources defined [ 5.865554] sun8i_ths: probe of 1c25000.thermal-sensor failed with error -22 [ 5.883193] thermal thermal_zone0: failed to read out thermal zone (-110) [ 5.906805] Registered IR keymap rc-empty [ 5.906957] rc rc0: sunxi-ir as /devices/platform/soc/1f02000.ir/rc/rc0 [ 5.907085] input: sunxi-ir as /devices/platform/soc/1f02000.ir/rc/rc0/input1 [ 5.907606] rc rc0: lirc_dev: driver sunxi-ir registered at minor = 0, raw IR receiver, no transmitter [ 5.931781] sun4i-codec 1c22c00.codec: ASoC: codec-analog@1f015c0 not registered [ 5.931793] sun4i-codec 1c22c00.codec: Failed to register our card [ 5.937961] sun4i-codec 1c22c00.codec: ASoC: Failed to create component debugfs directory [ 5.945489] sunxi-ir 1f02000.ir: initialized sunXi IR driver [ 5.948621] lima 1c40000.gpu: bus rate = 200000000 [ 5.948632] lima 1c40000.gpu: mod rate = 384000000 [ 5.956148] [TTM] Zone kernel: Available graphics memory: 375336 kiB [ 5.956163] [TTM] Zone highmem: Available graphics memory: 1030690 kiB [ 5.956169] [TTM] Initializing pool allocator [ 5.956934] sun4i-codec 1c22c00.codec: Codec <-> 1c22c00.codec mapping ok [ 5.957073] lima 1c40000.gpu: gp - mali400 version major 1 minor 1 [ 5.957126] lima 1c40000.gpu: pp0 - mali400 version major 1 minor 1 [ 5.957179] lima 1c40000.gpu: pp1 - mali400 version major 1 minor 1 [ 5.957218] lima 1c40000.gpu: l2 cache 64K, 4-way, 64byte cache line, 64bit external bus [ 5.981749] [drm] Initialized lima 1.0.0 20170325 for 1c40000.gpu on minor 1 [ 6.622646] Adding 1030688k swap on /dev/zram1. Priority:5 extents:1 across:1030688k SSFS [ 6.669213] asoc-simple-card soc:sound: i2s-hifi <-> 1c22800.i2s mapping ok [ 6.823905] zram0: detected capacity change from 0 to 52428800 [ 6.913057] random: crng init done [ 6.913068] random: 7 urandom warning(s) missed due to ratelimiting [ 9.255950] EXT4-fs (zram0): mounted filesystem without journal. Opts: discard [ 14.241422] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 14.242091] RTL8211E Gigabit Ethernet 0.2:01: attached PHY driver [RTL8211E Gigabit Ethernet] (mii_bus:phy_addr=0.2:01, irq=POLL) [ 14.244212] dwmac-sun8i 1c30000.ethernet eth0: No Safety Features support found [ 14.244226] dwmac-sun8i 1c30000.ethernet eth0: No MAC Management Counters available [ 14.244234] dwmac-sun8i 1c30000.ethernet eth0: PTP not supported by HW [ 14.244626] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 14.302378] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready [ 15.543905] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready [ 15.773736] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready [ 20.382120] dwmac-sun8i 1c30000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx [ 20.382178] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [ 51.984440] dwmac-sun8i 1c30000.ethernet eth0: Too many address, switching to promiscuous [59155.002962] TCP: request_sock_TCP: Possible SYN flooding on port 3659. Sending cookies. Check SNMP counters. [1665394.103392] hrtimer: interrupt took 893138 ns [2736263.499172] rcu: INFO: rcu_sched self-detected stall on CPU [2736263.499195] rcu: 0-...!: (1 ticks this GP) idle=642/0/0x1 softirq=68967472/68967472 fqs=0 [2736263.499197] rcu: (t=249107 jiffies g=147892009 q=729) [2736263.499208] rcu: rcu_sched kthread starved for 249107 jiffies! g147892009 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [2736263.499210] rcu: RCU grace-period kthread stack dump: [2736263.499214] rcu_sched I 0 10 2 0x00000000 [2736263.499249] [<c090a25b>] (__schedule) from [<c090a7c7>] (schedule+0x2f/0x68) [2736263.499261] [<c090a7c7>] (schedule) from [<c090d047>] (schedule_timeout+0x77/0x320) [2736263.499275] [<c090d047>] (schedule_timeout) from [<c016aed3>] (rcu_gp_kthread+0x41f/0x728) [2736263.499287] [<c016aed3>] (rcu_gp_kthread) from [<c0132ae5>] (kthread+0xfd/0x104) [2736263.499297] [<c0132ae5>] (kthread) from [<c01010f9>] (ret_from_fork+0x11/0x38) [2736263.499301] Exception stack(0xee54bfb0 to 0xee54bff8) [2736263.499307] bfa0: 00000000 00000000 00000000 00000000 [2736263.499313] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [2736263.499319] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [2736263.499334] NMI backtrace for cpu 0 [2736263.499341] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.19.57-sunxi #5.90 [2736263.499344] Hardware name: Allwinner sun8i Family [2736263.499355] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2736263.499366] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2736263.499375] [<c08fb6a1>] (dump_stack) from [<c08ff923>] (nmi_cpu_backtrace+0x8f/0x90) [2736263.499383] [<c08ff923>] (nmi_cpu_backtrace) from [<c08ff9d3>] (nmi_trigger_cpumask_backtrace+0xaf/0xe0) [2736263.499391] [<c08ff9d3>] (nmi_trigger_cpumask_backtrace) from [<c016c663>] (rcu_dump_cpu_stacks+0x7b/0x98) [2736263.499398] [<c016c663>] (rcu_dump_cpu_stacks) from [<c016bc91>] (rcu_check_callbacks+0x4f5/0x6c8) [2736263.499408] [<c016bc91>] (rcu_check_callbacks) from [<c0170ff3>] (update_process_times+0x2b/0x48) [2736263.499419] [<c0170ff3>] (update_process_times) from [<c017efbb>] (tick_sched_timer+0x37/0x74) [2736263.499428] [<c017efbb>] (tick_sched_timer) from [<c0171855>] (__hrtimer_run_queues+0x105/0x254) [2736263.499438] [<c0171855>] (__hrtimer_run_queues) from [<c0172315>] (hrtimer_interrupt+0xb5/0x200) [2736263.499451] [<c0172315>] (hrtimer_interrupt) from [<c07b0ea1>] (arch_timer_handler_phys+0x25/0x28) [2736263.499462] [<c07b0ea1>] (arch_timer_handler_phys) from [<c0162fd3>] (handle_percpu_devid_irq+0x57/0x19c) [2736263.499472] [<c0162fd3>] (handle_percpu_devid_irq) from [<c015f3d5>] (generic_handle_irq+0x1d/0x28) [2736263.499482] [<c015f3d5>] (generic_handle_irq) from [<c015f879>] (__handle_domain_irq+0x45/0x84) [2736263.499493] [<c015f879>] (__handle_domain_irq) from [<c05c9469>] (gic_handle_irq+0x39/0x68) [2736263.499502] [<c05c9469>] (gic_handle_irq) from [<c0101a65>] (__irq_svc+0x65/0x94) [2736263.499506] Exception stack(0xc0e01f18 to 0xc0e01f60) [2736263.499510] 1f00: 00000000 283f8640 [2736263.499517] 1f20: eed9e438 c0116441 ffffe000 c0e04d70 c0e04db8 00000001 00000000 c0e04d48 [2736263.499524] 1f40: c0dba870 00000000 00000000 c0e01f68 c01078f3 c01078f4 400f0033 ffffffff [2736263.499533] [<c0101a65>] (__irq_svc) from [<c01078f4>] (arch_cpu_idle+0x28/0x2c) [2736263.499543] [<c01078f4>] (arch_cpu_idle) from [<c013e993>] (do_idle+0x14b/0x1d8) [2736263.499551] [<c013e993>] (do_idle) from [<c013ec15>] (cpu_startup_entry+0x19/0x1c) [2736263.499562] [<c013ec15>] (cpu_startup_entry) from [<c0d00bad>] (start_kernel+0x3a7/0x3c6) [2736263.499573] Sending NMI from CPU 0 to CPUs 1: [2736263.499708] NMI backtrace for cpu 1 [2736263.499711] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.57-sunxi #5.90 [2736263.499713] Hardware name: Allwinner sun8i Family [2736263.499715] PC is at arch_cpu_idle+0x28/0x2c [2736263.499716] LR is at arch_cpu_idle+0x27/0x2c [2736263.499719] pc : [<c01078f4>] lr : [<c01078f3>] psr: 400f0033 [2736263.499720] sp : ee551fb0 ip : ffffffc9 fp : 00000000 [2736263.499722] r10: c0dba870 r9 : c0e04d48 r8 : 00000000 [2736263.499724] r7 : 00000002 r6 : c0e04db8 r5 : c0e04d70 r4 : ffffe000 [2736263.499726] r3 : c0116441 r2 : eedb0438 r1 : 0273f180 r0 : 00000000 [2736263.499729] Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA Thumb Segment none [2736263.499731] Control: 50c5387d Table: 6ab1806a DAC: 00000051 [2736263.499733] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.57-sunxi #5.90 [2736263.499735] Hardware name: Allwinner sun8i Family [2736263.499737] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2736263.499739] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2736263.499742] [<c08fb6a1>] (dump_stack) from [<c08ff8ed>] (nmi_cpu_backtrace+0x59/0x90) [2736263.499744] [<c08ff8ed>] (nmi_cpu_backtrace) from [<c010c5d9>] (handle_IPI+0x85/0x2c0) [2736263.499747] [<c010c5d9>] (handle_IPI) from [<c05c9497>] (gic_handle_irq+0x67/0x68) [2736263.499749] [<c05c9497>] (gic_handle_irq) from [<c0101a65>] (__irq_svc+0x65/0x94) [2736263.499751] Exception stack(0xee551f60 to 0xee551fa8) [2736263.499753] 1f60: 00000000 0273f180 eedb0438 c0116441 ffffe000 c0e04d70 c0e04db8 00000002 [2736263.499756] 1f80: 00000000 c0e04d48 c0dba870 00000000 ffffffc9 ee551fb0 c01078f3 c01078f4 [2736263.499757] 1fa0: 400f0033 ffffffff [2736263.499760] [<c0101a65>] (__irq_svc) from [<c01078f4>] (arch_cpu_idle+0x28/0x2c) [2736263.499762] [<c01078f4>] (arch_cpu_idle) from [<c013e993>] (do_idle+0x14b/0x1d8) [2736263.499765] [<c013e993>] (do_idle) from [<c013ec15>] (cpu_startup_entry+0x19/0x1c) [2736263.499767] [<c013ec15>] (cpu_startup_entry) from [<401024b1>] (0x401024b1) [2736263.500587] Sending NMI from CPU 0 to CPUs 2: [2736273.500626] Sending NMI from CPU 0 to CPUs 3: [2736273.500792] NMI backtrace for cpu 3 [2736273.500795] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.19.57-sunxi #5.90 [2736273.500797] Hardware name: Allwinner sun8i Family [2736273.500799] PC is at arch_cpu_idle+0x28/0x2c [2736273.500801] LR is at arch_cpu_idle+0x27/0x2c [2736273.500803] pc : [<c01078f4>] lr : [<c01078f3>] psr: 40070033 [2736273.500804] sp : ee555fb0 ip : c0e03d00 fp : 00000000 [2736273.500806] r10: c0dba870 r9 : c0e04d48 r8 : 00000000 [2736273.500808] r7 : 00000008 r6 : c0e04db8 r5 : c0e04d70 r4 : ffffe000 [2736273.500810] r3 : c0116441 r2 : eedd4438 r1 : ae02b048 r0 : 00000000 [2736273.500813] Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA Thumb Segment none [2736273.500815] Control: 50c5387d Table: 6959c06a DAC: 00000051 [2736273.500817] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.19.57-sunxi #5.90 [2736273.500819] Hardware name: Allwinner sun8i Family [2736273.500821] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2736273.500824] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2736273.500826] [<c08fb6a1>] (dump_stack) from [<c08ff8ed>] (nmi_cpu_backtrace+0x59/0x90) [2736273.500829] [<c08ff8ed>] (nmi_cpu_backtrace) from [<c010c5d9>] (handle_IPI+0x85/0x2c0) [2736273.500831] [<c010c5d9>] (handle_IPI) from [<c05c9497>] (gic_handle_irq+0x67/0x68) [2736273.500833] [<c05c9497>] (gic_handle_irq) from [<c0101a65>] (__irq_svc+0x65/0x94) [2736273.500835] Exception stack(0xee555f60 to 0xee555fa8) [2736273.500838] 5f60: 00000000 ae02b048 eedd4438 c0116441 ffffe000 c0e04d70 c0e04db8 00000008 [2736273.500840] 5f80: 00000000 c0e04d48 c0dba870 00000000 c0e03d00 ee555fb0 c01078f3 c01078f4 [2736273.500842] 5fa0: 40070033 ffffffff [2736273.500844] [<c0101a65>] (__irq_svc) from [<c01078f4>] (arch_cpu_idle+0x28/0x2c) [2736273.500846] [<c01078f4>] (arch_cpu_idle) from [<c013e993>] (do_idle+0x14b/0x1d8) [2736273.500848] [<c013e993>] (do_idle) from [<c013ec15>] (cpu_startup_entry+0x19/0x1c) [2736273.500851] [<c013ec15>] (cpu_startup_entry) from [<401024b1>] (0x401024b1) [2738018.318428] systemd-journald[236]: Failed to run event loop: Invalid argument [2751066.151818] rcu: INFO: rcu_sched self-detected stall on CPU [2751066.151838] rcu: 0-...!: (1 ticks this GP) idle=416/1/0x40000002 softirq=74660417/74660417 fqs=0 [2751066.151839] rcu: (t=245885 jiffies g=151959821 q=919) [2751066.151851] rcu: rcu_sched kthread starved for 245885 jiffies! g151959821 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=3 [2751066.151854] rcu: RCU grace-period kthread stack dump: [2751066.151857] rcu_sched I 0 10 2 0x00000000 [2751066.151888] [<c090a25b>] (__schedule) from [<c090a7c7>] (schedule+0x2f/0x68) [2751066.151899] [<c090a7c7>] (schedule) from [<c090d047>] (schedule_timeout+0x77/0x320) [2751066.151913] [<c090d047>] (schedule_timeout) from [<c016aed3>] (rcu_gp_kthread+0x41f/0x728) [2751066.151925] [<c016aed3>] (rcu_gp_kthread) from [<c0132ae5>] (kthread+0xfd/0x104) [2751066.151935] [<c0132ae5>] (kthread) from [<c01010f9>] (ret_from_fork+0x11/0x38) [2751066.151939] Exception stack(0xee54bfb0 to 0xee54bff8) [2751066.151945] bfa0: 00000000 00000000 00000000 00000000 [2751066.151951] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [2751066.151957] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [2751066.151979] NMI backtrace for cpu 0 [2751066.151987] CPU: 0 PID: 1303 Comm: qbittorrent-nox Not tainted 4.19.57-sunxi #5.90 [2751066.151989] Hardware name: Allwinner sun8i Family [2751066.152001] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751066.152011] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751066.152020] [<c08fb6a1>] (dump_stack) from [<c08ff923>] (nmi_cpu_backtrace+0x8f/0x90) [2751066.152028] [<c08ff923>] (nmi_cpu_backtrace) from [<c08ff9d3>] (nmi_trigger_cpumask_backtrace+0xaf/0xe0) [2751066.152036] [<c08ff9d3>] (nmi_trigger_cpumask_backtrace) from [<c016c663>] (rcu_dump_cpu_stacks+0x7b/0x98) [2751066.152043] [<c016c663>] (rcu_dump_cpu_stacks) from [<c016bc91>] (rcu_check_callbacks+0x4f5/0x6c8) [2751066.152053] [<c016bc91>] (rcu_check_callbacks) from [<c0170ff3>] (update_process_times+0x2b/0x48) [2751066.152064] [<c0170ff3>] (update_process_times) from [<c017efbb>] (tick_sched_timer+0x37/0x74) [2751066.152074] [<c017efbb>] (tick_sched_timer) from [<c0171855>] (__hrtimer_run_queues+0x105/0x254) [2751066.152084] [<c0171855>] (__hrtimer_run_queues) from [<c0172315>] (hrtimer_interrupt+0xb5/0x200) [2751066.152097] [<c0172315>] (hrtimer_interrupt) from [<c07b0ea1>] (arch_timer_handler_phys+0x25/0x28) [2751066.152108] [<c07b0ea1>] (arch_timer_handler_phys) from [<c0162fd3>] (handle_percpu_devid_irq+0x57/0x19c) [2751066.152118] [<c0162fd3>] (handle_percpu_devid_irq) from [<c015f3d5>] (generic_handle_irq+0x1d/0x28) [2751066.152127] [<c015f3d5>] (generic_handle_irq) from [<c015f879>] (__handle_domain_irq+0x45/0x84) [2751066.152139] [<c015f879>] (__handle_domain_irq) from [<c05c9469>] (gic_handle_irq+0x39/0x68) [2751066.152148] [<c05c9469>] (gic_handle_irq) from [<c0101e41>] (__irq_usr+0x61/0x80) [2751066.152152] Exception stack(0xeab21fb0 to 0xeab21ff8) [2751066.152157] 1fa0: 00000000 ad07dbe0 6510e700 000f144c [2751066.152164] 1fc0: b2efeaf8 b2efeb24 00000000 01574470 b6f90000 b6e74af5 ad07dbf8 b2efeac0 [2751066.152170] 1fe0: 00000000 b2efea40 b6eb83d9 b6e9ebae 600f0030 ffffffff [2751066.152181] Sending NMI from CPU 0 to CPUs 1: [2751076.174009] Sending NMI from CPU 0 to CPUs 2: [2751086.174051] Sending NMI from CPU 0 to CPUs 3: [2751096.174300] NMI backtrace for cpu 3 [2751096.174304] CPU: 3 PID: 32707 Comm: geth Not tainted 4.19.57-sunxi #5.90 [2751096.174306] Hardware name: Allwinner sun8i Family [2751096.174308] PC is at _raw_spin_unlock_irqrestore+0x1c/0x20 [2751096.174309] LR is at rcu_process_callbacks+0x34b/0x3dc [2751096.174311] pc : [<c090dd48>] lr : [<c016aa23>] psr: 000f0133 [2751096.174313] sp : c2389ec0 ip : 0000000f fp : c0e11880 [2751096.174315] r10: c0dbbe00 r9 : c0e11d80 r8 : c0e11880 [2751096.174317] r7 : c0e11880 r6 : 00000000 r5 : c0e11880 r4 : eedd9e00 [2751096.174319] r3 : 00001050 r2 : 00000000 r1 : 600f0113 r0 : c0e11880 [2751096.174321] Flags: nzcv IRQs on FIQs on Mode SVC_32 ISA Thumb Segment none [2751096.174323] Control: 50c5387d Table: 6959c06a DAC: 00000051 [2751096.174325] CPU: 3 PID: 32707 Comm: geth Not tainted 4.19.57-sunxi #5.90 [2751096.174327] Hardware name: Allwinner sun8i Family [2751096.174330] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751096.174332] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751096.174334] [<c08fb6a1>] (dump_stack) from [<c08ff8ed>] (nmi_cpu_backtrace+0x59/0x90) [2751096.174337] [<c08ff8ed>] (nmi_cpu_backtrace) from [<c010c5d9>] (handle_IPI+0x85/0x2c0) [2751096.174339] [<c010c5d9>] (handle_IPI) from [<c05c9497>] (gic_handle_irq+0x67/0x68) [2751096.174341] [<c05c9497>] (gic_handle_irq) from [<c0101a65>] (__irq_svc+0x65/0x94) [2751096.174343] Exception stack(0xc2389e70 to 0xc2389eb8) [2751096.174345] 9e60: c0e11880 600f0113 00000000 00001050 [2751096.174348] 9e80: eedd9e00 c0e11880 00000000 c0e11880 c0e11880 c0e11d80 c0dbbe00 c0e11880 [2751096.174350] 9ea0: 0000000f c2389ec0 c016aa23 c090dd48 000f0133 ffffffff [2751096.174353] [<c0101a65>] (__irq_svc) from [<c090dd48>] (_raw_spin_unlock_irqrestore+0x1c/0x20) [2751096.174355] [<c090dd48>] (_raw_spin_unlock_irqrestore) from [<c016aa23>] (rcu_process_callbacks+0x34b/0x3dc) [2751096.174358] [<c016aa23>] (rcu_process_callbacks) from [<c010226d>] (__do_softirq+0xd5/0x27c) [2751096.174360] [<c010226d>] (__do_softirq) from [<c011ff03>] (irq_exit+0x8f/0xc0) [2751096.174362] [<c011ff03>] (irq_exit) from [<c015f87d>] (__handle_domain_irq+0x49/0x84) [2751096.174365] [<c015f87d>] (__handle_domain_irq) from [<c05c9469>] (gic_handle_irq+0x39/0x68) [2751096.174368] [<c05c9469>] (gic_handle_irq) from [<c0101e41>] (__irq_usr+0x61/0x80) [2751096.174369] Exception stack(0xc2389fb0 to 0xc2389ff8) [2751096.174372] 9fa0: 00000001 0387cff0 00000000 00000001 [2751096.174374] 9fc0: 00000002 0387cf80 0387cfc0 ffffffff 00000007 00000008 03bfd420 015552b0 [2751096.174376] 9fe0: 0387cb80 0395fcdc 00024170 00024174 600f0010 ffffffff [2751321.729514] armbianmonitor invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=2, oom_score_adj=0 [2751321.729524] armbianmonitor cpuset=/ mems_allowed=0 [2751321.729540] CPU: 3 PID: 7416 Comm: armbianmonitor Not tainted 4.19.57-sunxi #5.90 [2751321.729543] Hardware name: Allwinner sun8i Family [2751321.729571] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751321.729584] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751321.729597] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751321.729609] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751321.729618] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751321.729629] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751321.729636] [<c01f12c5>] (__alloc_pages_nodemask) from [<c01f169d>] (__get_free_pages+0x15/0x20) [2751321.729648] [<c01f169d>] (__get_free_pages) from [<c0114631>] (pgd_alloc+0x19/0xf8) [2751321.729659] [<c0114631>] (pgd_alloc) from [<c01188d5>] (mm_init+0xad/0x12c) [2751321.729669] [<c01188d5>] (mm_init) from [<c011a16f>] (copy_process.part.5+0x109b/0x13e4) [2751321.729678] [<c011a16f>] (copy_process.part.5) from [<c011a5e1>] (_do_fork+0x89/0x328) [2751321.729687] [<c011a5e1>] (_do_fork) from [<c011a921>] (sys_clone+0x19/0x1c) [2751321.729695] [<c011a921>] (sys_clone) from [<c0101001>] (ret_fast_syscall+0x1/0x62) [2751321.729699] Exception stack(0xde5d3fa8 to 0xde5d3ff0) [2751321.729706] 3fa0: b6f3f238 b6f3f690 01200011 00000000 00000000 00000000 [2751321.729714] 3fc0: b6f3f238 b6f3f690 00000000 00000078 be8e1fd4 00000000 b6f3f1d0 019d1698 [2751321.729719] 3fe0: 00000078 be8e1f18 b6e53253 b6df5746 [2751321.729723] Mem-Info: [2751321.729738] active_anon:44298 inactive_anon:52193 isolated_anon:30 active_file:102328 inactive_file:90997 isolated_file:17 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16311 slab_unreclaimable:152338 mapped:12365 shmem:150 pagetables:701 bounce:0 free:6690 free_pcp:1708 free_cma:439 [2751321.729748] Node 0 active_anon:177192kB inactive_anon:208772kB active_file:409312kB inactive_file:363988kB unevictable:0kB isolated(anon):120kB isolated(file):68kB mapped:49460kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751321.729762] Normal free:16820kB min:3448kB low:4308kB high:5168kB active_anon:1112kB inactive_anon:1400kB active_file:0kB inactive_file:428kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8216kB pagetables:612kB bounce:0kB free_pcp:2548kB local_pcp:0kB free_cma:0kB [2751321.729764] lowmem_reserve[]: 0 10239 10239 [2751321.729783] HighMem free:9940kB min:512kB low:1880kB high:3248kB active_anon:175908kB inactive_anon:207916kB active_file:409072kB inactive_file:363792kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:2192kB bounce:0kB free_pcp:4284kB local_pcp:1200kB free_cma:1756kB [2751321.729785] lowmem_reserve[]: 0 0 0 [2751321.729794] Normal: 3966*4kB (UMEH) 151*8kB (UE) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 17072kB [2751321.729823] HighMem: 2322*4kB (UMC) 34*8kB (UMC) 1*16kB (C) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 9576kB [2751321.729854] 181565 total pagecache pages [2751321.729859] 190 pages in swap cache [2751321.729863] Swap cache stats: add 150616, delete 150421, find 40924368/40977975 [2751321.729866] Free swap = 812064kB [2751321.729868] Total swap = 1030688kB [2751321.729870] 524288 pages RAM [2751321.729872] 327680 pages HighMem/MovableOnly [2751321.729875] 8943 pages reserved [2751321.729877] 32768 pages cma reserved [2751321.729880] Tasks state (memory values in pages): [2751321.729882] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751321.729903] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751321.729910] [ 322] 0 322 1812 320 14336 642 0 haveged [2751321.729916] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751321.729925] [ 528] 0 528 742 242 12288 0 0 alsactl [2751321.729932] [ 537] 0 537 6117 499 24576 51 0 rsyslogd [2751321.729938] [ 543] 0 543 1830 369 14336 22 0 cron [2751321.729944] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751321.729951] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751321.729957] [ 570] 0 570 17691 1134 47104 249 0 NetworkManager [2751321.729964] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751321.729971] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751321.729977] [ 712] 104 712 806 323 12288 0 0 chronyd [2751321.729984] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751321.729992] [ 836] 0 836 8994 600 26624 55 0 polkitd [2751321.729999] [ 1071] 0 1071 1559 383 12288 167 0 dhclient [2751321.730006] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751321.730012] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751321.730019] [ 1153] 0 1153 1559 365 14336 185 0 dhclient [2751321.730026] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751321.730032] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751321.730039] [ 1278] 1000 1278 1346 572 14336 346 0 screen [2751321.730046] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751321.730053] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751321.730060] [ 1283] 0 1283 1045 245 12288 80 0 su [2751321.730066] [ 1284] 0 1284 818 207 14336 222 0 bash [2751321.730073] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751321.730079] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751321.730086] [ 1297] 1000 1297 819 172 12288 223 0 bash [2751321.730093] [ 1300] 1000 1300 113847 50343 376832 21423 0 qbittorrent-nox [2751321.730099] [ 1313] 1000 1313 819 364 8192 236 0 bash [2751321.730106] [ 1534] 1000 1534 883 342 12288 300 0 bash [2751321.730112] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751321.730118] [ 32670] 1000 32670 368 79 8192 15 0 run.sh [2751321.730124] [ 32671] 1000 32671 298277 65736 684032 22436 0 geth [2751321.730132] [ 759] 1000 759 819 403 10240 224 0 bash [2751321.730145] [ 7416] 0 7416 741 335 10240 109 0 armbianmonitor [2751321.730150] Out of memory: Kill process 32671 (geth) score 114 or sacrifice child [2751321.730332] Killed process 32671 (geth) total-vm:1193108kB, anon-rss:223300kB, file-rss:39644kB, shmem-rss:0kB [2751322.032138] oom_reaper: reaped process 32671 (geth), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [2751330.451997] systemd-logind invoked oom-killer: gfp_mask=0x6102c0(GFP_KERNEL|__GFP_NOWARN|__GFP_NOMEMALLOC), nodemask=(null), order=0, oom_score_adj=0 [2751330.452005] systemd-logind cpuset=/ mems_allowed=0 [2751330.452022] CPU: 0 PID: 547 Comm: systemd-logind Not tainted 4.19.57-sunxi #5.90 [2751330.452025] Hardware name: Allwinner sun8i Family [2751330.452052] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751330.452064] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751330.452079] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751330.452091] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751330.452100] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751330.452111] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751330.452123] [<c01f12c5>] (__alloc_pages_nodemask) from [<c022a225>] (new_slab+0x1fd/0x504) [2751330.452134] [<c022a225>] (new_slab) from [<c022b843>] (___slab_alloc.constprop.14+0x2f7/0x3cc) [2751330.452142] [<c022b843>] (___slab_alloc.constprop.14) from [<c022dbd9>] (__kmalloc_track_caller+0x129/0x1b4) [2751330.452155] [<c022dbd9>] (__kmalloc_track_caller) from [<c07d6431>] (__kmalloc_reserve.constprop.23+0x1d/0x48) [2751330.452166] [<c07d6431>] (__kmalloc_reserve.constprop.23) from [<c07d64bb>] (__alloc_skb+0x5f/0x110) [2751330.452175] [<c07d64bb>] (__alloc_skb) from [<c07d6d59>] (alloc_skb_with_frags+0x2d/0x108) [2751330.452184] [<c07d6d59>] (alloc_skb_with_frags) from [<c07d1ab1>] (sock_alloc_send_pskb+0x18d/0x1a4) [2751330.452197] [<c07d1ab1>] (sock_alloc_send_pskb) from [<c087eb1b>] (unix_dgram_sendmsg+0xef/0x4a0) [2751330.452209] [<c087eb1b>] (unix_dgram_sendmsg) from [<c07ce881>] (sock_sendmsg+0x21/0x2c) [2751330.452218] [<c07ce881>] (sock_sendmsg) from [<c07cedc5>] (___sys_sendmsg+0x159/0x17c) [2751330.452227] [<c07cedc5>] (___sys_sendmsg) from [<c07cf8d3>] (__sys_sendmsg+0x37/0x58) [2751330.452236] [<c07cf8d3>] (__sys_sendmsg) from [<c01011f9>] (__sys_trace_return+0x1/0xc) [2751330.452240] Exception stack(0xeb46dfa8 to 0xeb46dff0) [2751330.452247] dfa0: 00000074 b6d65404 00000012 beb6e7b0 00004000 00000000 [2751330.452254] dfc0: 00000074 b6d65404 00000001 00000128 00000012 00000000 00000000 00000000 [2751330.452259] dfe0: 00000128 beb6e770 b6e8951f b6e05746 [2751330.452348] Mem-Info: [2751330.452366] active_anon:30540 inactive_anon:22016 isolated_anon:30 active_file:87132 inactive_file:85739 isolated_file:17 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16342 slab_unreclaimable:157401 mapped:2693 shmem:150 pagetables:493 bounce:0 free:76751 free_pcp:2392 free_cma:6975 [2751330.452379] Node 0 active_anon:122160kB inactive_anon:88064kB active_file:348528kB inactive_file:342956kB unevictable:0kB isolated(anon):120kB isolated(file):68kB mapped:10772kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751330.452392] Normal free:3288kB min:3448kB low:4308kB high:5168kB active_anon:152kB inactive_anon:92kB active_file:100kB inactive_file:428kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8608kB pagetables:292kB bounce:0kB free_pcp:4656kB local_pcp:740kB free_cma:0kB [2751330.452393] lowmem_reserve[]: 0 10239 10239 [2751330.452411] HighMem free:303716kB min:512kB low:1880kB high:3248kB active_anon:122244kB inactive_anon:88548kB active_file:348872kB inactive_file:343496kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:1680kB bounce:0kB free_pcp:4912kB local_pcp:1088kB free_cma:27900kB [2751330.452412] lowmem_reserve[]: 0 0 0 [2751330.452421] Normal: 905*4kB (UEH) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3620kB [2751330.452449] HighMem: 53912*4kB (UMC) 9875*8kB (UMC) 557*16kB (UMC) 11*32kB (UM) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 303912kB [2751330.452480] 173221 total pagecache pages [2751330.452485] 190 pages in swap cache [2751330.452489] Swap cache stats: add 150771, delete 150576, find 40924745/40978385 [2751330.452491] Free swap = 900896kB [2751330.452493] Total swap = 1030688kB [2751330.452496] 524288 pages RAM [2751330.452498] 327680 pages HighMem/MovableOnly [2751330.452500] 8943 pages reserved [2751330.452502] 32768 pages cma reserved [2751330.452505] Tasks state (memory values in pages): [2751330.452507] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751330.452531] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751330.452538] [ 322] 0 322 1812 320 14336 642 0 haveged [2751330.452545] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751330.452554] [ 528] 0 528 742 242 12288 0 0 alsactl [2751330.452562] [ 537] 0 537 6117 505 24576 51 0 rsyslogd [2751330.452568] [ 543] 0 543 1830 369 14336 22 0 cron [2751330.452574] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751330.452581] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751330.452588] [ 570] 0 570 17691 1134 47104 249 0 NetworkManager [2751330.452595] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751330.452602] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751330.452608] [ 712] 104 712 806 323 12288 0 0 chronyd [2751330.452615] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751330.452622] [ 836] 0 836 8994 600 26624 55 0 polkitd [2751330.452628] [ 1071] 0 1071 1559 383 12288 167 0 dhclient [2751330.452635] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751330.452642] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751330.452648] [ 1153] 0 1153 1559 365 14336 185 0 dhclient [2751330.452655] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751330.452661] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751330.452668] [ 1278] 1000 1278 1346 558 14336 360 0 screen [2751330.452675] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751330.452681] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751330.452688] [ 1283] 0 1283 1045 245 12288 80 0 su [2751330.452694] [ 1284] 0 1284 818 207 14336 222 0 bash [2751330.452701] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751330.452708] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751330.452714] [ 1297] 1000 1297 819 172 12288 223 0 bash [2751330.452721] [ 1300] 1000 1300 113847 50240 376832 21526 0 qbittorrent-nox [2751330.452728] [ 1313] 1000 1313 819 364 8192 236 0 bash [2751330.452734] [ 1534] 1000 1534 883 342 12288 300 0 bash [2751330.452741] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751330.452748] [ 32670] 1000 32670 368 79 8192 15 0 run.sh [2751330.452756] [ 759] 1000 759 819 403 10240 224 0 bash [2751330.452770] [ 7455] 0 7455 354 81 10240 0 0 sleep [2751330.452775] Out of memory: Kill process 1300 (qbittorrent-nox) score 92 or sacrifice child [2751330.452906] Killed process 1300 (qbittorrent-nox) total-vm:455388kB, anon-rss:197596kB, file-rss:3364kB, shmem-rss:0kB [2751330.696027] oom_reaper: reaped process 1300 (qbittorrent-nox), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [2751333.517891] armbianmonitor invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=2, oom_score_adj=0 [2751333.517901] armbianmonitor cpuset=/ mems_allowed=0 [2751333.517917] CPU: 0 PID: 7472 Comm: armbianmonitor Not tainted 4.19.57-sunxi #5.90 [2751333.517920] Hardware name: Allwinner sun8i Family [2751333.517947] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751333.517958] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751333.517973] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751333.517985] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751333.517995] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751333.518005] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751333.518013] [<c01f12c5>] (__alloc_pages_nodemask) from [<c01f169d>] (__get_free_pages+0x15/0x20) [2751333.518024] [<c01f169d>] (__get_free_pages) from [<c0114631>] (pgd_alloc+0x19/0xf8) [2751333.518035] [<c0114631>] (pgd_alloc) from [<c01188d5>] (mm_init+0xad/0x12c) [2751333.518046] [<c01188d5>] (mm_init) from [<c0244655>] (__do_execve_file+0x1f1/0x5f0) [2751333.518056] [<c0244655>] (__do_execve_file) from [<c0244c3b>] (sys_execve+0x27/0x2c) [2751333.518065] [<c0244c3b>] (sys_execve) from [<c0101001>] (ret_fast_syscall+0x1/0x62) [2751333.518070] Exception stack(0xe1d9bfa8 to 0xe1d9bff0) [2751333.518077] bfa0: 00540000 00540000 019d0820 019cda28 019ad720 4c987c00 [2751333.518085] bfc0: 00540000 00540000 019d0820 0000000b 00547b74 019cda28 019ad720 b6f40968 [2751333.518090] bfe0: b6e533d1 be8e1fec 004aafa5 b6e533d8 [2751333.518094] Mem-Info: [2751333.518109] active_anon:1276 inactive_anon:1990 isolated_anon:30 active_file:3616 inactive_file:1023 isolated_file:18 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16342 slab_unreclaimable:158998 mapped:2259 shmem:150 pagetables:237 bounce:0 free:312693 free_pcp:2087 free_cma:31055 [2751333.518120] Node 0 active_anon:5104kB inactive_anon:7960kB active_file:14464kB inactive_file:4092kB unevictable:0kB isolated(anon):120kB isolated(file):72kB mapped:9036kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751333.518134] Normal free:10344kB min:3448kB low:4308kB high:5168kB active_anon:152kB inactive_anon:92kB active_file:192kB inactive_file:412kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8720kB pagetables:292kB bounce:0kB free_pcp:3312kB local_pcp:0kB free_cma:0kB [2751333.518135] lowmem_reserve[]: 0 10239 10239 [2751333.518154] HighMem free:1240428kB min:512kB low:1880kB high:3248kB active_anon:5284kB inactive_anon:8396kB active_file:14504kB inactive_file:3624kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:656kB bounce:0kB free_pcp:5036kB local_pcp:1280kB free_cma:124220kB [2751333.518156] lowmem_reserve[]: 0 0 0 [2751333.518164] Normal: 2380*4kB (UMEH) 156*8kB (UME) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 10768kB [2751333.518194] HighMem: 8516*4kB (UMC) 7608*8kB (UMC) 6386*16kB (UMC) 5327*32kB (UMC) 3845*64kB (UMC) 2230*128kB (UMC) 927*256kB (UMC) 184*512kB (MC) 10*1024kB (M) 0*2048kB 0*4096kB = 1240848kB [2751333.518235] 4993 total pagecache pages [2751333.518240] 190 pages in swap cache [2751333.518244] Swap cache stats: add 150772, delete 150577, find 40924878/40978589 [2751333.518247] Free swap = 987424kB [2751333.518249] Total swap = 1030688kB [2751333.518251] 524288 pages RAM [2751333.518253] 327680 pages HighMem/MovableOnly [2751333.518255] 8943 pages reserved [2751333.518257] 32768 pages cma reserved [2751333.518260] Unreclaimable slab info: [2751333.518262] Name Used Total [2751333.518577] zs_handle 234KB 356KB [2751333.518684] RAWv6 47KB 47KB [2751333.518688] UDPv6 47KB 47KB [2751333.518692] tw_sock_TCPv6 15KB 15KB [2751333.518696] request_sock_TCPv6 15KB 15KB [2751333.518699] TCPv6 123KB 123KB [2751333.518703] sgpool-128 30KB 30KB [2751333.518707] sgpool-64 31KB 31KB [2751333.518711] sgpool-32 15KB 15KB [2751333.518715] sgpool-16 7KB 7KB [2751333.518721] mqueue_inode_cache 15KB 15KB [2751333.518733] nfsd_drc 3KB 3KB [2751333.518747] posix_timers_cache 15KB 15KB [2751333.518751] rpc_buffers 32KB 32KB [2751333.518754] xfrm_state 31KB 31KB [2751333.518759] RAW 31KB 31KB [2751333.518763] UDP 61KB 61KB [2751333.518767] tw_sock_TCP 75KB 75KB [2751333.518771] request_sock_TCP 11KB 11KB [2751333.518775] TCP 182KB 182KB [2751333.518779] eventpoll_pwq 15KB 15KB [2751333.518783] inotify_inode_mark 15KB 15KB [2751333.518787] request_queue 92KB 92KB [2751333.518791] blkdev_ioc 44KB 44KB [2751333.518795] biovec-max 540KB 540KB [2751333.518799] biovec-128 126KB 126KB [2751333.518803] biovec-64 63KB 63KB [2751333.518810] biovec-16 69KB 126KB [2751333.518814] ksm_mm_slot 15KB 15KB [2751333.518820] skbuff_fclone_cache 283KB 315KB [2751333.518824] skbuff_head_cache 374KB 374KB [2751333.518828] configfs_dir_cache 3KB 3KB [2751333.518832] file_lock_cache 15KB 15KB [2751333.518836] fsnotify_mark_connector 16KB 16KB [2751333.518840] net_namespace 29KB 29KB [2751333.518853] shmem_inode_cache 807KB 960KB [2751333.518857] taskstats 30KB 30KB [2751333.518861] proc_dir_entry 108KB 108KB [2751333.518865] pde_opener 15KB 15KB [2751333.518869] seq_file 15KB 15KB [2751333.518873] sigqueue 15KB 15KB [2751333.518877] kernfs_node_cache 2563KB 2563KB [2751333.518881] filp 248790KB 248790KB [2751333.518885] names_cache 128KB 128KB [2751333.518890] key_jar 108KB 108KB [2751333.518895] vm_area_struct 248KB 248KB [2751333.518899] mm_struct 32KB 32KB [2751333.518903] files_cache 20KB 20KB [2751333.518907] signal_cache 204KB 204KB [2751333.518911] sighand_cache 315KB 315KB [2751333.518915] task_struct 401KB 401KB [2751333.518919] cred_jar 872KB 872KB [2751333.518925] anon_vma_chain 189KB 228KB [2751333.518929] anon_vma 287KB 287KB [2751333.518933] pid 32KB 32KB [2751333.518937] trace_event_file 79KB 79KB [2751333.518965] ftrace_event_field 403KB 748KB [2751333.518969] pool_workqueue 188KB 188KB [2751333.518973] kmalloc-8192 5024KB 5024KB [2751333.518977] kmalloc-4096 384KB 384KB [2751333.518982] kmalloc-2048 782KB 928KB [2751333.518990] kmalloc-1024 806KB 864KB [2751333.518996] kmalloc-512 1134KB 1164KB [2751333.519000] kmalloc-256 193KB 200KB [2751333.519004] kmalloc-192 1634KB 1634KB [2751333.519008] kmalloc-128 1912KB 1912KB [2751333.519011] kmalloc-64 84092KB 84092KB [2751333.519015] kmem_cache_node 188KB 188KB [2751333.519022] kmem_cache 605KB 620KB [2751333.519025] Tasks state (memory values in pages): [2751333.519028] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751333.519054] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751333.519062] [ 322] 0 322 1812 320 14336 642 0 haveged [2751333.519069] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751333.519079] [ 528] 0 528 742 242 12288 0 0 alsactl [2751333.519086] [ 537] 0 537 6117 505 24576 51 0 rsyslogd [2751333.519093] [ 543] 0 543 1830 369 14336 22 0 cron [2751333.519099] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751333.519107] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751333.519114] [ 570] 0 570 17691 1134 47104 249 0 NetworkManager [2751333.519121] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751333.519128] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751333.519135] [ 712] 104 712 806 323 12288 0 0 chronyd [2751333.519142] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751333.519151] [ 836] 0 836 8994 602 26624 53 0 polkitd [2751333.519158] [ 1071] 0 1071 1559 383 12288 167 0 dhclient [2751333.519165] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751333.519172] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751333.519179] [ 1153] 0 1153 1559 365 14336 185 0 dhclient [2751333.519185] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751333.519193] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751333.519199] [ 1278] 1000 1278 1346 558 14336 360 0 screen [2751333.519206] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751333.519214] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751333.519221] [ 1283] 0 1283 1045 245 12288 80 0 su [2751333.519228] [ 1284] 0 1284 818 207 14336 222 0 bash [2751333.519235] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751333.519242] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751333.519249] [ 1297] 1000 1297 819 375 12288 183 0 bash [2751333.519256] [ 1313] 1000 1313 819 364 8192 236 0 bash [2751333.519263] [ 1534] 1000 1534 883 342 12288 300 0 bash [2751333.519270] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751333.519277] [ 32670] 1000 32670 368 79 8192 15 0 run.sh [2751333.519287] [ 759] 1000 759 819 403 10240 224 0 bash [2751333.519302] [ 7471] 0 7471 741 335 10240 109 0 armbianmonitor [2751333.519309] [ 7472] 0 7472 741 247 12288 109 0 armbianmonitor [2751333.519313] Out of memory: Kill process 570 (NetworkManager) score 1 or sacrifice child [2751333.519351] Killed process 1071 (dhclient) total-vm:6236kB, anon-rss:184kB, file-rss:1348kB, shmem-rss:0kB [2751333.520445] oom_reaper: reaped process 1071 (dhclient), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [2751336.366839] systemd-logind invoked oom-killer: gfp_mask=0x6040c0(GFP_KERNEL|__GFP_COMP), nodemask=(null), order=0, oom_score_adj=0 [2751336.366847] systemd-logind cpuset=/ mems_allowed=0 [2751336.366863] CPU: 2 PID: 547 Comm: systemd-logind Not tainted 4.19.57-sunxi #5.90 [2751336.366866] Hardware name: Allwinner sun8i Family [2751336.366893] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751336.366905] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751336.366919] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751336.366930] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751336.366940] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751336.366951] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751336.366963] [<c01f12c5>] (__alloc_pages_nodemask) from [<c022a225>] (new_slab+0x1fd/0x504) [2751336.366974] [<c022a225>] (new_slab) from [<c022b843>] (___slab_alloc.constprop.14+0x2f7/0x3cc) [2751336.366982] [<c022b843>] (___slab_alloc.constprop.14) from [<c022be89>] (kmem_cache_alloc+0xd5/0x168) [2751336.366991] [<c022be89>] (kmem_cache_alloc) from [<c07d04dd>] (sk_prot_alloc+0x21/0xc8) [2751336.367001] [<c07d04dd>] (sk_prot_alloc) from [<c07d05a3>] (sk_alloc+0x1f/0x164) [2751336.367012] [<c07d05a3>] (sk_alloc) from [<c087dc6d>] (unix_create1+0x7d/0x184) [2751336.367022] [<c087dc6d>] (unix_create1) from [<c087dda7>] (unix_create+0x33/0x64) [2751336.367032] [<c087dda7>] (unix_create) from [<c07ce445>] (__sock_create+0xb9/0x154) [2751336.367042] [<c07ce445>] (__sock_create) from [<c07cef61>] (__sys_socket+0x51/0xb0) [2751336.367051] [<c07cef61>] (__sys_socket) from [<c01011f9>] (__sys_trace_return+0x1/0xc) [2751336.367055] Exception stack(0xeb46dfa8 to 0xeb46dff0) [2751336.367063] dfa0: 00000016 b6d65404 00000001 00080002 00000000 00796669 [2751336.367070] dfc0: 00000016 b6d65404 00000001 00000119 00000000 00000000 00000000 00000000 [2751336.367075] dfe0: b6deab78 beb6e78c b6ca15e9 b6e89668 [2751336.367079] Mem-Info: [2751336.367093] active_anon:1183 inactive_anon:1959 isolated_anon:30 active_file:3580 inactive_file:1002 isolated_file:18 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16374 slab_unreclaimable:160714 mapped:2259 shmem:150 pagetables:237 bounce:0 free:311041 free_pcp:2291 free_cma:31055 [2751336.367104] Node 0 active_anon:4732kB inactive_anon:7836kB active_file:14320kB inactive_file:4008kB unevictable:0kB isolated(anon):120kB isolated(file):72kB mapped:9036kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751336.367117] Normal free:3392kB min:3448kB low:4308kB high:5168kB active_anon:152kB inactive_anon:92kB active_file:188kB inactive_file:408kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8720kB pagetables:292kB bounce:0kB free_pcp:3828kB local_pcp:1088kB free_cma:0kB [2751336.367119] lowmem_reserve[]: 0 10239 10239 [2751336.367137] HighMem free:1240772kB min:512kB low:1880kB high:3248kB active_anon:5284kB inactive_anon:8052kB active_file:14504kB inactive_file:3624kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:656kB bounce:0kB free_pcp:5336kB local_pcp:1164kB free_cma:124220kB [2751336.367139] lowmem_reserve[]: 0 0 0 [2751336.367147] Normal: 829*4kB (UEH) 33*8kB (UE) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3580kB [2751336.367177] HighMem: 8465*4kB (UMC) 7555*8kB (UMC) 6341*16kB (UMC) 5291*32kB (UMC) 3826*64kB (UMC) 2229*128kB (UMC) 927*256kB (UMC) 190*512kB (MC) 11*1024kB (M) 0*2048kB 0*4096kB = 1241100kB [2751336.367217] 4884 total pagecache pages [2751336.367223] 188 pages in swap cache [2751336.367227] Swap cache stats: add 150772, delete 150579, find 40924940/40978651 [2751336.367229] Free swap = 987936kB [2751336.367231] Total swap = 1030688kB [2751336.367233] 524288 pages RAM [2751336.367235] 327680 pages HighMem/MovableOnly [2751336.367238] 8943 pages reserved [2751336.367240] 32768 pages cma reserved [2751336.367243] Unreclaimable slab info: [2751336.367245] Name Used Total [2751336.367561] zs_handle 234KB 352KB [2751336.367669] RAWv6 47KB 47KB [2751336.367673] UDPv6 47KB 47KB [2751336.367677] tw_sock_TCPv6 15KB 15KB [2751336.367681] request_sock_TCPv6 15KB 15KB [2751336.367685] TCPv6 123KB 123KB [2751336.367689] sgpool-128 30KB 30KB [2751336.367693] sgpool-64 31KB 31KB [2751336.367696] sgpool-32 15KB 15KB [2751336.367700] sgpool-16 7KB 7KB [2751336.367706] mqueue_inode_cache 15KB 15KB [2751336.367718] nfsd_drc 3KB 3KB [2751336.367732] posix_timers_cache 15KB 15KB [2751336.367737] rpc_buffers 32KB 32KB [2751336.367741] xfrm_state 31KB 31KB [2751336.367745] RAW 31KB 31KB [2751336.367749] UDP 61KB 61KB [2751336.367753] tw_sock_TCP 75KB 75KB [2751336.367757] request_sock_TCP 11KB 11KB [2751336.367761] TCP 182KB 182KB [2751336.367765] eventpoll_pwq 15KB 15KB [2751336.367769] inotify_inode_mark 15KB 15KB [2751336.367773] request_queue 92KB 92KB [2751336.367778] blkdev_ioc 44KB 44KB [2751336.367782] biovec-max 540KB 540KB [2751336.367786] biovec-128 126KB 126KB [2751336.367789] biovec-64 63KB 63KB [2751336.367796] biovec-16 76KB 126KB [2751336.367800] ksm_mm_slot 15KB 15KB [2751336.367807] skbuff_fclone_cache 283KB 315KB [2751336.367811] skbuff_head_cache 378KB 378KB [2751336.367815] configfs_dir_cache 3KB 3KB [2751336.367819] file_lock_cache 15KB 15KB [2751336.367823] fsnotify_mark_connector 16KB 16KB [2751336.367826] net_namespace 29KB 29KB [2751336.367840] shmem_inode_cache 807KB 960KB [2751336.367844] taskstats 30KB 30KB [2751336.367848] proc_dir_entry 108KB 108KB [2751336.367851] pde_opener 15KB 15KB [2751336.367856] seq_file 15KB 15KB [2751336.367859] sigqueue 15KB 15KB [2751336.367864] kernfs_node_cache 2563KB 2563KB [2751336.367868] filp 251748KB 251748KB [2751336.367872] names_cache 128KB 128KB [2751336.367877] key_jar 108KB 108KB [2751336.367882] vm_area_struct 248KB 248KB [2751336.367886] mm_struct 32KB 32KB [2751336.367890] files_cache 20KB 20KB [2751336.367957] signal_cache 204KB 204KB [2751336.367965] sighand_cache 315KB 315KB [2751336.367974] task_struct 401KB 401KB [2751336.367982] cred_jar 872KB 872KB [2751336.367996] anon_vma_chain 189KB 228KB [2751336.368004] anon_vma 287KB 287KB [2751336.368015] pid 32KB 32KB [2751336.368026] trace_event_file 79KB 79KB [2751336.368057] ftrace_event_field 403KB 748KB [2751336.368068] pool_workqueue 188KB 188KB [2751336.368079] kmalloc-8192 5024KB 5024KB [2751336.368088] kmalloc-4096 384KB 384KB [2751336.368098] kmalloc-2048 782KB 928KB [2751336.368111] kmalloc-1024 806KB 864KB [2751336.368123] kmalloc-512 1134KB 1164KB [2751336.368133] kmalloc-256 193KB 200KB [2751336.368144] kmalloc-192 1634KB 1634KB [2751336.368151] kmalloc-128 1912KB 1912KB [2751336.368158] kmalloc-64 85072KB 85072KB [2751336.368166] kmem_cache_node 188KB 188KB [2751336.368176] kmem_cache 605KB 620KB [2751336.368183] Tasks state (memory values in pages): [2751336.368190] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751336.368221] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751336.368233] [ 322] 0 322 1812 320 14336 642 0 haveged [2751336.368243] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751336.368258] [ 528] 0 528 742 242 12288 0 0 alsactl [2751336.368268] [ 537] 0 537 6117 505 24576 51 0 rsyslogd [2751336.368277] [ 543] 0 543 1830 369 14336 22 0 cron [2751336.368290] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751336.368299] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751336.368312] [ 570] 0 570 17691 1134 47104 249 0 NetworkManager [2751336.368322] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751336.368331] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751336.368341] [ 712] 104 712 806 323 12288 0 0 chronyd [2751336.368350] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751336.368358] [ 836] 0 836 8994 602 26624 53 0 polkitd [2751336.368366] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751336.368372] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751336.368379] [ 1153] 0 1153 1559 365 14336 185 0 dhclient [2751336.368386] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751336.368392] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751336.368399] [ 1278] 1000 1278 1346 558 14336 360 0 screen [2751336.368406] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751336.368413] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751336.368420] [ 1283] 0 1283 1045 245 12288 80 0 su [2751336.368427] [ 1284] 0 1284 818 207 14336 222 0 bash [2751336.368434] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751336.368441] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751336.368448] [ 1297] 1000 1297 819 375 12288 183 0 bash [2751336.368455] [ 1313] 1000 1313 819 364 8192 236 0 bash [2751336.368461] [ 1534] 1000 1534 883 342 12288 300 0 bash [2751336.368468] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751336.368475] [ 32670] 1000 32670 368 79 8192 15 0 run.sh [2751336.368484] [ 759] 1000 759 819 403 10240 224 0 bash [2751336.368498] [ 7476] 0 7476 354 93 10240 0 0 sleep [2751336.368503] Out of memory: Kill process 570 (NetworkManager) score 1 or sacrifice child [2751336.368540] Killed process 1153 (dhclient) total-vm:6236kB, anon-rss:112kB, file-rss:1348kB, shmem-rss:0kB [2751336.575030] systemd invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=0, oom_score_adj=0 [2751336.575038] systemd cpuset=/ mems_allowed=0 [2751336.575053] CPU: 3 PID: 1 Comm: systemd Not tainted 4.19.57-sunxi #5.90 [2751336.575056] Hardware name: Allwinner sun8i Family [2751336.575082] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751336.575094] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751336.575108] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751336.575119] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751336.575129] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751336.575140] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751336.575152] [<c01f12c5>] (__alloc_pages_nodemask) from [<c022a225>] (new_slab+0x1fd/0x504) [2751336.575163] [<c022a225>] (new_slab) from [<c022b843>] (___slab_alloc.constprop.14+0x2f7/0x3cc) [2751336.575171] [<c022b843>] (___slab_alloc.constprop.14) from [<c022be89>] (kmem_cache_alloc+0xd5/0x168) [2751336.575179] [<c022be89>] (kmem_cache_alloc) from [<c023f8bb>] (__alloc_file+0x23/0xa8) [2751336.575188] [<c023f8bb>] (__alloc_file) from [<c023fb1d>] (alloc_empty_file+0x49/0xe8) [2751336.575196] [<c023fb1d>] (alloc_empty_file) from [<c0249447>] (path_openat+0x3b/0xe0c) [2751336.575205] [<c0249447>] (path_openat) from [<c024af3b>] (do_filp_open+0x4f/0x94) [2751336.575215] [<c024af3b>] (do_filp_open) from [<c023cb0b>] (do_sys_open+0x13f/0x1a8) [2751336.575224] [<c023cb0b>] (do_sys_open) from [<c0101001>] (ret_fast_syscall+0x1/0x62) [2751336.575228] Exception stack(0xee529fa8 to 0xee529ff0) [2751336.575236] 9fa0: b6ee1000 b6f0d968 ffffff9c bec1abe0 000a0000 00000000 [2751336.575244] 9fc0: b6ee1000 b6f0d968 00000000 00000142 00000000 bec1abe0 bec1b8f0 bec1ac60 [2751336.575249] 9fe0: 00000142 bec1ab18 b6e7919f b6e02746 [2751336.575252] Mem-Info: [2751336.575267] active_anon:1090 inactive_anon:1804 isolated_anon:30 active_file:3642 inactive_file:1104 isolated_file:18 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16374 slab_unreclaimable:160869 mapped:2042 shmem:150 pagetables:237 bounce:0 free:311100 free_pcp:2273 free_cma:31141 [2751336.575278] Node 0 active_anon:4360kB inactive_anon:7216kB active_file:14568kB inactive_file:4416kB unevictable:0kB isolated(anon):120kB isolated(file):72kB mapped:8168kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751336.575292] Normal free:3284kB min:3448kB low:4308kB high:5168kB active_anon:152kB inactive_anon:92kB active_file:188kB inactive_file:192kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8720kB pagetables:292kB bounce:0kB free_pcp:3632kB local_pcp:1364kB free_cma:0kB [2751336.575294] lowmem_reserve[]: 0 10239 10239 [2751336.575312] HighMem free:1241116kB min:512kB low:1880kB high:3248kB active_anon:4940kB inactive_anon:7708kB active_file:14504kB inactive_file:4312kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:656kB bounce:0kB free_pcp:5460kB local_pcp:1424kB free_cma:124564kB [2751336.575314] lowmem_reserve[]: 0 0 0 [2751336.575323] Normal: 807*4kB (UEH) 44*8kB (UME) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3580kB [2751336.575352] HighMem: 8460*4kB (UMC) 7484*8kB (UMC) 6275*16kB (UMC) 5252*32kB (UMC) 3801*64kB (UMC) 2222*128kB (UMC) 933*256kB (UMC) 197*512kB (MC) 12*1024kB (M) 0*2048kB 0*4096kB = 1241856kB [2751336.575393] 5098 total pagecache pages [2751336.575398] 185 pages in swap cache [2751336.575402] Swap cache stats: add 150772, delete 150582, find 40924940/40978656 [2751336.575404] Free swap = 989728kB [2751336.575406] Total swap = 1030688kB [2751336.575409] 524288 pages RAM [2751336.575411] 327680 pages HighMem/MovableOnly [2751336.575413] 8943 pages reserved [2751336.575415] 32768 pages cma reserved [2751336.575418] Unreclaimable slab info: [2751336.575420] Name Used Total [2751336.575736] zs_handle 233KB 344KB [2751336.575842] RAWv6 47KB 47KB [2751336.575846] UDPv6 47KB 47KB [2751336.575850] tw_sock_TCPv6 15KB 15KB [2751336.575854] request_sock_TCPv6 15KB 15KB [2751336.575858] TCPv6 123KB 123KB [2751336.575861] sgpool-128 30KB 30KB [2751336.575865] sgpool-64 31KB 31KB [2751336.575868] sgpool-32 15KB 15KB [2751336.575873] sgpool-16 7KB 7KB [2751336.575878] mqueue_inode_cache 15KB 15KB [2751336.575890] nfsd_drc 3KB 3KB [2751336.575957] posix_timers_cache 15KB 15KB [2751336.575972] rpc_buffers 32KB 32KB [2751336.575980] xfrm_state 31KB 31KB [2751336.575988] RAW 31KB 31KB [2751336.575995] UDP 61KB 61KB [2751336.576007] tw_sock_TCP 75KB 75KB [2751336.576015] request_sock_TCP 11KB 11KB [2751336.576023] TCP 182KB 182KB [2751336.576031] eventpoll_pwq 15KB 15KB [2751336.576039] inotify_inode_mark 15KB 15KB [2751336.576050] request_queue 92KB 92KB [2751336.576071] blkdev_ioc 44KB 44KB [2751336.576081] biovec-max 540KB 540KB [2751336.576093] biovec-128 126KB 126KB [2751336.576102] biovec-64 63KB 63KB [2751336.576115] biovec-16 76KB 126KB [2751336.576124] ksm_mm_slot 15KB 15KB [2751336.576137] skbuff_fclone_cache 283KB 315KB [2751336.576144] skbuff_head_cache 378KB 378KB [2751336.576152] configfs_dir_cache 3KB 3KB [2751336.576159] file_lock_cache 15KB 15KB [2751336.576167] fsnotify_mark_connector 16KB 16KB [2751336.576174] net_namespace 29KB 29KB [2751336.576190] shmem_inode_cache 807KB 960KB [2751336.576200] taskstats 30KB 30KB [2751336.576211] proc_dir_entry 108KB 108KB [2751336.576221] pde_opener 15KB 15KB [2751336.576228] seq_file 15KB 15KB [2751336.576236] sigqueue 15KB 15KB [2751336.576246] kernfs_node_cache 2563KB 2563KB [2751336.576253] filp 251929KB 251929KB [2751336.576263] names_cache 128KB 128KB [2751336.576274] key_jar 108KB 108KB [2751336.576284] vm_area_struct 248KB 248KB [2751336.576294] mm_struct 32KB 32KB [2751336.576302] files_cache 20KB 20KB [2751336.576306] signal_cache 204KB 204KB [2751336.576309] sighand_cache 315KB 315KB [2751336.576313] task_struct 401KB 401KB [2751336.576317] cred_jar 872KB 872KB [2751336.576324] anon_vma_chain 189KB 228KB [2751336.576327] anon_vma 287KB 287KB [2751336.576331] pid 32KB 32KB [2751336.576335] trace_event_file 79KB 79KB [2751336.576364] ftrace_event_field 400KB 748KB [2751336.576368] pool_workqueue 188KB 188KB [2751336.576373] kmalloc-8192 5024KB 5024KB [2751336.576377] kmalloc-4096 384KB 384KB [2751336.576382] kmalloc-2048 782KB 928KB [2751336.576390] kmalloc-1024 805KB 864KB [2751336.576396] kmalloc-512 1134KB 1164KB [2751336.576400] kmalloc-256 193KB 200KB [2751336.576404] kmalloc-192 1634KB 1634KB [2751336.576407] kmalloc-128 1912KB 1912KB [2751336.576411] kmalloc-64 85136KB 85136KB [2751336.576415] kmem_cache_node 188KB 188KB [2751336.576427] kmem_cache 605KB 620KB [2751336.576432] Tasks state (memory values in pages): [2751336.576434] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751336.576459] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751336.576466] [ 322] 0 322 1812 320 14336 642 0 haveged [2751336.576476] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751336.576484] [ 528] 0 528 742 242 12288 0 0 alsactl [2751336.576490] [ 537] 0 537 6117 505 24576 51 0 rsyslogd [2751336.576497] [ 543] 0 543 1830 369 14336 22 0 cron [2751336.576503] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751336.576510] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751336.576518] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751336.576525] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751336.576532] [ 712] 104 712 806 323 12288 0 0 chronyd [2751336.576539] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751336.576546] [ 836] 0 836 8994 602 26624 53 0 polkitd [2751336.576554] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751336.576561] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751336.576567] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751336.576576] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751336.576584] [ 1278] 1000 1278 1346 558 14336 360 0 screen [2751336.576594] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751336.576601] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751336.576608] [ 1283] 0 1283 1045 245 12288 80 0 su [2751336.576614] [ 1284] 0 1284 818 207 14336 222 0 bash [2751336.576621] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751336.576627] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751336.576635] [ 1297] 1000 1297 819 375 12288 183 0 bash [2751336.576642] [ 1313] 1000 1313 819 364 8192 236 0 bash [2751336.576648] [ 1534] 1000 1534 883 342 12288 300 0 bash [2751336.576655] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751336.576662] [ 32670] 1000 32670 368 79 8192 15 0 run.sh [2751336.576670] [ 759] 1000 759 819 403 10240 224 0 bash [2751336.576684] [ 7476] 0 7476 354 93 10240 0 0 sleep [2751336.576691] Out of memory: Kill process 322 (haveged) score 1 or sacrifice child [2751336.576735] Killed process 322 (haveged) total-vm:7248kB, anon-rss:492kB, file-rss:788kB, shmem-rss:0kB [2751336.580922] oom_reaper: reaped process 322 (haveged), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [2751336.585332] systemd invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=0, oom_score_adj=0 [2751336.585340] systemd cpuset=/ mems_allowed=0 [2751336.585355] CPU: 3 PID: 1 Comm: systemd Not tainted 4.19.57-sunxi #5.90 [2751336.585358] Hardware name: Allwinner sun8i Family [2751336.585384] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751336.585396] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751336.585410] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751336.585421] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751336.585431] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751336.585441] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751336.585453] [<c01f12c5>] (__alloc_pages_nodemask) from [<c022a225>] (new_slab+0x1fd/0x504) [2751336.585464] [<c022a225>] (new_slab) from [<c022b843>] (___slab_alloc.constprop.14+0x2f7/0x3cc) [2751336.585472] [<c022b843>] (___slab_alloc.constprop.14) from [<c022be89>] (kmem_cache_alloc+0xd5/0x168) [2751336.585481] [<c022be89>] (kmem_cache_alloc) from [<c023f8bb>] (__alloc_file+0x23/0xa8) [2751336.585488] [<c023f8bb>] (__alloc_file) from [<c023fb1d>] (alloc_empty_file+0x49/0xe8) [2751336.585497] [<c023fb1d>] (alloc_empty_file) from [<c0249447>] (path_openat+0x3b/0xe0c) [2751336.585506] [<c0249447>] (path_openat) from [<c024af3b>] (do_filp_open+0x4f/0x94) [2751336.585516] [<c024af3b>] (do_filp_open) from [<c023cb0b>] (do_sys_open+0x13f/0x1a8) [2751336.585526] [<c023cb0b>] (do_sys_open) from [<c0101001>] (ret_fast_syscall+0x1/0x62) [2751336.585530] Exception stack(0xee529fa8 to 0xee529ff0) [2751336.585537] 9fa0: b6ee1000 b6f0d968 ffffff9c bec1abe0 000a0000 00000000 [2751336.585545] 9fc0: b6ee1000 b6f0d968 00000000 00000142 00000000 bec1abe0 bec1b8f0 bec1ac60 [2751336.585550] 9fe0: 00000142 bec1ab18 b6e7919f b6e02746 [2751336.585553] Mem-Info: [2751336.585568] active_anon:1028 inactive_anon:1742 isolated_anon:30 active_file:3642 inactive_file:1104 isolated_file:18 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16374 slab_unreclaimable:160869 mapped:2042 shmem:150 pagetables:237 bounce:0 free:311788 free_pcp:2306 free_cma:31141 [2751336.585579] Node 0 active_anon:4112kB inactive_anon:6968kB active_file:14568kB inactive_file:4416kB unevictable:0kB isolated(anon):120kB isolated(file):72kB mapped:8168kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751336.585592] Normal free:3284kB min:3448kB low:4308kB high:5168kB active_anon:152kB inactive_anon:92kB active_file:188kB inactive_file:192kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8720kB pagetables:292kB bounce:0kB free_pcp:3756kB local_pcp:1376kB free_cma:0kB [2751336.585594] lowmem_reserve[]: 0 10239 10239 [2751336.585613] HighMem free:1243868kB min:512kB low:1880kB high:3248kB active_anon:4596kB inactive_anon:7364kB active_file:14504kB inactive_file:4312kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:656kB bounce:0kB free_pcp:5468kB local_pcp:1476kB free_cma:124564kB [2751336.585615] lowmem_reserve[]: 0 0 0 [2751336.585624] Normal: 807*4kB (UEH) 44*8kB (UME) 1*16kB (U) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3596kB [2751336.585656] HighMem: 8089*4kB (UMC) 7196*8kB (UMC) 6091*16kB (UMC) 5133*32kB (UMC) 3775*64kB (UMC) 2237*128kB (UMC) 966*256kB (UMC) 206*512kB (MC) 12*1024kB (M) 0*2048kB 0*4096kB = 1244628kB [2751336.585696] 5098 total pagecache pages [2751336.585702] 185 pages in swap cache [2751336.585706] Swap cache stats: add 150772, delete 150582, find 40924940/40978656 [2751336.585708] Free swap = 992288kB [2751336.585710] Total swap = 1030688kB [2751336.585713] 524288 pages RAM [2751336.585715] 327680 pages HighMem/MovableOnly [2751336.585717] 8943 pages reserved [2751336.585719] 32768 pages cma reserved [2751336.585722] Unreclaimable slab info: [2751336.585724] Name Used Total [2751336.586039] zs_handle 231KB 344KB [2751336.586145] RAWv6 47KB 47KB [2751336.586149] UDPv6 47KB 47KB [2751336.586153] tw_sock_TCPv6 15KB 15KB [2751336.586157] request_sock_TCPv6 15KB 15KB [2751336.586161] TCPv6 123KB 123KB [2751336.586165] sgpool-128 30KB 30KB [2751336.586168] sgpool-64 31KB 31KB [2751336.586172] sgpool-32 15KB 15KB [2751336.586176] sgpool-16 7KB 7KB [2751336.586182] mqueue_inode_cache 15KB 15KB [2751336.586194] nfsd_drc 3KB 3KB [2751336.586207] posix_timers_cache 15KB 15KB [2751336.586212] rpc_buffers 32KB 32KB [2751336.586215] xfrm_state 31KB 31KB [2751336.586220] RAW 31KB 31KB [2751336.586224] UDP 61KB 61KB [2751336.586228] tw_sock_TCP 75KB 75KB [2751336.586231] request_sock_TCP 11KB 11KB [2751336.586235] TCP 182KB 182KB [2751336.586240] eventpoll_pwq 15KB 15KB [2751336.586244] inotify_inode_mark 15KB 15KB [2751336.586248] request_queue 92KB 92KB [2751336.586252] blkdev_ioc 44KB 44KB [2751336.586256] biovec-max 540KB 540KB [2751336.586260] biovec-128 126KB 126KB [2751336.586264] biovec-64 63KB 63KB [2751336.586270] biovec-16 76KB 126KB [2751336.586274] ksm_mm_slot 15KB 15KB [2751336.586281] skbuff_fclone_cache 283KB 315KB [2751336.586285] skbuff_head_cache 378KB 378KB [2751336.586289] configfs_dir_cache 3KB 3KB [2751336.586293] file_lock_cache 15KB 15KB [2751336.586297] fsnotify_mark_connector 16KB 16KB [2751336.586301] net_namespace 29KB 29KB [2751336.586314] shmem_inode_cache 807KB 960KB [2751336.586318] taskstats 30KB 30KB [2751336.586322] proc_dir_entry 108KB 108KB [2751336.586325] pde_opener 15KB 15KB [2751336.586330] seq_file 15KB 15KB [2751336.586333] sigqueue 15KB 15KB [2751336.586338] kernfs_node_cache 2563KB 2563KB [2751336.586342] filp 251929KB 251929KB [2751336.586346] names_cache 128KB 128KB [2751336.586351] key_jar 108KB 108KB [2751336.586356] vm_area_struct 248KB 248KB [2751336.586360] mm_struct 32KB 32KB [2751336.586363] files_cache 20KB 20KB [2751336.586367] signal_cache 204KB 204KB [2751336.586371] sighand_cache 315KB 315KB [2751336.586375] task_struct 401KB 401KB [2751336.586379] cred_jar 872KB 872KB [2751336.586385] anon_vma_chain 189KB 228KB [2751336.586389] anon_vma 287KB 287KB [2751336.586393] pid 32KB 32KB [2751336.586397] trace_event_file 79KB 79KB [2751336.586423] ftrace_event_field 382KB 740KB [2751336.586426] pool_workqueue 188KB 188KB [2751336.586432] kmalloc-8192 5024KB 5024KB [2751336.586435] kmalloc-4096 384KB 384KB [2751336.586440] kmalloc-2048 782KB 928KB [2751336.586448] kmalloc-1024 804KB 864KB [2751336.586454] kmalloc-512 1134KB 1164KB [2751336.586458] kmalloc-256 193KB 200KB [2751336.586462] kmalloc-192 1634KB 1634KB [2751336.586465] kmalloc-128 1912KB 1912KB [2751336.586469] kmalloc-64 85136KB 85136KB [2751336.586473] kmem_cache_node 188KB 188KB [2751336.586479] kmem_cache 605KB 620KB [2751336.586482] Tasks state (memory values in pages): [2751336.586485] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751336.586508] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751336.586516] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751336.586525] [ 528] 0 528 742 242 12288 0 0 alsactl [2751336.586532] [ 537] 0 537 6117 505 24576 51 0 rsyslogd [2751336.586538] [ 543] 0 543 1830 369 14336 22 0 cron [2751336.586545] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751336.586552] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751336.586559] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751336.586566] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751336.586573] [ 712] 104 712 806 323 12288 0 0 chronyd [2751336.586580] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751336.586587] [ 836] 0 836 8994 602 26624 53 0 polkitd [2751336.586596] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751336.586602] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751336.586609] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751336.586616] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751336.586623] [ 1278] 1000 1278 1346 558 14336 360 0 screen [2751336.586629] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751336.586635] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751336.586643] [ 1283] 0 1283 1045 245 12288 80 0 su [2751336.586649] [ 1284] 0 1284 818 207 14336 222 0 bash [2751336.586656] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751336.586663] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751336.586670] [ 1297] 1000 1297 819 375 12288 183 0 bash [2751336.586677] [ 1313] 1000 1313 819 364 8192 236 0 bash [2751336.586684] [ 1534] 1000 1534 883 342 12288 300 0 bash [2751336.586690] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751336.586698] [ 32670] 1000 32670 368 79 8192 15 0 run.sh [2751336.586706] [ 759] 1000 759 819 403 10240 224 0 bash [2751336.586719] [ 7476] 0 7476 354 93 10240 0 0 sleep [2751336.586723] Out of memory: Kill process 1278 (screen) score 1 or sacrifice child [2751336.586774] Killed process 1534 (bash) total-vm:3532kB, anon-rss:28kB, file-rss:1340kB, shmem-rss:0kB [2751336.588398] oom_reaper: reaped process 1534 (bash), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [2751336.594404] systemd invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=0, oom_score_adj=0 [2751336.594413] systemd cpuset=/ mems_allowed=0 [2751336.594431] CPU: 3 PID: 1 Comm: systemd Not tainted 4.19.57-sunxi #5.90 [2751336.594434] Hardware name: Allwinner sun8i Family [2751336.594465] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751336.594481] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751336.594497] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751336.594508] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751336.594519] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751336.594534] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751336.594549] [<c01f12c5>] (__alloc_pages_nodemask) from [<c022a225>] (new_slab+0x1fd/0x504) [2751336.594562] [<c022a225>] (new_slab) from [<c022b843>] (___slab_alloc.constprop.14+0x2f7/0x3cc) [2751336.594571] [<c022b843>] (___slab_alloc.constprop.14) from [<c022be89>] (kmem_cache_alloc+0xd5/0x168) [2751336.594580] [<c022be89>] (kmem_cache_alloc) from [<c023f8bb>] (__alloc_file+0x23/0xa8) [2751336.594588] [<c023f8bb>] (__alloc_file) from [<c023fb1d>] (alloc_empty_file+0x49/0xe8) [2751336.594597] [<c023fb1d>] (alloc_empty_file) from [<c0249447>] (path_openat+0x3b/0xe0c) [2751336.594607] [<c0249447>] (path_openat) from [<c024af3b>] (do_filp_open+0x4f/0x94) [2751336.594617] [<c024af3b>] (do_filp_open) from [<c023cb0b>] (do_sys_open+0x13f/0x1a8) [2751336.594627] [<c023cb0b>] (do_sys_open) from [<c0101001>] (ret_fast_syscall+0x1/0x62) [2751336.594631] Exception stack(0xee529fa8 to 0xee529ff0) [2751336.594639] 9fa0: b6ee1000 b6f0d968 ffffff9c bec1abe0 000a0000 00000000 [2751336.594646] 9fc0: b6ee1000 b6f0d968 00000000 00000142 00000000 bec1abe0 bec1b8f0 bec1ac60 [2751336.594652] 9fe0: 00000142 bec1ab18 b6e7919f b6e02746 [2751336.594655] Mem-Info: [2751336.594671] active_anon:1028 inactive_anon:1742 isolated_anon:30 active_file:3642 inactive_file:1135 isolated_file:18 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16374 slab_unreclaimable:160869 mapped:2042 shmem:150 pagetables:157 bounce:0 free:311960 free_pcp:2233 free_cma:31141 [2751336.594681] Node 0 active_anon:4112kB inactive_anon:6968kB active_file:14568kB inactive_file:4540kB unevictable:0kB isolated(anon):120kB isolated(file):72kB mapped:8168kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751336.594695] Normal free:3284kB min:3448kB low:4308kB high:5168kB active_anon:152kB inactive_anon:92kB active_file:188kB inactive_file:192kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8720kB pagetables:0kB bounce:0kB free_pcp:3896kB local_pcp:1468kB free_cma:0kB [2751336.594697] lowmem_reserve[]: 0 10239 10239 [2751336.594715] HighMem free:1244556kB min:512kB low:1880kB high:3248kB active_anon:4596kB inactive_anon:7364kB active_file:14504kB inactive_file:4312kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:656kB bounce:0kB free_pcp:5040kB local_pcp:1392kB free_cma:124564kB [2751336.594717] lowmem_reserve[]: 0 0 0 [2751336.594726] Normal: 807*4kB (UEH) 44*8kB (UME) 1*16kB (U) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3596kB [2751336.594758] HighMem: 8029*4kB (UMC) 7111*8kB (UMC) 6028*16kB (UMC) 5081*32kB (UMC) 3747*64kB (UMC) 2235*128kB (UMC) 974*256kB (UMC) 214*512kB (MC) 12*1024kB (M) 0*2048kB 0*4096kB = 1245132kB [2751336.594800] 5098 total pagecache pages [2751336.594805] 185 pages in swap cache [2751336.594809] Swap cache stats: add 150772, delete 150582, find 40924940/40978670 [2751336.594811] Free swap = 993560kB [2751336.594813] Total swap = 1030688kB [2751336.594816] 524288 pages RAM [2751336.594818] 327680 pages HighMem/MovableOnly [2751336.594820] 8943 pages reserved [2751336.594822] 32768 pages cma reserved [2751336.594827] Unreclaimable slab info: [2751336.594831] Name Used Total [2751336.595167] zs_handle 230KB 344KB [2751336.595282] RAWv6 47KB 47KB [2751336.595287] UDPv6 47KB 47KB [2751336.595291] tw_sock_TCPv6 15KB 15KB [2751336.595297] request_sock_TCPv6 15KB 15KB [2751336.595303] TCPv6 123KB 123KB [2751336.595307] sgpool-128 30KB 30KB [2751336.595311] sgpool-64 31KB 31KB [2751336.595314] sgpool-32 15KB 15KB [2751336.595318] sgpool-16 7KB 7KB [2751336.595324] mqueue_inode_cache 15KB 15KB [2751336.595339] nfsd_drc 3KB 3KB [2751336.595354] posix_timers_cache 15KB 15KB [2751336.595358] rpc_buffers 32KB 32KB [2751336.595362] xfrm_state 31KB 31KB [2751336.595366] RAW 31KB 31KB [2751336.595371] UDP 61KB 61KB [2751336.595374] tw_sock_TCP 75KB 75KB [2751336.595381] request_sock_TCP 11KB 11KB [2751336.595386] TCP 182KB 182KB [2751336.595391] eventpoll_pwq 15KB 15KB [2751336.595395] inotify_inode_mark 15KB 15KB [2751336.595399] request_queue 92KB 92KB [2751336.595404] blkdev_ioc 44KB 44KB [2751336.595407] biovec-max 540KB 540KB [2751336.595412] biovec-128 126KB 126KB [2751336.595415] biovec-64 63KB 63KB [2751336.595424] biovec-16 76KB 126KB [2751336.595430] ksm_mm_slot 15KB 15KB [2751336.595437] skbuff_fclone_cache 283KB 315KB [2751336.595441] skbuff_head_cache 378KB 378KB [2751336.595445] configfs_dir_cache 3KB 3KB [2751336.595451] file_lock_cache 15KB 15KB [2751336.595457] fsnotify_mark_connector 16KB 16KB [2751336.595462] net_namespace 29KB 29KB [2751336.595475] shmem_inode_cache 807KB 960KB [2751336.595479] taskstats 30KB 30KB [2751336.595483] proc_dir_entry 108KB 108KB [2751336.595486] pde_opener 15KB 15KB [2751336.595491] seq_file 15KB 15KB [2751336.595497] sigqueue 15KB 15KB [2751336.595502] kernfs_node_cache 2563KB 2563KB [2751336.595508] filp 251929KB 251929KB [2751336.595512] names_cache 128KB 128KB [2751336.595517] key_jar 108KB 108KB [2751336.595522] vm_area_struct 248KB 248KB [2751336.595526] mm_struct 32KB 32KB [2751336.595530] files_cache 20KB 20KB [2751336.595534] signal_cache 204KB 204KB [2751336.595538] sighand_cache 315KB 315KB [2751336.595542] task_struct 401KB 401KB [2751336.595546] cred_jar 872KB 872KB [2751336.595554] anon_vma_chain 189KB 228KB [2751336.595560] anon_vma 287KB 287KB [2751336.595565] pid 32KB 32KB [2751336.595569] trace_event_file 79KB 79KB [2751336.595596] ftrace_event_field 380KB 736KB [2751336.595603] pool_workqueue 188KB 188KB [2751336.595610] kmalloc-8192 5024KB 5024KB [2751336.595615] kmalloc-4096 384KB 384KB [2751336.595619] kmalloc-2048 782KB 928KB [2751336.595627] kmalloc-1024 800KB 864KB [2751336.595633] kmalloc-512 1134KB 1164KB [2751336.595638] kmalloc-256 193KB 200KB [2751336.595645] kmalloc-192 1634KB 1634KB [2751336.595649] kmalloc-128 1912KB 1912KB [2751336.595655] kmalloc-64 85136KB 85136KB [2751336.595659] kmem_cache_node 188KB 188KB [2751336.595666] kmem_cache 605KB 620KB [2751336.595670] Tasks state (memory values in pages): [2751336.595672] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751336.595702] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751336.595711] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751336.595721] [ 528] 0 528 742 242 12288 0 0 alsactl [2751336.595730] [ 537] 0 537 6117 552 24576 50 0 rsyslogd [2751336.595739] [ 543] 0 543 1830 369 14336 22 0 cron [2751336.595747] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751336.595755] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751336.595763] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751336.595773] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751336.595783] [ 712] 104 712 806 323 12288 0 0 chronyd [2751336.595791] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751336.595798] [ 836] 0 836 8994 602 26624 53 0 polkitd [2751336.595807] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751336.595816] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751336.595825] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751336.595833] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751336.595839] [ 1278] 1000 1278 1346 643 14336 317 0 screen [2751336.595847] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751336.595854] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751336.595863] [ 1283] 0 1283 1045 245 12288 80 0 su [2751336.595873] [ 1284] 0 1284 818 207 14336 222 0 bash [2751336.595880] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751336.595887] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751336.595895] [ 1297] 1000 1297 819 375 12288 183 0 bash [2751336.595962] [ 1313] 1000 1313 819 364 8192 236 0 bash [2751336.595973] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751336.595991] [ 32670] 1000 32670 368 79 8192 15 0 run.sh [2751336.596005] [ 759] 1000 759 819 403 10240 224 0 bash [2751336.596029] [ 7476] 0 7476 354 93 10240 0 0 sleep [2751336.596041] Out of memory: Kill process 1278 (screen) score 1 or sacrifice child [2751336.596098] Killed process 759 (bash) total-vm:3276kB, anon-rss:0kB, file-rss:1612kB, shmem-rss:0kB [2751336.604444] systemd invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=0, oom_score_adj=0 [2751336.604453] systemd cpuset=/ mems_allowed=0 [2751336.604469] CPU: 3 PID: 1 Comm: systemd Not tainted 4.19.57-sunxi #5.90 [2751336.604471] Hardware name: Allwinner sun8i Family [2751336.604498] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751336.604510] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751336.604525] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751336.604537] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751336.604547] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751336.604557] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751336.604570] [<c01f12c5>] (__alloc_pages_nodemask) from [<c022a225>] (new_slab+0x1fd/0x504) [2751336.604580] [<c022a225>] (new_slab) from [<c022b843>] (___slab_alloc.constprop.14+0x2f7/0x3cc) [2751336.604589] [<c022b843>] (___slab_alloc.constprop.14) from [<c022be89>] (kmem_cache_alloc+0xd5/0x168) [2751336.604598] [<c022be89>] (kmem_cache_alloc) from [<c023f8bb>] (__alloc_file+0x23/0xa8) [2751336.604606] [<c023f8bb>] (__alloc_file) from [<c023fb1d>] (alloc_empty_file+0x49/0xe8) [2751336.604615] [<c023fb1d>] (alloc_empty_file) from [<c0249447>] (path_openat+0x3b/0xe0c) [2751336.604624] [<c0249447>] (path_openat) from [<c024af3b>] (do_filp_open+0x4f/0x94) [2751336.604634] [<c024af3b>] (do_filp_open) from [<c023cb0b>] (do_sys_open+0x13f/0x1a8) [2751336.604644] [<c023cb0b>] (do_sys_open) from [<c0101001>] (ret_fast_syscall+0x1/0x62) [2751336.604648] Exception stack(0xee529fa8 to 0xee529ff0) [2751336.604656] 9fa0: b6ee1000 b6f0d968 ffffff9c bec1abe0 000a0000 00000000 [2751336.604663] 9fc0: b6ee1000 b6f0d968 00000000 00000142 00000000 bec1abe0 bec1b8f0 bec1ac60 [2751336.604668] 9fe0: 00000142 bec1ab18 b6e7919f b6e02746 [2751336.604672] Mem-Info: [2751336.604688] active_anon:1028 inactive_anon:1897 isolated_anon:30 active_file:3642 inactive_file:1135 isolated_file:18 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16374 slab_unreclaimable:160869 mapped:2042 shmem:150 pagetables:157 bounce:0 free:312100 free_pcp:2064 free_cma:31227 [2751336.604699] Node 0 active_anon:4112kB inactive_anon:7588kB active_file:14568kB inactive_file:4540kB unevictable:0kB isolated(anon):120kB isolated(file):72kB mapped:8168kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751336.604713] Normal free:3500kB min:3448kB low:4308kB high:5168kB active_anon:152kB inactive_anon:92kB active_file:188kB inactive_file:192kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8720kB pagetables:0kB bounce:0kB free_pcp:3884kB local_pcp:1420kB free_cma:0kB [2751336.604715] lowmem_reserve[]: 0 10239 10239 [2751336.604734] HighMem free:1244900kB min:512kB low:1880kB high:3248kB active_anon:4596kB inactive_anon:7708kB active_file:14504kB inactive_file:4312kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:656kB bounce:0kB free_pcp:4372kB local_pcp:1504kB free_cma:124908kB [2751336.604735] lowmem_reserve[]: 0 0 0 [2751336.604743] Normal: 862*4kB (UEH) 48*8kB (UME) 1*16kB (U) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3848kB [2751336.604775] HighMem: 7982*4kB (UMC) 7072*8kB (UMC) 5995*16kB (UMC) 5055*32kB (UMC) 3730*64kB (UMC) 2228*128kB (UMC) 980*256kB (UMC) 217*512kB (MC) 13*1024kB (M) 0*2048kB 0*4096kB = 1245384kB [2751336.604816] 5098 total pagecache pages [2751336.604821] 185 pages in swap cache [2751336.604825] Swap cache stats: add 150772, delete 150582, find 40924940/40978847 [2751336.604827] Free swap = 995104kB [2751336.604829] Total swap = 1030688kB [2751336.604832] 524288 pages RAM [2751336.604834] 327680 pages HighMem/MovableOnly [2751336.604836] 8943 pages reserved [2751336.604838] 32768 pages cma reserved [2751336.604841] Unreclaimable slab info: [2751336.604843] Name Used Total [2751336.605156] zs_handle 228KB 332KB [2751336.605261] RAWv6 47KB 47KB [2751336.605265] UDPv6 47KB 47KB [2751336.605270] tw_sock_TCPv6 15KB 15KB [2751336.605273] request_sock_TCPv6 15KB 15KB [2751336.605277] TCPv6 123KB 123KB [2751336.605281] sgpool-128 30KB 30KB [2751336.605285] sgpool-64 31KB 31KB [2751336.605289] sgpool-32 15KB 15KB [2751336.605293] sgpool-16 7KB 7KB [2751336.605299] mqueue_inode_cache 15KB 15KB [2751336.605311] nfsd_drc 3KB 3KB [2751336.605325] posix_timers_cache 15KB 15KB [2751336.605329] rpc_buffers 32KB 32KB [2751336.605333] xfrm_state 31KB 31KB [2751336.605338] RAW 31KB 31KB [2751336.605341] UDP 61KB 61KB [2751336.605345] tw_sock_TCP 75KB 75KB [2751336.605349] request_sock_TCP 11KB 11KB [2751336.605353] TCP 182KB 182KB [2751336.605357] eventpoll_pwq 15KB 15KB [2751336.605361] inotify_inode_mark 15KB 15KB [2751336.605365] request_queue 92KB 92KB [2751336.605370] blkdev_ioc 44KB 44KB [2751336.605374] biovec-max 540KB 540KB [2751336.605378] biovec-128 126KB 126KB [2751336.605382] biovec-64 63KB 63KB [2751336.605389] biovec-16 76KB 126KB [2751336.605393] ksm_mm_slot 15KB 15KB [2751336.605399] skbuff_fclone_cache 283KB 315KB [2751336.605403] skbuff_head_cache 378KB 378KB [2751336.605407] configfs_dir_cache 3KB 3KB [2751336.605411] file_lock_cache 15KB 15KB [2751336.605415] fsnotify_mark_connector 16KB 16KB [2751336.605419] net_namespace 29KB 29KB [2751336.605431] shmem_inode_cache 807KB 960KB [2751336.605435] taskstats 30KB 30KB [2751336.605439] proc_dir_entry 108KB 108KB [2751336.605443] pde_opener 15KB 15KB [2751336.605447] seq_file 15KB 15KB [2751336.605451] sigqueue 15KB 15KB [2751336.605455] kernfs_node_cache 2563KB 2563KB [2751336.605459] filp 251929KB 251929KB [2751336.605464] names_cache 128KB 128KB [2751336.605468] key_jar 108KB 108KB [2751336.605474] vm_area_struct 248KB 248KB [2751336.605478] mm_struct 32KB 32KB [2751336.605481] files_cache 20KB 20KB [2751336.605485] signal_cache 204KB 204KB [2751336.605489] sighand_cache 315KB 315KB [2751336.605493] task_struct 401KB 401KB [2751336.605497] cred_jar 872KB 872KB [2751336.605504] anon_vma_chain 189KB 228KB [2751336.605507] anon_vma 287KB 287KB [2751336.605511] pid 32KB 32KB [2751336.605515] trace_event_file 79KB 79KB [2751336.605540] ftrace_event_field 378KB 732KB [2751336.605544] pool_workqueue 188KB 188KB [2751336.605549] kmalloc-8192 5024KB 5024KB [2751336.605553] kmalloc-4096 384KB 384KB [2751336.605558] kmalloc-2048 782KB 928KB [2751336.605566] kmalloc-1024 797KB 864KB [2751336.605572] kmalloc-512 1134KB 1164KB [2751336.605576] kmalloc-256 193KB 200KB [2751336.605580] kmalloc-192 1634KB 1634KB [2751336.605584] kmalloc-128 1912KB 1912KB [2751336.605587] kmalloc-64 85136KB 85136KB [2751336.605591] kmem_cache_node 188KB 188KB [2751336.605598] kmem_cache 605KB 620KB [2751336.605601] Tasks state (memory values in pages): [2751336.605604] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751336.605628] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751336.605636] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751336.605645] [ 528] 0 528 742 242 12288 0 0 alsactl [2751336.605652] [ 537] 0 537 6117 552 24576 50 0 rsyslogd [2751336.605659] [ 543] 0 543 1830 369 14336 22 0 cron [2751336.605666] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751336.605673] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751336.605681] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751336.605687] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751336.605694] [ 712] 104 712 806 323 12288 0 0 chronyd [2751336.605701] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751336.605709] [ 836] 0 836 8994 602 26624 53 0 polkitd [2751336.605716] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751336.605723] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751336.605730] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751336.605737] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751336.605744] [ 1278] 1000 1278 1346 804 14336 156 0 screen [2751336.605750] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751336.605757] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751336.605764] [ 1283] 0 1283 1045 245 12288 80 0 su [2751336.605771] [ 1284] 0 1284 818 207 14336 222 0 bash [2751336.605778] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751336.605785] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751336.605792] [ 1297] 1000 1297 819 375 12288 183 0 bash [2751336.605798] [ 1313] 1000 1313 819 364 8192 236 0 bash [2751336.605805] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751336.605813] [ 32670] 1000 32670 368 79 8192 15 0 run.sh [2751336.605827] [ 7476] 0 7476 354 93 10240 0 0 sleep [2751336.605832] Out of memory: Kill process 1278 (screen) score 1 or sacrifice child [2751336.605885] Killed process 1313 (bash) total-vm:3276kB, anon-rss:4kB, file-rss:1452kB, shmem-rss:0kB [2751336.607295] oom_reaper: reaped process 1313 (bash), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [2751336.773638] systemd-logind invoked oom-killer: gfp_mask=0x6102c0(GFP_KERNEL|__GFP_NOWARN|__GFP_NOMEMALLOC), nodemask=(null), order=0, oom_score_adj=0 [2751336.773646] systemd-logind cpuset=/ mems_allowed=0 [2751336.773662] CPU: 2 PID: 547 Comm: systemd-logind Not tainted 4.19.57-sunxi #5.90 [2751336.773664] Hardware name: Allwinner sun8i Family [2751336.773691] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751336.773703] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751336.773718] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751336.773730] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751336.773740] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751336.773750] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751336.773762] [<c01f12c5>] (__alloc_pages_nodemask) from [<c022a225>] (new_slab+0x1fd/0x504) [2751336.773773] [<c022a225>] (new_slab) from [<c022b843>] (___slab_alloc.constprop.14+0x2f7/0x3cc) [2751336.773782] [<c022b843>] (___slab_alloc.constprop.14) from [<c022dbd9>] (__kmalloc_track_caller+0x129/0x1b4) [2751336.773794] [<c022dbd9>] (__kmalloc_track_caller) from [<c07d6431>] (__kmalloc_reserve.constprop.23+0x1d/0x48) [2751336.773805] [<c07d6431>] (__kmalloc_reserve.constprop.23) from [<c07d64bb>] (__alloc_skb+0x5f/0x110) [2751336.773814] [<c07d64bb>] (__alloc_skb) from [<c07d6d59>] (alloc_skb_with_frags+0x2d/0x108) [2751336.773824] [<c07d6d59>] (alloc_skb_with_frags) from [<c07d1ab1>] (sock_alloc_send_pskb+0x18d/0x1a4) [2751336.773837] [<c07d1ab1>] (sock_alloc_send_pskb) from [<c087eb1b>] (unix_dgram_sendmsg+0xef/0x4a0) [2751336.773850] [<c087eb1b>] (unix_dgram_sendmsg) from [<c07ce881>] (sock_sendmsg+0x21/0x2c) [2751336.773860] [<c07ce881>] (sock_sendmsg) from [<c07cedc5>] (___sys_sendmsg+0x159/0x17c) [2751336.773869] [<c07cedc5>] (___sys_sendmsg) from [<c07cf8d3>] (__sys_sendmsg+0x37/0x58) [2751336.773878] [<c07cf8d3>] (__sys_sendmsg) from [<c01011f9>] (__sys_trace_return+0x1/0xc) [2751336.773882] Exception stack(0xeb46dfa8 to 0xeb46dff0) [2751336.773890] dfa0: 00000074 b6d65404 00000012 beb6e7b0 00004000 00000000 [2751336.773897] dfc0: 00000074 b6d65404 00000001 00000128 00000012 00000000 00000000 00000000 [2751336.773902] dfe0: 00000128 beb6e770 b6e8951f b6e05746 [2751336.773906] Mem-Info: [2751336.773921] active_anon:1059 inactive_anon:1897 isolated_anon:30 active_file:3608 inactive_file:1166 isolated_file:18 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16374 slab_unreclaimable:160931 mapped:2042 shmem:150 pagetables:157 bounce:0 free:312077 free_pcp:2138 free_cma:31227 [2751336.773932] Node 0 active_anon:4236kB inactive_anon:7588kB active_file:14432kB inactive_file:4664kB unevictable:0kB isolated(anon):120kB isolated(file):72kB mapped:8168kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751336.773945] Normal free:3408kB min:3448kB low:4308kB high:5168kB active_anon:152kB inactive_anon:92kB active_file:188kB inactive_file:192kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8720kB pagetables:0kB bounce:0kB free_pcp:4020kB local_pcp:988kB free_cma:0kB [2751336.773947] lowmem_reserve[]: 0 10239 10239 [2751336.773966] HighMem free:1244900kB min:512kB low:1880kB high:3248kB active_anon:4596kB inactive_anon:7708kB active_file:14504kB inactive_file:4312kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:656kB bounce:0kB free_pcp:4532kB local_pcp:1084kB free_cma:124908kB [2751336.773967] lowmem_reserve[]: 0 0 0 [2751336.773975] Normal: 791*4kB (EH) 52*8kB (UME) 2*16kB (U) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3612kB [2751336.774007] HighMem: 7982*4kB (UMC) 7072*8kB (UMC) 5995*16kB (UMC) 5055*32kB (UMC) 3730*64kB (UMC) 2228*128kB (UMC) 980*256kB (UMC) 217*512kB (MC) 13*1024kB (M) 0*2048kB 0*4096kB = 1245384kB [2751336.774047] 5098 total pagecache pages [2751336.774053] 185 pages in swap cache [2751336.774057] Swap cache stats: add 150772, delete 150582, find 40924940/40978868 [2751336.774059] Free swap = 996128kB [2751336.774061] Total swap = 1030688kB [2751336.774064] 524288 pages RAM [2751336.774066] 327680 pages HighMem/MovableOnly [2751336.774068] 8943 pages reserved [2751336.774070] 32768 pages cma reserved [2751336.774073] Unreclaimable slab info: [2751336.774075] Name Used Total [2751336.774387] zs_handle 227KB 332KB [2751336.774493] RAWv6 47KB 47KB [2751336.774496] UDPv6 47KB 47KB [2751336.774500] tw_sock_TCPv6 15KB 15KB [2751336.774504] request_sock_TCPv6 15KB 15KB [2751336.774508] TCPv6 123KB 123KB [2751336.774512] sgpool-128 30KB 30KB [2751336.774516] sgpool-64 31KB 31KB [2751336.774520] sgpool-32 15KB 15KB [2751336.774523] sgpool-16 7KB 7KB [2751336.774529] mqueue_inode_cache 15KB 15KB [2751336.774541] nfsd_drc 3KB 3KB [2751336.774555] posix_timers_cache 15KB 15KB [2751336.774559] rpc_buffers 32KB 32KB [2751336.774563] xfrm_state 31KB 31KB [2751336.774568] RAW 31KB 31KB [2751336.774571] UDP 61KB 61KB [2751336.774575] tw_sock_TCP 75KB 75KB [2751336.774579] request_sock_TCP 11KB 11KB [2751336.774583] TCP 182KB 182KB [2751336.774588] eventpoll_pwq 15KB 15KB [2751336.774591] inotify_inode_mark 15KB 15KB [2751336.774595] request_queue 92KB 92KB [2751336.774600] blkdev_ioc 44KB 44KB [2751336.774604] biovec-max 540KB 540KB [2751336.774608] biovec-128 126KB 126KB [2751336.774611] biovec-64 63KB 63KB [2751336.774618] biovec-16 76KB 126KB [2751336.774622] ksm_mm_slot 15KB 15KB [2751336.774629] skbuff_fclone_cache 283KB 315KB [2751336.774633] skbuff_head_cache 378KB 378KB [2751336.774637] configfs_dir_cache 3KB 3KB [2751336.774641] file_lock_cache 15KB 15KB [2751336.774645] fsnotify_mark_connector 16KB 16KB [2751336.774649] net_namespace 29KB 29KB [2751336.774662] shmem_inode_cache 807KB 960KB [2751336.774666] taskstats 30KB 30KB [2751336.774670] proc_dir_entry 108KB 108KB [2751336.774673] pde_opener 15KB 15KB [2751336.774678] seq_file 15KB 15KB [2751336.774681] sigqueue 15KB 15KB [2751336.774686] kernfs_node_cache 2563KB 2563KB [2751336.774689] filp 252090KB 252090KB [2751336.774694] names_cache 128KB 128KB [2751336.774699] key_jar 108KB 108KB [2751336.774704] vm_area_struct 248KB 248KB [2751336.774708] mm_struct 32KB 32KB [2751336.774711] files_cache 20KB 20KB [2751336.774715] signal_cache 204KB 204KB [2751336.774719] sighand_cache 315KB 315KB [2751336.774723] task_struct 401KB 401KB [2751336.774727] cred_jar 872KB 872KB [2751336.774733] anon_vma_chain 189KB 228KB [2751336.774737] anon_vma 287KB 287KB [2751336.774741] pid 32KB 32KB [2751336.774745] trace_event_file 79KB 79KB [2751336.774771] ftrace_event_field 376KB 724KB [2751336.774775] pool_workqueue 188KB 188KB [2751336.774780] kmalloc-8192 5024KB 5024KB [2751336.774783] kmalloc-4096 384KB 384KB [2751336.774788] kmalloc-2048 782KB 928KB [2751336.774796] kmalloc-1024 792KB 864KB [2751336.774803] kmalloc-512 1134KB 1164KB [2751336.774807] kmalloc-256 193KB 200KB [2751336.774811] kmalloc-192 1634KB 1634KB [2751336.774814] kmalloc-128 1912KB 1912KB [2751336.774818] kmalloc-64 85188KB 85188KB [2751336.774822] kmem_cache_node 188KB 188KB [2751336.774829] kmem_cache 605KB 620KB [2751336.774832] Tasks state (memory values in pages): [2751336.774835] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751336.774857] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751336.774866] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751336.774874] [ 528] 0 528 742 242 12288 0 0 alsactl [2751336.774881] [ 537] 0 537 6117 552 24576 50 0 rsyslogd [2751336.774888] [ 543] 0 543 1830 369 14336 22 0 cron [2751336.774895] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751336.774902] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751336.774910] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751336.774917] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751336.774923] [ 712] 104 712 806 323 12288 0 0 chronyd [2751336.774931] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751336.774938] [ 836] 0 836 8994 602 26624 53 0 polkitd [2751336.774946] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751336.774953] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751336.774960] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751336.774967] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751336.774974] [ 1278] 1000 1278 1346 804 14336 156 0 screen [2751336.774981] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751336.774988] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751336.774994] [ 1283] 0 1283 1045 245 12288 80 0 su [2751336.775001] [ 1284] 0 1284 818 207 14336 222 0 bash [2751336.775008] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751336.775015] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751336.775022] [ 1297] 1000 1297 819 375 12288 183 0 bash [2751336.775029] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751336.775044] [ 7476] 0 7476 354 93 10240 0 0 sleep [2751336.775049] Out of memory: Kill process 1278 (screen) score 1 or sacrifice child [2751336.775097] Killed process 1297 (bash) total-vm:3276kB, anon-rss:164kB, file-rss:1336kB, shmem-rss:0kB [2751336.776314] oom_reaper: reaped process 1297 (bash), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [2751336.783377] systemd-logind invoked oom-killer: gfp_mask=0x6102c0(GFP_KERNEL|__GFP_NOWARN|__GFP_NOMEMALLOC), nodemask=(null), order=0, oom_score_adj=0 [2751336.783385] systemd-logind cpuset=/ mems_allowed=0 [2751336.783402] CPU: 2 PID: 547 Comm: systemd-logind Not tainted 4.19.57-sunxi #5.90 [2751336.783405] Hardware name: Allwinner sun8i Family [2751336.783431] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [2751336.783443] [<c010a2f1>] (show_stack) from [<c08fb6a1>] (dump_stack+0x69/0x78) [2751336.783456] [<c08fb6a1>] (dump_stack) from [<c01ed415>] (dump_header+0x55/0x1c0) [2751336.783469] [<c01ed415>] (dump_header) from [<c01ecbbf>] (oom_kill_process+0x25b/0x298) [2751336.783479] [<c01ecbbf>] (oom_kill_process) from [<c01ed2a1>] (out_of_memory+0x175/0x240) [2751336.783489] [<c01ed2a1>] (out_of_memory) from [<c01f12c5>] (__alloc_pages_nodemask+0x7fd/0xbc0) [2751336.783501] [<c01f12c5>] (__alloc_pages_nodemask) from [<c022a225>] (new_slab+0x1fd/0x504) [2751336.783511] [<c022a225>] (new_slab) from [<c022b843>] (___slab_alloc.constprop.14+0x2f7/0x3cc) [2751336.783521] [<c022b843>] (___slab_alloc.constprop.14) from [<c022dbd9>] (__kmalloc_track_caller+0x129/0x1b4) [2751336.783533] [<c022dbd9>] (__kmalloc_track_caller) from [<c07d6431>] (__kmalloc_reserve.constprop.23+0x1d/0x48) [2751336.783544] [<c07d6431>] (__kmalloc_reserve.constprop.23) from [<c07d64bb>] (__alloc_skb+0x5f/0x110) [2751336.783553] [<c07d64bb>] (__alloc_skb) from [<c07d6d59>] (alloc_skb_with_frags+0x2d/0x108) [2751336.783563] [<c07d6d59>] (alloc_skb_with_frags) from [<c07d1ab1>] (sock_alloc_send_pskb+0x18d/0x1a4) [2751336.783576] [<c07d1ab1>] (sock_alloc_send_pskb) from [<c087eb1b>] (unix_dgram_sendmsg+0xef/0x4a0) [2751336.783588] [<c087eb1b>] (unix_dgram_sendmsg) from [<c07ce881>] (sock_sendmsg+0x21/0x2c) [2751336.783599] [<c07ce881>] (sock_sendmsg) from [<c07cedc5>] (___sys_sendmsg+0x159/0x17c) [2751336.783608] [<c07cedc5>] (___sys_sendmsg) from [<c07cf8d3>] (__sys_sendmsg+0x37/0x58) [2751336.783617] [<c07cf8d3>] (__sys_sendmsg) from [<c01011f9>] (__sys_trace_return+0x1/0xc) [2751336.783621] Exception stack(0xeb46dfa8 to 0xeb46dff0) [2751336.783628] dfa0: 00000074 b6d65404 00000012 beb6e7b0 00004000 00000000 [2751336.783636] dfc0: 00000074 b6d65404 00000001 00000128 00000012 00000000 00000000 00000000 [2751336.783641] dfe0: 00000128 beb6e770 b6e8951f b6e05746 [2751336.783644] Mem-Info: [2751336.783659] active_anon:1090 inactive_anon:1897 isolated_anon:30 active_file:3639 inactive_file:1166 isolated_file:18 unevictable:0 dirty:4 writeback:29 unstable:0 slab_reclaimable:16374 slab_unreclaimable:160931 mapped:2011 shmem:150 pagetables:157 bounce:0 free:312077 free_pcp:2127 free_cma:31227 [2751336.783670] Node 0 active_anon:4360kB inactive_anon:7588kB active_file:14556kB inactive_file:4664kB unevictable:0kB isolated(anon):120kB isolated(file):72kB mapped:8044kB dirty:16kB writeback:116kB shmem:600kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [2751336.783683] Normal free:3408kB min:3448kB low:4308kB high:5168kB active_anon:152kB inactive_anon:92kB active_file:188kB inactive_file:192kB unevictable:0kB writepending:84kB present:786432kB managed:750672kB mlocked:0kB kernel_stack:8720kB pagetables:0kB bounce:0kB free_pcp:4020kB local_pcp:988kB free_cma:0kB [2751336.783685] lowmem_reserve[]: 0 10239 10239 [2751336.783704] HighMem free:1244900kB min:512kB low:1880kB high:3248kB active_anon:4596kB inactive_anon:7708kB active_file:14504kB inactive_file:4312kB unevictable:0kB writepending:0kB present:1310720kB managed:1310708kB mlocked:0kB kernel_stack:0kB pagetables:656kB bounce:0kB free_pcp:4488kB local_pcp:1112kB free_cma:124908kB [2751336.783705] lowmem_reserve[]: 0 0 0 [2751336.783713] Normal: 791*4kB (EH) 52*8kB (UME) 2*16kB (U) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3612kB [2751336.783745] HighMem: 7935*4kB (UMC) 7025*8kB (UMC) 5960*16kB (UMC) 5038*32kB (UMC) 3720*64kB (UMC) 2232*128kB (UMC) 982*256kB (UMC) 218*512kB (MC) 14*1024kB (M) 0*2048kB 0*4096kB = 1245636kB [2751336.783786] 5098 total pagecache pages [2751336.783791] 185 pages in swap cache [2751336.783795] Swap cache stats: add 150772, delete 150582, find 40924940/40978928 [2751336.783798] Free swap = 996896kB [2751336.783800] Total swap = 1030688kB [2751336.783802] 524288 pages RAM [2751336.783804] 327680 pages HighMem/MovableOnly [2751336.783807] 8943 pages reserved [2751336.783808] 32768 pages cma reserved [2751336.783812] Unreclaimable slab info: [2751336.783814] Name Used Total [2751336.784188] zs_handle 226KB 332KB [2751336.784302] RAWv6 47KB 47KB [2751336.784308] UDPv6 47KB 47KB [2751336.784312] tw_sock_TCPv6 15KB 15KB [2751336.784316] request_sock_TCPv6 15KB 15KB [2751336.784320] TCPv6 123KB 123KB [2751336.784324] sgpool-128 30KB 30KB [2751336.784328] sgpool-64 31KB 31KB [2751336.784331] sgpool-32 15KB 15KB [2751336.784335] sgpool-16 7KB 7KB [2751336.784344] mqueue_inode_cache 15KB 15KB [2751336.784356] nfsd_drc 3KB 3KB [2751336.784370] posix_timers_cache 15KB 15KB [2751336.784374] rpc_buffers 32KB 32KB [2751336.784378] xfrm_state 31KB 31KB [2751336.784382] RAW 31KB 31KB [2751336.784386] UDP 61KB 61KB [2751336.784390] tw_sock_TCP 75KB 75KB [2751336.784394] request_sock_TCP 11KB 11KB [2751336.784397] TCP 182KB 182KB [2751336.784402] eventpoll_pwq 15KB 15KB [2751336.784405] inotify_inode_mark 15KB 15KB [2751336.784409] request_queue 92KB 92KB [2751336.784414] blkdev_ioc 44KB 44KB [2751336.784417] biovec-max 540KB 540KB [2751336.784421] biovec-128 126KB 126KB [2751336.784425] biovec-64 63KB 63KB [2751336.784432] biovec-16 76KB 126KB [2751336.784436] ksm_mm_slot 15KB 15KB [2751336.784442] skbuff_fclone_cache 283KB 315KB [2751336.784445] skbuff_head_cache 378KB 378KB [2751336.784451] configfs_dir_cache 3KB 3KB [2751336.784456] file_lock_cache 15KB 15KB [2751336.784461] fsnotify_mark_connector 16KB 16KB [2751336.784466] net_namespace 29KB 29KB [2751336.784479] shmem_inode_cache 807KB 960KB [2751336.784483] taskstats 30KB 30KB [2751336.784487] proc_dir_entry 108KB 108KB [2751336.784491] pde_opener 15KB 15KB [2751336.784495] seq_file 15KB 15KB [2751336.784499] sigqueue 15KB 15KB [2751336.784503] kernfs_node_cache 2563KB 2563KB [2751336.784507] filp 252090KB 252090KB [2751336.784511] names_cache 128KB 128KB [2751336.784516] key_jar 108KB 108KB [2751336.784521] vm_area_struct 248KB 248KB [2751336.784525] mm_struct 32KB 32KB [2751336.784529] files_cache 20KB 20KB [2751336.784532] signal_cache 204KB 204KB [2751336.784536] sighand_cache 315KB 315KB [2751336.784540] task_struct 401KB 401KB [2751336.784544] cred_jar 872KB 872KB [2751336.784551] anon_vma_chain 189KB 228KB [2751336.784554] anon_vma 287KB 287KB [2751336.784558] pid 32KB 32KB [2751336.784564] trace_event_file 79KB 79KB [2751336.784594] ftrace_event_field 375KB 724KB [2751336.784598] pool_workqueue 188KB 188KB [2751336.784603] kmalloc-8192 5024KB 5024KB [2751336.784607] kmalloc-4096 384KB 384KB [2751336.784612] kmalloc-2048 782KB 928KB [2751336.784619] kmalloc-1024 790KB 864KB [2751336.784626] kmalloc-512 1134KB 1164KB [2751336.784630] kmalloc-256 193KB 200KB [2751336.784633] kmalloc-192 1634KB 1634KB [2751336.784637] kmalloc-128 1912KB 1912KB [2751336.784641] kmalloc-64 85188KB 85188KB [2751336.784645] kmem_cache_node 188KB 188KB [2751336.784652] kmem_cache 605KB 620KB [2751336.784655] Tasks state (memory values in pages): [2751336.784658] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [2751336.784686] [ 284] 0 284 4031 240 16384 146 -1000 systemd-udevd [2751336.784695] [ 324] 108 324 1344 410 14336 0 0 rpcbind [2751336.784704] [ 528] 0 528 742 242 12288 0 0 alsactl [2751336.784711] [ 537] 0 537 6117 618 24576 50 0 rsyslogd [2751336.784718] [ 543] 0 543 1830 369 14336 22 0 cron [2751336.784726] [ 547] 0 547 2655 512 20480 75 0 systemd-logind [2751336.784733] [ 563] 105 563 1381 509 10240 53 -900 dbus-daemon [2751336.784741] [ 572] 0 572 2241 657 16384 62 0 wpa_supplicant [2751336.784749] [ 710] 104 710 2753 323 12288 0 0 chronyd [2751336.784756] [ 712] 104 712 806 323 12288 0 0 chronyd [2751336.784763] [ 728] 0 728 2247 536 18432 150 -1000 sshd [2751336.784771] [ 836] 0 836 8994 602 26624 53 0 polkitd [2751336.784781] [ 1096] 0 1096 959 164 10240 0 0 agetty [2751336.784791] [ 1098] 0 1098 1536 272 14336 0 0 agetty [2751336.784798] [ 1173] 1000 1173 3146 573 18432 182 0 systemd [2751336.784805] [ 1174] 1000 1174 3599 14 20480 512 0 (sd-pam) [2751336.784812] [ 1278] 1000 1278 1346 868 14336 92 0 screen [2751336.784819] [ 1279] 1000 1279 819 173 14336 223 0 bash [2751336.784826] [ 1282] 0 1282 1153 266 12288 116 0 sudo [2751336.784833] [ 1283] 0 1283 1045 245 12288 80 0 su [2751336.784840] [ 1284] 0 1284 818 207 14336 222 0 bash [2751336.784847] [ 1287] 1000 1287 819 175 14336 223 0 bash [2751336.784854] [ 1296] 1000 1296 909 493 12288 23 0 htop [2751336.784861] [ 12362] 0 12362 741 445 12288 111 0 armbianmonitor [2751336.784880] [ 7476] 0 7476 354 93 10240 0 0 sleep [2751336.784886] Out of memory: Kill process 1278 (screen) score 1 or sacrifice child [2751336.784931] Killed process 1287 (bash) total-vm:3276kB, anon-rss:4kB, file-rss:696kB, shmem-rss:0kB [2751336.790855] Out of memory: Kill process 1278 (screen) score 1 or sacrifice child [2751336.790909] Killed process 1279 (bash) total-vm:3276kB, anon-rss:4kB, file-rss:688kB, shmem-rss:0kB [2751336.792602] oom_reaper: reaped process 1279 (bash), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB tesla@buster:~$ tesla@buster:~$ chronyc tracking Reference ID : C1465EB6 (time.taken.pl) Stratum : 3 Ref time (UTC) : Fri Aug 25 01:35:43 1978 System time : 0.000000002 seconds fast of NTP time Last offset : -1292662912.000000000 seconds RMS offset : 1289963136.000000000 seconds Frequency : 31.898 ppm fast Residual freq : +0.001 ppm Skew : 0.013 ppm Root delay : 0.023268534 seconds Root dispersion : 0.011201797 seconds Update interval : 259.0 seconds Leap status : Normal I got chrony instead of ntpd and it has a line "makestep 1 -1" in it's config allowing infinite number of big time corrections. For some reason it did not helped. There are relevant topis on the forum for A64: The only difference is that A64 jumps forward. I found no related issues on github or forum for H3 specifically. Does any one else experience this issue?
martinayotte Posted August 12, 2019 Posted August 12, 2019 21 hours ago, lomady said: Does any one else experience this issue? The original clock jump issue is really a jump in the future, not in the past ... Do you have any network/internet issue that prevent having NTP working properly ?
lomady Posted August 12, 2019 Author Posted August 12, 2019 Yes, I know that the original issue is about jumping in the future. No, I do not have any network issues. Both ntpd and chrony maintain correct time until it jumps back to 1978. After that rcu: INFO: rcu_sched self-detected stall on CPU OOM killer unhinges and starts killing processes left and right I also saw many cron processes and eMMC was thrashing. I just want t get to the bottom of these random time jumps. Sometimes it takes several days, sometimes it takes a month of uptime for it to happen.
drice Posted August 13, 2019 Posted August 13, 2019 On 8/11/2019 at 10:35 AM, lomady said: [2736263.499172] rcu: INFO: rcu_sched self-detected stall on CPU [2736263.499195] rcu: 0-...!: (1 ticks this GP) idle=642/0/0x1 softirq=68967472/68967472 fqs=0 [2736263.499197] rcu: (t=249107 jiffies g=147892009 q=729) [2736263.499208] rcu: rcu_sched kthread starved for 249107 jiffies! g147892009 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 This looks similar to the issue in this post: CPU stall followed by clock getting set to an incorrect time. I think the clock issue is a symptom rather than a cause. I didn't see anything obvious in the call stack of this issue or the referenced issue but I'm not very good at reading those. Maybe someone who knows more can compare these posts and see if a root cause is obvious. 1
AristoChen Posted August 30, 2019 Posted August 30, 2019 Hi, I have encountered this issue for many times on OrangePi PC, this issue happened by using following version Armbian version: 5.41, Kernel version: 4.14.32 Armbian version: 5.77, Kernel version: 4.19.25 Armbian version: 5.90, Kernel version: 4.19.57 I have more than 10 Orangepi PC running busy tasks at the same time, and this issue happened more than 15 times in the past year, still have no idea how to deal with this, it is still able to send data to my server somehow, which means that the internet connection is available, but i'm not able to ssh. I have tried the following methods: Wrote a udev file to copy all system log when usb pendrive is detected, this works fine when device runs normally, but not working when encounter this issue. Using watchdog, works fine if using fork bomb to test, but not able to recover from this issue Tried connect to a keyboard and hdmi monitor, this works fine when device runs normally, but not working when encounter this issue. Using TTL serial to usb adapter, I'm able to control device when it runs normally, but not working when encounter this issue. I found that dmesg record some error message as below when I encountered this again yesterday Quote [Thu Aug 29 01:45:44 2019] rcu: INFO: rcu_sched detected stalls on CPUs/tasks: [Thu Aug 29 01:45:44 2019] rcu: 0-...!: (4 GPs behind) idle=6ea/1/0x40000000 softirq=63333042/63333042 fqs=1 [Thu Aug 29 01:45:44 2019] rcu: (detected by 2, t=213675 jiffies, g=216788133, q=4) [Thu Aug 29 01:45:44 2019] Sending NMI from CPU 2 to CPUs 0: [Thu Aug 29 01:45:44 2019] NMI backtrace for cpu 0 [Thu Aug 29 01:45:44 2019] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 4.19.62-sunxi #5.92 [Thu Aug 29 01:45:44 2019] Hardware name: Allwinner sun8i Family [Thu Aug 29 01:45:44 2019] PC is at arch_cpu_idle+0x28/0x2c [Thu Aug 29 01:45:44 2019] LR is at arch_cpu_idle+0x27/0x2c [Thu Aug 29 01:45:44 2019] pc : [<c01078f4>] lr : [<c01078f3>] psr: 40010033 [Thu Aug 29 01:45:44 2019] sp : c0e01f68 ip : ef6a0780 fp : 00000000 [Thu Aug 29 01:45:44 2019] r10: c0dba870 r9 : c0e04d48 r8 : 00000000 [Thu Aug 29 01:45:44 2019] r7 : 00000001 r6 : c0e04db8 r5 : c0e04d70 r4 : ffffe000 [Thu Aug 29 01:45:44 2019] r3 : c0116441 r2 : ef69f438 r1 : 417ee6ec r0 : 00000000 [Thu Aug 29 01:45:44 2019] Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA Thumb Segment none [Thu Aug 29 01:45:44 2019] Control: 50c5387d Table: 6c03406a DAC: 00000051 [Thu Aug 29 01:45:44 2019] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 4.19.62-sunxi #5.92 [Thu Aug 29 01:45:44 2019] Hardware name: Allwinner sun8i Family [Thu Aug 29 01:45:44 2019] [<c010d74d>] (unwind_backtrace) from [<c010a2f1>] (show_stack+0x11/0x14) [Thu Aug 29 01:45:44 2019] [<c010a2f1>] (show_stack) from [<c08fc121>] (dump_stack+0x69/0x78) [Thu Aug 29 01:45:44 2019] [<c08fc121>] (dump_stack) from [<c090036d>] (nmi_cpu_backtrace+0x59/0x90) [Thu Aug 29 01:45:44 2019] [<c090036d>] (nmi_cpu_backtrace) from [<c010c5d9>] (handle_IPI+0x85/0x2c0) [Thu Aug 29 01:45:44 2019] [<c010c5d9>] (handle_IPI) from [<c05c9c7f>] (gic_handle_irq+0x67/0x68) [Thu Aug 29 01:45:44 2019] [<c05c9c7f>] (gic_handle_irq) from [<c0101a65>] (__irq_svc+0x65/0x94) [Thu Aug 29 01:45:44 2019] Exception stack(0xc0e01f18 to 0xc0e01f60) [Thu Aug 29 01:45:44 2019] 1f00: 00000000 417ee6ec [Thu Aug 29 01:45:44 2019] 1f20: ef69f438 c0116441 ffffe000 c0e04d70 c0e04db8 00000001 00000000 c0e04d48 [Thu Aug 29 01:45:44 2019] 1f40: c0dba870 00000000 ef6a0780 c0e01f68 c01078f3 c01078f4 40010033 ffffffff [Thu Aug 29 01:45:44 2019] [<c0101a65>] (__irq_svc) from [<c01078f4>] (arch_cpu_idle+0x28/0x2c) [Thu Aug 29 01:45:44 2019] [<c01078f4>] (arch_cpu_idle) from [<c013e96b>] (do_idle+0x14b/0x1d8) [Thu Aug 29 01:45:44 2019] [<c013e96b>] (do_idle) from [<c013ebed>] (cpu_startup_entry+0x19/0x1c) [Thu Aug 29 01:45:44 2019] [<c013ebed>] (cpu_startup_entry) from [<c0d00bad>] (start_kernel+0x3a7/0x3c6) [Thu Aug 29 01:45:44 2019] rcu: rcu_sched kthread starved for 213673 jiffies! g216788133 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=2 [Thu Aug 29 01:45:44 2019] rcu: RCU grace-period kthread stack dump: [Thu Aug 29 01:45:44 2019] rcu_sched I 0 10 2 0x00000000 [Thu Aug 29 01:45:44 2019] [<c090acdb>] (__schedule) from [<c090b247>] (schedule+0x2f/0x68) [Thu Aug 29 01:45:44 2019] [<c090b247>] (schedule) from [<c090daf7>] (schedule_timeout+0x77/0x320) [Thu Aug 29 01:45:44 2019] [<c090daf7>] (schedule_timeout) from [<c016af6f>] (rcu_gp_kthread+0x41f/0x728) [Thu Aug 29 01:45:44 2019] [<c016af6f>] (rcu_gp_kthread) from [<c0132ae9>] (kthread+0xfd/0x104) [Thu Aug 29 01:45:44 2019] [<c0132ae9>] (kthread) from [<c01010f9>] (ret_from_fork+0x11/0x38) [Thu Aug 29 01:45:44 2019] Exception stack(0xef14bfb0 to 0xef14bff8) [Thu Aug 29 01:45:44 2019] bfa0: 00000000 00000000 00000000 00000000 [Thu Aug 29 01:45:44 2019] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [Thu Aug 29 01:45:44 2019] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [Thu Aug 29 02:14:52 2019] systemd-journald[260]: Failed to run event loop: Invalid argument Currently, I might try to continuously detect whether this error message occurred in dmesg, if yes, I will try using this command "sudo systemctl --force --force reboot"(it seems that /sbin/reboot does not work at the moment) 2
Recommended Posts