Werner Posted July 15, 2020 Posted July 15, 2020 Armbianmonitor: http://ix.io/2rG7 Spoiler [55881.786168] 8<--- cut here --- [55881.789254] Unable to handle kernel paging request at virtual address bfd60e58 [55881.796482] pgd = 7d7d1ef9 [55881.799191] [bfd60e58] *pgd=00000000 [55881.802777] Internal error: Oops: 80000005 [#1] SMP THUMB2 [55881.808261] Modules linked in: rfkill zstd snd_soc_hdmi_codec dw_hdmi_cec dw_hdmi_i2s_audio sun8i_codec_analog snd_soc_simple_card sun8i_adda_pr_regmap sun4i_i2s snd_soc_simple_card_utils snd_soc_core ac97_bus sun8i_drm_hdmi snd_pcm_dmaengine snd_pcm dw_hdmi sun4i_gpadc_iio lima snd_timer industrialio cec sunxi_cedrus(C) snd gpu_sched v4l2_mem2mem sun8i_thermal soundcore videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videobuf2_common videodev zram mc zsmalloc sun4i_drm evdev sun4i_frontend sun8i_mixer sun4i_tcon sun8i_tcon_top uio_pdrv_genirq uio cpufreq_dt ip_tables x_tables autofs4 gpio_regulator fixed gpio_keys [55881.863216] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G C E 5.4.51-sunxi #trunk [55881.871391] Hardware name: Allwinner sun8i Family [55881.876101] PC is at 0xbfd60e58 [55881.879251] LR is at vfp_notifier+0x9/0xe4 [55881.883347] pc : [<bfd60e58>] lr : [<c0103a51>] psr: 80080093 [55881.889611] sp : c0e01ee8 ip : c0e00040 fp : d70fa1c0 [55881.894834] r10: c0902528 r9 : 00000000 r8 : 00000002 [55881.900057] r7 : d711e000 r6 : 00000000 r5 : ffffffff r4 : c0e09964 [55881.906582] r3 : c0103a49 r2 : d711e000 r1 : 00000002 r0 : c0e09964 [55881.913109] Flags: Nzcv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none [55881.920332] Control: 50c5387d Table: 4e39c06a DAC: 00000051 [55881.926078] Process swapper/0 (pid: 0, stack limit = 0xa8644cee) [55881.932083] Stack: (0xc0e01ee8 to 0xc0e02000) [55881.936445] 1ee0: c0103a49 c0e09964 ffffffff c0136a23 ffffffff d711e018 [55881.944625] 1f00: c0e08ac0 00000000 5eae9766 00000000 00000000 c0136a71 00000000 c0e01f48 [55881.952805] 1f20: c0902528 c01021ff 00000000 00000000 0000000a c0181695 8e37512a 1f1a2000 [55881.960986] 1f40: c0872d0b 00000000 c0e01f60 00000000 000032d2 c0e00000 c0e04fa4 c0e04fec [55881.969166] 1f60: c0e01f78 00000000 c0db99f0 c0ec4be5 00000000 c0872d0b ffffe000 c0e04fa4 [55881.977347] 1f80: 00000001 c0141249 c0e04f80 00000000 c0d5aa40 410fc075 50c5387d 000000ce [55881.985527] 1fa0: 00000001 c0e04f80 00000000 c0d5aa40 410fc075 50c5387d 00000000 c0141511 [55881.993708] 1fc0: 08000000 c0d00c81 ffffffff ffffffff 00000000 c0d004e9 00000000 c0d5aa40 [55882.001888] 1fe0: c0d00328 00000051 10c0387d 00001029 495b4000 00000000 00000000 00000000 [55882.010081] [<c0103a51>] (vfp_notifier) from [<c0136a23>] (notifier_call_chain+0x43/0x60) [55882.018264] [<c0136a23>] (notifier_call_chain) from [<c0136a71>] (atomic_notifier_call_chain+0x19/0x20) [55882.027660] [<c0136a71>] (atomic_notifier_call_chain) from [<c01021ff>] (__switch_to+0x33/0x48) [55882.036356] Exception stack(0xc0e01f28 to 0xc0e01f70) [55882.041409] 1f20: 00000000 00000000 0000000a c0181695 8e37512a 1f1a2000 [55882.049590] 1f40: c0872d0b 00000000 c0e01f60 00000000 000032d2 c0e00000 c0e04fa4 c0e04fec [55882.057767] 1f60: c0e01f78 00000000 c0db99f0 c0ec4be5 [55882.062820] Code: bad PC value [55882.065878] ---[ end trace 30b1e5b3d057aa18 ]--- [55882.070497] Kernel panic - not syncing: Attempted to kill the idle task! [55882.077204] CPU2: stopping [55882.079920] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D C E 5.4.51-sunxi #trunk [55882.088095] Hardware name: Allwinner sun8i Family [55882.092810] [<c010dc49>] (unwind_backtrace) from [<c010a245>] (show_stack+0x11/0x14) [55882.100564] [<c010a245>] (show_stack) from [<c086227b>] (dump_stack+0x6f/0x7c) [55882.107794] [<c086227b>] (dump_stack) from [<c010cacf>] (handle_IPI+0x293/0x2bc) [55882.115197] [<c010cacf>] (handle_IPI) from [<c0518395>] (gic_handle_irq+0x69/0x6c) [55882.122772] [<c0518395>] (gic_handle_irq) from [<c0101ae5>] (__irq_svc+0x65/0x94) [55882.130254] Exception stack(0xd7125f60 to 0xd7125fa8) [55882.135309] 5f60: 00000000 0005e84c dff85034 c01164c1 ffffe000 c0e04fa4 c0e04fec 00000004 [55882.143489] 5f80: 00000000 c0db99f0 c0ec4be5 00000000 c0f0d058 d7125fb0 c0107c6f c0107c70 [55882.151665] 5fa0: 40000033 ffffffff [55882.155161] [<c0101ae5>] (__irq_svc) from [<c0107c70>] (arch_cpu_idle+0x28/0x2c) [55882.162566] [<c0107c70>] (arch_cpu_idle) from [<c01412c3>] (do_idle+0x143/0x1b0) [55882.169968] [<c01412c3>] (do_idle) from [<c0141511>] (cpu_startup_entry+0x19/0x20) [55882.177542] [<c0141511>] (cpu_startup_entry) from [<40102531>] (0x40102531) [55883.106962] SMP: failed to stop secondary CPUs [55883.111415] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]--- A friend of mine told me that his OPi One freezes every couple of days for no reason. So I took it with me and see what I could find. I connected another SBC via serial to the board and wait until it freezes. This is what I have got and no idea what the reason is.... The armbianmonitor output is from a fresh restart after power cycle.
xwiggen Posted July 15, 2020 Posted July 15, 2020 Can you compare the dts of this kernel with earlier kernel? (v20.02 -- arch/arm/boot/dts especially cpu entries) Seems like upstream did refactoring, I didn't want to touch anything since I've no way of testing changes.
guidol Posted July 15, 2020 Posted July 15, 2020 I had these freezes also with my OPi One - no matter with armbian current or dev (buster & focal). I hadnt this in 2019. I did see it, because I used him as one of my piholes and when he did freeze some of my android-devices did act not normally while using the web (they seem to have a problem with using the 2nd dns in the network-settings)
Werner Posted July 15, 2020 Author Posted July 15, 2020 I have another One running for month now (also as Pihole) and never had issues with it. Issue recreated, though other virtual address this time. [16668.716765] Unable to handle kernel paging request at virtual address 3f00d700 [16668.723992] pgd = 61a9cd52 [16668.726701] [3f00d700] *pgd=00000000 [16668.730286] Internal error: Oops: 80000005 [#1] SMP THUMB2 [16668.735770] Modules linked in: rfkill zstd snd_soc_hdmi_codec dw_hdmi_cec dw_hdmi_i2s_audio snd_soc_simple_card sun8i_codec_analog snd_soc_simple_card_utils sun8i_adda_pr_regmap sun4i_i2s snd_soc_core ac97_bus sunxi_cedrus(C) snd_pcm_dmaengine v4l2_mem2mem videobuf2_dma_contig snd_pcm sun8i_drm_hdmi videobuf2_memops dw_hdmi lima snd_timer sun4i_gpadc_iio videobuf2_v4l2 cec snd gpu_sched industrialio videobuf2_common soundcore sun8i_thermal videodev mc sun4i_drm sun4i_frontend sun8i_mixer evdev sun4i_tcon sun8i_tcon_top uio_pdrv_genirq uio cpufreq_dt zram zsmalloc ip_tables x_tables autofs4 gpio_regulator fixed gpio_keys [16668.790725] CPU: 3 PID: 1 Comm: systemd Tainted: G C E 5.4.51-sunxi #trunk [16668.798726] Hardware name: Allwinner sun8i Family [16668.803435] PC is at 0x3f00d700 [16668.806577] LR is at 0xb6e8e746 [16668.809718] pc : [<3f00d700>] lr : [<b6e8e746>] psr: 20070093 [16668.815984] sp : d70edff8 ip : 000000c5 fp : 014df550 [16668.821207] r10: 00000000 r9 : bef6b288 r8 : b6f89c10 [16668.826431] r7 : 000000c5 r6 : b6f99968 r5 : b6f6bb1c r4 : 014df550 [16668.832957] r3 : b6ec6b45 r2 : bef6afe8 r1 : bef6afe8 r0 : 00000015 [16668.839484] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user [16668.846707] Control: 50c5387d Table: 5688806a DAC: 00000055 [16668.852452] Process systemd (pid: 1, stack limit = 0xc59b4d99) [16668.858283] Stack: (0xd70edff8 to 0xd70ee000) [16668.862642] dfe0: 00000000 00000000 [16668.870824] Code: bad PC value [16668.873883] ---[ end trace e1c9811577573273 ]--- [16668.878507] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b [16669.915929] SMP: failed to stop secondary CPUs [16669.920379] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---
Werner Posted July 15, 2020 Author Posted July 15, 2020 5 minutes ago, xwiggen said: cpufreq working? Yup.
xwiggen Posted July 15, 2020 Posted July 15, 2020 [ 7.102323] core: _opp_supported_by_regulators: OPP minuV: 1320000 maxuV: 1320000, not supported by regulator [ 7.102338] cpu cpu0: _opp_add: OPP not supported by regulators (1104000000) [ 7.102518] core: _opp_supported_by_regulators: OPP minuV: 1320000 maxuV: 1320000, not supported by regulator [ 7.102526] cpu cpu0: _opp_add: OPP not supported by regulators (1200000000) [ 7.102703] core: _opp_supported_by_regulators: OPP minuV: 1340000 maxuV: 1340000, not supported by regulator [ 7.102710] cpu cpu0: _opp_add: OPP not supported by regulators (1296000000) [ 7.102868] core: _opp_supported_by_regulators: OPP minuV: 1400000 maxuV: 1400000, not supported by regulator [ 7.102875] cpu cpu0: _opp_add: OPP not supported by regulators (1368000000) [ 7.103311] thermal thermal_zone0: binding zone cpu_thermal with cdev thermal-cpufreq-0 failed:-22
Werner Posted July 15, 2020 Author Posted July 15, 2020 Just now, xwiggen said: [ 7.102323] core: _opp_supported_by_regulators: OPP minuV: 1320000 maxuV: 1320000, not supported by regulator [ 7.102338] cpu cpu0: _opp_add: OPP not supported by regulators (1104000000) [ 7.102518] core: _opp_supported_by_regulators: OPP minuV: 1320000 maxuV: 1320000, not supported by regulator [ 7.102526] cpu cpu0: _opp_add: OPP not supported by regulators (1200000000) [ 7.102703] core: _opp_supported_by_regulators: OPP minuV: 1340000 maxuV: 1340000, not supported by regulator [ 7.102710] cpu cpu0: _opp_add: OPP not supported by regulators (1296000000) [ 7.102868] core: _opp_supported_by_regulators: OPP minuV: 1400000 maxuV: 1400000, not supported by regulator [ 7.102875] cpu cpu0: _opp_add: OPP not supported by regulators (1368000000) [ 7.103311] thermal thermal_zone0: binding zone cpu_thermal with cdev thermal-cpufreq-0 failed:-22 That is not related to this specific issue. Armbian cannot clock higher on H3 since there is no proper support for the voltage regulator.
Werner Posted July 29, 2020 Author Posted July 29, 2020 [182764.941115] Bad mode in undefined instruction handler detected [182764.952552] Internal error: Oops - bad mode: 0 [#1] SMP THUMB2 [182764.963861] Modules linked in: rfkill input_leds snd_soc_hdmi_codec sun4i_gpadc_iio industrialio sun8i_thermal sun8i_ce sunxi_cedrus(C) crypto_engine sun8i_di evdev uio_pdrv_genirq uio cpufreq_dt zram ip_tables x_tables autofs4 lima gpu_sched dw_hdmi_i2s_audio dw_hdmi_cec sunxi phy_generic gpio_keys display_connector [182765.004106] CPU: 2 PID: 1 Comm: systemd Tainted: G C 5.7.10-sunxi #trunk [182765.018311] Hardware name: Allwinner sun8i Family [182765.028826] PC is at 0xffff10ca [182765.037366] LR is at 0xb6e87be6 [182765.045482] pc : [<ffff10ca>] lr : [<b6e87be6>] psr: 800701b7 [182765.056862] sp : d70dffb0 ip : b6f67ba0 fp : 00008000 [182765.067193] r10: 00000068 r9 : 00000001 r8 : 00000008 [182765.077505] r7 : 01b72dd0 r6 : b6f6776c r5 : b6f677a0 r4 : 01b72dd0 [182765.089159] r3 : b6f67808 r2 : 01ae9248 r1 : 00000071 r0 : c0ff880c [182765.100871] Flags: Nzcv IRQs off FIQs on Mode ABT_32 ISA Thumb Segment none [182765.113591] Control: 50c5387d Table: 568d406a DAC: 00000000 [182765.124711] Process systemd (pid: 1, stack limit = 0x33bc84e2) [182765.135980] Stack: (0xd70dffb0 to 0xd70e0000) [182765.145799] ffa0: c0ff880c 00000071 01ae9248 b6f67808 [182765.159593] ffc0: 01b72dd0 b6f677a0 b6f6776c 01b72dd0 00000008 00000001 00000068 00008000 [182765.173468] ffe0: b6f67ba0 d70dffb0 b6e87be6 ffff10ca 800701b7 ffffffff 00000000 00000000 [182765.187425] Code: f850 e02e 4668 efca (f8f0) efee [182765.198035] ---[ end trace 2a7b257dd76743ba ]--- [182765.208522] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b [182766.252010] SMP: failed to stop secondary CPUs [182766.262576] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]--- I think I stop tinkering with this one. He's dead, Jim.
guidol Posted July 29, 2020 Posted July 29, 2020 4 hours ago, Werner said: I think I stop tinkering with this one. He's dead, Jim. But I think its only a software problem.....this One reacts like he has a bad armbian-image
Werner Posted July 29, 2020 Author Posted July 29, 2020 1 hour ago, guidol said: But I think its only a software problem.....this One reacts like he has a bad armbian-image In theory you were right, but unfortunately it is not. I have two OPi One side by side. One with the defect and one known to work. I swapped the sd cards and the sd card out of the defective board booted flawless on the known-to-work board while the known-to-work sd card did not boot in the defective board. After a dozen power cycles it came back once but I do not trust it anymore.
xwiggen Posted July 29, 2020 Posted July 29, 2020 9 hours ago, Werner said: In theory you were right, but unfortunately it is not. I have two OPi One side by side. One with the defect and one known to work. I swapped the sd cards and the sd card out of the defective board booted flawless on the known-to-work board while the known-to-work sd card did not boot in the defective board. After a dozen power cycles it came back once but I do not trust it anymore. Have you tried powering through GPIO pins?
Werner Posted July 30, 2020 Author Posted July 30, 2020 Not yet but now. I predict it will freeze in one or two days...
Mikhail Kulinich Posted August 11, 2020 Posted August 11, 2020 I notice similar freezes on some of my OrangePI One boards: Aug 11 04:01:10 kernel: [11045.901932] asix 3-1.4.1:1.0 asix: link up, 100Mbps, full-duplex, lpa 0x41E1 Aug 11 04:05:50 kernel: [11326.227650] INFO: rcu_sched detected stalls on CPUs/tasks: Aug 11 04:05:50 kernel: [11326.227679] 3-...: (1 ticks this GP) idle=992/1/0 softirq=363926/363926 fqs=1 Aug 11 04:05:50 kernel: [11326.227681] (detected by 2, t=41234 jiffies, g=193459, c=193458, q=5) Aug 11 04:05:50 kernel: [11326.227697] Sending NMI from CPU 2 to CPUs 3: Aug 11 04:05:50 kernel: [11326.227720] NMI backtrace for cpu 3 Aug 11 04:05:50 kernel: [11326.227728] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G W O 4.13.15-sunxi #1 Aug 11 04:05:50 kernel: [11326.227730] Hardware name: Allwinner sun8i Family Aug 11 04:05:50 kernel: [11326.227734] task: df4f2f40 task.stack: df51e000 Aug 11 04:05:50 kernel: [11326.227747] PC is at expire_timers+0x8e/0xe0 Aug 11 04:05:50 kernel: [11326.227752] LR is at run_timer_softirq+0x103/0x138 Aug 11 04:05:50 kernel: [11326.227755] pc : [] lr : [] psr: 000e0133 Aug 11 04:05:50 kernel: [11326.227758] sp : df51fe80 ip : df51fea8 fp : 4000001f Aug 11 04:05:50 kernel: [11326.227761] r10: 1ee84000 r9 : c0dd3430 r8 : ffffe000 Aug 11 04:05:50 kernel: [11326.227763] r7 : c0d03f6c r6 : df51fea4 r5 : dfb33440 r4 : df76f8a0 Aug 11 04:05:50 kernel: [11326.227767] r3 : 000003df r2 : df76f808 r1 : c0604f21 r0 : dfb33440 Aug 11 04:05:50 kernel: [11326.227771] Flags: nzcv IRQs on FIQs on Mode SVC_32 ISA Thumb Segment none Aug 11 04:05:50 kernel: [11326.227775] Control: 50c5387d Table: 4a5bc06a DAC: 00000051 Aug 11 04:05:50 kernel: [11326.227779] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G W O 4.13.15-sunxi #1 Aug 11 04:05:50 kernel: [11326.227781] Hardware name: Allwinner sun8i Family Aug 11 04:05:50 kernel: [11326.227801] [] (unwind_backtrace) from [] (show_stack+0x11/0x14) Aug 11 04:05:50 kernel: [11326.227811] [] (show_stack) from [] (dump_stack+0x69/0x78) Aug 11 04:05:50 kernel: [11326.227822] [] (dump_stack) from [] (nmi_cpu_backtrace+0x8b/0xd4) Aug 11 04:05:50 kernel: [11326.227831] [] (nmi_cpu_backtrace) from [] (handle_IPI+0x75/0x278) Aug 11 04:05:50 kernel: [11326.227840] [] (handle_IPI) from [] (gic_handle_irq+0x67/0x68) Aug 11 04:05:50 kernel: [11326.227847] [] (gic_handle_irq) from [] (__irq_svc+0x65/0x94) Aug 11 04:05:50 kernel: [11326.227851] Exception stack(0xdf51fe30 to 0xdf51fe78) Aug 11 04:05:50 kernel: [11326.227856] fe20: dfb33440 c0604f21 df76f808 000003df Aug 11 04:05:50 kernel: [11326.227863] fe40: df76f8a0 dfb33440 df51fea4 c0d03f6c ffffe000 c0dd3430 1ee84000 4000001f Aug 25 18:28:35 kernel: [11326.227869] fe60: df51fea8 df51fe80 c0168ca7 c0168b52 000e0133 ffffffff Aug 25 18:28:35 kernel: [11326.227877] [] (__irq_svc) from [] (expire_timers+0x8e/0xe0) Aug 25 18:28:35 kernel: [11326.227887] [] (expire_timers) from [] (run_timer_softirq+0x103/0x138) Aug 25 18:28:35 kernel: [11326.227899] [] (run_timer_softirq) from [] (__do_softirq+0xb9/0x25c) Aug 25 18:28:35 kernel: [11326.227908] [] (__do_softirq) from [] (irq_exit+0x93/0xe4) Aug 25 18:28:35 kernel: [11326.227919] [] (irq_exit) from [] (__handle_domain_irq+0x49/0x84) Aug 25 18:28:35 kernel: [11326.227928] [] (__handle_domain_irq) from [] (gic_handle_irq+0x39/0x68) Aug 25 18:28:35 kernel: [11326.227934] [] (gic_handle_irq) from [] (__irq_svc+0x65/0x94) Aug 25 18:28:35 kernel: [11326.227937] Exception stack(0xdf51ff78 to 0xdf51ffc0) Aug 25 18:28:35 kernel: [11326.227940] ff60: 00000001 00000000 Aug 25 18:28:35 kernel: [11326.227947] ff80: 00000000 c01165e1 ffffe000 c0d03fcc c0d03f6c c0cb33b8 c0dd2c3f 00000000 Aug 25 18:28:35 kernel: [11326.227953] ffa0: 00000000 00000000 01400000 df51ffc8 c0107087 c0107088 400e0033 ffffffff Aug 25 18:28:35 kernel: [11326.227962] [] (__irq_svc) from [] (arch_cpu_idle+0x28/0x2c) Aug 25 18:28:35 kernel: [11326.227973] [] (arch_cpu_idle) from [] (do_idle+0x115/0x16c) Aug 25 18:28:35 kernel: [11326.227982] [] (do_idle) from [] (cpu_startup_entry+0x19/0x1c) Aug 25 18:28:35 kernel: [11326.227990] [] (cpu_startup_entry) from [<40101491>] (0x40101491) Aug 25 18:28:35 kernel: [11326.228712] rcu_sched kthread starved for 41232 jiffies! g193459 c193458 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x0 Aug 25 18:28:35 kernel: [11326.228717] rcu_sched R running task 0 8 2 0x00000000 Aug 25 18:28:35 kernel: [11326.228736] [] (__schedule) from [] (schedule+0x2f/0x68) Aug 25 18:28:35 kernel: [11326.228746] [] (schedule) from [] (schedule_timeout+0x75/0x2ec) Aug 25 18:28:35 kernel: [11326.228757] [] (schedule_timeout) from [] (rcu_gp_kthread+0x425/0x690) Aug 25 18:28:35 kernel: [11326.228769] [] (rcu_gp_kthread) from [] (kthread+0xfd/0x104) Aug 25 18:28:35 kernel: [11326.228780] [] (kthread) from [] (ret_from_fork+0x11/0x20) Aug 25 18:32:47 kernel: [12265.081326] hrtimer: interrupt took 8625 ns Without a reason, it freezes, but: 1. A web server that runs on this OPI is still somehow functional for about several hours. I have a root shell via this web so I can run some commands and get results. 2. System time changed to Apr 25 1979 . This also noticable from the log above. 3. systemctl is not working like I can't get a list of units. It just does not produce any output. The kernel is 4.13.15-sunxi #1 SMP Tue Nov 21 23:35:46 MSK 2017 armv7l armv7l armv7l GNU/Linux. At the same time /var/log/syslog has other errors like that: Aug 11 04:07:56 systemd[1]: systemd-logind.service: Watchdog timeout (limit 3min)! Aug 11 04:07:56 systemd[1]: systemd-logind.service: Main process exited, code=killed, status=6/ABRT Aug 11 04:07:56 systemd[1]: systemd-logind.service: Unit entered failed state. Aug 11 04:07:56 systemd[1]: systemd-logind.service: Failed with result 'signal'. Aug 11 04:07:56 systemd[1]: systemd-logind.service: Service has no hold-off time, scheduling restart. Aug 11 04:07:56 systemd[1]: Stopped Login Service. Aug 25 18:28:35 wpa_supplicant[2055]: wlx74da38f15c54: CTRL-EVENT-SCAN-FAILED ret=-16 retry=1 Aug 25 18:28:35 systemd[1]: Starting Login Service... Aug 25 18:28:36 systemd[1]: systemd-logind.service: Main process exited, code=exited, status=1/FAILURE Aug 25 18:28:36 systemd[1]: Failed to start Login Service. Aug 25 18:28:36 systemd[1]: systemd-logind.service: Unit entered failed state. Aug 25 18:28:36 systemd[1]: systemd-logind.service: Failed with result 'exit-code'. Aug 25 18:28:36 systemd[1]: Time has been changed Aug 25 18:32:51 wpa_supplicant[2055]: message repeated 3 times: [ wlx74da38f15c54: CTRL-EVENT-SCAN-FAILED ret=-16 retry=1] Aug 25 18:33:11 systemd-udevd[286]: seq 3700 '/devices/platform/soc/1c20800.pinctrl/gpiochip0/gpio/gpio20' killed Aug 25 18:33:11 systemd-udevd[286]: seq 3700 '/devices/platform/soc/1c20800.pinctrl/gpiochip0/gpio/gpio20' is taking a long time Aug 25 18:33:11 systemd-udevd[286]: worker [5282] terminated by signal 9 (Killed) Aug 25 18:33:11 systemd-udevd[286]: worker [5282] failed while handling '/devices/platform/soc/1c20800.pinctrl/gpiochip0/gpio/gpio20' It is weird because this OrangePI was running fine for about half a year or year.
xwiggen Posted August 12, 2020 Posted August 12, 2020 21 hours ago, Mikhail Kulinich said: I notice similar freezes on some of my OrangePI One boards: Aug 11 04:01:10 kernel: [11045.901932] asix 3-1.4.1:1.0 asix: link up, 100Mbps, full-duplex, lpa 0x41E1 Aug 11 04:05:50 kernel: [11326.227650] INFO: rcu_sched detected stalls on CPUs/tasks: Aug 11 04:05:50 kernel: [11326.227679] 3-...: (1 ticks this GP) idle=992/1/0 softirq=363926/363926 fqs=1 Aug 11 04:05:50 kernel: [11326.227681] (detected by 2, t=41234 jiffies, g=193459, c=193458, q=5) Aug 11 04:05:50 kernel: [11326.227697] Sending NMI from CPU 2 to CPUs 3: Aug 11 04:05:50 kernel: [11326.227720] NMI backtrace for cpu 3 Aug 11 04:05:50 kernel: [11326.227728] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G W O 4.13.15-sunxi #1 Aug 11 04:05:50 kernel: [11326.227730] Hardware name: Allwinner sun8i Family Aug 11 04:05:50 kernel: [11326.227734] task: df4f2f40 task.stack: df51e000 Aug 11 04:05:50 kernel: [11326.227747] PC is at expire_timers+0x8e/0xe0 Aug 11 04:05:50 kernel: [11326.227752] LR is at run_timer_softirq+0x103/0x138 Aug 11 04:05:50 kernel: [11326.227755] pc : [] lr : [] psr: 000e0133 Aug 11 04:05:50 kernel: [11326.227758] sp : df51fe80 ip : df51fea8 fp : 4000001f Aug 11 04:05:50 kernel: [11326.227761] r10: 1ee84000 r9 : c0dd3430 r8 : ffffe000 Aug 11 04:05:50 kernel: [11326.227763] r7 : c0d03f6c r6 : df51fea4 r5 : dfb33440 r4 : df76f8a0 Aug 11 04:05:50 kernel: [11326.227767] r3 : 000003df r2 : df76f808 r1 : c0604f21 r0 : dfb33440 Aug 11 04:05:50 kernel: [11326.227771] Flags: nzcv IRQs on FIQs on Mode SVC_32 ISA Thumb Segment none Aug 11 04:05:50 kernel: [11326.227775] Control: 50c5387d Table: 4a5bc06a DAC: 00000051 Aug 11 04:05:50 kernel: [11326.227779] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G W O 4.13.15-sunxi #1 Aug 11 04:05:50 kernel: [11326.227781] Hardware name: Allwinner sun8i Family Aug 11 04:05:50 kernel: [11326.227801] [] (unwind_backtrace) from [] (show_stack+0x11/0x14) Aug 11 04:05:50 kernel: [11326.227811] [] (show_stack) from [] (dump_stack+0x69/0x78) Aug 11 04:05:50 kernel: [11326.227822] [] (dump_stack) from [] (nmi_cpu_backtrace+0x8b/0xd4) Aug 11 04:05:50 kernel: [11326.227831] [] (nmi_cpu_backtrace) from [] (handle_IPI+0x75/0x278) Aug 11 04:05:50 kernel: [11326.227840] [] (handle_IPI) from [] (gic_handle_irq+0x67/0x68) Aug 11 04:05:50 kernel: [11326.227847] [] (gic_handle_irq) from [] (__irq_svc+0x65/0x94) Aug 11 04:05:50 kernel: [11326.227851] Exception stack(0xdf51fe30 to 0xdf51fe78) Aug 11 04:05:50 kernel: [11326.227856] fe20: dfb33440 c0604f21 df76f808 000003df Aug 11 04:05:50 kernel: [11326.227863] fe40: df76f8a0 dfb33440 df51fea4 c0d03f6c ffffe000 c0dd3430 1ee84000 4000001f Aug 25 18:28:35 kernel: [11326.227869] fe60: df51fea8 df51fe80 c0168ca7 c0168b52 000e0133 ffffffff Aug 25 18:28:35 kernel: [11326.227877] [] (__irq_svc) from [] (expire_timers+0x8e/0xe0) Aug 25 18:28:35 kernel: [11326.227887] [] (expire_timers) from [] (run_timer_softirq+0x103/0x138) Aug 25 18:28:35 kernel: [11326.227899] [] (run_timer_softirq) from [] (__do_softirq+0xb9/0x25c) Aug 25 18:28:35 kernel: [11326.227908] [] (__do_softirq) from [] (irq_exit+0x93/0xe4) Aug 25 18:28:35 kernel: [11326.227919] [] (irq_exit) from [] (__handle_domain_irq+0x49/0x84) Aug 25 18:28:35 kernel: [11326.227928] [] (__handle_domain_irq) from [] (gic_handle_irq+0x39/0x68) Aug 25 18:28:35 kernel: [11326.227934] [] (gic_handle_irq) from [] (__irq_svc+0x65/0x94) Aug 25 18:28:35 kernel: [11326.227937] Exception stack(0xdf51ff78 to 0xdf51ffc0) Aug 25 18:28:35 kernel: [11326.227940] ff60: 00000001 00000000 Aug 25 18:28:35 kernel: [11326.227947] ff80: 00000000 c01165e1 ffffe000 c0d03fcc c0d03f6c c0cb33b8 c0dd2c3f 00000000 Aug 25 18:28:35 kernel: [11326.227953] ffa0: 00000000 00000000 01400000 df51ffc8 c0107087 c0107088 400e0033 ffffffff Aug 25 18:28:35 kernel: [11326.227962] [] (__irq_svc) from [] (arch_cpu_idle+0x28/0x2c) Aug 25 18:28:35 kernel: [11326.227973] [] (arch_cpu_idle) from [] (do_idle+0x115/0x16c) Aug 25 18:28:35 kernel: [11326.227982] [] (do_idle) from [] (cpu_startup_entry+0x19/0x1c) Aug 25 18:28:35 kernel: [11326.227990] [] (cpu_startup_entry) from [<40101491>] (0x40101491) Aug 25 18:28:35 kernel: [11326.228712] rcu_sched kthread starved for 41232 jiffies! g193459 c193458 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x0 Aug 25 18:28:35 kernel: [11326.228717] rcu_sched R running task 0 8 2 0x00000000 Aug 25 18:28:35 kernel: [11326.228736] [] (__schedule) from [] (schedule+0x2f/0x68) Aug 25 18:28:35 kernel: [11326.228746] [] (schedule) from [] (schedule_timeout+0x75/0x2ec) Aug 25 18:28:35 kernel: [11326.228757] [] (schedule_timeout) from [] (rcu_gp_kthread+0x425/0x690) Aug 25 18:28:35 kernel: [11326.228769] [] (rcu_gp_kthread) from [] (kthread+0xfd/0x104) Aug 25 18:28:35 kernel: [11326.228780] [] (kthread) from [] (ret_from_fork+0x11/0x20) Aug 25 18:32:47 kernel: [12265.081326] hrtimer: interrupt took 8625 ns Without a reason, it freezes, but: 1. A web server that runs on this OPI is still somehow functional for about several hours. I have a root shell via this web so I can run some commands and get results. 2. System time changed to Apr 25 1979 . This also noticable from the log above. 3. systemctl is not working like I can't get a list of units. It just does not produce any output. The kernel is 4.13.15-sunxi #1 SMP Tue Nov 21 23:35:46 MSK 2017 armv7l armv7l armv7l GNU/Linux. At the same time /var/log/syslog has other errors like that: Aug 11 04:07:56 systemd[1]: systemd-logind.service: Watchdog timeout (limit 3min)! Aug 11 04:07:56 systemd[1]: systemd-logind.service: Main process exited, code=killed, status=6/ABRT Aug 11 04:07:56 systemd[1]: systemd-logind.service: Unit entered failed state. Aug 11 04:07:56 systemd[1]: systemd-logind.service: Failed with result 'signal'. Aug 11 04:07:56 systemd[1]: systemd-logind.service: Service has no hold-off time, scheduling restart. Aug 11 04:07:56 systemd[1]: Stopped Login Service. Aug 25 18:28:35 wpa_supplicant[2055]: wlx74da38f15c54: CTRL-EVENT-SCAN-FAILED ret=-16 retry=1 Aug 25 18:28:35 systemd[1]: Starting Login Service... Aug 25 18:28:36 systemd[1]: systemd-logind.service: Main process exited, code=exited, status=1/FAILURE Aug 25 18:28:36 systemd[1]: Failed to start Login Service. Aug 25 18:28:36 systemd[1]: systemd-logind.service: Unit entered failed state. Aug 25 18:28:36 systemd[1]: systemd-logind.service: Failed with result 'exit-code'. Aug 25 18:28:36 systemd[1]: Time has been changed Aug 25 18:32:51 wpa_supplicant[2055]: message repeated 3 times: [ wlx74da38f15c54: CTRL-EVENT-SCAN-FAILED ret=-16 retry=1] Aug 25 18:33:11 systemd-udevd[286]: seq 3700 '/devices/platform/soc/1c20800.pinctrl/gpiochip0/gpio/gpio20' killed Aug 25 18:33:11 systemd-udevd[286]: seq 3700 '/devices/platform/soc/1c20800.pinctrl/gpiochip0/gpio/gpio20' is taking a long time Aug 25 18:33:11 systemd-udevd[286]: worker [5282] terminated by signal 9 (Killed) Aug 25 18:33:11 systemd-udevd[286]: worker [5282] failed while handling '/devices/platform/soc/1c20800.pinctrl/gpiochip0/gpio/gpio20' It is weird because this OrangePI was running fine for about half a year or year. use armbianmonitor -u, most likely rogue process eating all memory
Recommended Posts