Flole

There is an issue in recent u-boot versions that causes this behaviour. The commit that introduced it is a9bf024b2933bba0e23038892970a18b72dfaeb4. A possible solution is to disable all the EFI-stuff in u-boot, then it boots perfectly fine in recent u-boot versions aswell. So if armbian wants to update u-boot for this board, keep this in mind. I have reported this to u-boot aswell so hopefully it will be fixed at some point there.

As I've patched it for now I can wait until the current dev (5.13) becomes the new stable. Any future update of the kernel (and it's modules) should also not have this patch, so I should not need my patch ever again. Let's wait and see then.

It is in build/patch/kernel/archive/sunxi-5.4/0143-Bluetooth-btrtl-add-support-for-the-RTL8723CS.patch, don't ask me why it's applied to 5.10 though (or maybe that's how this build system works, no clue). It's not there upstream in 5.10 but the disassembler clearly tells me that it's applied to the module in the 5.10 module directory (and as it is, or at least was before I patched it, signed it's obviously built by something that has the signing keys).

Patched the binary in the meantime, issue is solved for me. Thanks. As I still don't know where that patch is coming from and where a PR has to go that's it for me. I just know who was the one who pulled it in (first as .disabled) and then enabled it later on. Don't tell me things about testing and so on when you are mass pulling in patches from somewhere, obviously without testing them or even looking at what they do and then introduce bugs due to that.... You even wrote "It's a mess which will be eventually sorted out or merged up.", yeah good you merged a mess so don't expect me to clean up your mess. Just throwing in some patches, breaking things for me and then expecting me to fix that stuff is not going to work, maybe you have more sucess with others though. By the way: Upstream it is done properly since a few months, so the mess you pulled in will be thrown out again eventually (hopefully).

Not entirely sure why you responded (or even are on this forum) if all you want to say is "find the issue and solution yourself" without providing any real help. Especially if your time is so valuable to you. Instead of writing your post you could've easily fixed this issue in the same time. The bug is not there upstream and gets added by a patch that can be found in the Armbian repository, that's why I asked here and not on an upstream mailing list, hoping to get it fixed easily. But if you want to keep buggy patches in your repository and ruin working software in doing so that's up to you and fine with me, I'll simply patch the binary then, easy fix to kick out that comparison again in the assembly. I just thought that it might be a good idea to have others benefit from this aswell but apparently that's just too complicated.

When I reboot my Orange Pi PC 2 with my Realtek USB Dongle so the firmware is already loaded onto the Dongle I am seeing this: [ 31.659072] Bluetooth: hci0: RTL: examining hci_ver=0a hci_rev=097b lmp_ver=0a lmp_subver=ec43 [ 31.659815] Bluetooth: hci0: RTL: unknown IC info, lmp subver ec43, hci rev 097b, hci ver 000a [ 31.659830] Bluetooth: hci0: RTL: assuming no firmware upload needed [ 31.659852] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000002 [ 31.668671] Mem abort info: [ 31.671488] ESR = 0x96000004 [ 31.674561] EC = 0x25: DABT (current EL), IL = 32 bits [ 31.679880] SET = 0, FnV = 0 [ 31.682937] EA = 0, S1PTW = 0 [ 31.686077] Data abort info: [ 31.688951] ISV = 0, ISS = 0x00000004 [ 31.692783] CM = 0, WnR = 0 [ 31.695752] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000043305000 [ 31.702186] [0000000000000002] pgd=0000000000000000, p4d=0000000000000000 [ 31.708978] Internal error: Oops: 96000004 [#1] SMP [ 31.713848] Modules linked in: btusb btrtl btbcm btintel sun4i_i2s bluetooth sun4i_gpadc_iio industrialio ecdh_generic rfkill ecc snd_rawmidi uvcvideo videobuf2_vmalloc snd_seq_device sunxi_cedrus(C) videobuf2_dma_contig v4l2_mem2mem videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc snd_soc_simple_card snd_soc_simple_card_utils display_connector cpufreq_dt zram sch_fq_codel nfsv3 nfs fscache realtek sy8106a_regulator i2c_mv64xxx dwmac_sun8i mdio_mux [ 31.754604] CPU: 0 PID: 313 Comm: kworker/u9:2 Tainted: G C 5.10.34-sunxi64 #21.05.1 [ 31.763633] Hardware name: Xunlong Orange Pi PC 2 (DT) [ 31.768813] Workqueue: hci0 hci_power_on [bluetooth] [ 31.773775] pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--) [ 31.779776] pc : btrtl_setup_realtek+0x64/0xe10 [btrtl] [ 31.784993] lr : btrtl_setup_realtek+0x54/0xe10 [btrtl] [ 31.790207] sp : ffff800011843c90 [ 31.793515] x29: ffff800011843c90 x28: 0000000000000000 [ 31.798820] x27: ffff000005782548 x26: ffff000005664100 [ 31.804124] x25: 0000000000000000 x24: ffff0000054d4698 [ 31.809429] x23: ffff0000054d4a38 x22: ffff0000054d4038 [ 31.814732] x21: 0000000000000000 x20: ffff0000054d4000 [ 31.820037] x19: ffff0000025ea880 x18: ffff8000114d17d0 [ 31.825341] x17: 0000000000000000 x16: 0000000000000000 [ 31.830646] x15: 0000000000000223 x14: ffff800011843870 [ 31.835950] x13: 00000000ffffffea x12: ffff000037d707a8 [ 31.841254] x11: 0000000000000003 x10: ffff000037d6a768 [ 31.846557] x9 : ffff000037d6a7c0 x8 : 0000000000005fe8 [ 31.851861] x7 : c0000000fffffbff x6 : 0000000000000001 [ 31.857165] x5 : 0000000000015fa8 x4 : 0000000000000000 [ 31.862468] x3 : 0000000000000000 x2 : 58ab9a179ad83900 [ 31.867772] x1 : 0000000000000000 x0 : 0000000000008703 [ 31.873078] Call trace: [ 31.875525] btrtl_setup_realtek+0x64/0xe10 [btrtl] [ 31.880425] hci_dev_do_open+0x298/0x700 [bluetooth] [ 31.885400] hci_power_on+0x54/0x2c0 [bluetooth] [ 31.890016] process_one_work+0x1f0/0x3c0 [ 31.894019] worker_thread+0x140/0x520 [ 31.897762] kthread+0x120/0x128 [ 31.900986] ret_from_fork+0x10/0x30 [ 31.904563] Code: 2a0003f5 350000c0 f9400261 5290e060 (79400421) [ 31.910650] ---[ end trace a2a6ba9ede4ddc6e ]--- The reason becomes obvious when looking at the relevant assembly and the cause seems to be this patch: 0143-Bluetooth-btrtl-add-support-for-the-RTL8723CS.patch It adds an additional btrtl_apply_quirks(hdev, btrtl_dev) without properly checking if btrtl_dev->ic_info is null (which it is in my case, that's why the "unknown IC info, lmp subver ec43, hci rev 097b, hci ver 000a" is printed). So either in btrtl_apply_quirks there should be an additional null check or before it is called a null-check needs to be added. What's the best and fastest way to get this fixed? As I've already found the cause it should be pretty straight forward?

It does not work in the legacy branch, it just used to work on 4.13.x before I upgraded to focal (I think I was on bionic before but I need to check that again). I tried the legacy branch as I was hoping that it could be better, but unfortunately it isn't.

Sure, I had to remove some sensitive information like IPv6 Addresses and NFS Paths though.

This is what happened right before a crash, looks like it was adjusting the frequency quite often at that time. I would assume that many frequency adjustments are the cause of it and also that is the only thing I changed and since then it's running super stable. Not even my selfmade power supply that provides 5.25V 3A through the dc jack made a change, I still had the same crashes while the voltage that comes out of the USB ports is super stable (checked with an oscilloscope). Only after switching to the performance governor it started to run stable (at least until now, but's thats already longer than usual).

I think I found the reason for my crashes: I used the ondemand power governor and apparently that caused the issues. Maybe the voltage regulator was set to an invalid level or something. Anyways, since I disabled it and used performance I now have had no crashes, that's an uptime of 8 days now. My absolute record was 14 days so I guess the next weeks will show if that was really the issue or not.

I am running Armbian Focal with Linux 5.4.43-sunxi64 (and now downgraded to 4.19.125-sunxi64) on my Orange Pi PC 2. I use motion software to provide a "18ec:5555 Arkmicro Technologies Inc. USB2.0 PC CAMERA" to the network. I also use pjsip to provide SIP functionality for the same device as audio input. As soon as I start pjsip for some reason I get a very high load average until the kernel detectets stalled CPUs and finally the entire device crashes. This worked fine in a previous version and started after I upgraded to focal. I did a kernel backtrace and this is what I got: [ 401.002517] sysrq: Show backtrace of all active CPUs [ 401.009965] sysrq: CPU1: [ 401.010084] Call trace: [ 401.010394] dump_backtrace+0x0/0x180 [ 401.010573] show_stack+0x14/0x20 [ 401.010811] showacpu+0x94/0xc8 [ 401.011039] flush_smp_call_function_queue+0x90/0x150 [ 401.011247] generic_smp_call_function_single_interrupt+0x10/0x18 [ 401.011471] handle_IPI+0xf8/0x180 [ 401.011634] gic_handle_irq+0x9c/0xa0 [ 401.011806] el0_irq_naked+0x4c/0x54 [ 401.011993] sysrq: CPU3: [ 401.012118] Call trace: [ 401.012372] dump_backtrace+0x0/0x180 [ 401.012533] show_stack+0x14/0x20 [ 401.012743] showacpu+0x94/0xc8 [ 401.012939] flush_smp_call_function_queue+0x90/0x150 [ 401.013137] generic_smp_call_function_single_interrupt+0x10/0x18 [ 401.013346] handle_IPI+0xf8/0x180 [ 401.013506] gic_handle_irq+0x9c/0xa0 [ 401.013675] el0_irq_naked+0x4c/0x54 [ 401.013834] sysrq: CPU0: [ 401.013941] Call trace: [ 401.014159] dump_backtrace+0x0/0x180 [ 401.014342] show_stack+0x14/0x20 [ 401.014548] showacpu+0x94/0xc8 [ 401.014937] flush_smp_call_function_queue+0x90/0x150 [ 401.015139] generic_smp_call_function_single_interrupt+0x10/0x18 [ 401.015345] handle_IPI+0xf8/0x180 [ 401.015501] gic_handle_irq+0x9c/0xa0 [ 401.015657] el1_irq+0xb8/0x140 [ 401.015915] arch_counter_get_cntpct+0x8/0x18 [ 401.016143] ktime_get+0x40/0xa0 [ 401.016633] uvc_video_decode_start+0x6d8/0x7e8 [uvcvideo] [ 401.017006] uvc_video_decode_isoc+0xac/0x168 [uvcvideo] [ 401.017359] uvc_video_complete+0x11c/0x1c8 [uvcvideo] [ 401.017613] __usb_hcd_giveback_urb+0x70/0x130 [ 401.017821] usb_giveback_urb_bh+0xf0/0x190 [ 401.018030] tasklet_action_common.isra.19+0xc4/0x188 [ 401.018182] tasklet_hi_action+0x24/0x30 [ 401.018349] __do_softirq+0x11c/0x230 [ 401.018497] irq_exit+0x9c/0xb8 [ 401.018750] __handle_domain_irq+0x64/0xb8 [ 401.018904] gic_handle_irq+0x50/0xa0 [ 401.019058] el1_irq+0xb8/0x140 [ 401.019279] clk_core_unprepare+0x0/0x100 [ 401.019529] clk_core_set_rate_nolock+0x1bc/0x1e8 [ 401.019725] clk_set_rate+0x34/0xa0 [ 401.019962] dev_pm_opp_set_rate+0x394/0x4f0 [ 401.020230] set_target+0x3c/0x80 [cpufreq_dt] [ 401.020428] __cpufreq_driver_target+0x2bc/0x678 [ 401.020611] od_dbs_update+0xb8/0x190 [ 401.020801] dbs_work_handler+0x3c/0x70 [ 401.021027] process_one_work+0x1ec/0x370 [ 401.021218] worker_thread+0x4c/0x4f8 [ 401.021401] kthread+0x120/0x128 [ 401.021601] ret_from_fork+0x10/0x18 [ 403.369173] sysrq: Show backtrace of all active CPUs [ 403.375878] sysrq: CPU0: [ 403.375993] Call trace: [ 403.376316] dump_backtrace+0x0/0x180 [ 403.376496] show_stack+0x14/0x20 [ 403.376721] showacpu+0x94/0xc8 [ 403.376942] flush_smp_call_function_queue+0x90/0x150 [ 403.377146] generic_smp_call_function_single_interrupt+0x10/0x18 [ 403.377354] handle_IPI+0xf8/0x180 [ 403.377667] gic_handle_irq+0x9c/0xa0 [ 403.378019] el1_irq+0xb8/0x140 [ 403.378650] uvc_video_decode_start+0x104/0x7e8 [uvcvideo] [ 403.379181] uvc_video_decode_isoc+0xac/0x168 [uvcvideo] [ 403.379899] uvc_video_complete+0x11c/0x1c8 [uvcvideo] [ 403.380312] __usb_hcd_giveback_urb+0x70/0x130 [ 403.380606] usb_giveback_urb_bh+0xf0/0x190 [ 403.380793] tasklet_action_common.isra.19+0xc4/0x188 [ 403.380945] tasklet_hi_action+0x24/0x30 [ 403.381110] __do_softirq+0x11c/0x230 [ 403.381460] irq_exit+0x9c/0xb8 [ 403.381862] __handle_domain_irq+0x64/0xb8 [ 403.382180] gic_handle_irq+0x50/0xa0 [ 403.382497] el1_irq+0xb8/0x140 [ 403.382876] tcp_sendmsg_locked+0x288/0xc90 [ 403.383202] tcp_sendmsg+0x34/0x58 [ 403.383488] inet_sendmsg+0x40/0x68 [ 403.383686] sock_sendmsg+0x44/0x50 [ 403.383869] __sys_sendto+0xcc/0x130 [ 403.384033] __arm64_sys_sendto+0x24/0x30 [ 403.384273] el0_svc_common.constprop.2+0x88/0x150 [ 403.384476] el0_svc_handler+0x20/0x80 [ 403.384636] el0_svc+0x8/0xc [ 403.384802] sysrq: CPU1: [ 403.384917] Call trace: [ 403.385150] dump_backtrace+0x0/0x180 [ 403.385323] show_stack+0x14/0x20 [ 403.385535] showacpu+0x94/0xc8 [ 403.385747] flush_smp_call_function_queue+0x90/0x150 [ 403.385950] generic_smp_call_function_single_interrupt+0x10/0x18 [ 403.386152] handle_IPI+0xf8/0x180 [ 403.386306] gic_handle_irq+0x9c/0xa0 [ 403.386477] el0_irq_naked+0x4c/0x54 [ 406.143125] sysrq: Show backtrace of all active CPUs [ 406.158279] sysrq: CPU2: [ 406.158407] Call trace: [ 406.158720] dump_backtrace+0x0/0x180 [ 406.158892] show_stack+0x14/0x20 [ 406.159119] showacpu+0x94/0xc8 [ 406.159340] flush_smp_call_function_queue+0x90/0x150 [ 406.159544] generic_smp_call_function_single_interrupt+0x10/0x18 [ 406.159763] handle_IPI+0xf8/0x180 [ 406.159931] gic_handle_irq+0x9c/0xa0 [ 406.160097] el1_irq+0xb8/0x140 [ 406.160303] __seccomp_filter+0x74/0x5e8 [ 406.160484] __secure_computing+0x38/0xc0 [ 406.160701] syscall_trace_enter+0x100/0x148 [ 406.160925] el0_svc_common.constprop.2+0x54/0x150 [ 406.161125] el0_svc_handler+0x20/0x80 [ 406.161282] el0_svc+0x8/0xc [ 406.161441] sysrq: CPU1: [ 406.161550] Call trace: [ 406.161773] dump_backtrace+0x0/0x180 [ 406.161949] show_stack+0x14/0x20 [ 406.162157] showacpu+0x94/0xc8 [ 406.162349] flush_smp_call_function_queue+0x90/0x150 [ 406.162711] generic_smp_call_function_single_interrupt+0x10/0x18 [ 406.162916] handle_IPI+0xf8/0x180 [ 406.163076] gic_handle_irq+0x9c/0xa0 [ 406.163240] el0_irq_naked+0x4c/0x54 [ 406.163399] sysrq: CPU0: [ 406.163511] Call trace: [ 406.163729] dump_backtrace+0x0/0x180 [ 406.163906] show_stack+0x14/0x20 [ 406.164110] showacpu+0x94/0xc8 [ 406.164311] flush_smp_call_function_queue+0x90/0x150 [ 406.164507] generic_smp_call_function_single_interrupt+0x10/0x18 [ 406.164710] handle_IPI+0xf8/0x180 [ 406.164871] gic_handle_irq+0x9c/0xa0 [ 406.165026] el1_irq+0xb8/0x140 [ 406.165610] uvc_video_decode_start+0x3d8/0x7e8 [uvcvideo] [ 406.166155] uvc_video_decode_isoc+0xac/0x168 [uvcvideo] [ 406.166672] uvc_video_complete+0x11c/0x1c8 [uvcvideo] [ 406.167086] __usb_hcd_giveback_urb+0x70/0x130 [ 406.167451] usb_giveback_urb_bh+0xf0/0x190 [ 406.167804] tasklet_action_common.isra.19+0xc4/0x188 [ 406.168128] tasklet_hi_action+0x24/0x30 [ 406.168458] __do_softirq+0x11c/0x230 [ 406.168691] irq_exit+0x9c/0xb8 [ 406.168922] __handle_domain_irq+0x64/0xb8 [ 406.169071] gic_handle_irq+0x50/0xa0 [ 406.169396] el1_irq+0xb8/0x140 [ 406.169755] tcp_sendmsg_locked+0x288/0xc90 [ 406.170081] tcp_sendmsg+0x34/0x58 [ 406.170441] inet_sendmsg+0x40/0x68 [ 406.170783] sock_sendmsg+0x44/0x50 [ 406.171117] __sys_sendto+0xcc/0x130 [ 406.171444] __arm64_sys_sendto+0x24/0x30 [ 406.171842] el0_svc_common.constprop.2+0x88/0x150 [ 406.172216] el0_svc_handler+0x20/0x80 [ 406.172541] el0_svc+0x8/0xc [ 408.204456] sysrq: Show backtrace of all active CPUs [ 408.214175] sysrq: CPU1: [ 408.214296] Call trace: [ 408.214624] dump_backtrace+0x0/0x180 [ 408.214813] show_stack+0x14/0x20 [ 408.215046] showacpu+0x94/0xc8 [ 408.215271] flush_smp_call_function_queue+0x90/0x150 [ 408.215477] generic_smp_call_function_single_interrupt+0x10/0x18 [ 408.215870] handle_IPI+0xf8/0x180 [ 408.216039] gic_handle_irq+0x9c/0xa0 [ 408.216216] el0_irq_naked+0x4c/0x54 [ 408.216351] sysrq: CPU2: [ 408.216453] Call trace: [ 408.216675] dump_backtrace+0x0/0x180 [ 408.216843] show_stack+0x14/0x20 [ 408.217054] showacpu+0x94/0xc8 [ 408.217250] flush_smp_call_function_queue+0x90/0x150 [ 408.217456] generic_smp_call_function_single_interrupt+0x10/0x18 [ 408.217666] handle_IPI+0xf8/0x180 [ 408.217824] gic_handle_irq+0x9c/0xa0 [ 408.217985] el1_irq+0xb8/0x140 [ 408.218219] kmsg_read+0x50/0x60 [ 408.218440] proc_reg_read+0x5c/0xc8 [ 408.218632] __vfs_read+0x18/0x40 [ 408.218802] vfs_read+0x9c/0x188 [ 408.218968] ksys_read+0x64/0xe8 [ 408.219142] __arm64_sys_read+0x18/0x20 [ 408.219375] el0_svc_common.constprop.2+0x88/0x150 [ 408.219581] el0_svc_handler+0x20/0x80 [ 408.219739] el0_svc+0x8/0xc [ 408.219893] sysrq: CPU0: [ 408.219995] Call trace: [ 408.220206] dump_backtrace+0x0/0x180 [ 408.220376] show_stack+0x14/0x20 [ 408.220588] showacpu+0x94/0xc8 [ 408.220782] flush_smp_call_function_queue+0x90/0x150 [ 408.220981] generic_smp_call_function_single_interrupt+0x10/0x18 [ 408.221184] handle_IPI+0xf8/0x180 [ 408.221336] gic_handle_irq+0x9c/0xa0 [ 408.221490] el1_irq+0xb8/0x140 [ 408.221688] arch_local_irq_restore+0x4/0x10 [ 408.221925] usb_hcd_submit_urb+0xdc/0xad0 [ 408.222133] usb_submit_urb+0x1e0/0x580 [ 408.222619] uvc_video_complete+0x188/0x1c8 [uvcvideo] [ 408.222971] __usb_hcd_giveback_urb+0x70/0x130 [ 408.223348] usb_giveback_urb_bh+0xf0/0x190 [ 408.223700] tasklet_action_common.isra.19+0xc4/0x188 [ 408.224029] tasklet_hi_action+0x24/0x30 [ 408.224350] __do_softirq+0x11c/0x230 [ 408.224589] irq_exit+0x9c/0xb8 [ 408.224822] __handle_domain_irq+0x64/0xb8 [ 408.224972] gic_handle_irq+0x50/0xa0 [ 408.225299] el1_irq+0xb8/0x140 [ 408.225673] tcp_sendmsg_locked+0x288/0xc90 [ 408.226005] tcp_sendmsg+0x34/0x58 [ 408.226364] inet_sendmsg+0x40/0x68 [ 408.226717] sock_sendmsg+0x44/0x50 [ 408.227058] __sys_sendto+0xcc/0x130 [ 408.227391] __arm64_sys_sendto+0x24/0x30 [ 408.227782] el0_svc_common.constprop.2+0x88/0x150 [ 408.228153] el0_svc_handler+0x20/0x80 [ 408.228465] el0_svc+0x8/0xc Does anybody know what might be going on here and how I can fix this issue? Or do I need to go back to Ubuntu Bionic with Armbian Linux 4.13.2-sun50iw2 which I was running before?

That indeed disabled those messages. I have a few more, are those normal and necessary aswell or can I disable something to get rid of them:

I assume there is no way to just disable the GPU completely? I don't need it at all, so if there is a way to disable it I would give that a try.

Is anybody else experiencing stability issues? I have replaced the power supply now and double-checked with an oscilloscope to make sure the voltage is stable even under load. I have stable 5.25V at the USB Ports. I am using an USB HDD as rootfs, is anybody else using this with the system running stable? There are quite some regulator entries in the dmesg, but I think those are normal? I tried to enable the watchdog but that doesn't fix the lockups. My HDDs LED goes off when that lockup happens, the Ethernet LEDs stay on but it doesn't respond anymore at all, just as if the CPU would hang. In that case the watchdog should kick in though, so I am not sure what is going on there. If anybody has an idea what's going on here I would appreciate any help.

But it is not "unsafe" to run it in this configuration, right? The regulators are all set now and there shouldn't be any need to change voltages now. Especially they shouldn't be set to an unsafe voltage, right? On the next reboot it'll become active again anyways but since last time I tried a reboot after unloading I got a complete lockup I would prefer to keep it running for 2 more days until I'm physically there again.