rcu_sched detected stalls on CPU

[Monsoft]

Hi,

Recently I reinstalled system on my Odroid C1+ from older Ubuntu with original 3.10.107 to newest Armbian with kernel 6.12.44. 
Generally it works quite well  but I see a lots of stalls detection warnings in logs:

 

Thu Oct  9 02:00:26 2025] rcu: INFO: rcu_sched self-detected stall on CPU
[Thu Oct  9 02:00:26 2025] rcu: 	0-....: (1799 ticks this GP) idle=a184/1/0x40000004 softirq=1089909/1089910 fqs=488
[Thu Oct  9 02:00:26 2025] rcu: 	(t=2100 jiffies g=2037329 q=10 ncpus=4)
[Thu Oct  9 02:00:26 2025] CPU: 0 UID: 0 PID: 7987 Comm: kworker/0:1 Not tainted 6.12.44-current-meson #2
[Thu Oct  9 02:00:26 2025] Hardware name: Amlogic Meson platform
[Thu Oct  9 02:00:26 2025] Workqueue: events dbs_work_handler
[Thu Oct  9 02:00:26 2025] PC is at __slab_free+0x0/0x314
[Thu Oct  9 02:00:26 2025] LR is at kmem_cache_free+0x120/0x378
[Thu Oct  9 02:00:26 2025] pc : [<c056fd04>]    lr : [<c05716fc>]    psr: 800f0113
[Thu Oct  9 02:00:26 2025] sp : f0801b58  ip : ef6a6398  fp : c2387fc0
[Thu Oct  9 02:00:26 2025] r10: c5844380  r9 : c2352740  r8 : 000e6e80
[Thu Oct  9 02:00:26 2025] r7 : c11dd1d4  r6 : c66a0000  r5 : ef7dfe80  r4 : c280ef80
[Thu Oct  9 02:00:26 2025] r3 : c66a0000  r2 : c66a0000  r1 : ef7dfe80  r0 : c280ef80
[Thu Oct  9 02:00:26 2025] Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[Thu Oct  9 02:00:26 2025] Control: 10c5387d  Table: 062a404a  DAC: 00000051
[Thu Oct  9 02:00:26 2025] Call trace: 
[Thu Oct  9 02:00:26 2025]  __slab_free from kmem_cache_free+0x120/0x378
[Thu Oct  9 02:00:26 2025]  kmem_cache_free from tcp_rcv_established+0x428/0x6cc
[Thu Oct  9 02:00:26 2025]  tcp_rcv_established from tcp_v4_do_rcv+0x180/0x284
[Thu Oct  9 02:00:26 2025]  tcp_v4_do_rcv from tcp_v4_rcv+0xbc0/0x1068
[Thu Oct  9 02:00:26 2025]  tcp_v4_rcv from ip_protocol_deliver_rcu+0x30/0x27c
[Thu Oct  9 02:00:26 2025]  ip_protocol_deliver_rcu from ip_local_deliver_finish+0x7c/0x98
[Thu Oct  9 02:00:26 2025]  ip_local_deliver_finish from ip_sublist_rcv_finish+0x3c/0x50
[Thu Oct  9 02:00:26 2025]  ip_sublist_rcv_finish from ip_sublist_rcv+0x174/0x1e4
[Thu Oct  9 02:00:26 2025]  ip_sublist_rcv from ip_list_rcv+0xe8/0x114
[Thu Oct  9 02:00:26 2025]  ip_list_rcv from __netif_receive_skb_list_core+0x188/0x1f4
[Thu Oct  9 02:00:26 2025]  __netif_receive_skb_list_core from netif_receive_skb_list_internal+0x180/0x288
[Thu Oct  9 02:00:26 2025]  netif_receive_skb_list_internal from napi_gro_receive+0x15c/0x264
[Thu Oct  9 02:00:26 2025]  napi_gro_receive from stmmac_napi_poll_rx+0x9f8/0xfa0
[Thu Oct  9 02:00:26 2025]  stmmac_napi_poll_rx from __napi_poll.constprop.0+0x2c/0x1c8
[Thu Oct  9 02:00:26 2025]  __napi_poll.constprop.0 from net_rx_action+0x328/0x3a4
[Thu Oct  9 02:00:26 2025]  net_rx_action from handle_softirqs+0x114/0x338
[Thu Oct  9 02:00:26 2025]  handle_softirqs from irq_exit+0x8c/0xbc
[Thu Oct  9 02:00:26 2025]  irq_exit from call_with_stack+0x18/0x20
[Thu Oct  9 02:00:26 2025]  call_with_stack from __irq_svc+0x98/0xb0
[Thu Oct  9 02:00:26 2025] Exception stack(0xf1851c60 to 0xf1851ca8)
[Thu Oct  9 02:00:26 2025] 1c60: c28f4d30 00000000 00000000 00000000 c28f4d80 c282bd80 c282b480 016e3600
[Thu Oct  9 02:00:26 2025] 1c80: 00000000 c28f4240 00000001 f1851d40 800f0013 f1851cb0 c0dec028 c0dec02c
[Thu Oct  9 02:00:26 2025] 1ca0: 600f0013 ffffffff
[Thu Oct  9 02:00:26 2025]  __irq_svc from __clk_recalc_accuracies+0x54/0x68
[Thu Oct  9 02:00:26 2025]  __clk_recalc_accuracies from clk_core_set_parent_nolock+0x204/0x274
[Thu Oct  9 02:00:26 2025]  clk_core_set_parent_nolock from meson8b_cpu_clk_notifier_cb+0x78/0x8c
[Thu Oct  9 02:00:26 2025]  meson8b_cpu_clk_notifier_cb from notifier_call_chain+0x8c/0x108
[Thu Oct  9 02:00:26 2025]  notifier_call_chain from srcu_notifier_call_chain+0x40/0x98
[Thu Oct  9 02:00:26 2025]  srcu_notifier_call_chain from __clk_notify+0x80/0xac
[Thu Oct  9 02:00:26 2025]  __clk_notify from clk_propagate_rate_change+0x98/0xb0
[Thu Oct  9 02:00:26 2025]  clk_propagate_rate_change from clk_propagate_rate_change+0x5c/0xb0
[Thu Oct  9 02:00:26 2025]  clk_propagate_rate_change from clk_propagate_rate_change+0x5c/0xb0
[Thu Oct  9 02:00:26 2025]  clk_propagate_rate_change from clk_propagate_rate_change+0x5c/0xb0
[Thu Oct  9 02:00:26 2025]  clk_propagate_rate_change from clk_core_set_rate_nolock+0xf4/0x31c
[Thu Oct  9 02:00:26 2025]  clk_core_set_rate_nolock from clk_set_rate+0x30/0x154
[Thu Oct  9 02:00:26 2025]  clk_set_rate from _opp_config_clk_single+0x24/0x84
[Thu Oct  9 02:00:26 2025]  _opp_config_clk_single from _set_opp+0x10c/0x41c
[Thu Oct  9 02:00:26 2025]  _set_opp from dev_pm_opp_set_rate+0x124/0x230
[Thu Oct  9 02:00:26 2025]  dev_pm_opp_set_rate from __cpufreq_driver_target+0x394/0x6ac
[Thu Oct  9 02:00:26 2025]  __cpufreq_driver_target from od_dbs_update+0x148/0x174
[Thu Oct  9 02:00:26 2025]  od_dbs_update from dbs_work_handler+0x2c/0x58
[Thu Oct  9 02:00:26 2025]  dbs_work_handler from process_one_work+0x168/0x3b8
[Thu Oct  9 02:00:26 2025]  process_one_work from worker_thread+0x264/0x42c
[Thu Oct  9 02:00:26 2025]  worker_thread from kthread+0xe0/0xfc
[Thu Oct  9 02:00:26 2025]  kthread from ret_from_fork+0x14/0x28
[Thu Oct  9 02:00:26 2025] Exception stack(0xf1851fb0 to 0xf1851ff8)
[Thu Oct  9 02:00:26 2025] 1fa0:                                     00000000 00000000 00000000 00000000
[Thu Oct  9 02:00:26 2025] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[Thu Oct  9 02:00:26 2025] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000

 

Is there a way to fix it ?
 

[torz77]

Assuming that you installed the standard image, then I suspect this is the same bug that I identified here:

I submitted a PR to patch this, which has been accepted into Armbian, but it won't have flowed through into the releases yet, so you will need to build the image (instructions also linked in the subsequent post)

[Monsoft]

Thanks for answer.

Yep, I installed standard image. 

Since I was using two storage devices (SD and MMC), I first noticed issues with the mounted file system on the SD card, which I eventually had to remove (only the MMC storage remained). Then I noticed errors causing the processor to freeze, which got me a little worried.

It was long time ago when last time I was compiling Linux kernel (FreeBSD more recently) , but maybe I will try to follow your instructions and try to do it.

Many Thanks

[torz77]

I didn't test eMMC, whilst the bug occurred (it definitely impacted the SD card), but I did try using a USB drive and I can confirm (for me at least) the issue went away (the issue impacts the MMC stack, specifically), so if you have a spare USB drive hanging around, and you want an easier interim "fix" you can try putting the rootfs on a USB drive in the meantime.

 

To be clear, the bug still did occur, but as it only impacted the SD card (which was for boot only), it only spammed my log files, but did not impact my day to day running (which with rootfs on SD, it definitely did. Made the device completely unusable pending a hard reset).

 

I'll put the build instructions here too, because it's very simple (This was using a Debian Bookworm/Trixie host):
 

sudo apt install git

git clone --depth=1 https://github.com/armbian/build.git

cd build

./compile.sh \
BOARD=odroidc1 \
BRANCH=current \
RELEASE=trixie \
BUILD_MINIMAL=yes \
BUILD_DESKTOP=no \
CLEAN_LEVEL= \
PREFER_DOCKER=no \
KERNEL_CONFIGURE=no \
COMPRESS_OUTPUTIMAGE=sha,gpg,img

 

And that's it. Image is saved in:

 

./output/images