K-worker problem on A20 based boards

[sgei]

Hi,

 

this fix does not work for BananaPi / BananaPi Pro.

I have blacklisted the module

 

nano /etc/modprobe.d/blacklist

blacklist sun4i_drm_hdmi

 

After that:

 

update-initramfs -u

 

But after a reboot, I still have ~10% cpu-load from a kworker process.

 

 

What I get with linux-perf-5.8 is:

 

 

Any idea?

 

Steffen

[jeloneal]

I can confirm what @sgei is reporting but on my cubietruck with the latest armbian buster release. See my screenshot:

 

top shows some kworker thread "events_freezable_power_" every few seconds with 10% load. Seems to be related somehow. At least this leads to the CPU being almost continually clocking @960MHz.

Edited December 16, 2020 by jeloneal
Updated with better screenshot, now animated

[jeloneal]

Can anyone else running a cubietruck confirm this findings? Is there anything we can do to fix or mitigate this behaviour?

[Madozu]

4 hours ago, jeloneal said:

Can anyone else running a cubietruck confirm this findings? Is there anything we can do to fix or mitigate this behaviour?

I am running a headless cubietruck with Armbian standard kernel (5.10.4-sunxi as of now). System is Ubuntu Focal Fossa with "linux-focal-root-current-cubietruck 20.11.6" installed. Cubietruck has uboot and /boot  on SD-Card and the system is on a SATA-attached SSD. CPU load average over 20s (vmstat 20) is like this:
 

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 1827484  18008 137776    0    0     0     0  121   78  0  5 94  0  0
 1  0      0 1827484  18008 137772    0    0     0     0  109   70  0  5 95  0  0
 0  0      0 1827232  18008 137776    0    0     0     0   96   57  0  5 95  0  0
 1  0      0 1827232  18008 137776    0    0     0     0  102   56  0  5 95  0  0
 1  0      0 1827232  18008 137776    0    0     0     0  109   72  0  5 95  0  0
 1  0      0 1827232  18008 137776    0    0     0     0   92   49  0  5 95  0  0
 2  0      0 1827232  18008 137776    0    0     0     0   95   49  0  5 95  0  0
 1  0      0 1827232  18008 137776    0    0     0     0  112   75  0  5 95  0  0
 0  0      0 1827232  18008 137776    0    0     0     0   92   52  0  5 95  0  0

 

This is better than what I saw with older (e.g. 4.x) kernels but still around 5% CPU usage coming from "kworker/1:3-events_freezable_power_". When removing the modules for ADC (sun4i_gpad and sun4i_gpad_iio) the CPU usage drops down to <1% in average:

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 1826784  18020 137772    0    0     0     9   41   46  0  0 100  0  0
 1  0      0 1826540  18020 137772    0    0     0     0   52   60  0  0 99  0  0
 1  0      0 1826540  18020 137772    0    0     0     0   39   44  0  0 100  0  0
 1  0      0 1826540  18020 137772    0    0     0     0   39   44  0  0 100  0  0
 1  0      0 1826532  18020 137776    0    0     0     1   61   69  0  0 99  0  0
 1  0      0 1826532  18020 137776    0    0     0     0   45   48  0  0 100  0  0
 1  0      0 1826532  18020 137776    0    0     0     0   34   40  0  0 100  0  0
 1  0      0 1826532  18020 137776    0    0     0     0   62   69  0  0 99  0  0
 1  0      0 1826532  18020 137776    0    0     0     0   35   41  0  0 100  0  0
 1  0      0 1826532  18020 137776    0    0     0     0   34   40  0  0 100  0  0

 

However as stated previously, removing those kernel modules breaks the CPU temp sensor display in the OS. What I recommend to do is running this command to remove the modules contributing to the CPU consumption:

sudo rmmod sun4i_gpadc sun4i_gpadc_iio

 

If you're happy with the result, put the above statement into /etc/rc.local to run at every boot. Blacklisting the above modules in /etc/modprobe.d only partially works as module sun4i_gpadc is loaded anyway.

[jeloneal]

Thanks for this info. Do I also need to put sudo in front of the statement in rc.local?

[Madozu]

15 hours ago, jeloneal said:

Thanks for this info. Do I also need to put sudo in front of the statement in rc.local?

 

No, that's not needed as rc.local runs with root privileges. The "Sudo Prefix" however is needed when you use your normal login user id to test.

[jeloneal]

Thanks. Works as expected now.

[mikhailai]

Actually, there is a better way to eliminate this CPU load without disabling the ADC driver and losing the ability to access the SOC temperature. The Linux thermal management periodically polls the SOC temperature, and you can disable it by adding the following line to /etc/rc.local:

echo disabled > /sys/devices/virtual/thermal/thermal_zone0/mode

 

Details/background:

• The A20 ADC driver is very inefficient: it seems to be using a busy loop (as mentioned here: https://linux-sunxi.narkive.com/1UetD5rh/bug-report-kworker-issue-for-mailline-kernel-4-12)
• The Linux thermal management periodically polls the SOC temperature (runs the ADC) at around 0.9Hz. I presume this would be useful for controlling the CPU fan, but probably not applicable to most A20 systems.
• Disabling this polling (as above) removes the CPU load, while still allowing inefficient one-off reads of the CPU temperature, as well as other ADC usage.

[wahlm]

Just fell over the same problem during reviving my ancient cubieboard 1 and Armbian community build "Armbian_community_25.2.0-trunk.195_Cubieboard_bookworm_current_6.6.62_minimal": around 10% cpu load of kworker events_freezable_power. I have tried mikhailai's solution but the issue remained. I then checked that the A10 has a "thermal_zone1", too. So I also disabled it and the load was gone. So the solution is correct, but not complete at least for the A10 based cubieboard 1.