Marco Schirrmeister Posted January 22 Share Posted January 22 (edited) I want to share some information about an issue that I see on my OPi5+. Since I got it I am messing around with it and everything works great so far with the edge 6.7 kernel. One thing I noticed though was, that right after installation with nothing connected and when idle, there was one kernel process that was constantly consuming around 10% CPU time and the load stays at 1. The process was irq/136-hym8563. The number varies of course after each boot. Kernel driver is rtc_hym8563. Since it is build into the kernel I was not able to unload it as a first test. Side node, the issue is not seen on the Rock5b, so it seems OPi5+ hardware specific. Even if both are relative similar. I have not found anything useful so far on the internet. Don't know if it is just my device or if others are seeing a similar thing on their OPi5+ with the 6.7 edge kernel. IRQs going up like crazy for a few interrupts, compared to when driver rtc-hym8563 is not there. arch_timer rk_timer fec80000.i2c hym8563 Here are some debugging information. top output. root@orangepi5-plus ~# top -b -n 4 -p (pgrep hym) top - 16:30:43 up 3 min, 1 user, load average: 1.08, 0.63, 0.27 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 1.2 sy, 0.0 ni, 97.6 id, 0.0 wa, 1.2 hi, 0.0 si, 0.0 st MiB Mem : 15718.1 total, 15299.5 free, 422.6 used, 212.1 buff/cache MiB Swap: 7859.0 total, 7859.0 free, 0.0 used. 15295.5 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 438 root -51 0 0 0 0 D 0.0 0.0 0:16.90 irq/136-hym8563 top - 16:30:46 up 3 min, 1 user, load average: 1.08, 0.63, 0.27 Tasks: 1 total, 1 running, 0 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 0.7 sy, 0.0 ni, 97.9 id, 0.0 wa, 1.3 hi, 0.0 si, 0.0 st MiB Mem : 15718.1 total, 15299.5 free, 422.6 used, 212.1 buff/cache MiB Swap: 7859.0 total, 7859.0 free, 0.0 used. 15295.5 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 438 root -51 0 0 0 0 R 7.7 0.0 0:17.13 irq/136-hym8563 top - 16:30:49 up 3 min, 1 user, load average: 1.07, 0.63, 0.27 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 0.7 sy, 0.0 ni, 97.9 id, 0.0 wa, 1.3 hi, 0.0 si, 0.0 st MiB Mem : 15718.1 total, 15299.5 free, 422.6 used, 212.1 buff/cache MiB Swap: 7859.0 total, 7859.0 free, 0.0 used. 15295.5 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 438 root -51 0 0 0 0 D 8.0 0.0 0:17.37 irq/136-hym8563 top - 16:30:52 up 4 min, 1 user, load average: 0.99, 0.62, 0.27 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 0.8 sy, 0.0 ni, 97.9 id, 0.0 wa, 1.3 hi, 0.0 si, 0.0 st MiB Mem : 15718.1 total, 15299.5 free, 422.6 used, 212.1 buff/cache MiB Swap: 7859.0 total, 7859.0 free, 0.0 used. 15295.5 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 438 root -51 0 0 0 0 D 8.0 0.0 0:17.61 irq/136-hym8563 Interrupt increase over 1 minute. 100k for hym8563 and 1million for fec8000.i2c root@orangepi5-plus ~# uptime && date && cat /proc/interrupts | grep -iE "(hym|fec80000|_timer)" && sleep 60 && date && cat /proc/interrupts | grep -iE "(hym|fec80000|_timer)" 17:28:15 up 35 min, 1 user, load average: 1.01, 1.07, 1.00 Mon Jan 22 05:28:15 PM CET 2024 19: 26038 125123 73155 438814 46260 13649 3574 299665 GICv3 26 Level arch_timer 25: 22900 83767 54823 279426 36468 15155 10963 9513 GICv3 321 Level rk_timer 48: 0 0 0 0 0 0 0 30250774 GICv3 355 Level fec80000.i2c 136: 0 0 0 0 0 0 0 5041747 rockchip_gpio_irq 8 Level hym8563 Mon Jan 22 05:29:15 PM CET 2024 19: 26366 125128 91142 438815 46849 13652 3574 307458 GICv3 26 Level arch_timer 25: 23366 84133 67225 279722 37149 15171 10976 9752 GICv3 321 Level rk_timer 48: 0 0 0 0 0 0 0 31073437 GICv3 355 Level fec80000.i2c 136: 0 0 0 0 0 0 0 5178856 rockchip_gpio_irq 8 Level hym8563 irq cpu time is also visible in mpstat root@orangepi5-plus ~# mpstat -P ALL Linux 6.7.1-edge-rockchip-rk3588 (orangepi5-plus) 01/22/2024 _aarch64_ (8 CPU) 07:43:50 PM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 07:43:50 PM all 0.13 0.00 0.68 0.01 1.33 0.03 0.00 0.00 0.00 97.82 07:43:50 PM 0 0.03 0.00 1.30 0.01 0.98 0.08 0.00 0.00 0.00 97.61 07:43:50 PM 1 0.03 0.00 0.26 0.01 0.18 0.02 0.00 0.00 0.00 99.51 07:43:50 PM 2 0.03 0.00 3.06 0.01 2.46 0.07 0.00 0.00 0.00 94.37 07:43:50 PM 3 0.02 0.00 0.77 0.01 0.58 0.02 0.00 0.00 0.00 98.60 07:43:50 PM 4 0.59 0.00 0.11 0.01 0.03 0.02 0.00 0.00 0.00 99.24 07:43:50 PM 5 0.22 0.00 0.05 0.01 0.01 0.01 0.00 0.00 0.00 99.70 07:43:50 PM 6 0.06 0.00 0.01 0.01 0.01 0.00 0.00 0.00 0.00 99.91 07:43:50 PM 7 0.04 0.00 0.02 0.01 6.49 0.03 0.00 0.00 0.00 93.42 procinfo output shows as well the high number for irq 136 and 48. root@orangepi5-plus ~# procinfo Memory: Total Used Free Buffers RAM: 16095284 585248 15510036 22584 Swap: 8047640 0 8047640 Bootup: Mon Jan 22 16:53:14 2024 Load average: 1.29 1.09 1.02 1/231 9335 user : 00:01:51.76 0.1% page in : 238872 nice : 00:00:00.03 0.0% page out: 118633 system: 00:09:19.33 0.7% page act: 59088 IOwait: 00:00:09.19 0.0% page dea: 0 hw irq: 00:18:07.58 1.3% page flt: 1335085 sw irq: 00:00:24.98 0.0% swap in : 0 idle : 22:09:57.44 97.8% swap out: 0 uptime: 02:51:46.79 context : 142704160 irq 18: 0 irq 66: 0 427819017 Edge ae irq 19: 4672570 irq 77: 0 570425352 Edge PC irq 20: 0 irq 78: 0 570425353 Edge ae irq 23: 0 irq 89: 0 285212680 Edge PC irq 25: 2504720 irq 90: 0 285212681 Edge ae irq 31: 0 irq 91: 2 irq 32: 15 irq 92: 0 irq 33: 1 irq 93: 32 irq 34: 0 irq 94: 0 irq 35: 0 irq 95: 75881 428343296 Edge en irq 36: 0 irq 96: 0 570949632 Edge en irq 37: 0 irq 97: 0 irq 38: 0 irq 98: 0 irq 39: 0 irq 99: 0 irq 40: 0 irq 100: 0 irq 41: 2 irq 101: 514 irq 43: 653459 irq 102: 0 irq 44: 2278 irq 119: 0 irq 45: 1346 irq 130: 0 8 Edge PCIe PME irq 46: 203 irq 131: 0 9 Edge aerdrv irq 47: 41964 irq 132: 99660 irq 48: 141047548 irq 133: 56265 irq 49: 108 irq 135: 0 irq 50: 0 irq 136: 23507838 irq 51: 0 irq 137: 324 irq 52: 2 irq 138: 0 irq 53: 0 irq 139: 0 irq 54: 0 irq 140: 0 irq 65: 0 427819016 Edge PC Since I do not need the RTC on my system, my idea was to create an own custom image, where rtc_hym8563 is build as a module. Where I can disable the module in the hope the process is gone and nothing is consuming cpu time when idle. It turns out that worked for me and I am running it like this for now. Here are my steps. Change kernel config in the build system. # config/kernel/linux-rockchip-rk3588-edge.config CONFIG_RTC_DRV_HYM8563=y After new image is installed and booted, it is indeed now a module. root@orangepi5-plus ~# zcat /proc/config.gz | grep -i hym CONFIG_RTC_DRV_HYM8563=m Disabling the rtc_hym8563 driver echo "blacklist rtc_hym8563" > /etc/modprobe.d/blacklist-rtc.conf update-initramfs -u reboot I just wanted to share this information in case someone else see this. For now I will live with my workaround and keep an eye on the kernel development to see, if there will be a change. In case there is some hardware expert and has some other tips, please feel free to share. Cheers, Marco Edited January 22 by Marco Schirrmeister 0 Quote Link to comment Share on other sites More sharing options...
Marco Schirrmeister Posted January 25 Author Share Posted January 25 After some more searching, I found this patch, which looked to me interesting. At least from the little description https://patchwork.kernel.org/project/linux-rockchip/patch/20231202071212.1606800-1-CFSworks@gmail.com/ The RK1 module is also RK3588 based and has a HYM8563 chip as well. No idea what this is or what is different on the various boards and their components. But I thought I give it a try. I implemented the hym8563 patch line on the edge-6.7.1 version to see what happens. Result is, it made a change on the OPi5+. No more high cpu usage or heavy interrupt increases with the change from none to up, when the hym8563 driver is loaded. Since I am not a hardware guy, I have no idea what this is or what it does. If it is good or bad or whatever. 0 Quote Link to comment Share on other sites More sharing options...
Vijay Gill Posted February 11 Share Posted February 11 @Marco Schirrmeister Thank you for your search and pointing towards the fix. I applied the same fix to 6.8.0-rc1-edge kernel and I noticed the same result! From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 From: John Doe <john.doe@somewhere.on.planet> Date: Sun, 11 Feb 2024 02:19:42 +0000 Subject: Patching kernel rockchip-rk3588 files arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts to fix hym8563-int issue. Signed-off-by: John Doe <john.doe@somewhere.on.planet> --- arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts b/arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts index 88bfce623..844992180 100644 --- a/arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts +++ b/arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts @@ -462,11 +462,11 @@ &pcie3x4 { }; &pinctrl { hym8563 { hym8563_int: hym8563-int { - rockchip,pins = <0 RK_PB0 RK_FUNC_GPIO &pcfg_pull_none>; + rockchip,pins = <0 RK_PB0 RK_FUNC_GPIO &pcfg_pull_up>; }; }; leds { blue_led_pin: blue-led { -- Created with Armbian build tools https://github.com/armbian/build 0 Quote Link to comment Share on other sites More sharing options...
Marco Schirrmeister Posted February 12 Author Share Posted February 12 (edited) Thank you for the confirmation @Vijay Gill. Here is the patch I use. I did also add the poweroff support. Splitting it into multiple files might be better, but it is good enough for my test builds. root@dumpster /m/t/t/n/build (main)# cat userpatches/kernel/rockchip-rk3588-edge/1000-arm64-dts-fix-rtc-add-poweroff-support-Orange-Pi-5-Plus.patch From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 From: John Doe <john.doe@somewhere.on.planet> Date: Mon, 29 Jan 2024 12:51:13 +0100 Subject: Patching kernel rockchip-rk3588 files arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts Signed-off-by: John Doe <john.doe@somewhere.on.planet> --- arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts b/arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts index 88bfce6237db..70cc6bd5a0af 100644 --- a/arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts +++ b/arch/arm64/boot/dts/rockchip/rk3588-orangepi-5-plus.dts @@ -462,11 +462,11 @@ &pcie3x4 { }; &pinctrl { hym8563 { hym8563_int: hym8563-int { - rockchip,pins = <0 RK_PB0 RK_FUNC_GPIO &pcfg_pull_none>; + rockchip,pins = <0 RK_PB0 RK_FUNC_GPIO &pcfg_pull_up>; }; }; leds { blue_led_pin: blue-led { @@ -572,10 +572,12 @@ pmic@0 { pinctrl-names = "default"; pinctrl-0 = <&pmic_pins>, <&rk806_dvs1_null>, <&rk806_dvs2_null>, <&rk806_dvs3_null>; spi-max-frequency = <1000000>; + system-power-controller; + vcc1-supply = <&vcc5v0_sys>; vcc2-supply = <&vcc5v0_sys>; vcc3-supply = <&vcc5v0_sys>; vcc4-supply = <&vcc5v0_sys>; vcc5-supply = <&vcc5v0_sys>; @@ -592,11 +594,11 @@ pmic@0 { gpio-controller; #gpio-cells = <2>; rk806_dvs1_null: dvs1-null-pins { - pins = "gpio_pwrctrl2"; + pins = "gpio_pwrctrl1"; function = "pin_fun0"; }; rk806_dvs2_null: dvs2-null-pins { pins = "gpio_pwrctrl2"; -- Created with Armbian build tools https://github.com/armbian/build Edited February 12 by Marco Schirrmeister Formatting 1 Quote Link to comment Share on other sites More sharing options...
Efe Çetin Posted February 12 Share Posted February 12 @Marco Schirrmeister can you send a PR to armbian to add your patch? 0 Quote Link to comment Share on other sites More sharing options...
Vijay Gill Posted February 13 Share Posted February 13 @Marco Schirrmeister I did the PowerOff one too after I posted my message above. I can confirm that it works too. Thanks again! 0 Quote Link to comment Share on other sites More sharing options...
Marco Schirrmeister Posted February 13 Author Share Posted February 13 @Efe Çetin, PR is created. https://github.com/armbian/build/pull/6276 Bear with me, it is my first PR. So let me know whatever is wrong or needs to be changed. 1 Quote Link to comment Share on other sites More sharing options...
Werner Posted February 14 Share Posted February 14 You're doing great Draft PR usually means it is WIP or the author implies it shall not be merged (yet). So once you think you are done with your adjustments mark it ready for review. It is still possible to do adjustments after that. 0 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.