2 2
kristsm

PREEMPT-RT PATCH for Allwinner H5

Recommended Posts

(edited)

Hi!

 

I didn't find much info in the forum on how to correctly apply RT patch to the Armbian (Orange Pi Zero Plus H5 board). 

Here is how I did:

Built Armbian as follows:

1) I downloaded patch-5.4.22-rt13.patch from https://mirrors.edge.kernel.org/pub/linux/kernel/projects/rt/5.4/ and placed into  ~/build/userpatches/kernel/sunxi-current ( I saw from the logs that this is the folder for the builder to search for user patches in my case)

2) Started ./compile.sh  --> FULL OS image for flashing --> Show a kernel configuration menu before  compilation --> orangepizeroplus (H5) as target board --> current (recommended , which is 5.4.23 as of today) --> bionic (18.04 LTS) --> standard image with console interface.

3) Build log shows:  "[ o.k. ] * [c] patch-5.4.22-rt13.patch"

So it seems the patch is included in the compilation.

4) In the kernel configuration screen I amended settings for low-latency kernel ( I chose the third item in the list which seems is full preemptive mode)

 

Armbian build finishes without errors.

The board starts fine, it shows Armbian 5.4.23-rt13 as the kernel version.  

uname -a shows that the kernel is in fact in the SMP PREEMPT mode.

 

Now the problem is that any cyclictest with any configuration shows much worse AVG latency times compared to the system with standard Armbian 5.4.23 kernel. In most cases AVG latency times exceed 1000us , therefore its far-far away from the times I see in the forum for other systems using even the H3 chipset (sub 50us AVG latency tims is not that uncommon).

 

What I did wrong, or what necessary/mandatory steps have I missed for having sub 50us AVG latency system?

 

Thanks for any help in advance!


Krists

 

 

 

Edited by kristsm
misstyped

Share this post


Link to post
Share on other sites

MX_Master - thanks for the tip!

 

How should I determine the last RT kernel patch version containing PREEMPT_RT Option for arm64? I am not an advanced programmer, but opening patch-5.4.22-rt I see there are configs for ARM64....

P.S. from packages.debian.org I also see there is a package with a name linux-image-5.4.0-4-rt-arm64-unsigned

 

 

 

Share this post


Link to post
Share on other sites

See the a/arch/arm64/Kconfig section and compare it with other popular platforms :)

lsdiff patch-5.4.22-rt13.patch | grep x86
filterdiff -i a/arch/x86/Kconfig patch-5.4.22-rt13.patch

lsdiff patch-5.4.22-rt13.patch | grep arm64
filterdiff -i a/arch/arm64/Kconfig patch-5.4.22-rt13.patch

 

Share this post


Link to post
Share on other sites

And what exactly should I look for? 

Both (x86 & arm64) have:

1. /include/asm/preempt.h

2. select ARCH_SUPPORTS_RT

3. select HAVE_PREEMPT_LAZY

 

Additionally , ARM64 have:

-       select ARCH_INLINE_READ_LOCK if !PREEMPT
-       select ARCH_INLINE_READ_LOCK_BH if !PREEMPT
-       select ARCH_INLINE_READ_LOCK_IRQ if !PREEMPT
-       select ARCH_INLINE_READ_LOCK_IRQSAVE if !PREEMPT
-       select ARCH_INLINE_READ_UNLOCK if !PREEMPT
-       select ARCH_INLINE_READ_UNLOCK_BH if !PREEMPT
-       select ARCH_INLINE_READ_UNLOCK_IRQ if !PREEMPT
-       select ARCH_INLINE_READ_UNLOCK_IRQRESTORE if !PREEMPT
-       select ARCH_INLINE_WRITE_LOCK if !PREEMPT
-       select ARCH_INLINE_WRITE_LOCK_BH if !PREEMPT
-       select ARCH_INLINE_WRITE_LOCK_IRQ if !PREEMPT
-       select ARCH_INLINE_WRITE_LOCK_IRQSAVE if !PREEMPT
-       select ARCH_INLINE_WRITE_UNLOCK if !PREEMPT
-       select ARCH_INLINE_WRITE_UNLOCK_BH if !PREEMPT
-       select ARCH_INLINE_WRITE_UNLOCK_IRQ if !PREEMPT
-       select ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE if !PREEMPT
-       select ARCH_INLINE_SPIN_TRYLOCK if !PREEMPT
-       select ARCH_INLINE_SPIN_TRYLOCK_BH if !PREEMPT
-       select ARCH_INLINE_SPIN_LOCK if !PREEMPT
-       select ARCH_INLINE_SPIN_LOCK_BH if !PREEMPT
-       select ARCH_INLINE_SPIN_LOCK_IRQ if !PREEMPT
-       select ARCH_INLINE_SPIN_LOCK_IRQSAVE if !PREEMPT
-       select ARCH_INLINE_SPIN_UNLOCK if !PREEMPT
-       select ARCH_INLINE_SPIN_UNLOCK_BH if !PREEMPT
-       select ARCH_INLINE_SPIN_UNLOCK_IRQ if !PREEMPT
-       select ARCH_INLINE_SPIN_UNLOCK_IRQRESTORE if !PREEMPT
+       select ARCH_INLINE_READ_LOCK if !PREEMPTION
+       select ARCH_INLINE_READ_LOCK_BH if !PREEMPTION
+       select ARCH_INLINE_READ_LOCK_IRQ if !PREEMPTION
+       select ARCH_INLINE_READ_LOCK_IRQSAVE if !PREEMPTION
+       select ARCH_INLINE_READ_UNLOCK if !PREEMPTION
+       select ARCH_INLINE_READ_UNLOCK_BH if !PREEMPTION
+       select ARCH_INLINE_READ_UNLOCK_IRQ if !PREEMPTION
+       select ARCH_INLINE_READ_UNLOCK_IRQRESTORE if !PREEMPTION
+       select ARCH_INLINE_WRITE_LOCK if !PREEMPTION
+       select ARCH_INLINE_WRITE_LOCK_BH if !PREEMPTION
+       select ARCH_INLINE_WRITE_LOCK_IRQ if !PREEMPTION
+       select ARCH_INLINE_WRITE_LOCK_IRQSAVE if !PREEMPTION
+       select ARCH_INLINE_WRITE_UNLOCK if !PREEMPTION
+       select ARCH_INLINE_WRITE_UNLOCK_BH if !PREEMPTION
+       select ARCH_INLINE_WRITE_UNLOCK_IRQ if !PREEMPTION
+       select ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE if !PREEMPTION
+       select ARCH_INLINE_SPIN_TRYLOCK if !PREEMPTION
+       select ARCH_INLINE_SPIN_TRYLOCK_BH if !PREEMPTION
+       select ARCH_INLINE_SPIN_LOCK if !PREEMPTION
+       select ARCH_INLINE_SPIN_LOCK_BH if !PREEMPTION
+       select ARCH_INLINE_SPIN_LOCK_IRQ if !PREEMPTION
+       select ARCH_INLINE_SPIN_LOCK_IRQSAVE if !PREEMPTION
+       select ARCH_INLINE_SPIN_UNLOCK if !PREEMPTION
+       select ARCH_INLINE_SPIN_UNLOCK_BH if !PREEMPTION
+       select ARCH_INLINE_SPIN_UNLOCK_IRQ if !PREEMPTION
+       select ARCH_INLINE_SPIN_UNLOCK_IRQRESTORE if !PREEMPTION

 

 

So what am I missing?

 

 

Share this post


Link to post
Share on other sites
On 3/5/2020 at 3:21 AM, kristsm said:

I didn't find much info in the forum on how to correctly apply RT patch to the Armbian (Orange Pi Zero Plus H5 board). 

 

Curious as to why one wants to apply the patches for preempt-rt in the first place? Do you have a reason/application that would benefit? Most times, one can see an overall decrease in application performance, and to take advantage of preempt-rt, the application must call pthreads in a very specific way - which means that many times, the app must also be rebuilt.

 

My reason for asking - if preempt-rt was so awesome, it would already be in the upstream linux kernel mainline and not require the patch. Avoiding politics around preempt-rt - many of the benefits here can be done without the patches, by selecting an appropriate scheduler - schedutil looks fairly interesting at the moment.

 

I work on networking oriented applications that have hard deadlines, on MIPS, PowerPC, ARM (both 32bit ARMv7a and 64-bit), and never had a need to apply the preempt-rt patches.

 

By networking oriented applications - I'm saying 20mSec framing for SIP-RTP (VoIP apps), along with 3GPP/UMTS signalling - if that ain't close to real-time, I don't know what is.

Share this post


Link to post
Share on other sites
On 3/7/2020 at 6:03 AM, sfx2000 said:

 

Curious as to why one wants to apply the patches for preempt-rt in the first place? Do you have a reason/application that would benefit? Most times, one can see an overall decrease in application performance, and to take advantage of preempt-rt, the application must call pthreads in a very specific way - which means that many times, the app must also be rebuilt.

 

My reason for asking - if preempt-rt was so awesome, it would already be in the upstream linux kernel mainline and not require the patch. Avoiding politics around preempt-rt - many of the benefits here can be done without the patches, by selecting an appropriate scheduler - schedutil looks fairly interesting at the moment.

 

I work on networking oriented applications that have hard deadlines, on MIPS, PowerPC, ARM (both 32bit ARMv7a and 64-bit), and never had a need to apply the preempt-rt patches.

 

By networking oriented applications - I'm saying 20mSec framing for SIP-RTP (VoIP apps), along with 3GPP/UMTS signalling - if that ain't close to real-time, I don't know what is.

 

Curious as to why you waste your time if you don't want to help?

 

 

Share this post


Link to post
Share on other sites

Hi there,

 

This topic interest me a lot. I'm using arm sbc for sound & music applications where low latency is important.

sfx2000's post reminds me other discussions on other forums (mostly computer "assisted" music oriented) where some thoughts/experiences are related to poor application performance regarding the low positive impact of a RT kernel.

From my own experience (x86, arm SBC's and RPI), RT kernel or not, good or bad audio interface in USB or I2S, depending on the load (cpu, memory, audio stream), I always have crack, glitches and xruns.

 

Without RT patch, what I get:

- With a H5 board with 2Gb memory (NanoPiK1plus) I can't go under 20ms of jack audio server latency without having a to unstable config but, as is, it is still "playable".

- With a NanoPiM4 and a HD USB audio interface, I have 5ms of latency, a lot of power for apps and a NvMe interface for a fast drive. That's great :)

 

So, I really ask myself for going further in kernel "hacking" with the RT patch.

 

 

Share this post


Link to post
Share on other sites

Tried to build Armbian Bionic today with sunxi-current ( I see the compiler have included patch 5.4.23-24 as of today, which seems the kernel should be @5.4.24?).  I moved 5.4.24 rt15 patch in userpatches/kernel/sunxi-current.

In kernel configuration I checked expert menu , which allowed me to choose fully preemptible kernel.

 

The compiler goes until CC [M]  kernel/kheaders.o, then it throws [error] ERROR in function compile_kernel [ compilation.sh: 378] and terminates the process.

 

Any tips on how to move further? 

Share this post


Link to post
Share on other sites
22 minutes ago, kristsm said:

Any tips on how to move further? 


You will find tips in debug logs: output/debug/compilation.log

Share this post


Link to post
Share on other sites
9 hours ago, Igor said:


You will find tips in debug logs: output/debug/compilation.log

Thanks! :)

 

Ok, log file shows following errors:

 

fs/aufs/i_op.c: In function ‘au_pin_hdir_set_owner’:
fs/aufs/i_op.c:636:45: error: ‘struct rw_semaphore’ has no member named ‘owner’
  atomic_long_set(&p->hdir->hi_inode->i_rwsem.owner, (long)task);
                                             ^
make[2]: *** [fs/aufs/i_op.o] Error 1
make[1]: *** [fs/aufs] Error 2
make: *** [fs] Error 2
make: *** Waiting for unfinished jobs....
scripts/Makefile.build:265: recipe for target 'fs/aufs/i_op.o'
scripts/Makefile.build:500: recipe for target 'fs/aufs' failed
Makefile:1692: recipe for target 'fs' failed

 

 

 

 

Share this post


Link to post
Share on other sites
54 minutes ago, kristsm said:

Ok, log file shows following errors:

 

fs/aufs/i_op.c: In function ‘au_pin_hdir_set_owner’:
fs/aufs/i_op.c:636:45: error: ‘struct rw_semaphore’ has no member named ‘owner’
  atomic_long_set(&p->hdir->hi_inode->i_rwsem.owner, (long)task);
                                             ^
make[2]: *** [fs/aufs/i_op.o] Error 1
make[1]: *** [fs/aufs] Error 2
make: *** [fs] Error 2
make: *** Waiting for unfinished jobs....
scripts/Makefile.build:265: recipe for target 'fs/aufs/i_op.o'
scripts/Makefile.build:500: recipe for target 'fs/aufs' failed
Makefile:1692: recipe for target 'fs' failed


Now its clear what to do. Build with AUFS="no" since it looks like it doesn't want to operate with RT or fix errors in the code if you need AUFS.

Share this post


Link to post
Share on other sites
On 3/9/2020 at 5:47 AM, kristsm said:

Curious as to why you waste your time if you don't want to help?

 

My time is valuable - so let's spend it wisely...

 

Do you have a real-world use-case where there is value or not?

Share this post


Link to post
Share on other sites
14 hours ago, Igor said:


Now its clear what to do. Build with AUFS="no" since it looks like it doesn't want to operate with RT or fix errors in the code if you need AUFS.

Thanks Igor!

After AUFS disabled, the compiler threw at me another error regarding NFS, disabled that too and was greeted with completed build. Flashed the image in 64gb sd-card and after boot uname -a greeted me with the message:

Linux orangepizeroplus 5.4.24-rt13-sunxi64 #trunk SMP PREEMPT_RT Wed Mar 11 07:28:42 EET 2020 aarch64 aarch64 aarch64 GNU/Linux ;)

 

cyclic test results with all cores running in performance mode (no mods for ISOLCPUS):

 

sudo cyclictest -a -t -n -p80
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.96 0.85 0.57 1/165 3526

T: 0 ( 3518) P:80 I:1000 C: 161625 Min:      7 Act:   10 Avg:   10 Max:      49
T: 1 ( 3519) P:80 I:1500 C: 107750 Min:      7 Act:   14 Avg:   11 Max:      49
T: 2 ( 3520) P:80 I:2000 C:  80812 Min:      7 Act:   10 Avg:   10 Max:      35
T: 3 ( 3521) P:80 I:2500 C:  64650 Min:      8 Act:   10 Avg:   10 Max:      25

 

 

 

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
2 2