Jump to content

Help needed: We want to improve performance on exotic boards

tkaiser

By tkaiser
in Allwinner sunxi

 Share

Recommended Posts

tkaiser

tkaiser

Posted
Posted (edited)

Dear all,

 

we need some help from Banana Pi M2 (not that important), Cubox-i/UDOO and ODROID XU4 users.

 

Please look through this Github issue and also have look how we currently handle IRQ redistribution on various board families (example 1 and example 2)

 

Since irqbalanced on ARM is broken (and if it would work it's often counterproductive) assigning IRQs statically to specific CPU cores increases overall system performance slightly for free. We do this for A20 boards for years (dual core --> sending Ethernet to second CPU core), extended this on H3 boards a few months ago (quad core and somewhat complicated due to the count of different boards exposing different interfaces) and added other board families just recently.

 

Now we need you!

 

In case you are somewhat experienced please get back to us with the output of

cat /proc/interrupts

(when you're running a more recent Armbian version then output of 'sudo armbianmonitor -u' is preferred!)

 

And in case you're able to do some testing please have a look at the examples above and the Github issue (Amlogic kernels seem to need different procfs calls for example), test yourself and provide code snippets already known to work!

 

Thx in advance!

Edited by wildcat_paris
minor typo (since=>for)
1
wildcat_paris

wildcat_paris

Posted
Posted

XU4

 

  Reveal hidden contents

 

sudo armbianmonitor -u is not very useful http://sprunge.us/WYTL

(note: I am managing XU4 fan PWM myself so it outputs a lots of logs)

tkaiser

tkaiser

Posted
  • Author
Posted
  On 6/10/2016 at 9:03 AM, wildcat_paris said:

sudo armbianmonitor -u is not very useful http://sprunge.us/WYTL

 

Hmm... can you please replace the contents of /usr/local/bin/armbianmonitor with https://github.com/igorpecovnik/lib/blob/master/scripts/armbianmonitor/armbianmonitor then run the following code (and report back if error messages appear), generate some activity and try again?

echo 2 >/proc/irq/$(awk -F":" "/usb2/ {print \$1}" </proc/interrupts | sed 's/\ //g')/smp_affinity
echo 4 >/proc/irq/$(awk -F":" "/usb3/ {print \$1}" </proc/interrupts | sed 's/\ //g')/smp_affinity
echo 8 >/proc/irq/$(awk -F":" "/usb5/ {print \$1}" </proc/interrupts | sed 's/\ //g')/smp_affinity
echo 16 >/proc/irq/$(awk -F":" "/dw-mci/ {print \$1}" </proc/interrupts | sed 's/\ //g' | tail -1)/smp_affinity
echo 32 >/proc/irq/$(awk -F":" "/dw-mci/ {print \$1}" </proc/interrupts | sed 's/\ //g' | head -1)/smp_affinity
for i in $(awk -F':' '/11800000.mali/{print $1}' /proc/interrupts); do
	echo 64 >/proc/irq/$i/smp_affinity
done

(does the XU4 have no Ethernet? Or is this USB?!)

 

EDIT: It's Realtek 8153 USB3 chipset.

1
Igor

Igor

Posted
Posted

 

  Reveal hidden contents

XU4, big heatsink, no fan :)

 

  Reveal hidden contents

 

 
http://sprunge.us/DDVC

1
wildcat_paris

wildcat_paris

Posted
Posted (edited)

@Tk,

 

the issue with "armbianmonitor -u" when dmesg is too large you only get dmesg in the log

so I have rebooted & it is better

http://sprunge.us/jTFh

 

note: need to add activity

gr@odroidxu4:~$ sudo ./tkActivityTest
gr@odroidxu4:~$ sudo armbianmonitor -u
/var/log/armhwinfo.log has been uploaded to http://sprunge.us/hNgL

http://sprunge.us/hNgL

 

 

note: I have badly named your script : tkActivityTest => tkChangeIRQCPUaffinity

 

I have done HDD reading (cloudshell NAS uses USB3=>SATA), reading from /dev/TrueRNG (on USB3 port #2), find / for MMC

(no USB2, no Mali)

 

  Reveal hidden contents

 

 

to compare before reboot

 

  Reveal hidden contents

 

--

so my tkChangeIRQCPUaffinity script

 

 

  Reveal hidden contents

 

Edited by wildcat_paris
"added" activity
1
tkaiser

tkaiser

Posted
  • Author
Posted

Would be great if you can replace the contents of /etc/init.d/armhwinfo with latest version from Github, then add the IRQ redirection stuff to the XU4 case construct, restart and report back after some activity.

 

And if you've some spare time maybe investigating what's wrong with the dmesg call in armbianmonitor's line 666 ;) -- maybe playing around with '-s, --buffer-size' option helps?

1
wildcat_paris

wildcat_paris

Posted
Posted (edited)
  On 6/10/2016 at 10:23 AM, tkaiser said:

Would be great if you can replace the contents of /etc/init.d/armhwinfo with latest version from Github, then add the IRQ redirection stuff to the XU4 case construct, restart and report back after some activity.

 

And if you've some spare time maybe investigating what's wrong with the dmesg call in armbianmonitor's line 666 ;) -- maybe playing around with '-s, --buffer-size' option helps?

 

part 1/

 

updated armhwinfo, added the IRQ redirection in /etc/rc.local and rebooted

 

 

  Reveal hidden contents

 

 

part 2/

 

Let's see what I can do with "dmesg"

 

I guess line 666 of armbianmonitor is not cursed

I had an old version of the script and I guess Igor too

 

up-to-date "armbianmonitor -u" is working

Edited by wildcat_paris
dmesg line 666 not cursed
Schwemmlandebene

Schwemmlandebene

Posted
Posted (edited)

Banana Pi M2

PS: I don't use the latest Armbian toolchain. I'm using the .ignore_changes feature.

root@m2-bare:~# uname -a
Linux m2-bare 4.5.4-sunxi #2 SMP Sat Jun 11 13:05:21 CEST 2016 armv7l GNU/Linux
root@m2-bare:~# cat /proc/interrupts 
           CPU0       CPU1       CPU2       CPU3       
 17:          0          0          0          0     GIC-0  29 Edge      arch_timer
 18:      11870     265608      31223      36550     GIC-0  30 Edge      arch_timer
 21:          0          0          0          0     GIC-0  50 Level     sun4i_timer0
 22:          0          0          0          0     GIC-0  83 Level     sun5i_timer0
 27:          0          0          0          0     GIC-0  82 Level     1c02000.dma-controller
 28:       1653          0          0          0     GIC-0  92 Level     sunxi-mmc
 29:       8549          0          0          0     GIC-0  94 Level     sunxi-mmc
 30:         27          0          0          0     GIC-0 104 Level     ehci_hcd:usb1
 31:          0          0          0          0     GIC-0 105 Level     ohci_hcd:usb2
 40:       2251          0          0          0     GIC-0  60 Level     sun4i-ts
 41:        425          0          0          0     GIC-0  32 Level     serial
 42:      43646          0          0          0     GIC-0 114 Level     eth0
 48:          0          0          0          0     GIC-0  72 Level     1f00000.rtc
 57:          0          0          0          0  sunxi_pio_edge   4 Edge      1c0f000.mmc cd
181:         43          0          0          0  sunxi_pio_level   0 Level     brcmf_oob_intr
IPI0:          0          0          0          0  CPU wakeup interrupts
IPI1:          0          0          0          0  Timer broadcast interrupts
IPI2:       3407      15690      13225       7318  Rescheduling interrupts
IPI3:          6          8          3          6  Function call interrupts
IPI4:          0          0          0          0  CPU stop interrupts
IPI5:          0          0          0          0  IRQ work interrupts
IPI6:          0          0          0          0  completion interrupts
Err:          0
root@m2-bare:~#
Edited by Schwemmlandebene
1
tkaiser

tkaiser

Posted
  • Author
Posted
  On 6/12/2016 at 9:24 AM, Schwemmlandebene said:

Banana Pi M2

PS: I don't use the latest Armbian toolchain.

 

Doesn't matter since it's just about testing out IRQ redistribution. Could you please exchange the contents of /etc/init.d/armhwinfo with this here, reboot and try again (doing some stuff and then checking/posting /proc/interrupts)? Or simply try to execute the statements in the BPi M2 case manually, do some stuff and get back with results (just to ensure that the code works and does something useful since it would be great if we could starting with 5.15 do reasonable IRQ distribution on every board we support). Thx!

Schwemmlandebene

Schwemmlandebene

Posted
Posted

I did

- copy armhwinfo

- reboot

- copy some GB over NFS

root@m2-bare:~# cat /boot/.verbose 
1465726540 sun6i armv7l 4.5.4-sunxi   Linux version 4.5.4-sunxi (root@ubuarm) (gcc version 4.8.4 (Ubuntu/Linaro 4.8.4-2ubuntu1~14.04.1) ) #2 SMP Sat Jun 11 13:05:21 CEST 2016
root@m2-bare:~# cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       
 17:          0          0          0          0     GIC-0  29 Edge      arch_timer
 18:      53622      41095     142718      94383     GIC-0  30 Edge      arch_timer
 21:          0          0          0          0     GIC-0  50 Level     sun4i_timer0
 22:          0          0          0          0     GIC-0  83 Level     sun5i_timer0
 27:          0          0          0          0     GIC-0  82 Level     1c02000.dma-controller
 28:       2095          0          0          0     GIC-0  92 Level     sunxi-mmc
 29:       8580          0          0          0     GIC-0  94 Level     sunxi-mmc
 30:         27          0          0          0     GIC-0 104 Level     ehci_hcd:usb1
 31:          0          0          0          0     GIC-0 105 Level     ohci_hcd:usb2
 40:        932          0          0          0     GIC-0  60 Level     sun4i-ts
 41:        423          0          0          0     GIC-0  32 Level     serial
 42:      30792     257702          0          0     GIC-0 114 Level     eth0
 48:          0          0          0          0     GIC-0  72 Level     1f00000.rtc
 57:          0          0          0          0  sunxi_pio_edge   4 Edge      1c0f000.mmc cd
181:         44          0          0          0  sunxi_pio_level   0 Level     brcmf_oob_intr
IPI0:          0          0          0          0  CPU wakeup interrupts
IPI1:          0          0          0          0  Timer broadcast interrupts
IPI2:      43231      21388      45126      41832  Rescheduling interrupts
IPI3:          7          5          6          7  Function call interrupts
IPI4:          0          0          0          0  CPU stop interrupts
IPI5:          0          0          0          0  IRQ work interrupts
IPI6:          0          0          0          0  completion interrupts
Err:          0
root@m2-bare:~#
tkaiser

tkaiser

Posted
  • Author
Posted

Thx, seems to work but unfortunately when testing locally I made a mistake that went on pastebin.com too:

		sun6i) # Banana Pi M2: do network on cpu1, SDIO on cpu2, USB on cpu3
			for i in $(awk -F':' '/eth0|brcmf_oob_intr/{print $1}' /proc/interrupts); do
				echo 2 >/proc/irq/$i/smp_affinity
			done
			for i in $(awk -F':' '/sunxi-mmc/{print $1}' lala.txt); do
				echo 4 >/proc/irq/$i/smp_affinity
			done
			for i in $(awk -F':' '/hcd:usb/{print $1}' lala.txt); do
				echo 8 >/proc/irq/$i/smp_affinity
			done
			;;

lala.txt should be replaced with /proc/interrupts then USB and SDIO IRQs should be handled by 3rd and 4th CPU core :)

tkaiser

tkaiser

Posted
  • Author
Posted
  On 6/12/2016 at 11:25 AM, Schwemmlandebene said:

/proc/interrupts or </proc/interrupts?

if </proc/interrupts, what about line 232?

 

Doesn't really matter since awk either uses stdin or the file argument. Since it works (eth0 on cpu1) in your setup just with /proc/interrupts I would give this a try.

Schwemmlandebene

Schwemmlandebene

Posted
Posted

Ahh, I didn't mind the awk syntax.

But bash complains.

root@m2-bare:/etc/init.d# /etc/init.d/armhwinfo start
[ ok ] Setting noop I/O scheduler for mmcblk0
[ ok ] Starting ARM hardware info:  (Linux version 4.5.4-sunxi (root@ubuarm) (gcc version 4.8.4 (Ubuntu/Linaro 4.8.4-2ubuntu1~14.04.1) ) #2 SMP Sat Jun 11 13:05:21 CEST 2016)
/etc/init.d/armhwinfo: line 233: echo: write error: Input/output error
                sun6i) # Banana Pi M2: do network on cpu1, SDIO on cpu2, USB on cpu3
                        for i in $(awk -F':' '/eth0|brcmf_oob_intr/{print $1}' /proc/interrupts); do
                                echo 2 >/proc/irq/$i/smp_affinity
                        done
                        for i in $(awk -F':' '/sunxi-mmc/{print $1}' /proc/interrupts); do
                                echo 4 >/proc/irq/$i/smp_affinity
                        done
                        for i in $(awk -F':' '/hcd:usb/{print $1}' /proc/interrupts); do
                                echo 8 >/proc/irq/$i/smp_affinity
                        done
                        ;;
tkaiser

tkaiser

Posted
  • Author
Posted
  On 6/12/2016 at 11:48 AM, Schwemmlandebene said: 
/etc/init.d/armhwinfo: line 233: echo: write error: Input/output error

 

Weird since eth0 has been sent successfully to cpu1 before? Can you please check state of /proc/interrupts now and maybe do the manual call again prefixing everything with

/bin/bash -x /etc/init.d/armhwinfo start

and upload the whole output on pastebin.com or similar? Thx!

Schwemmlandebene

Schwemmlandebene

Posted
Posted

Hmm, I tried the lala version, too, and that's the output:

root@m2-bare:~# journalctl --no-pager | grep  armhwinfo
Jun 12 14:09:52 m2-bare armhwinfo[645]: [ ok ] Setting noop I/O scheduler for mmcblk0
Jun 12 14:09:52 m2-bare armhwinfo[645]: [ ok ] Starting ARM hardware info:  (Linux version 4.5.4-sunxi (root@ubuarm) (gcc version 4.8.4 (Ubuntu/Linaro 4.8.4-2ubuntu1~14.04.1) ) #2 SMP Sat Jun 11 13:05:21 CEST 2016)
Jun 12 14:09:52 m2-bare armhwinfo[645]: /etc/init.d/armhwinfo: line 233: echo: write error: Input/output error
Jun 12 14:09:52 m2-bare armhwinfo[645]: awk: cannot open lala.txt (No such file or directory)
Jun 12 14:09:52 m2-bare armhwinfo[645]: awk: cannot open lala.txt (No such file or directory)
root@m2-bare:~#
Schwemmlandebene

Schwemmlandebene

Posted
Posted

I don't use postebin, etc. I could post the whole output here.

 

Here is a snippet:

+ ID='Banana Pi M2'
+ [[ Sinovoip BPI-M2 == *AMLOGIC* ]]
+ [[ Sinovoip BPI-M2 == *HummingBoard2* ]]
+ [[ -f /etc/armbian-release ]]
+ case ${HARDWARE} in
+ chmod 644 /etc/logrotate.d/armhwinfo
++ awk -F: '/eth0|brcmf_oob_intr/{print $1}' /proc/interrupts
+ for i in '$(awk -F'\'':'\'' '\''/eth0|brcmf_oob_intr/{print $1}'\'' /proc/interrupts)'
+ echo 2
+ for i in '$(awk -F'\'':'\'' '\''/eth0|brcmf_oob_intr/{print $1}'\'' /proc/interrupts)'
+ echo 2
/etc/init.d/armhwinfo: line 233: echo: write error: Input/output error
+ '[' -f /var/log/armhwinfo.log ']'
++ awk -F: '/sunxi-mmc/{print $1}' /proc/interrupts
++ date
+ for i in '$(awk -F'\'':'\'' '\''/sunxi-mmc/{print $1}'\'' /proc/interrupts)'
+ echo 4
+ for i in '$(awk -F'\'':'\'' '\''/sunxi-mmc/{print $1}'\'' /proc/interrupts)'
+ echo 4
+ echo -e '\n\n\nSun Jun 12 14:21:33 CEST 2016 sun6i armv7l 4.5.4-sunxi   Linux version 4.5.4-sunxi (root@ubuarm) (gcc version 4.8.4 (Ubuntu/Linaro 4.8.4-2ubuntu1~14.04.1) ) #2 SMP Sat Jun 11 13:05:21 CEST 2016'
++ awk -F: '/hcd:usb/{print $1}' /proc/interrupts
+ echo -e '\n### dmesg:\n'
+ cat /tmp/armhwinfo.OwXgVA
+ for i in '$(awk -F'\'':'\'' '\''/hcd:usb/{print $1}'\'' /proc/interrupts)'
+ echo 8
+ for i in '$(awk -F'\'':'\'' '\''/hcd:usb/{print $1}'\'' /proc/interrupts)'
+ echo 8
+ echo -e '\n### lsusb:'
+ '[' -f /boot/.verbose ']'

Does armhwinfo do not exit?


root@m2-bare:~# cat /proc/interrupts 
           CPU0       CPU1       CPU2       CPU3       
 17:          0          0          0          0     GIC-0  29 Edge      arch_timer
 18:       3795       8165      75263       8169     GIC-0  30 Edge      arch_timer
 21:          0          0          0          0     GIC-0  50 Level     sun4i_timer0
 22:          0          0          0          0     GIC-0  83 Level     sun5i_timer0
 27:          0          0          0          0     GIC-0  82 Level     1c02000.dma-controller
 28:        625          0       1910          0     GIC-0  92 Level     sunxi-mmc
 29:       8266          0          0          0     GIC-0  94 Level     sunxi-mmc
 30:         27          0          0        262     GIC-0 104 Level     ehci_hcd:usb1
 31:          0          0          0          0     GIC-0 105 Level     ohci_hcd:usb2
 40:        498          0          0          0     GIC-0  60 Level     sun4i-ts
 41:        430          0          0          0     GIC-0  32 Level     serial
 42:      31217      10822          0          0     GIC-0 114 Level     eth0
 48:          0          0          0          0     GIC-0  72 Level     1f00000.rtc
 57:          0          0          0          0  sunxi_pio_edge   4 Edge      1c0f000.mmc cd
181:         44          0          0          0  sunxi_pio_level   0 Level     brcmf_oob_intr
IPI0:          0          0          0          0  CPU wakeup interrupts
IPI1:          0          0          0          0  Timer broadcast interrupts
IPI2:       4199       8013      14879       8066  Rescheduling interrupts
IPI3:          9          9          6          4  Function call interrupts
IPI4:          0          0          0          0  CPU stop interrupts
IPI5:          0          0          0          0  IRQ work interrupts
IPI6:          0          0          0          0  completion interrupts
Err:          0
root@m2-bare:~#
tkaiser

tkaiser

Posted
  • Author
Posted

Hmm... since the code does what it's supposed to (sending IRQ processing to dedicated CPU cores) I'll simply suffix all the calls with ' 2>/dev/null'. Thx for your help, new settings will then be active starting with Armbian 5.15 :)

Schwemmlandebene

Schwemmlandebene

Posted
Posted

Are you sure?

root@m2-bare:~# cat /proc/irq/181/smp_affinity
f
root@m2-bare:~# cat /proc/irq/42/smp_affinity
2
root@m2-bare:~# cat /proc/irq/31/smp_affinity
8
root@m2-bare:~# cat /proc/irq/30/smp_affinity
8
root@m2-bare:~# cat /proc/irq/29/smp_affinity
4
root@m2-bare:~# cat /proc/irq/28/smp_affinity
4
root@m2-bare:~# echo 2 > /proc/irq/181/smp_affinity
-bash: echo: write error: Input/output error
root@m2-bare:~#
tkaiser

tkaiser

Posted
  • Author
Posted
  On 6/12/2016 at 1:21 PM, Schwemmlandebene said:

Are you sure?

 

Regarding eth0 I was pretty sure but thanks for confirmation that 118 isn't accessible. Will remove this and commit the stuff for BPi M2 within the next minutes.

Guest
This topic is now closed to further replies.
 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept