0
tkaiser

Benchmarking CPUs (not yet a research article)

Recommended Posts

With this commit I added 7-zip benchmark reporting to Armbian now. Will be available after next updates and with next batch of new images.

 

Why not recommending to just do an 'apt install p7zip ; 7zr b'? Since 'fire and forget' benchmarking is always BS. You need some monitoring in parallel to know whether your system was really idle and at which clockspeeds the CPU cores were operating (throttling occuring or not?). 

 

Most recent 7-zip contains an own routine to 'pre-heat' the system prior to starting the benchmark (to let cpufreq scaling switch from low clockspeeds to highest ones and e.g. on Intel systems let the system enter TurboBoost modes). This 7-zip code runs single threaded so based on the kernel's scheduler sometimes ending up on the 'wrong' CPU core (e.g. a little core on big.LITTLE SoCs)

 

On a NanoPC T4 with conservative settings (limiting big CPU cores to 1.8 GHz and little cores to 1.4 GHz) this looks like this:

root@nanopct4:/home/tk# armbianmonitor -z
Preparing benchmark. Be patient please...

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,6 CPUs LE)

LE
CPU Freq:  1413  1414  1414  1411  1413  1414  1414  1414  1415

RAM size:    3878 MB,  # CPU hardware threads:   6
RAM usage:   1323 MB,  # Benchmark threads:      6

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       3642   363    976   3543  |      98020   543   1540   8359
23:       3691   365   1030   3761  |      95217   541   1522   8239
24:       3606   354   1094   3878  |      92662   535   1520   8133
25:       4597   451   1164   5249  |      89079   529   1498   7928
----------------------------------  | ------------------------------
Avr:             383   1066   4108  |              537   1520   8165
Tot:             460   1293   6136

Monitoring output recorded while running the benchmark:

Time       big.LITTLE   load %cpu %sys %usr %nice %io %irq   CPU  C.St.
10:16:19: 1800/1416MHz  0.12  12%   1%   7%   2%   1%   0% 44.4°C  0/5
10:16:25:  408/ 600MHz  0.11   0%   0%   0%   0%   0%   0% 43.9°C  0/5
10:16:30:  600/1416MHz  0.10   1%   0%   0%   0%   0%   0% 45.0°C  0/5
10:16:35: 1800/1416MHz  0.17  40%   0%  39%   0%   0%   0% 49.4°C  0/5
10:16:40: 1800/1416MHz  0.32  77%   0%  77%   0%   0%   0% 55.0°C  0/5
10:16:45: 1800/1416MHz  0.94  73%   0%  72%   0%   0%   0% 51.1°C  0/5
10:16:50: 1800/1416MHz  0.94  65%   0%  65%   0%   0%   0% 53.3°C  0/5
10:16:55: 1800/1416MHz  1.19  68%   0%  67%   0%   0%   0% 56.1°C  0/5
10:17:00: 1800/1416MHz  1.49  79%   1%  78%   0%   0%   0% 53.9°C  0/5
10:17:06: 1800/1416MHz  1.45  31%   0%  31%   0%   0%   0% 57.8°C  0/5
10:17:11: 1800/1416MHz  2.07  68%   0%  67%   0%   0%   0% 57.2°C  0/5
10:17:17: 1800/1416MHz  2.30  78%   0%  77%   0%   0%   0% 58.9°C  0/5
10:17:22: 1800/1416MHz  2.52  90%   1%  89%   0%   0%   0% 57.8°C  0/5
10:17:27: 1800/1416MHz  2.72  81%   0%  80%   0%   0%   0% 57.2°C  0/5
Time       big.LITTLE   load %cpu %sys %usr %nice %io %irq   CPU  C.St.
10:17:32: 1800/1416MHz  2.66  61%   0%  60%   0%   0%   0% 60.6°C  0/5

We get an overall score of above 6100 and 7-zip's 'CPU Freq' line reports CPU0 (a little core) being clocked at 1.4 GHz. But since this is a big.LITTLE design we need the monitoring output that gets displayed below 7-zip benchmark numbers.

 

By looking at the 2nd line we see that the system was totally idle prior to starting the benchmark (I implemented a 10 second sleep between starting monitoring and firing up the benchmark for this reason -- to control whether the system was already busy or not). As a comparison 7-zip numbers of another RK3399 board that allowed the CPU cores to clock slightly higher (2.0/1.5 GHz): ODROID-N1 scored 6500.

 

As a reference some other boards.

 

Rock64 with new 1.4 GHz settings:

 

root@rock64:~# armbianmonitor -z
Preparing benchmark. Be patient please...

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs LE)

LE
CPU Freq:  1377  1380  1381  1381  1381  1374  1380  1381  1381

RAM size:    1927 MB,  # CPU hardware threads:   4
RAM usage:    882 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       2128   322    643   2071  |      60495   397   1298   5161
23:       2085   330    645   2125  |      58242   392   1286   5039
24:       2058   341    649   2213  |      56036   386   1276   4919
25:       2024   352    657   2311  |      55218   397   1237   4914
----------------------------------  | ------------------------------
Avr:             336    649   2180  |              393   1274   5009
Tot:             365    961   3594

Below Monitoring output recorded while running the benchmark:

Time        CPU    load %cpu %sys %usr %nice %io %irq   CPU  C.St.
09:57:52: 1392MHz  0.23   1%   0%   0%   0%   0%   0% 54.2°C  0/6
09:57:57:  600MHz  0.21   0%   0%   0%   0%   0%   0% 49.1°C  0/6
09:58:02: 1392MHz  0.19   2%   0%   1%   0%   0%   0% 54.6°C  0/6
09:58:08: 1392MHz  0.58  50%   1%  48%   0%   0%   0% 62.9°C  0/6
09:58:13: 1392MHz  0.85  93%   0%  92%   0%   0%   0% 68.1°C  0/6
09:58:18: 1392MHz  1.02  79%   2%  77%   0%   0%   0% 68.5°C  0/6
09:58:25: 1392MHz  1.26  92%   1%  90%   0%   0%   0% 73.3°C  0/6
09:58:31: 1392MHz  1.24  75%   1%  73%   0%   0%   0% 68.1°C  0/6
09:58:36: 1392MHz  1.66  86%   2%  83%   0%   0%   0% 66.9°C  0/6
09:58:41: 1392MHz  1.85  92%   2%  89%   0%   0%   0% 70.8°C  0/6
09:58:46: 1392MHz  1.78  88%   1%  86%   0%   0%   0% 68.1°C  0/6
09:58:51: 1392MHz  1.96  69%   2%  67%   0%   0%   0% 68.8°C  0/6
09:58:56: 1392MHz  2.04  83%   2%  81%   0%   0%   0% 68.1°C  0/6
09:59:02: 1392MHz  2.20  87%   2%  84%   0%   0%   0% 68.1°C  0/6
Time        CPU    load %cpu %sys %usr %nice %io %irq   CPU  C.St.
09:59:07: 1392MHz  2.42  90%   2%  88%   0%   0%   0% 71.7°C  0/6
09:59:12: 1392MHz  2.47  94%   2%  91%   0%   0%   0% 72.1°C  0/6
09:59:18: 1392MHz  2.75  97%   2%  94%   0%   0%   0% 73.8°C  0/6

 

 

 

NanoPi NEO with cpufreq scaling limited to 816 MHz to keep the board always at lowest DVFS voltage (results irrelevant)

 

root@backupsrv:/home/tk# armbianmonitor -z
Preparing benchmark. Be patient please...
Selecting previously unselected package p7zip.
(Reading database ... 50467 files and directories currently installed.)
Preparing to unpack .../p7zip_16.02+dfsg-3+deb9u1_armhf.deb ...
Unpacking p7zip (16.02+dfsg-3+deb9u1) ...
Setting up p7zip (16.02+dfsg-3+deb9u1) ...
Processing triggers for man-db (2.7.6.1-2) ...

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

LE
CPU Freq:   806   808   808   808   770   806   808   802

RAM size:     493 MB,  # CPU hardware threads:   4
RAM usage:    450 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1078   312    336   1050  |      29263   376    664   2497
23:       1071   321    340   1092  |      30503   398    663   2639
24:        720   228    339    775  |      29312   395    651   2573
----------------------------------  | ------------------------------
Avr:             287    339    972  |              390    659   2570
Tot:             339    499   1771

Monitoring output recorded while running the benchmark:

Time        CPU    load %cpu %sys %usr %nice %io %irq   CPU  C.St.
12:09:50:  816MHz  0.67   0%   0%   0%   0%   0%   0% 35.5°C  0/7
12:09:55:  816MHz  0.62   1%   0%   0%   0%   0%   0% 33.8°C  0/7
12:10:00:  816MHz  0.57   4%   0%   1%   0%   1%   0% 33.6°C  0/7
12:10:05:  816MHz  0.60  58%   2%  52%   0%   3%   0% 38.0°C  0/7
12:10:15:  816MHz  0.79  86%   2%  83%   0%   0%   0% 42.0°C  0/7
12:10:20:  816MHz  1.20  79%   2%  77%   0%   0%   0% 41.3°C  0/7
12:10:26:  816MHz  1.51  82%   2%  79%   0%   0%   0% 42.1°C  0/7
12:10:31:  816MHz  1.71  85%   3%  81%   0%   0%   0% 42.4°C  0/7
12:10:36:  816MHz  1.89  89%   1%  88%   0%   0%   0% 43.9°C  0/7
12:10:41:  816MHz  1.82  71%   2%  68%   0%   0%   0% 42.7°C  0/7
12:10:47:  816MHz  2.07  81%   3%  78%   0%   0%   0% 42.8°C  0/7
12:10:52:  816MHz  2.31  83%   3%  79%   0%   0%   0% 43.6°C  0/7
12:10:57:  816MHz  2.58  87%   4%  83%   0%   0%   0% 43.8°C  0/7
12:11:19:  816MHz  2.61  67%  28%  37%   0%   1%   0% 41.4°C  0/7
Time        CPU    load %cpu %sys %usr %nice %io %irq   CPU  C.St.
12:11:27:  816MHz  3.63  73%   6%  67%   0%   0%   0% 44.9°C  0/7

 

 

Please keep in mind that benchmarks that run fully multi threaded are NOT representative for most workloads running on computers (they're single threaded). Also please keep in mind that while 7-zip is not that much affected by different compiler settings (like the infamous sysbench) of course it is somewhat. So when you see 7-zip benchmark numbers generated few years ago when the 7z binary has been built with a GCC 4.x most probably with today's software and a binary built by GCC 7.x you see higher scores.

 

So take these comparison numbers with a grain of salt: https://s1.hoffart.de/7zip-bench/

 

To get new armbianmonitor with -z functionality today it's as easy as

wget -O /usr/bin/armbianmonitor https://raw.githubusercontent.com/armbian/build/master/packages/bsp/common/usr/bin/armbianmonitor

 

Share this post


Link to post
Share on other sites

PineH64 with Allwinner H6 using mainline kernel which has no working cpufreq/dvfs/thermal drivers, therefore clocking H6 with 912 MHz (results irrelevant):

 

root@pineh64:/home/tk# armbianmonitor -z
Preparing benchmark. Be patient please...

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs LE)

LE
CPU Freq:   910   908   909   910   910   910   910   910

RAM size:    2001 MB,  # CPU hardware threads:   4
RAM usage:    882 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1625   302    525   1582  |      39546   385    876   3374
23:       1607   308    531   1637  |      40696   398    885   3521
24:       1596   316    542   1716  |      40068   399    882   3517
25:       1583   326    555   1808  |      38533   397    864   3429
----------------------------------  | ------------------------------
Avr:             313    538   1686  |              395    877   3461
Tot:             354    708   2573

Monitoring output recorded while running the benchmark:

Time      CPU n/a    load %cpu %sys %usr %nice %io %irq
10:48:15:   ---      0.56  13%   1%   3%   0%   9%   0%
10:48:20:   ---      0.52   0%   0%   0%   0%   0%   0%
10:48:26:   ---      0.48   0%   0%   0%   0%   0%   0%
10:48:31:   ---      0.76  51%   0%  50%   0%   0%   0%
10:48:37:   ---      1.10  88%   1%  86%   0%   0%   0%
10:48:42:   ---      1.25  75%   1%  74%   0%   0%   0%
10:48:47:   ---      1.47  81%   0%  80%   0%   0%   0%
10:48:52:   ---      1.75  92%   0%  91%   0%   0%   0%
10:48:57:   ---      1.93  67%   1%  65%   0%   0%   0%
10:49:02:   ---      2.02  77%   1%  76%   0%   0%   0%
10:49:07:   ---      2.02  81%   1%  80%   0%   0%   0%
10:49:12:   ---      2.10  85%   1%  84%   0%   0%   0%
10:49:17:   ---      2.09  83%   1%  81%   0%   0%   0%
10:49:22:   ---      2.00  62%   1%  60%   0%   0%   0%
Time      CPU n/a    load %cpu %sys %usr %nice %io %irq
10:49:28:   ---      2.16  75%   1%  74%   0%   0%   0%
10:49:33:   ---      2.23  78%   1%  77%   0%   0%   0%
10:49:38:   ---      2.37  80%   1%  79%   0%   0%   0%
10:49:43:   ---      2.34  83%   1%  81%   0%   0%   0%
10:49:48:   ---      2.31  85%   1%  84%   0%   0%   0%
10:49:53:   ---      2.45  87%   1%  86%   0%   0%   0%
10:49:58:   ---      2.49  89%   1%  87%   0%   0%   0%
10:50:03:   ---      2.61  90%   2%  87%   0%   0%   0%

root@pineh64:/home/tk# uname -a
Linux pineh64 4.17.6-sunxi64 #113 SMP Thu Jul 12 23:54:08 UTC 2018 aarch64 GNU/Linux

 

 

PineH64 scores close to 2600 which as said already is irrelevant for the SoC's performance since with current mainline kernel the SoC runs at a fixed clockspeed reported by 7-zip as ~910 MHz -- it's 912 MHz). We know that the chip can clock up to 1.8 GHz (confirmed by @wtarreau's cool mhz tool) so once cpufreq/dvfs is working we see almost twice the numbers than these 2600. Why only 'almost twice'? Since the 7-zip benchmark depends also on memory bandwidth (like any real workload -- that's just another reason why sysbench sucks as CPU benchmark since sysbench does not depend on memory bandwidth at all).

 

Those numbers are made with 4 tasks stressing all 4 CPU cores in parallel. Most real world workloads look differently and are single threaded. So even if a H6 clocked at 1.8 GHz produces 7-zip scores above 5000 any big.LITTLE ARM SoC with a similar score will be way faster in reality since when single threaded loads are running on the big cores they perform much faster.

Share this post


Link to post
Share on other sites

NanoPi Fire3 with Samsung/Nexell S5P6818 which is an octa-core A53 clocked by default with 1.4 GHz (results irrelevant):

 

root@nanopifire3:~# armbianmonitor -z
Preparing benchmark. Be patient please...

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs LE)

LE
CPU Freq:  1396  1396  1397  1396  1396  1396  1396  1396  1396

RAM size:     990 MB,  # CPU hardware threads:   8
RAM usage:    901 MB,  # Benchmark threads:      8

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       4512   626    702   4390  |     121164   793   1303  10335
23:       4478   630    724   4564  |     112375   792   1228   9725
24:       4340   638    731   4667  |     108191   787   1206   9496
----------------------------------  | ------------------------------
Avr:             631    719   4540  |              791   1246   9852
Tot:             711    982   7196

Monitoring output recorded while running the benchmark:

Time       big.LITTLE   load %cpu %sys %usr %nice %io %irq   CPU  C.St.
11:05:59: 1400/1400MHz  0.31   1%   1%   0%   0%   0%   0% 66.0°C  0/10
11:06:04: 1400/1400MHz  0.28   1%   1%   0%   0%   0%   0% 66.0°C  0/10
11:06:09: 1400/1400MHz  0.34   1%   1%   0%   0%   0%   0% 66.0°C  0/10
11:06:14: 1400/1400MHz  0.39  39%   0%  38%   0%   0%   0% 73.0°C  0/10
11:06:20: 1400/1400MHz  1.00  92%   0%  92%   0%   0%   0% 79.0°C  0/10
11:06:25: 1400/1400MHz  1.32  71%   0%  70%   0%   0%   0% 77.0°C  0/10
11:06:32: 1300/1300MHz  1.70  90%   0%  89%   0%   0%   0% 80.0°C  1/10
11:06:37: 1400/1400MHz  2.42  64%   0%  63%   0%   0%   0% 79.0°C  0/10
11:06:42: 1300/1300MHz  2.87  80%   1%  79%   0%   0%   0% 79.0°C  0/10
11:06:48: 1300/1300MHz  3.36  88%   3%  84%   0%   0%   0% 80.0°C  1/10

root@nanopifire3:~# for in in 1 2 3 4 5 ; do
> armbianmonitor -z
> done
Preparing benchmark. Be patient please...

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs LE)

LE
CPU Freq:  1395  1395  1396  1396  1395  1396  1396  1396  1396

RAM size:     990 MB,  # CPU hardware threads:   8
RAM usage:    901 MB,  # Benchmark threads:      8

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       4534   626    705   4412  |     112720   786   1223   9615
23:       4253   621    698   4334  |     109786   787   1207   9501
24:       4210   630    718   4527  |     102781   789   1144   9021
----------------------------------  | ------------------------------
Avr:             626    707   4424  |              787   1191   9379
Tot:             707    949   6902

Monitoring output recorded while running the benchmark:

Time       big.LITTLE   load %cpu %sys %usr %nice %io %irq   CPU  C.St.
11:07:51: 1400/1400MHz  1.55   5%   1%   3%   0%   0%   0% 71.0°C  0/10
11:07:56:  400/1400MHz  1.43   1%   1%   0%   0%   0%   0% 71.0°C  0/10
11:08:01: 1400/1400MHz  1.31   1%   1%   0%   0%   0%   0% 71.0°C  0/10
11:08:06: 1400/1400MHz  1.29  39%   0%  38%   0%   0%   0% 77.0°C  0/10
11:08:12: 1300/1300MHz  1.74  93%   0%  92%   0%   0%   0% 82.0°C  1/10
11:08:17: 1300/1300MHz  2.62  69%   0%  68%   0%   0%   0% 80.0°C  1/10
11:08:24: 1300/1300MHz  3.05  87%   0%  87%   0%   0%   0% 84.0°C  1/10
11:08:29: 1300/1300MHz  3.21  67%   0%  66%   0%   0%   0% 81.0°C  1/10
11:08:34: 1300/1300MHz  3.67  78%   1%  77%   0%   0%   0% 82.0°C  1/10
11:08:40: 1300/1300MHz  3.86  83%   1%  82%   0%   0%   0% 83.0°C  1/10

Preparing benchmark. Be patient please...

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs LE)

LE
CPU Freq:  1087  1397  1396  1397  1396  1396  1396  1397  1397

RAM size:     990 MB,  # CPU hardware threads:   8
RAM usage:    901 MB,  # Benchmark threads:      8

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       4316   617    680   4199  |     106923   793   1151   9120
23:       4174   614    693   4253  |      96915   789   1063   8387
24:       4086   628    700   4393  |      91520   774   1038   8033
----------------------------------  | ------------------------------
Avr:             620    691   4282  |              785   1084   8513
Tot:             702    887   6397

Monitoring output recorded while running the benchmark:

Time       big.LITTLE   load %cpu %sys %usr %nice %io %irq   CPU  C.St.
11:08:46: 1300/1300MHz  3.87   8%   1%   7%   0%   0%   0% 82.0°C  1/10
11:08:51: 1400/1400MHz  3.56   1%   1%   0%   0%   0%   0% 76.0°C  0/10
11:08:56: 1400/1400MHz  3.27   1%   1%   0%   0%   0%   0% 76.0°C  0/10
11:09:01: 1300/1300MHz  3.09  39%   0%  38%   0%   0%   0% 81.0°C  1/10
11:09:07: 1200/1200MHz  3.24  92%   0%  92%   0%   0%   0% 85.0°C  2/10
11:09:13: 1300/1300MHz  3.89  69%   1%  68%   0%   0%   0% 84.0°C  1/10
11:09:20: 1100/1100MHz  4.22  86%   0%  85%   0%   0%   0% 85.0°C  3/10
11:09:26: 1200/1200MHz  3.96  69%   0%  69%   0%   0%   0% 84.0°C  2/10
11:09:31: 1200/1200MHz  4.20  77%   1%  76%   0%   0%   0% 85.0°C  2/10
11:09:36: 1200/1200MHz  4.43  82%   0%  81%   0%   0%   0% 85.0°C  2/10
11:09:42: 1100/1100MHz  4.72  91%   0%  90%   0%   0%   0% 86.0°C  3/10

Preparing benchmark. Be patient please...

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs LE)

LE
CPU Freq:  1394  1397  1396  1397  1395  1396  1396  1397  1396

RAM size:     990 MB,  # CPU hardware threads:   8
RAM usage:    901 MB,  # Benchmark threads:      8

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       4243   617    669   4128  |      91915   795    986   7840
23:       3888   614    646   3962  |      86758   782    960   7508
24:       3760   619    653   4043  |      83425   777    942   7322
----------------------------------  | ------------------------------
Avr:             617    656   4044  |              785    963   7557
Tot:             701    809   5801

Monitoring output recorded while running the benchmark:

Time       big.LITTLE   load %cpu %sys %usr %nice %io %irq   CPU  C.St.
11:09:43: 1100/1100MHz  5.06  11%   1%  10%   0%   0%   0% 84.0°C  2/10
11:09:48: 1400/1400MHz  4.65   1%   1%   0%   0%   0%   0% 79.0°C  1/10
11:09:53: 1400/1400MHz  4.28   1%   0%   0%   0%   0%   0% 79.0°C  0/10
11:09:58: 1300/1300MHz  4.58  38%   0%  38%   0%   0%   0% 84.0°C  1/10
11:10:05: 1000/1000MHz  4.85  92%   0%  91%   0%   0%   0% 85.0°C  4/10
11:10:10: 1100/1100MHz  4.95  72%   0%  71%   0%   0%   0% 85.0°C  3/10
11:10:17: 1000/1000MHz  5.27  84%   0%  84%   0%   0%   0% 86.0°C  4/10
11:10:23: 1200/1200MHz  5.92  72%   0%  71%   0%   0%   0% 85.0°C  2/10
11:10:29: 1100/1100MHz  5.85  75%   0%  74%   0%   0%   0% 85.0°C  3/10
11:10:34: 1100/1100MHz  5.94  78%   0%  78%   0%   0%   0% 85.0°C  3/10
11:10:40: 1000/1000MHz  6.11  82%   1%  80%   0%   0%   0% 86.0°C  4/10

Preparing benchmark. Be patient please...

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs LE)

LE
CPU Freq:  1295  1297  1296  1296  1295  1296  1296  1297  1297

RAM size:     990 MB,  # CPU hardware threads:   8
RAM usage:    901 MB,  # Benchmark threads:      8

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       4095   609    654   3984  |      87864   786    953   7494
23:       3486   599    593   3552  |      86797   789    952   7511
24:       3523   610    621   3788  |      84785   787    946   7441
----------------------------------  | ------------------------------
Avr:             606    623   3775  |              787    950   7482
Tot:             697    787   5629

Monitoring output recorded while running the benchmark:

Time       big.LITTLE   load %cpu %sys %usr %nice %io %irq   CPU  C.St.
11:10:44: 1000/1000MHz  6.26  14%   1%  13%   0%   0%   0% 85.0°C  4/10
11:10:49: 1300/1300MHz  5.76   1%   1%   0%   0%   0%   0% 80.0°C  1/10
11:10:54: 1300/1300MHz  5.30   1%   1%   0%   0%   0%   0% 80.0°C  1/10
11:10:59: 1200/1200MHz  4.95  35%   0%  34%   0%   0%   0% 85.0°C  2/10
11:11:06: 1000/1000MHz  5.20  90%   0%  89%   0%   0%   0% 87.0°C  4/10
11:11:12: 1000/1000MHz  5.62  70%   0%  70%   0%   0%   0% 85.0°C  4/10
11:11:17: 1000/1000MHz  5.49  76%   0%  76%   0%   0%   0% 85.0°C  4/10
11:11:22: 1000/1000MHz  5.69  97%   0%  97%   0%   0%   0% 87.0°C  4/10
11:11:27: 1100/1100MHz  5.56  54%   1%  53%   0%   0%   0% 85.0°C  3/10
11:11:32: 1000/1000MHz  5.59  74%   0%  73%   0%   0%   0% 85.0°C  4/10
11:11:37: 1000/1000MHz  5.70  78%   0%  77%   0%   0%   0% 85.0°C  4/10
11:11:43: 1000/1000MHz  5.89  81%   1%  80%   0%   0%   0% 86.0°C  4/10

Preparing benchmark. Be patient please...

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs LE)

LE
CPU Freq:  1047  1296  1295  1295  1286  1296  1296  1296  1297

RAM size:     990 MB,  # CPU hardware threads:   8
RAM usage:    901 MB,  # Benchmark threads:      8

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       3965   608    635   3858  |      88320   790    953   7533
23:       3498   602    592   3564  |      86291   789    947   7467
24:       3452   604    614   3713  |      85275   792    946   7485
----------------------------------  | ------------------------------
Avr:             605    614   3712  |              790    949   7495
Tot:             697    781   5603

Monitoring output recorded while running the benchmark:

Time       big.LITTLE   load %cpu %sys %usr %nice %io %irq   CPU  C.St.
11:11:47: 1000/1000MHz  6.06  17%   1%  15%   0%   0%   0% 86.0°C  4/10
11:11:52: 1300/1300MHz  5.57   1%   1%   0%   0%   0%   0% 81.0°C  1/10
11:11:57: 1300/1300MHz  5.13   1%   1%   0%   0%   0%   0% 80.0°C  1/10
11:12:02: 1200/1200MHz  5.12  35%   0%  34%   0%   0%   0% 85.0°C  2/10
11:12:10: 1000/1000MHz  5.35  90%   0%  89%   0%   0%   0% 87.0°C  4/10
11:12:15: 1000/1000MHz  5.48  70%   0%  70%   0%   0%   0% 85.0°C  4/10
11:12:20: 1000/1000MHz  5.44  76%   0%  75%   0%   0%   0% 85.0°C  4/10
11:12:26: 1000/1000MHz  5.65  98%   0%  98%   0%   0%   0% 88.0°C  4/10
11:12:31: 1000/1000MHz  5.27  54%   0%  53%   0%   0%   0% 85.0°C  4/10
11:12:36: 1000/1000MHz  5.25  74%   1%  73%   0%   0%   0% 86.0°C  4/10
11:12:41: 1000/1000MHz  5.31  77%   0%  77%   0%   0%   0% 86.0°C  4/10
11:12:46: 1000/1000MHz  5.61  79%   1%  77%   0%   0%   0% 86.0°C  4/10

 

 

Why are results irrelevant? For two reasons:

  • Throttling occured. The board with vendor's standard heatsink starts to overheat badly when running demanding loads. Active cooling is needed and that's why monitoring when running benchmarks is that important!
  • This is an octa-core Cortex-A53 SoC showing with this benchmark a score of well above 7000 when no throttling happens. Once again: such multithreaded results are BS wrt most real world workloads. An RK3399 board like an ODROID-N1 scoring 'just' 6500 will be the faster performing board with almost all usual workloads since equipped with 2 fast A72 cores while the Fire3 only has 8 slow A53 cores. Most workloads do not scale linearly with count of CPU cores. This has to be taken into account.

Share this post


Link to post
Share on other sites

Olimex Lime2 (dual core Allwinner A20) capped to 912 MHz with slight background activity:

 

root@lime2:~# armbianmonitor -z
Preparing benchmark. Be patient please...

7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs)

RAM size:    1000 MB,  # CPU hardware threads:   2
RAM usage:    425 MB,  # Benchmark threads:      2

Dict        Compressing          |        Decompressing
      Speed Usage    R/U Rating  |    Speed Usage    R/U Rating
       KB/s     %   MIPS   MIPS  |     KB/s     %   MIPS   MIPS

22:     526   154    333    512  |    13531   196    624   1221
23:     519   156    338    529  |    13339   196    623   1221
24:     502   157    344    539  |    12996   194    621   1206
25:     506   163    355    578  |    12783   195    617   1202
----------------------------------------------------------------
Avr:          157    343    539               195    621   1212
Tot:          176    482    876

Monitoring output recorded while running the benchmark:

Time        CPU    load %cpu %sys %usr %nice %io %irq   CPU   PMIC   DC-IN  C.St.
17:38:04:  720MHz  0.00   1%   0%   0%   0%   0%   0% 37.6°C 37.6°C   5.12V  0/6
17:38:09:  528MHz  0.00   2%   1%   0%   0%   0%   0% 38.0°C 37.7°C   5.13V  0/6
17:38:15:  912MHz  0.00  19%   3%  16%   0%   0%   0% 39.0°C 38.2°C   5.08V  0/6
17:38:20:  912MHz  0.08  82%   4%  76%   0%   0%   0% 39.8°C 38.5°C   5.07V  0/6
17:38:29:  912MHz  0.23  94%   1%  92%   0%   0%   0% 41.0°C 39.2°C   5.09V  0/6
17:38:37:  912MHz  0.66  86%   2%  84%   0%   0%   0% 40.7°C 39.3°C   5.08V  0/6
17:38:43:  912MHz  0.77  82%   4%  78%   0%   0%   0% 41.0°C 39.3°C   5.08V  0/6
17:38:48:  912MHz  0.88  82%   4%  77%   0%   0%   0% 41.3°C 39.4°C   5.06V  0/6
17:38:59:  912MHz  0.97  99%   0%  98%   0%   0%   0% 41.4°C 40.1°C   5.09V  0/6
17:39:05:  912MHz  1.20  86%  10%  75%   0%   0%   0% 41.6°C 40.1°C   5.08V  0/6
17:39:10:  912MHz  1.18  78%   3%  74%   0%   0%   0% 41.4°C 40.1°C   5.08V  0/6
17:39:16:  912MHz  1.25  81%   4%  77%   0%   0%   0% 41.8°C 40.2°C   5.08V  0/6
17:39:22:  912MHz  1.23  85%   4%  79%   0%   0%   0% 42.0°C 40.6°C   5.07V  0/6
17:39:28:  912MHz  1.37  88%   6%  81%   0%   0%   0% 41.9°C 40.5°C   5.08V  0/6
Time        CPU    load %cpu %sys %usr %nice %io %irq   CPU   PMIC   DC-IN  C.St.
17:39:37:  912MHz  1.47  89%   4%  83%   0%   0%   0% 42.2°C 41.4°C   5.06V  0/6
17:39:47:  912MHz  1.62  94%   0%  93%   0%   0%   0% 41.9°C 41.1°C   5.09V  0/6
17:39:53:  912MHz  1.65  74%   4%  69%   0%   0%   0% 42.2°C 41.3°C   5.08V  0/6
17:39:58:  912MHz  1.60  77%   3%  73%   0%   0%   0% 42.0°C 41.3°C   5.07V  0/6
17:40:05:  912MHz  1.89  92%   5%  74%  12%   0%   0% 42.4°C 41.5°C   5.07V  0/6
17:40:11:  912MHz  1.98  83%   5%  76%   2%   0%   0% 42.6°C 41.6°C   5.08V  0/6
17:40:17:  912MHz  1.98  84%   5%  78%   0%   0%   0% 42.3°C 41.7°C   5.07V  0/6
17:40:23:  912MHz  2.06  85%   4%  80%   0%   0%   0% 42.5°C 41.7°C   5.07V  0/6
17:40:29:  912MHz  1.97  87%   5%  81%   0%   0%   0% 42.4°C 42.1°C   5.07V  0/6
17:40:35:  912MHz  1.89  87%   5%  82%   0%   0%   0% 42.6°C 41.8°C   5.07V  0/6
17:40:41:  912MHz  1.98  90%   4%  85%   0%   0%   0% 42.4°C 41.9°C   5.07V  0/6
17:40:47:  912MHz  1.99  91%   5%  86%   0%   0%   0% 42.8°C 42.3°C   5.07V  0/6
17:40:57:  912MHz  1.99  90%   4%  85%   0%   0%   0% 43.3°C 42.5°C   5.07V  0/6
17:41:08:  912MHz  2.13 100%   1%  98%   0%   0%   0% 43.7°C 42.8°C

 

 

NanoPi K1 Plus (quad core Allwinner H5) capped to 816 MHz due to missing cpufreq/dvfs/thermal support in mainline kernel:

 

root@nanopik1plus:~# armbianmonitor -z

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs LE)

LE
CPU Freq:   807   806   804   807   807   807   807   807

RAM size:    2002 MB,  # CPU hardware threads:   4
RAM usage:    882 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1369   310    430   1332  |      35956   395    777   3068
23:       1364   321    433   1390  |      35606   398    774   3081
24:       1346   328    441   1448  |      34967   399    770   3070
25:       1332   339    449   1522  |      32824   390    749   2921
----------------------------------  | ------------------------------
Avr:             325    438   1423  |              395    767   3035
Tot:             360    603   2229

Monitoring output recorded while running the benchmark:

Time      CPU n/a    load %cpu %sys %usr %nice %io %irq
16:02:15:   ---      0.19   6%   1%   3%   0%   0%   0%
16:02:20:   ---      0.17   0%   0%   0%   0%   0%   0%
16:02:25:   ---      0.16   1%   0%   0%   0%   0%   0%
16:02:30:   ---      0.23  53%   1%  48%   0%   3%   0%
16:02:38:   ---      0.61  86%   1%  84%   0%   0%   0%
16:02:43:   ---      1.20  80%   1%  78%   0%   0%   0%
16:02:48:   ---      1.35  83%   1%  81%   0%   0%   0%
16:02:55:   ---      1.56  91%   1%  90%   0%   0%   0%
16:03:00:   ---      1.92  66%   1%  64%   0%   0%   0%
16:03:05:   ---      2.08  78%   1%  77%   0%   0%   0%
16:03:10:   ---      2.16  83%   1%  81%   0%   0%   0%
16:03:15:   ---      2.22  87%   1%  85%   0%   0%   0%
16:03:20:   ---      2.29  85%   1%  82%   0%   0%   0%
16:03:25:   ---      2.42  80%   0%  79%   0%   0%   0%
Time      CPU n/a    load %cpu %sys %usr %nice %io %irq
16:03:30:   ---      2.31  68%   2%  65%   0%   0%   0%
16:03:35:   ---      2.37  77%   1%  76%   0%   0%   0%
16:03:40:   ---      2.58  80%   1%  79%   0%   0%   0%
16:03:45:   ---      2.53  82%   1%  81%   0%   0%   0%
16:03:51:   ---      2.65  85%   1%  83%   0%   0%   0%
16:03:56:   ---      2.68  87%   1%  85%   0%   0%   0%
16:04:01:   ---      2.70  89%   1%  87%   0%   0%   0%
16:04:06:   ---      2.81  91%   1%  89%   0%   0%   0%
16:04:11:   ---      2.98  93%   1%  91%   0%   0%   0%
16:04:19:   ---      3.07  89%   2%  87%   0%   0%   0%

 

 

NanoPi NEO (quad core Allwinner H3) tested again this time with 1.1 GHz:

 

root@backupsrv:/home/tk# armbianmonitor -z
Preparing benchmark. Be patient please...

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

LE
CPU Freq:  1093  1094  1094  1094  1038  1094  1094  1094  1094

RAM size:     493 MB,  # CPU hardware threads:   4
RAM usage:    450 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1338   328    397   1302  |      40214   391    878   3431
23:       1316   335    401   1342  |      39936   397    871   3456
24:       1082   306    381   1164  |      20888   212    866   1834
----------------------------------  | ------------------------------
Avr:             323    393   1269  |              333    872   2907
Tot:             328    632   2088

Monitoring output recorded while running the benchmark:

Time        CPU    load %cpu %sys %usr %nice %io %irq   CPU  C.St.
17:39:56: 1104MHz  0.04   5%   1%   3%   0%   0%   0% 33.3°C  0/7
17:40:01: 1104MHz  0.04   0%   0%   0%   0%   0%   0% 33.8°C  0/7
17:40:07: 1104MHz  0.03   2%   0%   1%   0%   0%   0% 33.5°C  0/7
17:40:12: 1104MHz  0.11  41%   1%  40%   0%   0%   0% 37.0°C  0/7
17:40:20: 1104MHz  0.26  88%   1%  86%   0%   0%   0% 44.9°C  0/7
17:40:27: 1104MHz  0.99  84%   1%  83%   0%   0%   0% 43.8°C  0/7
17:40:32: 1104MHz  1.23  89%   3%  86%   0%   0%   0% 44.9°C  0/7
17:40:39: 1104MHz  1.81  94%   2%  91%   0%   0%   0% 47.6°C  0/7
17:40:45: 1104MHz  1.91  73%   3%  70%   0%   0%   0% 45.6°C  0/7
17:40:50: 1104MHz  2.16  87%   4%  82%   0%   0%   0% 45.9°C  0/7
17:40:56: 1104MHz  2.46  91%   5%  85%   0%   0%   0% 46.7°C  0/7
17:41:09: 1104MHz  2.67  97%  25%  62%   0%   8%   0% 46.9°C  0/7
17:41:16: 1104MHz  3.87  94%  28%  63%   0%   1%   0% 49.5°C  0/7

 

Share this post


Link to post
Share on other sites

So now a quick overview of the relevant SoC families with Armbian -- results in 7-zip-MIPS:

     A20 @ 912 MHz:  ~900
    A64 @ 1152 MHz: ~2550
     H3 @ 1100 MHz: ~2100
      H5 @ 816 MHz: ~2200
      H6 @ 912 MHz: ~2550
  i.MX6 @ 1000 MHz: ~2100
 RK3288 @ 1800 MHz: ~5400
 RK3328 @ 1392 MHz: ~3600   
S5P6818 @ 1400 MHz: ~7200
   S905 @ 1500 MHz: ~3700

(for A20, H3, H5, H6, RK3328 and S5P6818 results see above, for A64 see here and there, for i.MX6 see here (search for my Wandboard), for RK3288 see MiQi results here and for S905 see ODROID-C2 numbers there)

 

All the SoCs above are quad-core except A20 (dual) and S5P6818 (octa). And it's all about the type of CPU cores: A20 and H3 are Cortex-A7, A64/H5/H6/RK3328/S5P6818/S905 are Cortex-A53, i.MX6 is Cortex-A9 and RK3288 is Cortex-A17. So let's look at all the results, take count of cores into account and MHz also. The following is a table of 7-zip-MIPS per single core at 1GHz clockspeed:

 A7: 475
 A9: 525
A53: 625
A15: 700
A17: 750
A72: 850

(yeah, A15 and A72 also exist -- see below).

 

So that's roughly what you can expect from each individual Cortex core running at 1 GHz. As expected if a SoC contains more cores specific workloads that benefit from parallel code execution get faster (once again: most workloads are single-threaded!). Also as expected clockspeeds matter: if you buy an H3 or H5 board without voltage regulation limiting the maximum clockspeed then obviously this board will perform slower compared to another H3/H5 board with sophisticated voltage regulation allowing the CPU cores to clock much higher.

 

What also matters with this benchmark and most if not all real world workloads: memory bandwidth. Boards with just a 16-bit memory interface are slower than those with 32- or even 64-bit memory interfaces (something that the incapable sysbench pseudo cpu test is not able to report since whole execution happens inside the CPU cores). Boards that use 'better' DRAM (DDR4 vs DDR3) can be faster as long as available software/settings are available (and that's often not the case -- for example we're still waiting for Rockchip releasing new BLOBs with faster DRAM initialization for (L)PPDR4 equipped Rockchip boards).

 

Speaking about software/settings it should also be obvious that in the meantime we always also have to take care about heat dissipation of ARM SoCs used today. Heat dissipation is an issue to prevent damaging the SoCs due to overheating under load. But without fully functioning cpufreq/dvfs/thermal drivers we can not allow the CPU cores to clock at their upper limits since we need working throttling to protect the chips. And that's why the results for Allwinner H5 and H6 boards look that bad: since linux-sunxi community still is working on upstreaming driver support and/or we at Armbian have not incorporated latest patches flying around into the build system. Once cpufreq/dvfs/thermal is ready for those newer Allwinner SoCs H5 boards will get 1.5 as fast and H6 boards almost twice as fast as today.

 

Software/settings matter. Always. That's why it's so disappointing to see all those benchmark numbers flying around not taking this into account.

 

What about Cortex-A15 and A72? When we look at boards Armbian supports we find those cores in SoCs implementing big.LITTLE: ODROID XU4/HC1/HC2 use Exynos 5442 which consists of 4 fast A15 and 4 slow A7 cores (32-bit ARMv7). Boards based on RK3399 have 2 fast A72 cores and 4 slow A53 cores (64-bit ARMv8)

 

Does it make sense to run the very same 7-zip benchmark on those big.LITTLE designs? Not that much since we can not easily draw any conclusion for normal workloads from such a benchmark number. For example when executing '7z b' on all 6 cores of an ODROID-N1 at the same time we get an overall score of ~6550 7-zip MIPS. When limiting benchmark execution to only the 2 fast A72 cores at 2 GHz we get ~3350 (that's ~1700 7-zip MIPS per core), when we execute the benchmark on the 4 little cores only (1.5GHz) then it's ~3900 (~975 7-zip MIPS per core). A single threaded task running on one of the two big cores will perform almost twice as fast compared to running on a little core. That's important to keep in mind since based on the workload running in reality some of the benchmark numbers are simply misleading or just... numbers without meaning.

 

Same with ODROID-XU4/HC1/HC2: when running on the A15 big cores ~4950 7-zip MIPS are reported at around 1.8GHz (~1250 7-zip MIPS per big core), when running only on the little A7 cores at 1.4 GHz it's ~2725 (~675 per core). Same situation as with RK3399: single threaded stuff moved to the big cores performs almost twice as fast as on the little cores. I never measured 7-zip running on all cores together since 'numbers without meaning' but I would assume we get something similar as with RK3399: not the addition of big+little numbers (3350+3900=7250 vs. 6550 in reality) but something lower since all cores have to fight for memory bandwidth.

 

Other things to keep in mind:

  • When looking at the above benchmark numbers we see A53 cores performing with this specific benchmark 30% better compared to A7 cores at the same clockspeed (so there's a slight advantage ARMv8/64-bit has over ARMv7/32-bit). But as soon as we use other software/benchmarks that make heavy use of NEON optimizations we usually see a performance increase much higher (A53 usually performing twice as fast as A7 -- can be easily checked with cpuminer). So as always it depends on the use case.
  • Speaking of 'use case' we should also keep in mind that all those ARM SoCs have special engines for this and that. Almost all ARMv8/64-bit SoCs for example contain a cryptographic acceleration engine called 'ARMv8 Crypto Extensions' that make a massive difference with AES for example compared to 32-bit/ARMv7 SoCs that have to do crypto stuff on the CPU cores (see here for numbers). So again: it's about the use case: if you're interested in VPN stuff or disk encryption looking at generic CPU benchmarks is BS since you want an ARMv8 SoC with crypto support (almost all have, the only exceptions are RPi 3/3+, ODROID-C2 and NanoPi K2)
  • CPU performance with many use cases isn't that important. With Marvell based boards (EspressoBin, Clearfogs, Helios4) CPU benchmarks look rather low but these SoCs are designed for highest I/O and networking throughput and even if the SoC in question scores low in CPU benchmarks those boards outperform everything else if it's about fast storage and network

 

Share this post


Link to post
Share on other sites

Clearfog Pro based on Marvell Armada 388 SoC (dual core A9):

 

root@clearfogpro:~# armbianmonitor -z
Preparing benchmark. Be patient please...

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,32 bits,2 CPUs LE)

LE
CPU Freq:   793  1372  1559  1521  1584  1590  1584  1587  1586

RAM size:    1003 MB,  # CPU hardware threads:   2
RAM usage:    441 MB,  # Benchmark threads:      2

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1455   155    912   1416  |      33604   199   1444   2869
23:       1449   158    934   1476  |      33042   199   1439   2860
24:       1436   161    961   1545  |      32479   200   1428   2851
25:       1418   163    994   1619  |      31054   197   1406   2764
----------------------------------  | ------------------------------
Avr:             159    950   1514  |              198   1429   2836
Tot:             179   1190   2175

Monitoring output recorded while running the benchmark:

Time        CPU    load %cpu %sys %usr %nice %io %irq   CPU
11:32:38: 1332MHz  0.00   2%   1%   1%   0%   0%   0% 70.4°C
11:32:43:  666MHz  0.00   0%   0%   0%   0%   0%   0% 69.9°C
11:32:48: 1332MHz  0.00   2%   0%   1%   0%   0%   0% 70.8°C
11:32:53: 1332MHz  0.08  64%   2%  62%   0%   0%   0% 77.5°C
11:32:58: 1332MHz  0.24  94%   1%  92%   0%   0%   0% 77.0°C
11:33:04: 1332MHz  0.46  80%   3%  77%   0%   0%   0% 77.0°C
11:33:09: 1332MHz  0.58  92%   2%  89%   0%   0%   0% 78.0°C
11:33:14: 1332MHz  0.62  78%   3%  75%   0%   0%   0% 78.5°C
11:33:19: 1332MHz  0.81  85%   2%  82%   0%   0%   0% 78.5°C
11:33:24: 1332MHz  0.90  92%   1%  90%   0%   0%   0% 76.5°C
11:33:29: 1332MHz  1.07  74%   4%  70%   0%   0%   0% 77.5°C
11:33:34: 1332MHz  1.23  79%   2%  76%   0%   0%   0% 78.9°C
11:33:39: 1332MHz  1.37  83%   2%  80%   0%   0%   0% 78.5°C
11:33:44: 1332MHz  1.42  86%   2%  83%   0%   0%   0% 78.9°C
Time        CPU    load %cpu %sys %usr %nice %io %irq   CPU
11:33:51: 1332MHz  1.47  91%   3%  88%   0%   0%   0% 79.4°C

 

 

Results need an explanation since

  • 7-zip reports the board running at 1600 MHz ('CPU Freq:   793  1372  1559  1521  1584  1590  1584  1587  1586') while cpufreq driver tells us switching between 666MHz and 1332MHz
  • Even if the CPU cores were running at 1600 MHz the numbers are still too high compared to the i.MX6 numbers generated above with a Wandboard (i.MX6 is also Cortex-A9). So maybe someone with one of the i.MX6 boards supported by Armbian can provide recent results with 'armbianmonitor -z'?

 

Edit: Trying to confirm clockspeeds using @wtarreau's mhz tool (git clone http://git.1wt.eu/git/mhz.git/ ; cd mhz ; make ; ./mhz):

root@clearfogpro:~/mhz# cpufreq-set -g performance
root@clearfogpro:~/mhz# ./mhz 
count=645643 us50=20196 us250=100998 diff=80802 cpu_MHz=1598.087
root@clearfogpro:~/mhz# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq 
1332000

So for whatever reasons we have a nice mismatch between clockspeeds reported via sysfs and real clockspeeds with Armada 38x :)

 

BTW: there is more stuff that needs an explanation, for example behaviour with Amlogic S912: https://forum.khadas.com/t/cpu-frequency-up-to-2ghz/2010/20?u=tkaiser

 

Share this post


Link to post
Share on other sites

Is it worth to move this thread to the research guides & tutorials section? There's IMO a bit to often a 'results irrelevant' (most likely cause DVFS doesn't work on those boards?) inside, but can we summarize this work to make something relevant out of it? 'CPU' benchmarking seems to be something people are interested in..  I would like to see a nice 'research guide' out of it. Maybe with a short example why multicore numbers are often 'worthless' in real world scenarios and how to properly interpret those 'numbers'.

 

Share this post


Link to post
Share on other sites
11 minutes ago, chwe said:

Is it worth to move this thread to the research guides & tutorials section?

 

That was my intention (to split everything starting from post #5 above to a new thread I'm preparing an intro post for within the next 2 weeks). Top post will then focus on an overview about CPU performance capabilities of different boards and when (or with which use cases) this matters.

 

No need to hurry :) 

Share this post


Link to post
Share on other sites
2 hours ago, tkaiser said:

 


root@clearfogpro:~/mhz# cpufreq-set -g performance
root@clearfogpro:~/mhz# ./mhz 
count=645643 us50=20196 us250=100998 diff=80802 cpu_MHz=1598.087
root@clearfogpro:~/mhz# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq 
1332000

So for whatever reasons we have a nice mismatch between clockspeeds reported via sysfs and real clockspeeds with Armada 38x :)

Please note that the operating points is usually fed via the DT while the operating frequency is defined by the jumpers on the board. It's very possible that the DT doesn't reference the correct frequencies here. From what I've apparently seen till now, the Armada 38x has limited ability to do frequency scaling, something like full speed or half speed possibly. When I was running mine at 1.6 GHz, I remember seeing only 1600 or 800 being effectively used. I didn't check since I upgraded to 2 GHz (well 1.992 to be precise) but I suspect I'm now doing either 2000 or 1000 and nothing else. Thus if you have a smaller number of operating points it would be possible that they are incorrectly mapped. Just my two cents :-)

Share this post


Link to post
Share on other sites
7 minutes ago, tkaiser said:

No need to hurry :) 

To late...  :P I think it makes more sense to split it yet otherwise it will be a hard nut to distinguish between what should be part of the research article and what's the left over from the original thread (this splitting often leads in headache for the one who has to do it). Feel free to change the title as soon as you think it's ready. :) 

Share this post


Link to post
Share on other sites
0