5 5
pbies

Benchmarking CPUs

Recommended Posts

I've run some sysbench (v0.4.12) tests. More will come later as I will get Raspberry Pi 3 Model B+ and Orange Pi RK3399.

 

Banana Pi = ARMv71 A20 @ 960 MHz (not sure about current clock, not overclocked)

 

Banana Pi Armbian (2 cores, 2 threads) - 1 thread - 1057 events in 60 seconds, which gives 17.62 events per second

Banana Pi Armbian (2 cores, 2 threads) - 2 threads - 2003 events in 60 seconds, which gives 33.38 events per second

i7-8700K @ 4.7 GHz VM with Mint 18.3 x64 in VirtualBox on Windows 10 (4 cores, 4 threads) - 1 thread - 10000 events in 4 seconds, which gives 2500 events per second

i7-8700K @ 4.7 GHz Mint 18.3 x64 stand-alone (6 cores, 12 threads) - 12 threads - 10000 events in 1.47 second, which gives 6802 events per second

 

For the above and only CPU - i7-8700K is from 75 to 386 times faster than Banana Pi CPU.

Share this post


Link to post
Share on other sites
1 hour ago, pbies said:

sysbench

...is crap. It's not a CPU but only a compiler benchmark. You chose the worst 'benchmark' possible.

 

Sysbench numbers change with compiler version and settings or even with sysbench version (higher version number --> lower scores). There's no 'benchmark' known producing more unreliable results wrt hardware/CPU. Use 'Google site search' here to get the details.

 

If it's about a quick and rough CPU performance estimate I would recommend 7-zip's benchmark mode (7z b). 

Share this post


Link to post
Share on other sites

@tkaiser I don't understand your negativity. Banana Pi's are bad, sysbench is bad... The whole world is bad, isn't it?

 

I'm sorry but I won't be arguing with you.

Share this post


Link to post
Share on other sites

For people not rejecting reality... again why sysbench is unrealiable (not able to indicate CPU performance AT ALL).

For people loving to 'shoot the messenger'... it's not only me who describes sysbench as the totally wrong tool, e.g. https://www.raspberrypi.org/forums/viewtopic.php?t=208314&start=25

 

Again: 7-zip's benchmark mode is not just using an insanely limited synthetic benchmark routine like sysbench (calculating prime numbers only involving CPU caches but no memory access), 7-zip is not dependent on compiler versions or platform switches and 7-zip allows for cross-platform comparisons. You'll find a lot of numbers here in the forum and some comparisons on the net e.g. https://s1.hoffart.de/7zip-bench/ (again: it's just a rough estimate but at least something somewhat reliable related to CPU performance)

 

The most important thing with benchmarking is 'benchmarking the benchmark'. Since most tools (especially the popular ones) do not do what they pretend to do. Active benchmarking is needed and not just throwing out numbers without meaning.

 

BTW: sysbench is part of MySQL and when used correctly a great tool to provide insights. It's just the 'cpu test' that is not a CPU test at all. And it's about people firing up sysbench in 'passive benchmarking' mode generating numbers without meaning and praising insufficient tools.

Share this post


Link to post
Share on other sites

The command I have used (sysbench v0.4.12-1.2):

sysbench --test=cpu --cpu-max-prime=20000 --num-threads=1 --max-time=60 run

For Raspberry Pi 3 model B+ the results are:

(4 cores, 4 threads CPU) - 1 thread - 1619 events in 60 seconds, which gives 26.98 events per second

(4 cores, 4 threads CPU) - 4 threads - 6457 events in 60 seconds, which gives 107.62 events per second

Share this post


Link to post
Share on other sites
25 minutes ago, pbies said:

The command I have used (sysbench v0.4.12-1.2)

 

To keep other readers informed: The sysbench approach is BS since this pseudo benchmark does not test 'CPU performance' at all. It's a compiler benchmark but not a hardware benchmark.


When using a distro on the RPi 3B+ that brings up the ARMv8 cores in ARMv8/64-bit mode the funny numbers sysbench spits out are 15 times better so naive users even think the hardware would perform 15 times faster. See results with Fedora here, see changing sysbench scores when switching/upgrading distros there, see explanation already posted again: https://www.raspberrypi.org/forums/viewtopic.php?t=208314&start=25

 

Sysbench's 'cpu test' is not a 'CPU performance' benchmark. It's producing only numbers without meaning. What we need instead is Active Benchmarking generating insights and not silly numbers that are wrong anyway.

Share this post


Link to post
Share on other sites
3 hours ago, pbies said:

For Raspberry Pi 3 model B+

 

Just curious: are you able to provide output from the 7-zip benchmark on the new RPi 3 B+? Since you're using Raspbian (judging by crappy sysbench numbers that indicate you're running a 32-bit distro on this 64-bit board) it's as easy as:

sudo apt install p7zip
7zr b ; 7zr b

Full output would be great. In case you updated all packages to latest version probability that you're affected by throttling (lowering sustained performance) is very high since the RPi Trading guys decided to cripple performance on all RPi 3+ to hide instability issues a few users experience:  https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=217056

 

It's also explained there that the 'firmware' is now cheating even more and even if throttling occured it can not be read out later with the 'vcgencmd' command (the cpufreq values obtained by reading /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq on any RPi 2, 3 or 3+ can never be trusted since they are faked).

 

As usual monitoring in parallel is the most basic requirement for benchmarking (since otherwise it's just generating numbers without meaning) so you might want to think about downloading my raspimon tool and running it in parallel:

wget https://raw.githubusercontent.com/ThomasKaiser/OMV4_for_Raspberries/master/overlay/raspimon
/bin/bash ./raspimon

This tool allows to check for real clockspeeds and throttling occuring. Output looks similar to what's shown here in the right column or in the above referenced thread in RPi forum (@jahboater is using my code from raspimon)

Share this post


Link to post
Share on other sites
8 hours ago, tkaiser said:

Just curious: are you able to provide output from the 7-zip benchmark on the new RPi 3 B+?

I'll check that. I've used 7-zip to compare the Bpi-zero to the rpi zero/3b. But the versions are different. I think because of that the numbers aren't exactly comparable. But it is a lot better than most other tools.
The best way for me is to use as many different programs. Some do better with 32-bit architecture, some better with 64-bit. Then take an average of all the results and your close.
Any way of installing exactly the same version of 7zip on all sbc's?
 

Here's the Rasp 3B+ result you wanted. All default settings.

nicod@nicod-desktop:~$ 7zr b ; 7zr b

7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)

RAM size:     927 MB,  # CPU hardware threads:   4
RAM usage:    850 MB,  # Benchmark threads:      4

Dict        Compressing          |        Decompressing
      Speed Usage    R/U Rating  |    Speed Usage    R/U Rating
       KB/s     %   MIPS   MIPS  |     KB/s     %   MIPS   MIPS

22:    1623   299    528   1579  |    45650   371   1110   4118
23:    1553   300    527   1582  |    45169   373   1108   4133
24:    1555   306    546   1672  |    40410   372   1008   3749
Killed

7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)

RAM size:     927 MB,  # CPU hardware threads:   4
RAM usage:    850 MB,  # Benchmark threads:      4

Dict        Compressing          |        Decompressing
      Speed Usage    R/U Rating  |    Speed Usage    R/U Rating
       KB/s     %   MIPS   MIPS  |     KB/s     %   MIPS   MIPS

22:    1681   320    511   1635  |    39773   371    966   3588
23:    1483   310    487   1511  |    38959   370    963   3565
24:    1408   306    495   1514  |    38199   370    958   3543
Killed

Here my results of the Bpi-zero(i know you don't care for that one :)
 

pi@bpi-iot-ros-ai:~$ taskset -c 0-3 7zr b -mmt4

7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)

RAM size:     494 MB,  # CPU hardware threads:   4
RAM usage:    434 MB,  # Benchmark threads:      4

Dict        Compressing          |        Decompressing
      Speed Usage    R/U Rating  |    Speed Usage    R/U Rating
       KB/s     %   MIPS   MIPS  |     KB/s     %   MIPS   MIPS

22:    1224   328    363   1190  |    34817   393    799   3141
23:    1203   335    366   1226  |    33868   389    796   3099
24:    1078   328    353   1159  |    30588   359    789   2837
----------------------------------------------------------------
Avr:          330    361   1192               380    795   3026
Tot:          355    578   2109


--------------------------------------------------------------------
Single Thread
--------------------------------------------------------------------
pi@bpi-iot-ros-ai:~$ taskset -c 0 7zr b -mmt1

7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)

RAM size:     494 MB,  # CPU hardware threads:   4
RAM usage:    419 MB,  # Benchmark threads:      1

Dict        Compressing          |        Decompressing
      Speed Usage    R/U Rating  |    Speed Usage    R/U Rating
       KB/s     %   MIPS   MIPS  |     KB/s     %   MIPS   MIPS

22:     435    99    426    423  |     9036   100    819    815
23:     425    99    436    433  |     8884    99    817    813
24:     403    99    436    433  |     8753   100    815    812
25:     349    97    411    399  |     7673    90    804    721
----------------------------------------------------------------
Avr:           99    427    422                97    814    790
Tot:           98    620    606

And here the same for the Rpi 3b
 

pi@Raspi:~ $ taskset -c 0-3 7zr b -mmt4

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

LE
CPU Freq:   765  1192  1193  1193  1191  1193  1193  1193  1193

RAM size:     858 MB,  # CPU hardware threads:   4
RAM usage:    450 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1813   319    553   1764  |      56458   391   1232   4817
23:       1756   320    559   1790  |      54822   389   1218   4743
24:       1734   328    569   1865  |      53375   390   1203   4686
----------------------------------  | ------------------------------
Avr:             322    560   1806  |              390   1218   4749
Tot:             356    889   3277

-------------------------------------
Single Thread
-------------------------------------

pi@Raspi:~ $ taskset -c 0 7zr b -mmt1

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

LE
CPU Freq:  1016  1192  1192  1192  1192  1192  1192  1192  1192

RAM size:     858 MB,  # CPU hardware threads:   4
RAM usage:    435 MB,  # Benchmark threads:      1

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:        684   100    666    666  |      15163   100   1295   1295
23:        663   100    676    676  |      14808   100   1282   1282
24:        637   100    686    685  |      14440   100   1268   1268
25:        609   100    696    696  |      13990   100   1245   1245
----------------------------------  | ------------------------------
Avr:             100    681    681  |              100   1273   1272
Tot:             100    977    977

As you can see, bpi-zero was :
7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)
Rpi 3b :
7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

The result of the Bpi-zero seemed a bit on the low side. I did cool it so it wouldn't throttle.

The result of the rpi 3b+ also look lower than 3b. I'll try again with the same command on the 3b.
@tkaiser
Also yesterday I've made a new video about the Rpi 3b+ where I overclock it in Ubuntu. It clocks quite a bit higher in Ubuntu than in Raspbian. How comes? Did the people of canonical do a better job in managing the cpu? It also didn't throttle in Ubuntu. I've closed the case completely and let it run on the max to test that. More than 80°C and still at the max frequency.
Here's the video
Video install Ubuntu + Overclock

Share this post


Link to post
Share on other sites

Here the new result of the rpi 3b
A lot more in line with what I would expect. Also the same version now. That shows that different versions of 7zip also are not comparable.
 

7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)

RAM size:     927 MB,  # CPU hardware threads:   4
RAM usage:    850 MB,  # Benchmark threads:      4

Dict        Compressing          |        Decompressing
      Speed Usage    R/U Rating  |    Speed Usage    R/U Rating
       KB/s     %   MIPS   MIPS  |     KB/s     %   MIPS   MIPS

22:    1551   304    497   1509  |    42124   386    985   3800
23:    1466   296    505   1493  |    41299   385    982   3779
24:    1497   312    516   1609  |    40645   386    977   3770
Killed

7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)

RAM size:     927 MB,  # CPU hardware threads:   4
RAM usage:    850 MB,  # Benchmark threads:      4

Dict        Compressing          |        Decompressing
      Speed Usage    R/U Rating  |    Speed Usage    R/U Rating
       KB/s     %   MIPS   MIPS  |     KB/s     %   MIPS   MIPS

22:    1501   293    498   1460  |    42208   386    985   3808
23:    1552   313    505   1582  |    41542   387    982   3801
24:    1496   312    515   1608  |    40726   387    977   3778
Killed

 

Share this post


Link to post
Share on other sites
19 minutes ago, NicoD said:

Killed

 

The old 7-zip versions did not cope that good with low memory conditions. As we all know Debian and Ubuntu package update policies could be best described as crap 'outdated as hell' (nice rant). So only with Debian Stretch or Ubuntu Bionic distros we get reasonable results.

 

 

Share this post


Link to post
Share on other sites
40 minutes ago, tkaiser said:

Killed

Yeah, no idea why it's killed. Just tried again. I'll try tomorrow with a fresh install of raspbian.
Just tried the same with my highest "stable" overclock, and I don't get better results. While a lot better results with Blender.

All bit weird. I'll check tomorrow what's happening. Cheers

 

Share this post


Link to post
Share on other sites
10 hours ago, NicoD said:

Here my results of the Bpi-zero

 

BPi Zero means: Allwinner H2+ with 16-bit memory interface. So results should be almost the same as with NanoPi NEO (Allwinner H3 which is the same as H2+ and also 16-bit memory interface): https://forum.armbian.com/topic/7776-benchmarking-cpus-not-yet-a-research-article/?do=findComment&comment=58554 (what also matters is DRAM clockspeed and kernel version). Your BPi Zero scores 2109 while my NEO limited to 1.1 GHz scores 2088 --> same numbers. So most probably your Zero started to throttle down (quite common with small boards if they don't wear huge heatsinks)

 

10 hours ago, NicoD said:

And here the same for the Rpi 3b

 

There's a reason I asked for the output of '7zr b ; 7zr b' above (executing the benchmark at least 2 times -- 'fire and forget' benchmarking is bad). Since users obviously hate monitoring (so only generating numbers without meaning) the 2nd 7z execution would show actual clockspeeds. The new 7-zip versions contain a clockspeed measuring algorithm producing something like this at the start of the benchmark:

CPU Freq:   765  1192  1193  1193  1191  1193  1193  1193  1193

This confirms that the CPU cores were running at 1.2 GHz when starting the benchmark. The 2nd 7z call would run this again so if the RPi firmware in the meantime started to throttle it would be obvious by looking at these numbers. Also 2nd benchmark run would produce lower numbers so 'throttling had happened' can not be overlooked.

 

Your RPi 3 scored 3277, my Rock64 scored 3594. Both times 4 Cortex-A53 cores. RPi 3 set to 1.2 GHz, Rock64 set to 1.4 GHz. Results more or less as expected. The RPi 3+ when really clocking with 1.4 GHz will perform also better. But as explained above RPi Trading guys rolled out a new ThreadX release (AKA 'firmware') that cripples all RPi 3+ around to throttle down to 1200 MHz even with medium loads.

 

BTW: I'm asking here for 7-zip numbers not since I'm interested in 'RPi 3+ performance' (this is predictable since it's just a boring quad-core A53) but to demonstrate that MONITORING is important when benchmarking and that passive benchmarking always only generates numbers without meaning :) 

Share this post


Link to post
Share on other sites
3 hours ago, tkaiser said:

So most probably your Zero started to throttle down (quite common with small boards if they don't wear huge heatsinks)

I was using a fan + heatsink on the Bpi zero and rpi 3b.
 

 

3 hours ago, tkaiser said:

BTW: I'm asking here for 7-zip numbers not since I'm interested in 'RPi 3+ performance' (this is predictable since it's just a boring quad-core A53) but to demonstrate that MONITORING is important when benchmarking and that passive benchmarking always only generates numbers without meaning :) 

I've just done the 7-zip bench in Raspbian Lite, now installing the desktop to compare. After that I'll to the same with ubuntu and with the rpi 3b+. I'll add all the numbers to this post.
Here's the Raspbian Lite results of the Rpi 3B

rpi 3b Raspbian Lite result

1st time
Avr :  329    563      1853  |   393     1208     4746
Tot:   361    886      3300  
2nd time
Avr :  328    564      1851  |   393     1210     4752
Tot :  360    887      3301      

3th and 4th times almost the same as 1st and 2nd

Results Raspbian Desktop Rpi 3b

pi@RPI3B:~ $ 7zr b ; 7zr b

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

LE
CPU Freq:   949  1197  1198  1196  1198  1198  1198  1199  1198

RAM size:     911 MB,  # CPU hardware threads:   4
RAM usage:    882 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1793   317    550   1745  |      56845   394   1231   4850
23:       1759   323    555   1792  |      55419   394   1218   4795
24:       1733   329    566   1863  |      54065   394   1205   4746
25:       1280   252    580   1462  |      50155   384   1162   4464
----------------------------------  | ------------------------------
Avr:             305    563   1716  |              391   1204   4714
Tot:             348    883   3215

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

LE
CPU Freq:   801  1089  1126  1059  1167  1184  1186  1186  1186

RAM size:     911 MB,  # CPU hardware threads:   4
RAM usage:    882 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1785   316    550   1736  |      56654   392   1232   4834
23:       1757   322    556   1791  |      55426   394   1219   4796
24:       1737   329    568   1868  |      53818   392   1205   4724
25:       1710   337    579   1953  |      51503   387   1184   4584
----------------------------------  | ------------------------------
Avr:             326    563   1837  |              391   1210   4734
Tot:             359    887   3286

Desktop is a little bit slower.

 

Raspberry Pi 3b+ Raspbian Lite
 

rpi 3b+ Raspbian Lite result

1st
Avr :   336    610     2051  |   390     1381      5388
Tot :   363    996     3720

2nd
Avr :   329    608     2000  |   390     1382      5394
Tot :   360    995     3697

Raspberry Pi 3b+ Raspbian Desktop

i@raspberrypi:~ $ 7zr b ; 7zr b

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

LE
CPU Freq:   753  1393  1393  1393  1393  1393  1393  1392  1388

RAM size:     911 MB,  # CPU hardware threads:   4
RAM usage:    882 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1915   314    594   1863  |      64588   391   1410   5510
23:       1893   322    599   1930  |      62188   388   1388   5381
24:       1889   332    612   2032  |      60452   388   1368   5307
25:       1508   277    621   1722  |      57029   386   1313   5075
----------------------------------  | ------------------------------
Avr:             311    607   1887  |              388   1370   5318
Tot:             350    988   3603

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

LE
CPU Freq:   922  1317  1270  1178  1279  1299  1341  1349  1346

RAM size:     911 MB,  # CPU hardware threads:   4
RAM usage:    882 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1978   325    592   1925  |      63685   387   1404   5433
23:       1977   335    602   2015  |      62343   389   1387   5394
24:       1888   332    611   2031  |      61095   390   1374   5363
25:       1891   350    617   2160  |      56111   375   1333   4994
----------------------------------  | ------------------------------
Avr:             335    606   2032  |              385   1374   5296
Tot:             360    990   3664

08:33:52: 39.7'C  600/ 600 MHz 0000000000000000000 1.2V
08:33:58: 41.3'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:03: 44.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:08: 47.8'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:13: 47.2'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:18: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:23: 48.3'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:28: 48.3'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:33: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:39: 49.9'C 1400/1400 MHz 0000000000000000000 1.3250V
Time       Temp  CPU fake/real     Health state    Vcore
08:34:44: 48.9'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:49: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:54: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V
08:34:59: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V
08:35:04: 49.9'C 1400/1400 MHz 0000000000000000000 1.3250V
08:35:10: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:35:30: 47.2'C 1400/1400 MHz 0000000000000000000 1.3250V
08:35:35: 48.3'C 1400/1400 MHz 0000000000000000000 1.3250V
08:35:40: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:35:45: 49.9'C 1400/1400 MHz 0000000000000000000 1.3250V
08:35:50: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:35:56: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:01: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:06: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:11: 51.0'C 1400/1400 MHz 0000000000000000000 1.3250V
Time       Temp  CPU fake/real     Health state    Vcore
08:36:16: 49.9'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:21: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:26: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:31: 51.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:37: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:42: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:47: 51.0'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:53: 52.1'C 1400/1400 MHz 0000000000000000000 1.3250V
08:36:58: 46.2'C  600/ 600 MHz 0000000000000000000 1.2V

@tkaiser
Again a little slower than Lite. In the afternoon I'll check Ubuntu.
More to come. Cheers
 

Share this post


Link to post
Share on other sites

@tkaiser
In Ubuntu it again got killed every time. Something's definitely wrong there. It also seems a lot slower.
It is a fresh install of Ubuntu on the Rasp3B. I've added the data of your raspimon so you see it's not overheating or so. For all the tests I've used my fan.
Here's the Ubuntu result of the RPi 3b
 

nicod@nicod-desktop:~$ 7zr b ; 7zr b

7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)

RAM size:     897 MB,  # CPU hardware threads:   4
RAM usage:    850 MB,  # Benchmark threads:      4

Dict        Compressing          |        Decompressing
      Speed Usage    R/U Rating  |    Speed Usage    R/U Rating
       KB/s     %   MIPS   MIPS  |     KB/s     %   MIPS   MIPS

22:    1592   320    484   1549  |    39719   369    971   3583
23:    1568   324    492   1598  |    39481   373    967   3612
24:    1425   302    507   1533  |    38455   370    963   3567
Killed

7-Zip (A) 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)

RAM size:     897 MB,  # CPU hardware threads:   4
RAM usage:    850 MB,  # Benchmark threads:      4

Dict        Compressing          |        Decompressing
      Speed Usage    R/U Rating  |    Speed Usage    R/U Rating
       KB/s     %   MIPS   MIPS  |     KB/s     %   MIPS   MIPS

22:    1588   317    488   1545  |    39872   370    971   3597
23:    1312   274    487   1337  |    35413   342    948   3240
24:    1492   316    507   1605  |    39071   376    964   3624
Killed

12:43:16: 44.5'C  600/ 600 MHz 0000000000000000000 1.2V
12:43:21: 50.5'C 1200/1200 MHz 0000000000000000000 1.2688V
12:43:26: 53.2'C 1200/1200 MHz 0000000000000000000 1.2688V
12:43:31: 53.7'C 1200/1200 MHz 0000000000000000000 1.2688V
12:43:37: 54.8'C 1200/1200 MHz 0000000000000000000 1.2688V
12:43:42: 55.3'C 1200/1200 MHz 0000000000000000000 1.2688V
12:43:47: 56.9'C 1200/1200 MHz 0000000000000000000 1.2688V
12:43:52: 55.8'C 1200/1200 MHz 0000000000000000000 1.2688V
12:43:57: 56.4'C 1200/1200 MHz 0000000000000000000 1.2688V
12:44:02: 55.8'C 1200/1200 MHz 0000000000000000000 1.2688V
12:44:07: 56.4'C 1200/1200 MHz 0000000000000000000 1.2688V
12:44:12: 55.8'C 1200/1200 MHz 0000000000000000000 1.2688V
12:44:18: 58.5'C 1200/1200 MHz 0000000000000000000 1.2688V
12:44:23: 55.8'C 1200/1200 MHz 0000000000000000000 1.2688V
Time       Temp  CPU fake/real     Health state    Vcore
12:44:28: 56.9'C 1200/1200 MHz 0000000000000000000 1.2688V
12:44:33: 56.9'C 1200/1200 MHz 0000000000000000000 1.2688V
12:44:38: 56.9'C 1200/1200 MHz 0000000000000000000 1.2688V
12:44:43: 57.5'C 1200/1200 MHz 0000000000000000000 1.2688V
12:44:48: 58.0'C 1200/1200 MHz 0000000000000000000 1.2688V

@pbies

As tkaiser told you. Sysbench isn't very good to compare cpu scores.
And what my findings are, is that there's no real perfect way of measuring cpu's between each other. 7zip is very good for that task, but again many factors come into play.
There's sd-card speed, ram-speed, distro, distro-version, version of the benchmark tool, 32-bit vs 64-bit, temperature, and many more things. It's always comparing apples and oranges(and raspberry's and banana's and ....)
My tip is, use the program you want to use you sbc for as benchmark. That's the only information you need. All the rest is just numbers without much meaning.
I use my sbc's for video editing and as 3d-render farm. So I use those tools to benchmark my different sbc's. I only need to know which sbc does the fastest render.
If I then compare my render results with sysbench results, then I find there's no correlation. Some do some tasks better, some do other tasks better.
My best video render sbc isn't my best 3d-Blender render sbc...
Greetings.

 

 

 

Share this post


Link to post
Share on other sites

@tkaiser @NicoD I understand your posts, but don't agree with them. As 7z is not enough reliable in testing only CPU, I didn't found other benchmark to test CPU. If you have any other benchmark that would be reliable - just tell me the name.

 

7z is killed by kernel, because it is using too much resources for its tests. And this is normal for kernel. At later test it is using over 2GB of RAM, so it is using swapfile (much worse results) then and gets killed. That's enough for me to assume that it is not a reliable benchmark.

And this is the reason why I am not putting here results of 7z benchmarks.

 

Still I don't see any other better only-CPU test for SBCs.

 

And I assume that each compiler do its best for each architecture, so still sysbench stays with me much more than 7z.

Share this post


Link to post
Share on other sites
19 minutes ago, NicoD said:

In Ubuntu it again got killed every time. Something's definitely wrong there

 

Most probably related to OOM killer settings (see here). The problem is that Debian and Ubuntu love to keep horribly outdated program versions in their repos (nice rant again). Version 9.20 that's used by Ubuntu Xenial and Debian Jessie is from 2010 (see changelog -- it's a shame to not package a more recent version). Version 16.02 now used by more recent distros is 'just' 2 years old.

 

So to get a 'bigger picture' execution with Debian Stretch on every board would be needed.

Share this post


Link to post
Share on other sites
16 minutes ago, pbies said:

reliable in testing only CPU

An SBC/computer isn't a CPU alone. So you can not test the cpu alone. Why do you need to know?
There are other tools, but again none are perfect. Or you use as many as possible and make averages, or you benchmark with the programs you want to use. Or just 7zip to get an idea of how it compares. But it doesn't mean that the fastest 7zip sbc will be the fastest for what you want to do.

Share this post


Link to post
Share on other sites
1 minute ago, tkaiser said:

to keep horribly outdated program versions in their repos (nice rant again)

Very true. I always need to compile Kdenlive myself because they're too lazy to update the repo's.
I tried to use Ubuntu to show it gives different results.(with every tool there is, not just 7zip)
Tho the differences shouldn't be too big. Cheers

Share this post


Link to post
Share on other sites

@NicoD SBCs aren't CPU alone, but I need to test only CPU. 7z is dependant on RAM mainly. This way I can use MemTest and get the same level-reliable results.

 

So what kind of software do you propose then? Do you have any software that tests only CPU and isn't memory-dependant? And please don't tell me that it doesn't exist - any such software can operate only on registers inside CPU.

Share this post


Link to post
Share on other sites
12 minutes ago, pbies said:

So what kind of software do you propose then?

https://haydenjames.io/linux-benchmark-scripts-tools/
https://www.makeuseof.com/tag/benchmark-linux-pcs-performance/

As said, there are a lot of tools for that. You could also use a miner. Or anything you want. But not all cpu's work the same. Some have more cache, faster memory, other architecture. So some will perform better for some tasks, other for other tasks.
What do you want it for?
You can use sysbench too. But always make sure you use the same version, the same distro/version. It's not going to tell you everything about a cpu. It's just numbers.
That's why I don't use tools like that for benchmarks, but real live tasks to see what's best for the right use.
Greetings

Share this post


Link to post
Share on other sites

And also always check the temperature of the CPU, the frequency, the voltage...
If a cpu throttle's a lot, then many times it's better to underclock it for better performance. For example the XU4 at max freq is always overheating. If you go 200Mhz(1800Mhz) lower with the big cores your results will be better than at 2Ghz.
 

Share this post


Link to post
Share on other sites
43 minutes ago, pbies said:

Phoronix Test Suite

 

LOL, the Moronix Test Suite: https://forum.pine64.org/showthread.php?tid=389&pid=3259#pid3259

 

1 hour ago, NicoD said:
1 hour ago, pbies said:

So what kind of software do you propose then?

https://haydenjames.io/linux-benchmark-scripts-tools/

 

Unixbench: http://www.brendangregg.com/blog/2014-05-02/compilers-love-messing-with-benchmarks.html

Bonnie: http://www.brendangregg.com/ActiveBenchmarking/bonnie++.html

 

And so on... all those tools usually recommended are popular since they're easy to execute, do not care about correct benchmarking but people are happy with numbers without meaning. Good luck...

Share this post


Link to post
Share on other sites
39 minutes ago, pbies said:

@NicoD still these rely on RAM speed

Every process rely's on the RAM. You can not run a program without memory. A program loads into the RAM. The cpu takes blocks of data/instuctions out the ram. It computes what's asked from it, and writes the data back to the ram. Ram and the CPU are a whole. A cpu can't do anything without ram, and you're nothing with memory without a cpu(except for other applications than pc's).
So you can not benchmark a cpu without using ram. If you calculate prime like with sysbench. The instruction is red out of the memory into the cpu, the calculations of the cpu are written into ram, and a new instruction is passed to the cpu. So having only cpu benchmarks are impossible.

Just so you understand.

Share this post


Link to post
Share on other sites

@tkaiser Really, I am not interested in your negative way of thinking. Please leave me alone out of your ideas.

 

@NicoD You didn't understood what I said: I need benchmark for CPU, not RAM. And to benchmark CPU you don't need tons of RAM as for 7z. You understand now direction in which I want to go?

I don't need to test tons of RAM, I agree that nothing without it would exist, but still operations on registers is enough to test CPU, and NOT on RAM.

Share this post


Link to post
Share on other sites

Even when a program uses less ram it will read and write as much. Then just always the same data. I could write something for you that uses 100kb of ram, but the spu would constantly need to read the ram. Maybe you could write something in assembler where you send the cpu in a loop without reading much, and then count the loops/s. But still the ram would be used to count. You can`t count anything without saving that to ram. Else you wouldn`t know any number.

So last time. A program can not run without ram. All instructions for the cpu come from ram. Otherwise just check the clockspeed with a great ocyloscope. And you`ll know how many clocks it did. But that doesn`t say anything.
You are searching for something unexisting. If you were a programmer you would understand.

Have a nice day. And tkaiser is a great guy. He`s trying to help you, but you don`t understand the facts.

Share this post


Link to post
Share on other sites

 

@pbies

My last words about this. See the cpu as an engine, and the ram is the fuel pump. The engine can not run without the fuelpump because it needs gasoline to run. Just as a cpu can`t do anything without data what the ram is providing.
With every test you do, overclock the ram and it will run faster. Overclock the cpu and it will run faster. Both are as important.
That`s why the raspberry pi is the slowest sbc, while the cpu is ok. The ram is too slow.
Greetings

Share this post


Link to post
Share on other sites
(edited)
On 7/26/2018 at 2:16 PM, NicoD said:

always check the temperature of the CPU, the frequency, the voltage...

 

First attempt on sbc-bench.sh (I'll put this later on Github)

 

In case you still have the RPi 3B+ around it would be really great if you could let the script run without a fan (I need the throttling situation to see whether everything is reported well). The script is designed to work on normal ARM SBC as well as on the crippled Raspberries (VideoCore main CPU controlling the hardware and all the proprietary TheadX stuff). Basic requirement is either Debian Stretch or Ubuntu Bionic (or something similar with somewhat recent packages)

 

Output from a RPi 2 with Raspbian Stretch I had access to:

root@raspberrypi:/home/pi# /usr/local/bin/sbc-bench.sh
Installing needed tools. This may take some time... Done.
Executing tinymembench. This will take a long time... Done.
Executing 7-zip benchmark. This will take a long time... Done.
Executing OpenSSL benchmark. This will take a long time... Done.


Below benchmark results:

Memory performance:
memcpy: 1013.7 MB/s 
memset: 1167.9 MB/s 

7-zip total scores (three runs): 2134,2129,2125

OpenSSL results:
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc      14216.59k    19191.70k    21097.90k    21513.56k    21793.45k    21785.26k
aes-128-cbc      14147.59k    19138.28k    21094.49k    21632.68k    21673.30k    21796.18k
aes-192-cbc      12802.24k    16565.27k    18059.09k    18446.34k    18352.81k    18568.53k
aes-192-cbc      12807.21k    16660.76k    17955.16k    18443.61k    18557.61k    18459.31k
aes-256-cbc      11806.24k    14964.78k    16007.42k    16251.22k    16460.46k    16465.92k
aes-256-cbc      11738.95k    14962.37k    16062.21k    16279.21k    16455.00k    16460.46k

Full results uploaded to http://ix.io/1irc. Please check the log for anomalies (e.g. swapping
or throttling happenend) and otherwise share this URL.

 

Output from a heavily throttled NanoPC-T4 with some preliminary Armbian I built few weeks ago:

root@nanopct4:/tmp# /usr/local/bin/sbc-bench.sh 
Installing needed tools. This may take some time... Done.
Executing tinymembench. This will take a long time... Done.
Executing 7-zip benchmark. This will take a long time... Done.
Executing OpenSSL benchmark. This will take a long time... Done.

ATTENTION: Throttling occured on CPU cluster 4. Check the uploaded log for details.

Below benchmark results:

Memory performance (on big.LITTLE systems measured individually):
memcpy: 2060.7 MB/s (0.3%)
memset: 8440.3 MB/s (0.6%)
memcpy: 4000.9 MB/s (0.6%)
memset: 9011.0 MB/s (0.7%)

7-zip total scores (three runs): 4496,3980,3837

OpenSSL results (on big.LITTLE systems measured individually):
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc      91970.98k   292346.01k   630755.93k   914955.95k  1053515.78k  1065336.83k
aes-128-cbc     272534.69k   638176.02k   956555.26k  1068235.78k  1137390.93k  1144930.30k
aes-192-cbc      92993.86k   274024.58k   532484.78k   715422.38k   794593.96k   797059.75k
aes-192-cbc     260573.74k   581688.00k   797185.54k   940168.87k   982846.12k   981581.82k
aes-256-cbc      90710.94k   257319.98k   466747.22k   601770.33k   656547.84k   654611.80k
aes-256-cbc     235117.07k   531296.70k   735206.31k   807839.74k   850635.43k   852792.66k

Full results uploaded to http://ix.io/1irf. Please check the log for anomalies (e.g. swapping
or throttling happenend) and otherwise share this URL.

 

 

To let it run it's as easy as:

wget -O /usr/local/bin/sbc-bench.sh https://raw.githubusercontent.com/ThomasKaiser/sbc-bench/master/sbc-bench.sh
chmod 755 /usr/local/bin/sbc-bench.sh
sudo /usr/local/bin/sbc-bench.sh

(unfortunately needs root privileges to manipulate cpufreq governor and access monitoring data)

Edited by tkaiser
Adjusted download URL to Github

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
5 5