pbies Posted July 22, 2018 Posted July 22, 2018 I've run some sysbench (v0.4.12) tests. More will come later as I will get Raspberry Pi 3 Model B+ and Orange Pi RK3399. Banana Pi = ARMv71 A20 @ 960 MHz (not sure about current clock, not overclocked) Banana Pi Armbian (2 cores, 2 threads) - 1 thread - 1057 events in 60 seconds, which gives 17.62 events per second Banana Pi Armbian (2 cores, 2 threads) - 2 threads - 2003 events in 60 seconds, which gives 33.38 events per second i7-8700K @ 4.7 GHz VM with Mint 18.3 x64 in VirtualBox on Windows 10 (4 cores, 4 threads) - 1 thread - 10000 events in 4 seconds, which gives 2500 events per second i7-8700K @ 4.7 GHz Mint 18.3 x64 stand-alone (6 cores, 12 threads) - 12 threads - 10000 events in 1.47 second, which gives 6802 events per second For the above and only CPU - i7-8700K is from 75 to 386 times faster than Banana Pi CPU. 0 Quote
tkaiser Posted July 22, 2018 Posted July 22, 2018 1 hour ago, pbies said: sysbench ...is crap. It's not a CPU but only a compiler benchmark. You chose the worst 'benchmark' possible. Sysbench numbers change with compiler version and settings or even with sysbench version (higher version number --> lower scores). There's no 'benchmark' known producing more unreliable results wrt hardware/CPU. Use 'Google site search' here to get the details. If it's about a quick and rough CPU performance estimate I would recommend 7-zip's benchmark mode (7z b). 1 Quote
pbies Posted July 22, 2018 Author Posted July 22, 2018 @tkaiser I don't understand your negativity. Banana Pi's are bad, sysbench is bad... The whole world is bad, isn't it? I'm sorry but I won't be arguing with you. 0 Quote
tkaiser Posted July 22, 2018 Posted July 22, 2018 For people not rejecting reality... again why sysbench is unrealiable (not able to indicate CPU performance AT ALL). Sysbench not able to compare different CPU architectures since only a compiler benchmark (you get 15 times higher 'CPU performance' reported with a 64-bit distro than a 32-bit distro on 64-bit ARM boards) Switch the distro and your sysbench numbers differ (in fact it's just different distros building their packages with different GCC versions) Update your distro and your sysbench numbers improve (since it's just a compiler benchmark) Different sysbench version, different 'benchmark' results (start at 'Performance with legacy Armbian image') Why sysbench's method to 'calculate CPU performance' is that weird and does not apply to anything performance relevant in the real world For people loving to 'shoot the messenger'... it's not only me who describes sysbench as the totally wrong tool, e.g. https://www.raspberrypi.org/forums/viewtopic.php?t=208314&start=25 Again: 7-zip's benchmark mode is not just using an insanely limited synthetic benchmark routine like sysbench (calculating prime numbers only involving CPU caches but no memory access), 7-zip is not dependent on compiler versions or platform switches and 7-zip allows for cross-platform comparisons. You'll find a lot of numbers here in the forum and some comparisons on the net e.g. https://s1.hoffart.de/7zip-bench/ (again: it's just a rough estimate but at least something somewhat reliable related to CPU performance) The most important thing with benchmarking is 'benchmarking the benchmark'. Since most tools (especially the popular ones) do not do what they pretend to do. Active benchmarking is needed and not just throwing out numbers without meaning. BTW: sysbench is part of MySQL and when used correctly a great tool to provide insights. It's just the 'cpu test' that is not a CPU test at all. And it's about people firing up sysbench in 'passive benchmarking' mode generating numbers without meaning and praising insufficient tools. 1 Quote
chwe Posted July 23, 2018 Posted July 23, 2018 Thread was splitted, benchmarking from @tkaiser can be found here as a 'not yet ready' research tutorial: 0 Quote
pbies Posted July 25, 2018 Author Posted July 25, 2018 The command I have used (sysbench v0.4.12-1.2): sysbench --test=cpu --cpu-max-prime=20000 --num-threads=1 --max-time=60 run For Raspberry Pi 3 model B+ the results are: (4 cores, 4 threads CPU) - 1 thread - 1619 events in 60 seconds, which gives 26.98 events per second (4 cores, 4 threads CPU) - 4 threads - 6457 events in 60 seconds, which gives 107.62 events per second 0 Quote
tkaiser Posted July 25, 2018 Posted July 25, 2018 25 minutes ago, pbies said: The command I have used (sysbench v0.4.12-1.2) To keep other readers informed: The sysbench approach is BS since this pseudo benchmark does not test 'CPU performance' at all. It's a compiler benchmark but not a hardware benchmark. When using a distro on the RPi 3B+ that brings up the ARMv8 cores in ARMv8/64-bit mode the funny numbers sysbench spits out are 15 times better so naive users even think the hardware would perform 15 times faster. See results with Fedora here, see changing sysbench scores when switching/upgrading distros there, see explanation already posted again: https://www.raspberrypi.org/forums/viewtopic.php?t=208314&start=25 Sysbench's 'cpu test' is not a 'CPU performance' benchmark. It's producing only numbers without meaning. What we need instead is Active Benchmarking generating insights and not silly numbers that are wrong anyway. 0 Quote
tkaiser Posted July 25, 2018 Posted July 25, 2018 3 hours ago, pbies said: For Raspberry Pi 3 model B+ Just curious: are you able to provide output from the 7-zip benchmark on the new RPi 3 B+? Since you're using Raspbian (judging by crappy sysbench numbers that indicate you're running a 32-bit distro on this 64-bit board) it's as easy as: sudo apt install p7zip 7zr b ; 7zr b Full output would be great. In case you updated all packages to latest version probability that you're affected by throttling (lowering sustained performance) is very high since the RPi Trading guys decided to cripple performance on all RPi 3+ to hide instability issues a few users experience: https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=217056 It's also explained there that the 'firmware' is now cheating even more and even if throttling occured it can not be read out later with the 'vcgencmd' command (the cpufreq values obtained by reading /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq on any RPi 2, 3 or 3+ can never be trusted since they are faked). As usual monitoring in parallel is the most basic requirement for benchmarking (since otherwise it's just generating numbers without meaning) so you might want to think about downloading my raspimon tool and running it in parallel: wget https://raw.githubusercontent.com/ThomasKaiser/OMV4_for_Raspberries/master/overlay/raspimon /bin/bash ./raspimon This tool allows to check for real clockspeeds and throttling occuring. Output looks similar to what's shown here in the right column or in the above referenced thread in RPi forum (@jahboater is using my code from raspimon) 0 Quote
NicoD Posted July 25, 2018 Posted July 25, 2018 8 hours ago, tkaiser said: Just curious: are you able to provide output from the 7-zip benchmark on the new RPi 3 B+? I'll check that. I've used 7-zip to compare the Bpi-zero to the rpi zero/3b. But the versions are different. I think because of that the numbers aren't exactly comparable. But it is a lot better than most other tools. The best way for me is to use as many different programs. Some do better with 32-bit architecture, some better with 64-bit. Then take an average of all the results and your close. Any way of installing exactly the same version of 7zip on all sbc's? Here's the Rasp 3B+ result you wanted. All default settings. nicod@nicod-desktop:~$ 7zr b ; 7zr b 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs) RAM size: 927 MB, # CPU hardware threads: 4 RAM usage: 850 MB, # Benchmark threads: 4 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 1623 299 528 1579 | 45650 371 1110 4118 23: 1553 300 527 1582 | 45169 373 1108 4133 24: 1555 306 546 1672 | 40410 372 1008 3749 Killed 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs) RAM size: 927 MB, # CPU hardware threads: 4 RAM usage: 850 MB, # Benchmark threads: 4 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 1681 320 511 1635 | 39773 371 966 3588 23: 1483 310 487 1511 | 38959 370 963 3565 24: 1408 306 495 1514 | 38199 370 958 3543 Killed Here my results of the Bpi-zero(i know you don't care for that one pi@bpi-iot-ros-ai:~$ taskset -c 0-3 7zr b -mmt4 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs) RAM size: 494 MB, # CPU hardware threads: 4 RAM usage: 434 MB, # Benchmark threads: 4 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 1224 328 363 1190 | 34817 393 799 3141 23: 1203 335 366 1226 | 33868 389 796 3099 24: 1078 328 353 1159 | 30588 359 789 2837 ---------------------------------------------------------------- Avr: 330 361 1192 380 795 3026 Tot: 355 578 2109 -------------------------------------------------------------------- Single Thread -------------------------------------------------------------------- pi@bpi-iot-ros-ai:~$ taskset -c 0 7zr b -mmt1 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs) RAM size: 494 MB, # CPU hardware threads: 4 RAM usage: 419 MB, # Benchmark threads: 1 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 435 99 426 423 | 9036 100 819 815 23: 425 99 436 433 | 8884 99 817 813 24: 403 99 436 433 | 8753 100 815 812 25: 349 97 411 399 | 7673 90 804 721 ---------------------------------------------------------------- Avr: 99 427 422 97 814 790 Tot: 98 620 606 And here the same for the Rpi 3b pi@Raspi:~ $ taskset -c 0-3 7zr b -mmt4 7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE) LE CPU Freq: 765 1192 1193 1193 1191 1193 1193 1193 1193 RAM size: 858 MB, # CPU hardware threads: 4 RAM usage: 450 MB, # Benchmark threads: 4 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 1813 319 553 1764 | 56458 391 1232 4817 23: 1756 320 559 1790 | 54822 389 1218 4743 24: 1734 328 569 1865 | 53375 390 1203 4686 ---------------------------------- | ------------------------------ Avr: 322 560 1806 | 390 1218 4749 Tot: 356 889 3277 ------------------------------------- Single Thread ------------------------------------- pi@Raspi:~ $ taskset -c 0 7zr b -mmt1 7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE) LE CPU Freq: 1016 1192 1192 1192 1192 1192 1192 1192 1192 RAM size: 858 MB, # CPU hardware threads: 4 RAM usage: 435 MB, # Benchmark threads: 1 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 684 100 666 666 | 15163 100 1295 1295 23: 663 100 676 676 | 14808 100 1282 1282 24: 637 100 686 685 | 14440 100 1268 1268 25: 609 100 696 696 | 13990 100 1245 1245 ---------------------------------- | ------------------------------ Avr: 100 681 681 | 100 1273 1272 Tot: 100 977 977 As you can see, bpi-zero was : 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs) Rpi 3b : 7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE) The result of the Bpi-zero seemed a bit on the low side. I did cool it so it wouldn't throttle. The result of the rpi 3b+ also look lower than 3b. I'll try again with the same command on the 3b.@tkaiser Also yesterday I've made a new video about the Rpi 3b+ where I overclock it in Ubuntu. It clocks quite a bit higher in Ubuntu than in Raspbian. How comes? Did the people of canonical do a better job in managing the cpu? It also didn't throttle in Ubuntu. I've closed the case completely and let it run on the max to test that. More than 80°C and still at the max frequency. Here's the videoVideo install Ubuntu + Overclock 1 Quote
NicoD Posted July 25, 2018 Posted July 25, 2018 Here the new result of the rpi 3b A lot more in line with what I would expect. Also the same version now. That shows that different versions of 7zip also are not comparable. 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs) RAM size: 927 MB, # CPU hardware threads: 4 RAM usage: 850 MB, # Benchmark threads: 4 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 1551 304 497 1509 | 42124 386 985 3800 23: 1466 296 505 1493 | 41299 385 982 3779 24: 1497 312 516 1609 | 40645 386 977 3770 Killed 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs) RAM size: 927 MB, # CPU hardware threads: 4 RAM usage: 850 MB, # Benchmark threads: 4 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 1501 293 498 1460 | 42208 386 985 3808 23: 1552 313 505 1582 | 41542 387 982 3801 24: 1496 312 515 1608 | 40726 387 977 3778 Killed 0 Quote
tkaiser Posted July 25, 2018 Posted July 25, 2018 19 minutes ago, NicoD said: Killed The old 7-zip versions did not cope that good with low memory conditions. As we all know Debian and Ubuntu package update policies could be best described as crap 'outdated as hell' (nice rant). So only with Debian Stretch or Ubuntu Bionic distros we get reasonable results. 0 Quote
NicoD Posted July 25, 2018 Posted July 25, 2018 40 minutes ago, tkaiser said: Killed Yeah, no idea why it's killed. Just tried again. I'll try tomorrow with a fresh install of raspbian. Just tried the same with my highest "stable" overclock, and I don't get better results. While a lot better results with Blender. All bit weird. I'll check tomorrow what's happening. Cheers 0 Quote
tkaiser Posted July 26, 2018 Posted July 26, 2018 10 hours ago, NicoD said: Here my results of the Bpi-zero BPi Zero means: Allwinner H2+ with 16-bit memory interface. So results should be almost the same as with NanoPi NEO (Allwinner H3 which is the same as H2+ and also 16-bit memory interface): https://forum.armbian.com/topic/7776-benchmarking-cpus-not-yet-a-research-article/?do=findComment&comment=58554 (what also matters is DRAM clockspeed and kernel version). Your BPi Zero scores 2109 while my NEO limited to 1.1 GHz scores 2088 --> same numbers. So most probably your Zero started to throttle down (quite common with small boards if they don't wear huge heatsinks) 10 hours ago, NicoD said: And here the same for the Rpi 3b There's a reason I asked for the output of '7zr b ; 7zr b' above (executing the benchmark at least 2 times -- 'fire and forget' benchmarking is bad). Since users obviously hate monitoring (so only generating numbers without meaning) the 2nd 7z execution would show actual clockspeeds. The new 7-zip versions contain a clockspeed measuring algorithm producing something like this at the start of the benchmark: CPU Freq: 765 1192 1193 1193 1191 1193 1193 1193 1193 This confirms that the CPU cores were running at 1.2 GHz when starting the benchmark. The 2nd 7z call would run this again so if the RPi firmware in the meantime started to throttle it would be obvious by looking at these numbers. Also 2nd benchmark run would produce lower numbers so 'throttling had happened' can not be overlooked. Your RPi 3 scored 3277, my Rock64 scored 3594. Both times 4 Cortex-A53 cores. RPi 3 set to 1.2 GHz, Rock64 set to 1.4 GHz. Results more or less as expected. The RPi 3+ when really clocking with 1.4 GHz will perform also better. But as explained above RPi Trading guys rolled out a new ThreadX release (AKA 'firmware') that cripples all RPi 3+ around to throttle down to 1200 MHz even with medium loads. BTW: I'm asking here for 7-zip numbers not since I'm interested in 'RPi 3+ performance' (this is predictable since it's just a boring quad-core A53) but to demonstrate that MONITORING is important when benchmarking and that passive benchmarking always only generates numbers without meaning 0 Quote
NicoD Posted July 26, 2018 Posted July 26, 2018 3 hours ago, tkaiser said: So most probably your Zero started to throttle down (quite common with small boards if they don't wear huge heatsinks) I was using a fan + heatsink on the Bpi zero and rpi 3b. 3 hours ago, tkaiser said: BTW: I'm asking here for 7-zip numbers not since I'm interested in 'RPi 3+ performance' (this is predictable since it's just a boring quad-core A53) but to demonstrate that MONITORING is important when benchmarking and that passive benchmarking always only generates numbers without meaning I've just done the 7-zip bench in Raspbian Lite, now installing the desktop to compare. After that I'll to the same with ubuntu and with the rpi 3b+. I'll add all the numbers to this post. Here's the Raspbian Lite results of the Rpi 3B rpi 3b Raspbian Lite result 1st time Avr : 329 563 1853 | 393 1208 4746 Tot: 361 886 3300 2nd time Avr : 328 564 1851 | 393 1210 4752 Tot : 360 887 3301 3th and 4th times almost the same as 1st and 2nd Results Raspbian Desktop Rpi 3b pi@RPI3B:~ $ 7zr b ; 7zr b 7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE) LE CPU Freq: 949 1197 1198 1196 1198 1198 1198 1199 1198 RAM size: 911 MB, # CPU hardware threads: 4 RAM usage: 882 MB, # Benchmark threads: 4 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 1793 317 550 1745 | 56845 394 1231 4850 23: 1759 323 555 1792 | 55419 394 1218 4795 24: 1733 329 566 1863 | 54065 394 1205 4746 25: 1280 252 580 1462 | 50155 384 1162 4464 ---------------------------------- | ------------------------------ Avr: 305 563 1716 | 391 1204 4714 Tot: 348 883 3215 7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE) LE CPU Freq: 801 1089 1126 1059 1167 1184 1186 1186 1186 RAM size: 911 MB, # CPU hardware threads: 4 RAM usage: 882 MB, # Benchmark threads: 4 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 1785 316 550 1736 | 56654 392 1232 4834 23: 1757 322 556 1791 | 55426 394 1219 4796 24: 1737 329 568 1868 | 53818 392 1205 4724 25: 1710 337 579 1953 | 51503 387 1184 4584 ---------------------------------- | ------------------------------ Avr: 326 563 1837 | 391 1210 4734 Tot: 359 887 3286 Desktop is a little bit slower. Raspberry Pi 3b+ Raspbian Lite rpi 3b+ Raspbian Lite result 1st Avr : 336 610 2051 | 390 1381 5388 Tot : 363 996 3720 2nd Avr : 329 608 2000 | 390 1382 5394 Tot : 360 995 3697 Raspberry Pi 3b+ Raspbian Desktop i@raspberrypi:~ $ 7zr b ; 7zr b 7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE) LE CPU Freq: 753 1393 1393 1393 1393 1393 1393 1392 1388 RAM size: 911 MB, # CPU hardware threads: 4 RAM usage: 882 MB, # Benchmark threads: 4 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 1915 314 594 1863 | 64588 391 1410 5510 23: 1893 322 599 1930 | 62188 388 1388 5381 24: 1889 332 612 2032 | 60452 388 1368 5307 25: 1508 277 621 1722 | 57029 386 1313 5075 ---------------------------------- | ------------------------------ Avr: 311 607 1887 | 388 1370 5318 Tot: 350 988 3603 7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE) LE CPU Freq: 922 1317 1270 1178 1279 1299 1341 1349 1346 RAM size: 911 MB, # CPU hardware threads: 4 RAM usage: 882 MB, # Benchmark threads: 4 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 1978 325 592 1925 | 63685 387 1404 5433 23: 1977 335 602 2015 | 62343 389 1387 5394 24: 1888 332 611 2031 | 61095 390 1374 5363 25: 1891 350 617 2160 | 56111 375 1333 4994 ---------------------------------- | ------------------------------ Avr: 335 606 2032 | 385 1374 5296 Tot: 360 990 3664 08:33:52: 39.7'C 600/ 600 MHz 0000000000000000000 1.2V 08:33:58: 41.3'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:03: 44.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:08: 47.8'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:13: 47.2'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:18: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:23: 48.3'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:28: 48.3'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:33: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:39: 49.9'C 1400/1400 MHz 0000000000000000000 1.3250V Time Temp CPU fake/real Health state Vcore 08:34:44: 48.9'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:49: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:54: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V 08:34:59: 49.4'C 1400/1400 MHz 0000000000000000000 1.3250V 08:35:04: 49.9'C 1400/1400 MHz 0000000000000000000 1.3250V 08:35:10: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:35:30: 47.2'C 1400/1400 MHz 0000000000000000000 1.3250V 08:35:35: 48.3'C 1400/1400 MHz 0000000000000000000 1.3250V 08:35:40: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:35:45: 49.9'C 1400/1400 MHz 0000000000000000000 1.3250V 08:35:50: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:35:56: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:01: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:06: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:11: 51.0'C 1400/1400 MHz 0000000000000000000 1.3250V Time Temp CPU fake/real Health state Vcore 08:36:16: 49.9'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:21: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:26: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:31: 51.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:37: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:42: 50.5'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:47: 51.0'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:53: 52.1'C 1400/1400 MHz 0000000000000000000 1.3250V 08:36:58: 46.2'C 600/ 600 MHz 0000000000000000000 1.2V @tkaiser Again a little slower than Lite. In the afternoon I'll check Ubuntu. More to come. Cheers 1 Quote
NicoD Posted July 26, 2018 Posted July 26, 2018 @tkaiser In Ubuntu it again got killed every time. Something's definitely wrong there. It also seems a lot slower. It is a fresh install of Ubuntu on the Rasp3B. I've added the data of your raspimon so you see it's not overheating or so. For all the tests I've used my fan. Here's the Ubuntu result of the RPi 3b nicod@nicod-desktop:~$ 7zr b ; 7zr b 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs) RAM size: 897 MB, # CPU hardware threads: 4 RAM usage: 850 MB, # Benchmark threads: 4 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 1592 320 484 1549 | 39719 369 971 3583 23: 1568 324 492 1598 | 39481 373 967 3612 24: 1425 302 507 1533 | 38455 370 963 3567 Killed 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,4 CPUs) RAM size: 897 MB, # CPU hardware threads: 4 RAM usage: 850 MB, # Benchmark threads: 4 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 1588 317 488 1545 | 39872 370 971 3597 23: 1312 274 487 1337 | 35413 342 948 3240 24: 1492 316 507 1605 | 39071 376 964 3624 Killed 12:43:16: 44.5'C 600/ 600 MHz 0000000000000000000 1.2V 12:43:21: 50.5'C 1200/1200 MHz 0000000000000000000 1.2688V 12:43:26: 53.2'C 1200/1200 MHz 0000000000000000000 1.2688V 12:43:31: 53.7'C 1200/1200 MHz 0000000000000000000 1.2688V 12:43:37: 54.8'C 1200/1200 MHz 0000000000000000000 1.2688V 12:43:42: 55.3'C 1200/1200 MHz 0000000000000000000 1.2688V 12:43:47: 56.9'C 1200/1200 MHz 0000000000000000000 1.2688V 12:43:52: 55.8'C 1200/1200 MHz 0000000000000000000 1.2688V 12:43:57: 56.4'C 1200/1200 MHz 0000000000000000000 1.2688V 12:44:02: 55.8'C 1200/1200 MHz 0000000000000000000 1.2688V 12:44:07: 56.4'C 1200/1200 MHz 0000000000000000000 1.2688V 12:44:12: 55.8'C 1200/1200 MHz 0000000000000000000 1.2688V 12:44:18: 58.5'C 1200/1200 MHz 0000000000000000000 1.2688V 12:44:23: 55.8'C 1200/1200 MHz 0000000000000000000 1.2688V Time Temp CPU fake/real Health state Vcore 12:44:28: 56.9'C 1200/1200 MHz 0000000000000000000 1.2688V 12:44:33: 56.9'C 1200/1200 MHz 0000000000000000000 1.2688V 12:44:38: 56.9'C 1200/1200 MHz 0000000000000000000 1.2688V 12:44:43: 57.5'C 1200/1200 MHz 0000000000000000000 1.2688V 12:44:48: 58.0'C 1200/1200 MHz 0000000000000000000 1.2688V @pbies As tkaiser told you. Sysbench isn't very good to compare cpu scores. And what my findings are, is that there's no real perfect way of measuring cpu's between each other. 7zip is very good for that task, but again many factors come into play. There's sd-card speed, ram-speed, distro, distro-version, version of the benchmark tool, 32-bit vs 64-bit, temperature, and many more things. It's always comparing apples and oranges(and raspberry's and banana's and ....) My tip is, use the program you want to use you sbc for as benchmark. That's the only information you need. All the rest is just numbers without much meaning. I use my sbc's for video editing and as 3d-render farm. So I use those tools to benchmark my different sbc's. I only need to know which sbc does the fastest render. If I then compare my render results with sysbench results, then I find there's no correlation. Some do some tasks better, some do other tasks better. My best video render sbc isn't my best 3d-Blender render sbc... Greetings. 0 Quote
pbies Posted July 26, 2018 Author Posted July 26, 2018 @tkaiser @NicoD I understand your posts, but don't agree with them. As 7z is not enough reliable in testing only CPU, I didn't found other benchmark to test CPU. If you have any other benchmark that would be reliable - just tell me the name. 7z is killed by kernel, because it is using too much resources for its tests. And this is normal for kernel. At later test it is using over 2GB of RAM, so it is using swapfile (much worse results) then and gets killed. That's enough for me to assume that it is not a reliable benchmark. And this is the reason why I am not putting here results of 7z benchmarks. Still I don't see any other better only-CPU test for SBCs. And I assume that each compiler do its best for each architecture, so still sysbench stays with me much more than 7z. 0 Quote
tkaiser Posted July 26, 2018 Posted July 26, 2018 19 minutes ago, NicoD said: In Ubuntu it again got killed every time. Something's definitely wrong there Most probably related to OOM killer settings (see here). The problem is that Debian and Ubuntu love to keep horribly outdated program versions in their repos (nice rant again). Version 9.20 that's used by Ubuntu Xenial and Debian Jessie is from 2010 (see changelog -- it's a shame to not package a more recent version). Version 16.02 now used by more recent distros is 'just' 2 years old. So to get a 'bigger picture' execution with Debian Stretch on every board would be needed. 0 Quote
NicoD Posted July 26, 2018 Posted July 26, 2018 16 minutes ago, pbies said: reliable in testing only CPU An SBC/computer isn't a CPU alone. So you can not test the cpu alone. Why do you need to know? There are other tools, but again none are perfect. Or you use as many as possible and make averages, or you benchmark with the programs you want to use. Or just 7zip to get an idea of how it compares. But it doesn't mean that the fastest 7zip sbc will be the fastest for what you want to do. 0 Quote
NicoD Posted July 26, 2018 Posted July 26, 2018 1 minute ago, tkaiser said: to keep horribly outdated program versions in their repos (nice rant again) Very true. I always need to compile Kdenlive myself because they're too lazy to update the repo's. I tried to use Ubuntu to show it gives different results.(with every tool there is, not just 7zip) Tho the differences shouldn't be too big. Cheers 0 Quote
pbies Posted July 26, 2018 Author Posted July 26, 2018 @NicoD SBCs aren't CPU alone, but I need to test only CPU. 7z is dependant on RAM mainly. This way I can use MemTest and get the same level-reliable results. So what kind of software do you propose then? Do you have any software that tests only CPU and isn't memory-dependant? And please don't tell me that it doesn't exist - any such software can operate only on registers inside CPU. 0 Quote
NicoD Posted July 26, 2018 Posted July 26, 2018 12 minutes ago, pbies said: So what kind of software do you propose then? https://haydenjames.io/linux-benchmark-scripts-tools/https://www.makeuseof.com/tag/benchmark-linux-pcs-performance/ As said, there are a lot of tools for that. You could also use a miner. Or anything you want. But not all cpu's work the same. Some have more cache, faster memory, other architecture. So some will perform better for some tasks, other for other tasks. What do you want it for? You can use sysbench too. But always make sure you use the same version, the same distro/version. It's not going to tell you everything about a cpu. It's just numbers. That's why I don't use tools like that for benchmarks, but real live tasks to see what's best for the right use. Greetings 0 Quote
NicoD Posted July 26, 2018 Posted July 26, 2018 And also always check the temperature of the CPU, the frequency, the voltage... If a cpu throttle's a lot, then many times it's better to underclock it for better performance. For example the XU4 at max freq is always overheating. If you go 200Mhz(1800Mhz) lower with the big cores your results will be better than at 2Ghz. 0 Quote
pbies Posted July 26, 2018 Author Posted July 26, 2018 @NicoD still these rely on RAM speed, compression. I would go for Phoronix Test Suite and stress-ng. Thanks. 1 Quote
tkaiser Posted July 26, 2018 Posted July 26, 2018 43 minutes ago, pbies said: Phoronix Test Suite LOL, the Moronix Test Suite: https://forum.pine64.org/showthread.php?tid=389&pid=3259#pid3259 1 hour ago, NicoD said: 1 hour ago, pbies said: So what kind of software do you propose then? https://haydenjames.io/linux-benchmark-scripts-tools/ Unixbench: http://www.brendangregg.com/blog/2014-05-02/compilers-love-messing-with-benchmarks.html Bonnie: http://www.brendangregg.com/ActiveBenchmarking/bonnie++.html And so on... all those tools usually recommended are popular since they're easy to execute, do not care about correct benchmarking but people are happy with numbers without meaning. Good luck... 1 Quote
NicoD Posted July 26, 2018 Posted July 26, 2018 39 minutes ago, pbies said: @NicoD still these rely on RAM speed Every process rely's on the RAM. You can not run a program without memory. A program loads into the RAM. The cpu takes blocks of data/instuctions out the ram. It computes what's asked from it, and writes the data back to the ram. Ram and the CPU are a whole. A cpu can't do anything without ram, and you're nothing with memory without a cpu(except for other applications than pc's). So you can not benchmark a cpu without using ram. If you calculate prime like with sysbench. The instruction is red out of the memory into the cpu, the calculations of the cpu are written into ram, and a new instruction is passed to the cpu. So having only cpu benchmarks are impossible. Just so you understand. 0 Quote
pbies Posted July 26, 2018 Author Posted July 26, 2018 @tkaiser Really, I am not interested in your negative way of thinking. Please leave me alone out of your ideas. @NicoD You didn't understood what I said: I need benchmark for CPU, not RAM. And to benchmark CPU you don't need tons of RAM as for 7z. You understand now direction in which I want to go? I don't need to test tons of RAM, I agree that nothing without it would exist, but still operations on registers is enough to test CPU, and NOT on RAM. 0 Quote
tkaiser Posted July 26, 2018 Posted July 26, 2018 14 minutes ago, pbies said: Really, I am not interested in your negative way of thinking. Please leave me alone out of your ideas LOL! And once again: http://www.brendangregg.com/activebenchmarking.html 0 Quote
NicoD Posted July 26, 2018 Posted July 26, 2018 Even when a program uses less ram it will read and write as much. Then just always the same data. I could write something for you that uses 100kb of ram, but the spu would constantly need to read the ram. Maybe you could write something in assembler where you send the cpu in a loop without reading much, and then count the loops/s. But still the ram would be used to count. You can`t count anything without saving that to ram. Else you wouldn`t know any number. So last time. A program can not run without ram. All instructions for the cpu come from ram. Otherwise just check the clockspeed with a great ocyloscope. And you`ll know how many clocks it did. But that doesn`t say anything. You are searching for something unexisting. If you were a programmer you would understand. Have a nice day. And tkaiser is a great guy. He`s trying to help you, but you don`t understand the facts. 0 Quote
NicoD Posted July 26, 2018 Posted July 26, 2018 @pbies My last words about this. See the cpu as an engine, and the ram is the fuel pump. The engine can not run without the fuelpump because it needs gasoline to run. Just as a cpu can`t do anything without data what the ram is providing. With every test you do, overclock the ram and it will run faster. Overclock the cpu and it will run faster. Both are as important. That`s why the raspberry pi is the slowest sbc, while the cpu is ok. The ram is too slow. Greetings 0 Quote
tkaiser Posted July 26, 2018 Posted July 26, 2018 (edited) On 7/26/2018 at 2:16 PM, NicoD said: always check the temperature of the CPU, the frequency, the voltage... First attempt on sbc-bench.sh (I'll put this later on Github) In case you still have the RPi 3B+ around it would be really great if you could let the script run without a fan (I need the throttling situation to see whether everything is reported well). The script is designed to work on normal ARM SBC as well as on the crippled Raspberries (VideoCore main CPU controlling the hardware and all the proprietary TheadX stuff). Basic requirement is either Debian Stretch or Ubuntu Bionic (or something similar with somewhat recent packages) Output from a RPi 2 with Raspbian Stretch I had access to: root@raspberrypi:/home/pi# /usr/local/bin/sbc-bench.sh Installing needed tools. This may take some time... Done. Executing tinymembench. This will take a long time... Done. Executing 7-zip benchmark. This will take a long time... Done. Executing OpenSSL benchmark. This will take a long time... Done. Below benchmark results: Memory performance: memcpy: 1013.7 MB/s memset: 1167.9 MB/s 7-zip total scores (three runs): 2134,2129,2125 OpenSSL results: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes aes-128-cbc 14216.59k 19191.70k 21097.90k 21513.56k 21793.45k 21785.26k aes-128-cbc 14147.59k 19138.28k 21094.49k 21632.68k 21673.30k 21796.18k aes-192-cbc 12802.24k 16565.27k 18059.09k 18446.34k 18352.81k 18568.53k aes-192-cbc 12807.21k 16660.76k 17955.16k 18443.61k 18557.61k 18459.31k aes-256-cbc 11806.24k 14964.78k 16007.42k 16251.22k 16460.46k 16465.92k aes-256-cbc 11738.95k 14962.37k 16062.21k 16279.21k 16455.00k 16460.46k Full results uploaded to http://ix.io/1irc. Please check the log for anomalies (e.g. swapping or throttling happenend) and otherwise share this URL. Output from a heavily throttled NanoPC-T4 with some preliminary Armbian I built few weeks ago: root@nanopct4:/tmp# /usr/local/bin/sbc-bench.sh Installing needed tools. This may take some time... Done. Executing tinymembench. This will take a long time... Done. Executing 7-zip benchmark. This will take a long time... Done. Executing OpenSSL benchmark. This will take a long time... Done. ATTENTION: Throttling occured on CPU cluster 4. Check the uploaded log for details. Below benchmark results: Memory performance (on big.LITTLE systems measured individually): memcpy: 2060.7 MB/s (0.3%) memset: 8440.3 MB/s (0.6%) memcpy: 4000.9 MB/s (0.6%) memset: 9011.0 MB/s (0.7%) 7-zip total scores (three runs): 4496,3980,3837 OpenSSL results (on big.LITTLE systems measured individually): type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes aes-128-cbc 91970.98k 292346.01k 630755.93k 914955.95k 1053515.78k 1065336.83k aes-128-cbc 272534.69k 638176.02k 956555.26k 1068235.78k 1137390.93k 1144930.30k aes-192-cbc 92993.86k 274024.58k 532484.78k 715422.38k 794593.96k 797059.75k aes-192-cbc 260573.74k 581688.00k 797185.54k 940168.87k 982846.12k 981581.82k aes-256-cbc 90710.94k 257319.98k 466747.22k 601770.33k 656547.84k 654611.80k aes-256-cbc 235117.07k 531296.70k 735206.31k 807839.74k 850635.43k 852792.66k Full results uploaded to http://ix.io/1irf. Please check the log for anomalies (e.g. swapping or throttling happenend) and otherwise share this URL. To let it run it's as easy as: wget -O /usr/local/bin/sbc-bench.sh https://raw.githubusercontent.com/ThomasKaiser/sbc-bench/master/sbc-bench.sh chmod 755 /usr/local/bin/sbc-bench.sh sudo /usr/local/bin/sbc-bench.sh (unfortunately needs root privileges to manipulate cpufreq governor and access monitoring data) Edited July 27, 2018 by tkaiser Adjusted download URL to Github 1 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.