NicoD Posted February 22, 2019 Author Posted February 22, 2019 14 minutes ago, TonyMac32 said: He also hung a lantern on it, so to speak:. "silly kitchen sink benchmarking". He's illustrating how numbers can be used to selectively represent reality. But isn't that exactly the same reason why rooted did the benchmarks? To check if the numbers from Hardkernel were fictional or not? 1+1= still 2 On 2/21/2019 at 10:48 AM, tkaiser said: Funny place here... Bout time to come back to make it more funny maybe?
balbes150 Posted February 22, 2019 Posted February 22, 2019 On 2/21/2019 at 2:31 PM, rooted said: Test for yourselves, Hardkernel has 4 N2 on the bench ready for testing. You have the ability to perform an N2 check with Blender (which you have already done), but with as many kernel maximum frequency settings as you can get ? A prerequisite is an excellent cooling system with a guaranteed maximum temperature from the N2 sensors, throughout the test, no higher than 60-65 degrees. The cooling system and the surrounding conditions are entirely at your choice, passive or active with or without an additional fan, i.e. any option without limitation.
rooted Posted February 22, 2019 Posted February 22, 2019 You have the ability to perform an N2 check with Blender (which you have already done), but with as many kernel maximum frequency settings as you can get ? A prerequisite is an excellent cooling system with a guaranteed maximum temperature from the N2 sensors, throughout the test, no higher than 60-65 degrees. The cooling system and the surrounding conditions are entirely at your choice, passive or active with or without an additional fan, i.e. any option without limitation.I ran the BMW benchmark at stock frequencies, I haven't ran it overclocked and my AC was off.
tkaiser Posted February 22, 2019 Posted February 22, 2019 (edited) Blender test. Checking the relevance of SETTINGS instead of focusing on irrelevant hardware details like DDR3 vs. DDR4: RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:15:31 RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:06:59 RockPro64, Ubuntu Cosmic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:01:11 That's 8:30 minutes difference due to switching from CONFIG_HZ=1000 to CONFIG_HZ=250. Check %sys vs. %user below (iostat 60 output). And why is Cosmic faster than Bionic if it's exactly same Blender version? Since Cosmic (18.10) uses GCC 8.2 while Bionic (18.04) uses GCC 7.3 to build the packages. So by switching from default SoC vendor kernel settings to something better and by letting modern compilers do their job we get almost 25% performance 'for free'. RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:15:31 avg-cpu: %user %nice %system %iowait %steal %idle 37.36 0.00 2.82 0.60 0.00 59.22 97.34 0.00 2.26 0.00 0.00 0.41 97.45 0.00 2.25 0.00 0.00 0.31 97.48 0.00 2.21 0.00 0.00 0.31 97.44 0.00 2.22 0.00 0.00 0.34 97.23 0.02 2.44 0.00 0.00 0.32 97.28 0.00 2.39 0.00 0.00 0.33 97.32 0.01 2.33 0.00 0.00 0.34 97.40 0.00 2.22 0.00 0.00 0.38 97.33 0.01 2.18 0.00 0.00 0.49 97.38 0.00 2.29 0.00 0.00 0.33 97.29 0.00 2.33 0.00 0.00 0.38 97.32 0.00 2.32 0.00 0.00 0.36 97.40 0.00 2.29 0.00 0.00 0.31 97.47 0.01 2.24 0.00 0.00 0.28 97.34 0.00 2.35 0.00 0.00 0.31 97.34 0.00 2.37 0.00 0.00 0.29 97.39 0.00 2.29 0.00 0.00 0.32 97.46 0.00 2.26 0.00 0.00 0.28 97.61 0.00 2.11 0.00 0.00 0.28 97.38 0.00 2.30 0.00 0.00 0.32 97.22 0.00 2.40 0.00 0.00 0.38 97.47 0.00 2.20 0.00 0.00 0.33 97.45 0.00 2.24 0.00 0.00 0.31 97.52 0.00 2.15 0.00 0.00 0.33 97.36 0.00 2.27 0.00 0.00 0.37 97.39 0.00 2.33 0.00 0.00 0.28 97.28 0.00 2.24 0.00 0.00 0.48 97.53 0.00 2.16 0.00 0.00 0.30 97.45 0.00 2.19 0.00 0.00 0.35 97.54 0.00 2.16 0.00 0.00 0.30 97.39 0.00 2.28 0.00 0.00 0.33 97.40 0.01 2.21 0.00 0.00 0.39 97.50 0.00 2.16 0.00 0.00 0.34 97.67 0.00 1.98 0.00 0.00 0.35 97.54 0.00 2.10 0.00 0.00 0.36 97.33 0.00 2.23 0.00 0.00 0.43 97.40 0.00 2.30 0.00 0.00 0.30 97.47 0.00 2.15 0.00 0.00 0.37 97.60 0.00 2.09 0.00 0.00 0.31 97.54 0.00 2.14 0.00 0.00 0.32 97.61 0.00 2.13 0.00 0.00 0.26 97.51 0.00 2.15 0.00 0.00 0.35 97.48 0.00 2.16 0.00 0.00 0.36 97.45 0.00 2.22 0.00 0.00 0.33 97.34 0.00 2.32 0.00 0.00 0.34 97.61 0.00 2.10 0.00 0.00 0.30 97.37 0.00 2.34 0.00 0.00 0.30 97.32 0.00 2.37 0.00 0.00 0.30 97.40 0.00 2.25 0.00 0.00 0.34 97.46 0.00 2.23 0.00 0.00 0.31 97.45 0.00 2.20 0.00 0.00 0.36 97.49 0.00 2.18 0.00 0.00 0.34 97.34 0.00 2.35 0.00 0.00 0.31 97.41 0.00 2.32 0.00 0.00 0.27 97.42 0.00 2.25 0.00 0.00 0.33 97.49 0.00 2.23 0.00 0.00 0.28 97.51 0.00 2.15 0.00 0.00 0.34 97.52 0.00 2.17 0.00 0.00 0.31 97.42 0.00 2.24 0.00 0.00 0.34 97.42 0.00 2.26 0.00 0.00 0.31 97.47 0.00 2.16 0.00 0.00 0.37 97.52 0.00 2.19 0.00 0.00 0.29 97.55 0.00 2.08 0.00 0.00 0.37 97.36 0.00 2.30 0.00 0.00 0.34 97.60 0.00 2.03 0.00 0.00 0.36 97.56 0.01 2.05 0.00 0.00 0.39 97.74 0.00 1.99 0.00 0.00 0.28 97.63 0.00 1.93 0.00 0.00 0.44 97.57 0.00 2.14 0.00 0.00 0.29 97.63 0.00 2.03 0.00 0.00 0.34 97.65 0.00 2.03 0.00 0.00 0.32 97.40 0.00 2.20 0.00 0.00 0.39 98.03 0.00 1.63 0.00 0.00 0.34 97.66 0.00 2.03 0.00 0.00 0.30 50.37 0.00 2.53 0.00 0.00 47.09 4.41 0.00 3.20 0.00 0.00 92.40 root@rockpro64:~# zgrep CONFIG_HZ /proc/config.gz # CONFIG_HZ_PERIODIC is not set # CONFIG_HZ_100 is not set # CONFIG_HZ_250 is not set # CONFIG_HZ_300 is not set CONFIG_HZ_1000=y CONFIG_HZ=1000 RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:06:59 avg-cpu: %user %nice %system %iowait %steal %idle 0.48 0.00 0.30 0.01 0.00 99.21 65.25 0.00 0.75 0.01 0.00 33.99 99.24 0.00 0.52 0.00 0.00 0.24 98.99 0.00 0.54 0.00 0.00 0.46 98.89 0.00 0.55 0.00 0.00 0.56 98.74 0.00 0.59 0.00 0.00 0.66 98.97 0.00 0.57 0.00 0.00 0.46 99.07 0.00 0.53 0.00 0.00 0.40 99.11 0.00 0.51 0.00 0.00 0.38 98.80 0.00 0.57 0.00 0.00 0.63 98.95 0.00 0.53 0.00 0.00 0.52 98.87 0.00 0.60 0.00 0.00 0.53 99.00 0.00 0.53 0.00 0.00 0.47 99.04 0.00 0.51 0.00 0.00 0.46 98.95 0.00 0.55 0.00 0.00 0.50 98.91 0.00 0.57 0.00 0.00 0.52 98.99 0.00 0.54 0.00 0.00 0.47 99.07 0.00 0.59 0.00 0.00 0.34 99.00 0.00 0.50 0.00 0.00 0.50 98.98 0.00 0.55 0.00 0.00 0.47 98.87 0.00 0.60 0.00 0.00 0.53 99.00 0.00 0.58 0.00 0.00 0.41 99.00 0.00 0.52 0.00 0.00 0.48 98.96 0.00 0.55 0.00 0.00 0.48 99.04 0.00 0.52 0.00 0.00 0.44 99.10 0.00 0.56 0.00 0.00 0.34 99.05 0.00 0.50 0.00 0.00 0.45 99.07 0.00 0.50 0.00 0.00 0.43 98.98 0.00 0.53 0.00 0.00 0.49 99.15 0.00 0.48 0.00 0.00 0.36 98.93 0.00 0.55 0.00 0.00 0.51 99.06 0.00 0.54 0.00 0.00 0.40 98.98 0.00 0.53 0.00 0.00 0.49 98.90 0.00 0.60 0.00 0.00 0.50 99.10 0.00 0.49 0.00 0.00 0.41 99.02 0.00 0.54 0.00 0.00 0.44 98.97 0.00 0.58 0.00 0.00 0.45 99.00 0.00 0.55 0.00 0.00 0.45 98.93 0.00 0.57 0.00 0.00 0.50 99.07 0.00 0.51 0.00 0.00 0.42 99.05 0.00 0.51 0.00 0.00 0.44 99.04 0.00 0.56 0.00 0.00 0.40 98.90 0.00 0.56 0.00 0.00 0.53 98.92 0.00 0.56 0.00 0.00 0.52 99.05 0.00 0.50 0.00 0.00 0.45 98.94 0.00 0.58 0.00 0.00 0.49 98.96 0.00 0.55 0.00 0.00 0.49 98.83 0.00 0.56 0.00 0.00 0.60 99.07 0.00 0.50 0.00 0.00 0.43 98.89 0.00 0.56 0.00 0.00 0.55 98.98 0.00 0.54 0.00 0.00 0.48 99.11 0.00 0.49 0.00 0.00 0.39 98.97 0.00 0.55 0.00 0.00 0.48 99.05 0.00 0.54 0.00 0.00 0.41 99.07 0.00 0.54 0.00 0.00 0.39 99.02 0.00 0.51 0.00 0.00 0.47 98.93 0.00 0.55 0.00 0.00 0.52 98.83 0.00 0.54 0.00 0.00 0.63 99.00 0.00 0.53 0.00 0.00 0.48 99.03 0.00 0.47 0.00 0.00 0.50 98.93 0.00 0.53 0.00 0.00 0.55 99.05 0.00 0.47 0.00 0.00 0.48 98.84 0.00 0.57 0.00 0.00 0.60 99.10 0.00 0.48 0.00 0.00 0.42 99.05 0.00 0.52 0.00 0.00 0.43 98.98 0.00 0.52 0.00 0.00 0.50 98.96 0.00 0.50 0.00 0.00 0.54 98.88 0.00 0.51 0.00 0.00 0.61 27.79 0.00 0.52 0.23 0.00 71.46 0.45 0.00 0.24 0.01 0.00 99.30 root@rockpro64:~# zgrep CONFIG_HZ /proc/config.gz # CONFIG_HZ_PERIODIC is not set # CONFIG_HZ_100 is not set CONFIG_HZ_250=y # CONFIG_HZ_300 is not set # CONFIG_HZ_1000 is not set CONFIG_HZ=250 RockPro64, Ubuntu Cosmic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:01:11 avg-cpu: %user %nice %system %iowait %steal %idle 76.64 0.00 0.60 0.17 0.00 22.59 99.42 0.00 0.28 0.00 0.00 0.30 99.48 0.00 0.26 0.00 0.00 0.25 99.26 0.00 0.33 0.00 0.00 0.41 99.34 0.00 0.33 0.00 0.00 0.33 99.41 0.00 0.31 0.00 0.00 0.28 99.28 0.00 0.34 0.00 0.00 0.38 99.25 0.00 0.35 0.00 0.00 0.39 99.41 0.00 0.31 0.00 0.00 0.27 99.28 0.00 0.34 0.00 0.00 0.38 99.28 0.00 0.30 0.00 0.00 0.42 99.20 0.00 0.40 0.00 0.00 0.40 99.38 0.00 0.31 0.00 0.00 0.31 99.25 0.00 0.32 0.00 0.00 0.43 99.27 0.00 0.35 0.00 0.00 0.38 99.36 0.00 0.32 0.00 0.00 0.32 99.38 0.00 0.29 0.00 0.00 0.34 99.24 0.00 0.33 0.00 0.00 0.43 99.26 0.00 0.32 0.00 0.00 0.42 99.32 0.00 0.32 0.00 0.00 0.36 99.45 0.00 0.29 0.00 0.00 0.26 99.47 0.00 0.28 0.00 0.00 0.24 99.42 0.00 0.33 0.00 0.00 0.25 99.23 0.00 0.30 0.00 0.00 0.47 99.31 0.00 0.34 0.00 0.00 0.35 99.34 0.00 0.33 0.00 0.00 0.33 99.24 0.00 0.33 0.00 0.00 0.43 99.22 0.00 0.37 0.00 0.00 0.41 99.25 0.00 0.35 0.00 0.00 0.40 99.28 0.00 0.36 0.00 0.00 0.36 99.16 0.00 0.42 0.00 0.00 0.43 99.31 0.00 0.33 0.00 0.00 0.36 99.31 0.00 0.33 0.00 0.00 0.36 99.37 0.00 0.36 0.00 0.00 0.28 99.36 0.00 0.31 0.00 0.00 0.33 99.42 0.00 0.30 0.00 0.00 0.28 99.22 0.00 0.37 0.00 0.00 0.41 99.29 0.00 0.32 0.00 0.00 0.39 99.24 0.00 0.36 0.00 0.00 0.40 99.23 0.00 0.38 0.00 0.00 0.39 99.33 0.00 0.33 0.00 0.00 0.34 99.26 0.00 0.40 0.00 0.00 0.34 99.34 0.00 0.34 0.00 0.00 0.33 99.19 0.00 0.31 0.00 0.00 0.51 99.33 0.00 0.32 0.00 0.00 0.35 99.17 0.00 0.38 0.00 0.00 0.46 99.17 0.00 0.37 0.00 0.00 0.47 99.47 0.00 0.33 0.00 0.00 0.20 99.22 0.00 0.31 0.00 0.00 0.47 99.28 0.00 0.33 0.00 0.00 0.39 99.25 0.00 0.36 0.00 0.00 0.39 99.23 0.00 0.31 0.00 0.00 0.46 99.25 0.00 0.33 0.00 0.00 0.42 18.80 0.00 0.43 0.10 0.00 80.67 root@rockpro64:~# gcc --version gcc (Ubuntu 8.2.0-7ubuntu1) 8.2.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. root@rockpro64:~# lsb_release -c Codename: cosmic root@rockpro64:~# zgrep CONFIG_HZ /proc/config.gz # CONFIG_HZ_PERIODIC is not set # CONFIG_HZ_100 is not set CONFIG_HZ_250=y # CONFIG_HZ_300 is not set # CONFIG_HZ_1000 is not set CONFIG_HZ=250 Edited February 22, 2019 by tkaiser Added GCC 8.2 results 3
AreaScout Posted February 22, 2019 Posted February 22, 2019 3 hours ago, TonyMac32 said: He also hung a lantern on it, so to speak:. "silly kitchen sink benchmarking". He's illustrating how numbers can be used to selectively represent reality. Sent from my Pixel using Tapatalk Yes the GPU Benchmark for example must be taken with doubts, it shows a result from ~255fps with glmark2 and --off-screen rendering switch with fbdev drivers on XU4 and 285fps on N1 and 305fps on N2, but the XU4 was able to do the same test with X11 drivers and 1200fps, but we do not have X11 drivers for N2, but from my experience the N1 and XU4 was relatively equal when it comes to real live gaming / emulation tests, maybe slightly better and not so stressed out like the XU4 with the same tasks in terms of temperature, also the Mali-G52 GPU is very new (Q1 2018) and maybe not fully supported from software side, so it's too early to make a clear statement.
NicoD Posted February 22, 2019 Author Posted February 22, 2019 34 minutes ago, tkaiser said: Blender test. Checking the relevance of SETTINGS instead of focusing on irrelevant hardware details like DDR3 vs. DDR4: RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:15:31 RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:06:59 That's 8:30 minutes difference due to switching from CONFIG_HZ=1000 to CONFIG_HZ=250. Check %sys vs. %user below (iostat 60 output): Reveal hidden contents RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:15:31 avg-cpu: %user %nice %system %iowait %steal %idle 37.36 0.00 2.82 0.60 0.00 59.22 97.34 0.00 2.26 0.00 0.00 0.41 97.45 0.00 2.25 0.00 0.00 0.31 97.48 0.00 2.21 0.00 0.00 0.31 97.44 0.00 2.22 0.00 0.00 0.34 97.23 0.02 2.44 0.00 0.00 0.32 97.28 0.00 2.39 0.00 0.00 0.33 97.32 0.01 2.33 0.00 0.00 0.34 97.40 0.00 2.22 0.00 0.00 0.38 97.33 0.01 2.18 0.00 0.00 0.49 97.38 0.00 2.29 0.00 0.00 0.33 97.29 0.00 2.33 0.00 0.00 0.38 97.32 0.00 2.32 0.00 0.00 0.36 97.40 0.00 2.29 0.00 0.00 0.31 97.47 0.01 2.24 0.00 0.00 0.28 97.34 0.00 2.35 0.00 0.00 0.31 97.34 0.00 2.37 0.00 0.00 0.29 97.39 0.00 2.29 0.00 0.00 0.32 97.46 0.00 2.26 0.00 0.00 0.28 97.61 0.00 2.11 0.00 0.00 0.28 97.38 0.00 2.30 0.00 0.00 0.32 97.22 0.00 2.40 0.00 0.00 0.38 97.47 0.00 2.20 0.00 0.00 0.33 97.45 0.00 2.24 0.00 0.00 0.31 97.52 0.00 2.15 0.00 0.00 0.33 97.36 0.00 2.27 0.00 0.00 0.37 97.39 0.00 2.33 0.00 0.00 0.28 97.28 0.00 2.24 0.00 0.00 0.48 97.53 0.00 2.16 0.00 0.00 0.30 97.45 0.00 2.19 0.00 0.00 0.35 97.54 0.00 2.16 0.00 0.00 0.30 97.39 0.00 2.28 0.00 0.00 0.33 97.40 0.01 2.21 0.00 0.00 0.39 97.50 0.00 2.16 0.00 0.00 0.34 97.67 0.00 1.98 0.00 0.00 0.35 97.54 0.00 2.10 0.00 0.00 0.36 97.33 0.00 2.23 0.00 0.00 0.43 97.40 0.00 2.30 0.00 0.00 0.30 97.47 0.00 2.15 0.00 0.00 0.37 97.60 0.00 2.09 0.00 0.00 0.31 97.54 0.00 2.14 0.00 0.00 0.32 97.61 0.00 2.13 0.00 0.00 0.26 97.51 0.00 2.15 0.00 0.00 0.35 97.48 0.00 2.16 0.00 0.00 0.36 97.45 0.00 2.22 0.00 0.00 0.33 97.34 0.00 2.32 0.00 0.00 0.34 97.61 0.00 2.10 0.00 0.00 0.30 97.37 0.00 2.34 0.00 0.00 0.30 97.32 0.00 2.37 0.00 0.00 0.30 97.40 0.00 2.25 0.00 0.00 0.34 97.46 0.00 2.23 0.00 0.00 0.31 97.45 0.00 2.20 0.00 0.00 0.36 97.49 0.00 2.18 0.00 0.00 0.34 97.34 0.00 2.35 0.00 0.00 0.31 97.41 0.00 2.32 0.00 0.00 0.27 97.42 0.00 2.25 0.00 0.00 0.33 97.49 0.00 2.23 0.00 0.00 0.28 97.51 0.00 2.15 0.00 0.00 0.34 97.52 0.00 2.17 0.00 0.00 0.31 97.42 0.00 2.24 0.00 0.00 0.34 97.42 0.00 2.26 0.00 0.00 0.31 97.47 0.00 2.16 0.00 0.00 0.37 97.52 0.00 2.19 0.00 0.00 0.29 97.55 0.00 2.08 0.00 0.00 0.37 97.36 0.00 2.30 0.00 0.00 0.34 97.60 0.00 2.03 0.00 0.00 0.36 97.56 0.01 2.05 0.00 0.00 0.39 97.74 0.00 1.99 0.00 0.00 0.28 97.63 0.00 1.93 0.00 0.00 0.44 97.57 0.00 2.14 0.00 0.00 0.29 97.63 0.00 2.03 0.00 0.00 0.34 97.65 0.00 2.03 0.00 0.00 0.32 97.40 0.00 2.20 0.00 0.00 0.39 98.03 0.00 1.63 0.00 0.00 0.34 97.66 0.00 2.03 0.00 0.00 0.30 50.37 0.00 2.53 0.00 0.00 47.09 4.41 0.00 3.20 0.00 0.00 92.40 root@rockpro64:~# zgrep CONFIG_HZ /proc/config.gz # CONFIG_HZ_PERIODIC is not set # CONFIG_HZ_100 is not set # CONFIG_HZ_250 is not set # CONFIG_HZ_300 is not set CONFIG_HZ_1000=y CONFIG_HZ=1000 RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:06:59 avg-cpu: %user %nice %system %iowait %steal %idle 0.48 0.00 0.30 0.01 0.00 99.21 65.25 0.00 0.75 0.01 0.00 33.99 99.24 0.00 0.52 0.00 0.00 0.24 98.99 0.00 0.54 0.00 0.00 0.46 98.89 0.00 0.55 0.00 0.00 0.56 98.74 0.00 0.59 0.00 0.00 0.66 98.97 0.00 0.57 0.00 0.00 0.46 99.07 0.00 0.53 0.00 0.00 0.40 99.11 0.00 0.51 0.00 0.00 0.38 98.80 0.00 0.57 0.00 0.00 0.63 98.95 0.00 0.53 0.00 0.00 0.52 98.87 0.00 0.60 0.00 0.00 0.53 99.00 0.00 0.53 0.00 0.00 0.47 99.04 0.00 0.51 0.00 0.00 0.46 98.95 0.00 0.55 0.00 0.00 0.50 98.91 0.00 0.57 0.00 0.00 0.52 98.99 0.00 0.54 0.00 0.00 0.47 99.07 0.00 0.59 0.00 0.00 0.34 99.00 0.00 0.50 0.00 0.00 0.50 98.98 0.00 0.55 0.00 0.00 0.47 98.87 0.00 0.60 0.00 0.00 0.53 99.00 0.00 0.58 0.00 0.00 0.41 99.00 0.00 0.52 0.00 0.00 0.48 98.96 0.00 0.55 0.00 0.00 0.48 99.04 0.00 0.52 0.00 0.00 0.44 99.10 0.00 0.56 0.00 0.00 0.34 99.05 0.00 0.50 0.00 0.00 0.45 99.07 0.00 0.50 0.00 0.00 0.43 98.98 0.00 0.53 0.00 0.00 0.49 99.15 0.00 0.48 0.00 0.00 0.36 98.93 0.00 0.55 0.00 0.00 0.51 99.06 0.00 0.54 0.00 0.00 0.40 98.98 0.00 0.53 0.00 0.00 0.49 98.90 0.00 0.60 0.00 0.00 0.50 99.10 0.00 0.49 0.00 0.00 0.41 99.02 0.00 0.54 0.00 0.00 0.44 98.97 0.00 0.58 0.00 0.00 0.45 99.00 0.00 0.55 0.00 0.00 0.45 98.93 0.00 0.57 0.00 0.00 0.50 99.07 0.00 0.51 0.00 0.00 0.42 99.05 0.00 0.51 0.00 0.00 0.44 99.04 0.00 0.56 0.00 0.00 0.40 98.90 0.00 0.56 0.00 0.00 0.53 98.92 0.00 0.56 0.00 0.00 0.52 99.05 0.00 0.50 0.00 0.00 0.45 98.94 0.00 0.58 0.00 0.00 0.49 98.96 0.00 0.55 0.00 0.00 0.49 98.83 0.00 0.56 0.00 0.00 0.60 99.07 0.00 0.50 0.00 0.00 0.43 98.89 0.00 0.56 0.00 0.00 0.55 98.98 0.00 0.54 0.00 0.00 0.48 99.11 0.00 0.49 0.00 0.00 0.39 98.97 0.00 0.55 0.00 0.00 0.48 99.05 0.00 0.54 0.00 0.00 0.41 99.07 0.00 0.54 0.00 0.00 0.39 99.02 0.00 0.51 0.00 0.00 0.47 98.93 0.00 0.55 0.00 0.00 0.52 98.83 0.00 0.54 0.00 0.00 0.63 99.00 0.00 0.53 0.00 0.00 0.48 99.03 0.00 0.47 0.00 0.00 0.50 98.93 0.00 0.53 0.00 0.00 0.55 99.05 0.00 0.47 0.00 0.00 0.48 98.84 0.00 0.57 0.00 0.00 0.60 99.10 0.00 0.48 0.00 0.00 0.42 99.05 0.00 0.52 0.00 0.00 0.43 98.98 0.00 0.52 0.00 0.00 0.50 98.96 0.00 0.50 0.00 0.00 0.54 98.88 0.00 0.51 0.00 0.00 0.61 27.79 0.00 0.52 0.23 0.00 71.46 0.45 0.00 0.24 0.01 0.00 99.30 root@rockpro64:~# zgrep CONFIG_HZ /proc/config.gz # CONFIG_HZ_PERIODIC is not set # CONFIG_HZ_100 is not set CONFIG_HZ_250=y # CONFIG_HZ_300 is not set # CONFIG_HZ_1000 is not set CONFIG_HZ=250 Thank you for checking. That indeed also makes a big difference. How do I know if it's CONFIG_HZ=250 or not? I imagine all my tests were with CONFIG_HZ=1000 since it's a new setting. Could you explain to the ignorant (me) what exactly this does?
balbes150 Posted February 22, 2019 Posted February 22, 2019 1 hour ago, rooted said: I ran the BMW benchmark at stock frequencies, I haven't ran it overclocked and my AC was off. Yes, I remember that you did this test with the default CPU frequency settings. I am interested in the technical capability of the S922 to operate at higher frequencies than is limited in the default settings (I assume that the default values are the result of the limitation from AML). In RK, with an increase in the frequency of the processor (ceteris paribus), the result changes significantly, indicating a real increase in frequency (system performance). I fear that The s922 has the same situation as it did with the S905, limiting the limiting frequencies is not related to limiting the thermal state. Perhaps this architecture is not able to work at higher frequencies with all the cores and will require them to be disabled (ie 2000\1900 will not work 4 cores, but only 1-2 cores).
tkaiser Posted February 22, 2019 Posted February 22, 2019 10 minutes ago, balbes150 said: In RK, with an increase in the frequency of the processor (ceteris paribus), the result changes significantly, indicating a real increase in frequency (system performance) Same with S922X, simply check @rooted's sbc-bench results (where even the cpufreq OPP are checked and confirmed). The problem is not thermal but reliability/undervoltage due to the firmware running on the Cortex-M3 controlling DVFS which can be clearly seen by running the most heavy load called cpuminer. Quoting myself: ''Overclocked' executions with both CPU clusters set to 2.0 GHz showed reliability issues most probably due to DVFS undervoltage (cpuminer quit almost immediately here while it ran only 50 seconds there -- this tool since being a load generator checking for data corruption can also be used for reliability testing but I would prefer our StabilityTester instead)' I already suggested to Justin and Dongjin to take our StabilityTester approach to provide N2 users with an easy way to check for undervoltage/instabilities since a lot of those users will activate the 'overclocking' settings (most probably these are undervolting settings at the same time based on results) and then end up with silent data corruption and/or crashes (all the great results of trying to get laughable 7%-8% performance gain almost nobody is able to notice). 45 minutes ago, NicoD said: Could you explain to the ignorant (me) what exactly this does? Simply do a web search for CONFIG_HZ 1
rooted Posted February 22, 2019 Posted February 22, 2019 That indeed also makes a big difference. How do I know if it's CONFIG_HZ=250 or not? I imagine all my tests were with CONFIG_HZ=1000 since it's a new setting. The stock N2 kernel is running at CONFIG_HZ=250 so this is what my BMW benchmark was using. It's not a new setting, it's been around for a long time.
tkaiser Posted February 22, 2019 Posted February 22, 2019 13 minutes ago, rooted said: The stock N2 kernel is running at CONFIG_HZ=250 so this is what my BMW benchmark was using. It's not a new setting, it's been around for a long time 'Around for a long time'? The point is not since when settings exist but what the defaults are and why people interested in maximum performance should care about such settings. This was the context: All the RK3399 'benchmark' results collected so far were done with CONFIG_HZ=1000 (good for a responsive UI in Android and Linux, not so good for normal computing or server tasks) while everywhere around CONFIG_HZ=250 is the default. Is this affecting Blender or not? Anyone tested so far (latest ayufan images switched to CONFIG_HZ=250)? And by looking at both tinymembench and Blender scores it should be pretty obvious that this makes a difference for workloads that utilize CPU cores fully... just to explain why comparing RK3399 and S922X benchmark scores doesn't make that much sense as long as such essential stuff is not also considered. @NicoD: I explained in my former post how to check for such settings, simply check the spoiler. 1
NicoD Posted February 22, 2019 Author Posted February 22, 2019 20 minutes ago, tkaiser said: @NicoD: I explained in my former post how to check for such settings, simply check the spoiler. Thanks, I'm already doing a test with my M4 with Armbian Bionic. It seems to be set to 250. I'll do some tests with different settings. I didn't know about this, it indeed seems very important. That's another thing I have to add to "reasons why benchmarks can be misleading". Spoiler I received the OPi3 today, that's another vat of issue's it seems. unable to keep it cool with heatsink+fan and underclocks at 60°C to seemingly 1.5Ghz. It seems to underperform to the Rock64, OdroidC2. But that's another topic for another day. Spoiler Reasons why benchmarks can be misleading ---------------------------------------- throttling 32-bit/64-bit Difference in cores A53/A7/A15/A72 distro (ubuntu/debian...) distro version kernel version driver versions compiler version software version/outdated repositories desktop Mate/Xfce/LXDE/... display resolution/headless background processes cpu clockspeed ram clockspeed/latency ram useage/swap/zram process sheduler optimizations for the system/distro crypto engine for encryption Undervoltage config settings Wifi dongle CONFIG_HZ=250 - 1000 or any other 7-zip works a bit better on 32-bit vs 64-bit, it doesn't use all cores at 100% in multi-core scores. The percentage differs with different distro's and boards. So it's not completely exact. Blender works a lot better on 64-bit than on 32-bit. It uses 100% of the cores. Sysbench works 10x better on 64-bit, different versions give different results. Version 1.xx does the test 10s and gives the amount of events. GIMP only uses 1 core, works better on 64-bit. GTKPerf tests desktop speed. Works 10x better on 64-bit. It only uses 1 core. CPU Miner only works on 64-bit. Works better in Ubuntu Bionic than in Debian Stretch. Blender : BMW render @ 1080p Gimp : BMW render result 1080p Filters -> Artistic -> Van Gogh -> ok Sysbench : sysbench --test=cpu --cpu-max-prime=20000 --num-threads="number of threads" run 7-zip : Numbers are average of 3 of decompressing only All tests are done with a fan when necessary so no throttling occurs. 64-bit SBC's Odroid N2 |SBC bench result |CPU Miner |7-zip s/c A53 |7-zip b/c A73 |7-zip multi avg. of 3 |Blender |GIMP |GTKPerf |Sysbench Ubuntu Bionic http://ix.io/1Brv 11.35kH/s 1564 1879 9988 50m28s NanoPC T3+ |SBC bench result |CPU Miner |7-zip s/c |7-zip b/c |7-zip multi avg. of 3 |Blender |GIMP |GTKPerf |Sysbench Armbian Bionic http://ix.io/1iRJ 10.99kH/s 1290 10254 1h10m25s 1m24s 10.11s 21692 Arbmian Stretch http://ix.io/1qiF 8.55kH/s 1275 10149 1h13m55s 1m32s 11.06s 3.2s NanoPi M4 |SBC bench result |CPU Miner |7-zip s/c |7-zip b/c |7-zip multi avg. of 3 |Blender |GIMP |GTKPerf |Sysbench Armbian bionic http://ix.io/1nLh 10.23kH/s 1335 2005 8352 1h13m50s 0m29s5 5.06s 26763 Armbian bionic nightly http://ix.io/1pDo 10.24kH/s 1329 1990 8292 1h13m28s 0m29s 5.12s 26733 Armbian stretch desktop http://ix.io/1odF 8.66kH/s 1350 1977 8400 1h14m12s 0m31s 5.24s 3.1s Armbian stretch dsk nightly //ix.io/1pM0 8.80kH/s 1359 1993 8500 1h15m04s 0m31s 5.32s 3.3s Armbian stretch core no fan //ix.io/1pKU 8.80-8.65kH/s 1353 1989 8461 Armbian stretch core //ix.io/1pL9 8.76kH/s 1354 1988 8456 Armbian stretch core nightly //ix.io/1pLf 8.82kH/s 1357 1994 8494 Lubuntu Bionic arm64 http://ix.io/1oGJ 9.24kH/s CPU Miner 1056 1551 6943 1h28m13s Lubuntu Bionic armhf http://ix.io/1pJ1 1111 1769 7705 2h02m54s 0m57s 6.97s 1666 32-bit Lubuntu Xenial armhf http://ix.io/1oCb 989 1507 6339 2h20m51s 0m59s 49.77s 49.7s 32-bit Khadas Vim2 Max |SBC bench result |CPU Miner |7-zip s/c |7-zip b/c |7-zip multi avg. of 3 |Blender |GIMP |GTKPerf |Sysbench Ubuntu Xenial http://ix.io/1qkA 6.86kH/s 823 1134 6682 1h14m39s 1m53s 16.26s 3.8s 7-zip only 600% of 800% used Odroid C2 |SBC bench result |CPU Miner |7-zip big core |7-zip multi avg. of 3 |Blender |GIMP |GTKPerf |Sysbench Armbian Stretch Core http://ix.io/1pZu 4.65kH/s 1390 5342 Armbian Stretch Core Nightly //ix.io/1pZJ 4.66kH/s 1391 5340 Armbian Stretch Desktop http://ix.io/1q1C 4.65kH/s 1394 5363 1m23s 11.66s 5.96s Armbian Stretch Desktop NGHT //ix.io/1p02 4.59kH/s 1394 5356 2h38m18s 1m23s 12s 6.0s Meveric Stetch No-OC 1337 5223 2h40m00s 1m25s 10.41s 5.99s Meveric Stretch Only RAM OC 1361 5292 1m25s 5.99s Meveric Stretch OC 1548 6049 2h14m17s 1m14s 8.80s 5.24s Ubuntu Mate Bionic http://ix.io/1q2S clocked to 100Mhz 2h35m10s 1m17s 10.01s 12026 Ubuntu Mate Bionic OC Doesn't work/Clocked to 100Mhz 1607 5960 2h10m21s 1m09s 8.94s 13755 Rock64 |SBC bench result |CPU Miner |7-zip small core |7-zip multi avg. of 3 |Blender |GIMP |GTKPerf |Sysbench Armbian Stretch 1.5Ghz http://ix.io/1nCj 4.06kH/s 1406 5407 3h00n32s 1m39s 15.91s 7.0s OLD Armbian Stretch 1.3Ghz //ix.io/1iHB 3.80kH/s 1211 4904 Armbian Bionic 1.5Ghz core //ix.io/1qbK 5.00kH/s 1384 5379 10.0s Armbian Bionic 1.5Ghz dsk //ix.io/1qcb 4.94kH/s 1379 5326 2h55m56s 1m31s 15.00s 10172 32-bit SBC's Odroid XU4 |SBC bench result |7-zip s/c |7-zip b/c |7-zip multi avg. of 3 |Blender |GIMP |GTKPerf |Sysbench Debian Jessie http://ix.io/1q6X 950 1653 8823 1h12m19s 1m08s 18.53s 41.3s Ubuntu Bionic http://ix.io/1qbL 1219 2094 9395 1h44m19s 1m10s 14.36s 2200 Asus Tinker board |SBC bench result |7-zip big core|7-zip multi avg. of 3 |Blender |GIMP |GTKPerf |Sysbench Tinker OS 9.5 Stretch http://ix.io/1pRN 1983 7536 2h55m00s 1m19s 189.82s 63.7s Raspberry Pi 3B+ |SBC bench result |7-zip small core |7-zip multi avg. of 3 |Blender |GIMP |GTKPerf |Sysbench Raspbian Default no fan http://ix.io/1q10 1471 5027 2m09s 9.85s 88.2s Raspbian Default http://ix.io/1q1Q 1411 5371 5h47m31s 2m09s 10.04 79.5s Raspbian OC http://ix.io/1q5J 1591 6141 1m55s 8.81s 70.8s Ubuntu Mate Xenial http://ix.io/1q65 7-zip didn't work 2m17s 11.71s 90.5s Software versions ----------------- GIMP Blender GTKPerf SysBench SBC-bench M4 : Lubuntu Xenial armhf 2.8.18 2.79b 0.40 0.4.12 0.6.1 Lubuntu Bionic armhf : 2.8.22 2.79b 0.40 1.0.11 LuaJIT 2.1.0-beta3 0.6.1 Armbian Stretch desktop 9.5 : 2.8.18 2.79b 0.40 0.4.12 0.6.1 Armbian Bionic : 2.8.22 2.79b 0.40 1.0.11 LuaJIT 2.1.0-beta3 0.6.1 Tinker : TinkerOS 9.5 Stretch : 2.8.18 2.79b 0.40 0.4.12 0.6.1 Odroid C2 : Armbian Stretch 9.5 : 2.8.18 2.79b 0.40 0.4.12 0.6.1 : Ubuntu Mate Bionic : 2.8.22 2.79b 0.40 1.0.11 LuqJIT 2.1.0-beta3 0.6.1 Doesn't work clocks to 100Mhz Meveric Stretch 9.5 : 2.8.18 2.79b 0.40 0.4.12 Doesn't work Rock64 : Armbian Stretch 9.5: 2.8.18 2.79b 0.40 0.4.12 0.6.1 : Armbian Bionic : 2.8.22 2.79b 0.40 1.0.11 0.6.2 RPi 3b+ : Raspbian Stretch 9.5 : 2.8.18 2.78a 0.40 0.4.12 0.6.1 Ubuntu Mate Xenial : 2.8.16 0.40 0.4.12 0.6.1 Odroid XU4 : Debian Jessie : 2.8.14 2.72b 0.40 0.4.12 0.6.1 7-zip doesn't work : Ubuntu Bionic : 2.8.22 2.79b 0.40 1.0.11 0.6.2 NanoPC T3+ : Armbian Bionic : 2.8.22 2.79b 0.40 1.0.11 0.4.6 Armbian Stretch : 2.79b 0.40 0.4.12 0.6.2 Khadas Vim2 Max : Ubuntu Xenial : 2.8.16 2.76b 0.40 0.4.12 0.6.2 CPU Clocks ---------- Odroid N2 : Ubuntu Bionic : 2x1.9Ghz(A53) + 4x1.8Ghz(A73) 64-bit NanoPi M4 : Armbian Bionic/Stretch : 2x2Ghz + 4X1.5Ghz 64-bit Lubuntu armhf/ARM64 : 2x1.8Ghz + 4X1.4Ghz armhf 32-bit / ARM64 64-bit Tinker Board : TinkerOS Stretch : 4x1.8Ghz 32-bit Odroid C2 : Armbian Stretch : 4x1.5Ghz 64-bit Ubuntu Mate Bionic : 4x1.5Ghz RAM 912Mhz 64-bit Ubuntu Mate Bionic OC : 4x1.75Ghz + RAM 1104Mhz 64-bit Rock64 : Armbian Stretch : 4x1.5Ghz 64-bit Armbian Bionic : 4x1.5Ghz 64-bit RPi 3B+ : Raspbian Stretch : 4x1.4Ghz no fan 4x1.2Ghz above 60°C 32-bit Raspbian Stretch OC : 4x1.570Ghz over_voltage=4 core_freq=500 sd_freq=510 32-bit Ubuntu Xenial : 4x1.4Ghz 32-bit Odroid XU4 : Debian Stretch : 4x1.4Ghz + 4x1.9Ghz 32-bit : Ubuntu Mate Bionic : 4x1.5Ghz + 4x2Ghz Underclocks when above 75°C 32-bit NanoPC T3+ : Armbian Bionic : 8x1.4Ghz 64-bit Some benchmark tools can give an estimate of the performance. But they are never an exact reflection.
tkaiser Posted February 22, 2019 Posted February 22, 2019 24 minutes ago, NicoD said: That's another thing I have to add to "reasons why benchmarks can be misleading". There's a lot more. I just updated my post above with GCC 8.2 results. When Blender is built with a more recent compiler rendering gets faster (this is one of the many reasons why this Phoronix stuff is so bad -- Michael Larabel doesn't educate his users about such basics but throws a bunch of meaningless numbers and graphs at them to create the impression benchmarking would be something magic). And if you build Blender from source with appropriate compiler flags (not those ultra conservative distro defaults, especially not with 'stable' distros like Debian and Ubuntu) then it will be even faster. 27 minutes ago, NicoD said: OPi3 ... it seems to underperform to the Rock64, OdroidC2 Very unlikely. And performance is already known, check sbc-results for PineH64 (board vendors don't matter, it's only about the SoC in question). And of course settings matter. If you use one of those crappy Xunlong images there's no need to test further since they're known for using crappy Allwinner defaults that suck. 1
NicoD Posted February 22, 2019 Author Posted February 22, 2019 5 minutes ago, tkaiser said: There's a lot more. I just updated my post above with GCC 8.2 results. When Blender is built with a more recent compiler rendering gets faster (this is one of the many reasons why this Phoronix stuff is so bad -- Michael Larabel doesn't educate his users about such basics but throws a bunch of meaningless numbers and graphs at them to create the impression benchmarking would be something magic). And if you build Blender from source with appropriate compiler flags (not those ultra conservative distro defaults, especially not with 'stable' distros like Debian and Ubuntu) then it will be even faster. Very unlikely. And performance is already known, check sbc-results for PineH64 (board vendors don't matter, it's only about the SoC in question). And of course settings matter. If you use one of those crappy Xunlong images there's no need to test further since they're known for using crappy Allwinner defaults that suck. Indeed there's a lot more. I also showed that xenial vs bionic performed worse on blender(now the same with bionic -> cosmic) and that different versions of Blender perform different. I tried to keep all that in mind, and only compare bionic -> bionic and the same versions. But even all that isn't sufficient to have a good comparison. I also said there's no perfect benchmark tool. I still stand behind that. We did see that the N2 seems to perform well. Even if it's not as much as first thought. A day not learned is a day not lived. 11 minutes ago, tkaiser said: Very unlikely. And performance is already known, check sbc-results for PineH64 (board vendors don't matter, it's only about the SoC in question). And of course settings matter. If you use one of those crappy Xunlong images there's no need to test further since they're known for using crappy Allwinner defaults that suck. Indeed, the images are aweful. It's a quest for a good image. I've also got the Pine H64 model b. Both companies that really can use a good Armbian. I just done my first Blender test with the M4. 1h08m23s. +5 minutes faster than my previous tests. In the margines of your test. (1 minute difference is no difference...) You were right. (I don't hate to say it)
rooted Posted February 22, 2019 Posted February 22, 2019 2 hours ago, NicoD said: I imagine all my tests were with CONFIG_HZ=1000 since it's a new setting. No this was the context to my post @tkaiser
tkaiser Posted February 22, 2019 Posted February 22, 2019 26 minutes ago, NicoD said: just done my first Blender test with the M4. 1h08m23s. +5 minutes faster than my previous tests. In the margines of your test. (1 minute difference is no difference...) Another thing to consider: SoC temperature. Even if no throttling happens higher temperature will result in both slightly lower performance and slightly higher consumption (details). With a task running over an hour SoC temperature 20°C vs. 70°C might result in ~1 minute difference (maybe even more -- at least that's another reason why sbc-bench always does temperature monitoring and this also explains why N2 benchmarks currently running on "ODROID bench" in a container but with active cooling seem to be faster compared to running bare metal with passive cooling only at higher temperatures).
NicoD Posted February 22, 2019 Author Posted February 22, 2019 33 minutes ago, rooted said: No this was the context to my post @tkaiser 2 hours ago, tkaiser said: 'Around for a long time'? The point is not since when settings exist but what the defaults are I did all my tests with the M4 4 months ago for a video Benchmarking single board computers Video I did my tests with the RockPi4B 2 months later. I'm now redoing all the tests with both. The better result of the RockPi4B could be because of this. So then the default CONFIG_HZ could have changed between 2 - 4 months for Bionic(guess). I'll know when I'm finished with the RockPi4B.@rooted So what he said is also correct (as is what you said) The difference was too big to be true. I'm happy to know this. It's still 15minutes difference. 24 minutes ago, tkaiser said: Another thing to consider: SoC temperature. Even if no throttling happens higher temperature will result in both slightly lower performance and slightly higher consumption (details). With a task running over an hour SoC temperature 20°C vs. 70°C might result in ~1 minute difference (maybe even more I am noticing this. A difference of 6°C lower, while it's faster. But that's compared to my previous test of 4 months ago, now SBC-bench @ HZ=250 with M4. Then everything again with HZ=1000.... RockPi4B ... I know what to do this weekend. 24 minutes ago, tkaiser said: at least that's another reason why sbc-bench always does temperature monitoring and this also explains why N2 benchmarks currently running on "ODROID bench" with active cooling seem to be faster compared to running bare metal with passive cooling only at higher temperatures). @rooted Are you willing/able to do some more tests with a fan?(sbc-bench/blender) I also wonder if Blender would work at 2Ghz, it crashes quickly on unstable systems. We've seen on your 2Ghz SBC-Bench that CPUMiner didn't work either.
rooted Posted February 22, 2019 Posted February 22, 2019 [mention=8966]rooted[/mention] Are you willing/able to do some more tests with a fan?(sbc-bench/blender) I also wonder if Blender would work at 2Ghz, it crashes quickly on unstable systems. We've seen on your 2Ghz SBC-Bench that CPUMiner didn't work either. My son is in the hospital right now so I won't be able to test anything.
NicoD Posted February 22, 2019 Author Posted February 22, 2019 Just now, rooted said: My son is in the hospital right now so I won't be able to test anything. Take good care of him. More important than an sbc. Greetings. 2
rooted Posted February 22, 2019 Posted February 22, 2019 Take good care of him. More important than an sbc. Greetings.Thanks NicoD.
chwe Posted February 22, 2019 Posted February 22, 2019 2 hours ago, TonyMac32 said: He's illustrating how numbers can be used to selectively represent reality. The concept of 'alternative facts' isn't it? Sine I'm barely interested in benchmarks I didn't even read the whole benchmarking tests from odroid (https://www.hardkernel.com/blog-2/odroid-n2/).. But a few things which are for sure misleading. a graphical trick which is quite often used by populists to make a thing bigger than it is.. There are only a very few cases where a graph shouldn't go thorough zero as a nomination.. If we do so here: the whole story looks different.. doesn't it? (using 3d-ish plots for 2d data is also useless, but different story.) Besides the benchmarking which is somehow boring for me.. I still think about use-case for this board.. Lightweight desktop replacement? TV-box on linux? Or 'cluster like' replacement? For the first I would prefer to have at least on USB on the other side, for the second I don't care cause not my field and for the third others may comment on this. Btw. the same counts for this graph: which would better be summarized in a table not a graph.. 1
tkaiser Posted February 23, 2019 Posted February 23, 2019 13 hours ago, NicoD said: The better result of the RockPi4B could be because of this. So then the default CONFIG_HZ could have changed between 2 - 4 months for Bionic(guess). I'll know when I'm finished with the RockPi4B No, you do NOT know when you're finished firing up your next round of passive benchmarking tests spitting out some numbers. You need to know what's going on to generate answers to the question 'why is A faster than B' and also 'what is the limiter on A and why can't A not be twice as fast'? You missed this change and attributed faster RockPi scores to DDR4 vs. DDR3 memory while in reality you compared old Armbian kernel config with newer one. This whole passive benchmarking approach (and IMO 'technical' Youtube videos in general) always only contributes to confusion and generates zero insights. The general rule of passive benchmarking and what in fact happened here is: you benchmark A, but actually measure B, and conclude you've measured C (what's needed instead is Active Benchmarking) What matters are insights, settings and software. And this not only applies to RK3399 but to N2/S922X too of course. So with your use case that utilizes all CPU cores in parallel you surely should go with S922X (since being definitely faster than RK3399 with everything that's multi-threaded), then use a kernel with CONFIG_HZ=100 and an optimized software stack. With Gentoo for example, most recent GCC, optimal/aggressive compiler flags for the A53/A73 combo and a CONFIG_HZ=100 kernel on the N2 your Blender job might finish in less than 40 minutes (maybe just 20 or even less if a programmer equipped with knowledge starts to look into Blender code and adds NEON optimizations to the performance critical code parts -- see here for a great example of Active Benchmarking and adding NEON optimizations on ARM to a software that utilizes SIMD Extensions only on x86 so far just like SSE2 optimized Blender does) The price you'll pay is a few weeks of your life needed to become a Linux expert since there's nothing ready. Choosing Armbian is usually a matter of convenience but if you want software that performs as fast as possible it's a problematic choice since the software stack is not meant to be performant but to be compatible and stable (funny joke since Armbian's own approach with kernel/bootloader updates is the opposite). The two distro variants Armbian provides are Ubuntu also known as 'everything outdated' Debian also known as 'everything outdated as hell' (there are areas where Armbian is in fact faster than other Ubuntu/Debian based images or distros using a more modern software stack like Gentoo or Fedora but this is due to what separates Armbian from the stock Debian/Ubuntu stuff: settings like CPU/IRQ affinity, thermal/DVFS tuning, zram, log2ram and such stuff. But two guys who mostly took care of this contributed nothing to Armbian for a longer time now and no idea how Armbian evolves here in the future. On EoL platforms like Allwinner H5 IRQ affinity is already broken but nobody cares) 1
NicoD Posted February 23, 2019 Author Posted February 23, 2019 3 hours ago, tkaiser said: You missed this change and attributed faster RockPi scores to DDR4 vs. DDR3 memory while Yes Thomas, you are right. I had no idea. But I do know why you knew. Evidence No 1 Not many others will have known this changed. 3 hours ago, tkaiser said: The price you'll pay is a few weeks of your life needed to become a Linux expert If only it was this simple. I've got the weeks, but that wont make me an expert at anything. It's strange, I don't believe in benchmaking, I've showed most/all benchmarking is off. But still I need to do it, and I hate it. 3 hours ago, tkaiser said: But two guys who mostly took care of this contributed nothing to Armbian for a longer time now and no idea how Armbian evolves here in the future. On EoL platforms like Allwinner H5 IRQ affinity is already broken but nobody cares) That's why Armbian could do well with more well informed and knowledgeable people like yourself. You know your place is here. It's too hard for me searching the whole web for all the posts you've scattered over all other forums. Cheers.
tkaiser Posted February 23, 2019 Posted February 23, 2019 2 hours ago, NicoD said: I don't believe in benchmaking, I've showed most/all benchmarking is off. But still I need to do it, and I hate it. Benchmarking is great! You simply fire up a bunch of tests in uncontrolled manner without monitoring what's happening, then generate bar charts out of the generated numbers and can then show that your devices are the fastest SBC around (even faster than those expensive NVIDIA Jetson thingies!!1!1!!): (full results) Only problem: these bar charts above create the impression ODROID-N1 would be faster than N2 which is something Hardkernel clearly wants to avoid. Last year they cancelled their RK3399 based N1 for three reasons (two of them their customers don't want to hear) so they need to choose a bunch of benchmarks where N2 looks like an improvement compared to N1 and this pretty much explains why they only chose multi-threaded CPU benchmarks since with single-core stuff RK3399 usually is as fast as S922X or even slightly faster (this whole CPU benchmarking crap boils down to exactly this: S922X wins over RK3399 with multi-threaded loads while single-core stuff is usually faster on RK3399. Whether this is important or not always depends on the use case only. Majority of users staring at CPU benchmark charts simply don't understand that the relevant stuff happens somewhere else than stupid multi-core '100% CPU utilization' tests) So how to benchmark properly: if you're coming from the developer/researcher perspective then you need Active Benchmarking. All this kitchen-sink stuff is pretty useless. And then you do not benchmark to show how product A is faster than B but to get B as fast as A or even faster. From a user or consumer point of view it's always 'use case first'. Let's take your 'Blender' test here since this is also a real use case you're interested in (rendering stuff on slow SBC for whatever reasons). I mentioned Blender using SIMD Extensions only on x86 for a reason: to illustrate that if you're not a developer able to code and familiar with NEON2 on ARM you might be better off looking at x86 instead. The Gemini Lake thing on ODROID H2 for example is not equipped with Intel's latest and greatest extensions like AVX but at least fully supports SSE2 so maybe @rooted is so kind providing you with Blender numbers from H2? Maybe then your excitement for N2 is already gone and you're a future H2 buyer? Your stuff (a rather special application making use of special instructions) is an exception so how should benchmarks that test entirely different stuff show what's going on? You can't appropriately benchmark without knowledge and without being focused on the use case. Otherwise you're all the time just collecting numbers without meaning. Here I won't comment on why further contributing to Armbian (or using this forum) doesn't make that much sense for me but you should be aware that https://www.cnx-software.com is a great source of knowledge (especially in the comments section where insiders share details and experts explain so much stuff like how/why A72 and A73 differ and so on)
NicoD Posted February 23, 2019 Author Posted February 23, 2019 3 hours ago, tkaiser said: you might be better off looking at x86 instead. I've got a very special use case that I don't think anybody else uses it for. I use them (among many other things) to edit and render videos on location(while traveling, while visiting interesting stuff...) I must do this with power banks charged with solar panels. So for that reason ARM is the best choice(power efficient). I've got my "render bread box" for a long time with the C2(fastest while best passive cooled) for editing and rendering. The RPi2B with small display to record sound(because it's most power efficient). And I also just love sbc's... Spoiler Build to last... not to look good. It's 3 years I use this setup, I took it on different cycling trips of many weeks. It had to endure a lot of punishment. Cold, wet, heat, milions of shocks... I'm now putting together something better with a 13" 5V display and probably with the RockPi4B for this summer. The case will need to be the cooling for it. I've got more solar panels, so I can power more. And I hope in the future to be able to make a new more powerful handheld sbc with the Pine H64.(if it's ever released, I've seen TL Lim on FOSDEM and he said it would be out first half of Februari) Video Pine64@FOSDEM Since I couldn't get the right info for my use case(because nobody else uses it like this) I filmed some tests when I bought the Tinker Board and put it on Youtube. That video got a few thousand views. So I just kept making videos about new boards. I try to do this a good as I can. That's why I love it when somebody comes with information I didn't know about what you often do. Many people get their info on Youtube, it is mostly another public then we meet here in the Armbian forum, that uses SBC's for other tasks(gaming, desktop use). So that's what I show. 3 hours ago, tkaiser said: you should be aware that https://www.cnx-software.com is a great source of knowledge I'll follow it. Thanks for the info. 3 hours ago, tkaiser said: and then you do not benchmark to show how product A is faster than B but to get B as fast as A or even faster. But that's what people want. They want to see them lined up in an order. So it's the reviewer his choice what order he shows to the viewers/readers. That's why I try to find the better tools for the job, but none of those I've used are even near to perfect.(as I've shown in my videos) So I try to minimise the variables as much as possible by using the same distro, no throttlin, the same software version, the same settings, only ARMV7->V7 - V8->V8 ... But it's never possible to do this well. Most Youtubers don't even care about that, and show very weird results of sysbench or so. I'll keep using 7zip, CPUMiner and blender for now because even with their faults. They still manage to make the most sensible line up. And I'll keep saying it only shows how well it does that task only. And there's a lot more to SBC's than how powerful the CPU is. 3 hours ago, tkaiser said: Here I won't comment on why further contributing to Armbian (or using this forum) Well the last 2 days we've managed to keep you bussy here. And I enjoyed it, and I bet many readers will have too. So it's only a tiny step to stay and end the self-censorship. Have a great day. Read you soon. 1
rooted Posted February 23, 2019 Posted February 23, 2019 I will do the BMW benchmark on an H2 if you are interested in that, it may be a few days as my son is still in the hospital.I would like to know how it stacks up against the N2 anyway.
chwe Posted February 24, 2019 Posted February 24, 2019 7 hours ago, NicoD said: But that's what people want. They want to see them lined up in an order. So it's the reviewer his choice what order he shows to the viewers/readers. I'm not sure on this one.. I seriously don't read many benchmarks.. I bought my notebook without looking at any benchmark.. I decided against another one cause it's a known not properly working one on linux.. I had a few other requirements which a bunch of them didn't fulfill (e.g. browsers are ram hungry and no way in heaven I go back to a 8gb ram one).. Whats about showing how a random board with the current available OS's perform on the use-cases you and others might have? E.g. lightweight desktop scenarios etc. Personally, I think benchmarking makes sense to compare the same board with various distros or find bottlenecks.. e.g. is my SBC capable of saturating a GbE link with encrypted storage.. and when not why? Can this part be improved or not? If I've a desktop usage in mind should I use a Debian, an Ubuntu or might something like a Arch or Gentoo be the better option. For sure this is difficult due to Linux on Arm (especially in desktop scenarios, except android) is nowhere compared to x86 world. This approach doesn't benchmark the max. performance a *random SoC* is capable to deliver (and should also not be sold as such) but instead it represents the current state of what you can expect by buying random product. The majority of users don't spend as much time as some of us do to tweak settings. Such 'benchmarks' have then to be repeated over time to see if *random distribution for random board* improves or not. A board might be badly supported in the beginnings but hopefully they get it and pick up such tweaks so that it gets better over time. As a customer of a board I'm interested that the board producer makes my board better over time (e.g. as a beginner I would never buy a xunlong board, cause they're notoriously known to provide no support at all for their SBCs - the whole story differs as a more experienced guy, I can fix some flaws of a OPi as long as they're only software wise). But you target audience is more the end-user side right? So it might be interesting for them what they can expect by buying *random board* from *random boardmaker*. This approach differs from @tkaiser benchmarking. He tries to benchmark the hardware and tweaks settings to ensure that the software impact is as marginal as it can be. Don't get me wrong, this is important as well, but as long as I don't have a distro which delivers those tweaks in their stock Images, I can't benefit from a powerful SoC (e.g. if thermal settings are badly on every distro for a random board I can't benefit from better settings as a end-user). Somehow the same counts for the blender benchmark: On 2/22/2019 at 4:21 PM, tkaiser said: Blender test. Checking the relevance of SETTINGS instead of focusing on irrelevant hardware details like DDR3 vs. DDR4: RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:15:31 RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:06:59 RockPro64, Ubuntu Cosmic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:01:11 How often did you run those tests? Roughly 25% looks indeed like a real difference but I would prefer to have at least a triplicate to get a clue how much error margin I expect for such a test. I'm quite sure you'll still see a difference but the dataset locks not complete (e.g. 'RockPro64, Ubuntu Cosmic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:01:11' is missing). So you could twice see a difference between config_hz and twice a difference between cosmic and bionic. By a triplicate with an error margin you could also show that those numbers are somehow reliable (my current work outside SBCs has error margins of ~30-100%, means I run a lot of triplicates to ensure that there's a real difference before I get excited about something). This isn't grass root benchmarking, it's more a treetops benchmarking and it should also be sold as such. Means that you've to be honest to your viewers that such numbers can change if distributions tweaks their settings and you can elaborate why a board-distribution combination might perform worse compared to another combination. 1
chwe Posted February 24, 2019 Posted February 24, 2019 16 hours ago, rooted said: I will do the BMW benchmark on an H2 if you are interested in that, it may be a few days as my son is still in the hospital. I wish you and your family the best that your son recovers soon. Just ignore all wishes from other users as long as you've more important issues. 1
rooted Posted February 24, 2019 Posted February 24, 2019 I wish you and your family the best that your son recovers soon. Just ignore all wishes from other users as long as you've more important issues.Thanks@chwe it's appreciated. I just read the forum to keep my mind busy instead of worrying, anyone that has spent much time in a hospital likely knows what I mean.
TonyMac32 Posted February 24, 2019 Posted February 24, 2019 Thanks[mention=5623]chwe[/mention] it's appreciated. I just read the forum to keep my mind busy instead of worrying, anyone that has spent much time in a hospital likely knows what I mean.Yes. Best wishes from my side as well.Sent from my Pixel using Tapatalk 1
balbes150 Posted February 24, 2019 Posted February 24, 2019 1 hour ago, rooted said: I just read the forum to keep my mind busy instead of worrying, anyone that has spent much time in a hospital likely knows what I mean. We create our own future, if you firmly believe that you will be all right-then everything will be fine. 1
Recommended Posts