Jump to content

Recommended Posts

Posted
  On 2/22/2019 at 12:14 PM, TonyMac32 said:

He also hung a lantern on it, so to speak:. "silly kitchen sink benchmarking". He's illustrating how numbers can be used to selectively represent reality.

Expand  

But isn't that exactly the same reason why rooted did the benchmarks? To check if the numbers from Hardkernel were fictional or not?
1+1= still 2
 

 

  On 2/21/2019 at 9:48 AM, tkaiser said:

Funny place here... 

Expand  

Bout time to come back to make it more funny maybe? ;)

Posted
  On 2/21/2019 at 11:31 AM, rooted said:

Test for yourselves, Hardkernel has 4 N2 on the bench ready for testing.

Expand  

You have the ability to perform an N2 check with Blender (which you have already done), but with as many kernel maximum frequency settings as you can get ? A prerequisite is an excellent cooling system with a guaranteed maximum temperature from the N2 sensors, throughout the test, no higher than 60-65 degrees. The cooling system and the surrounding conditions are entirely at your choice, passive or active with or without an additional fan, i.e. any option without limitation.

Posted
  On 2/22/2019 at 1:56 PM, balbes150 said:
You have the ability to perform an N2 check with Blender (which you have already done), but with as many kernel maximum frequency settings as you can get ? A prerequisite is an excellent cooling system with a guaranteed maximum temperature from the N2 sensors, throughout the test, no higher than 60-65 degrees. The cooling system and the surrounding conditions are entirely at your choice, passive or active with or without an additional fan, i.e. any option without limitation.
I ran the BMW benchmark at stock frequencies, I haven't ran it overclocked and my AC was off.
Posted (edited)

Blender test. Checking the relevance of SETTINGS instead of focusing on irrelevant hardware details like DDR3 vs. DDR4:

  • RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:15:31
  • RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:06:59
  • RockPro64, Ubuntu Cosmic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:01:11

 

That's 8:30 minutes difference due to switching from CONFIG_HZ=1000 to CONFIG_HZ=250. Check %sys vs. %user below (iostat 60 output). And why is Cosmic faster than Bionic if it's exactly same Blender version? Since Cosmic (18.10) uses GCC 8.2 while Bionic (18.04) uses GCC 7.3 to build the packages. So by switching from default SoC vendor kernel settings to something better and by letting modern compilers do their job we get almost 25% performance 'for free'.

 

  Reveal hidden contents
Edited by tkaiser
Added GCC 8.2 results
Posted
  On 2/22/2019 at 12:14 PM, TonyMac32 said:

He also hung a lantern on it, so to speak:. "silly kitchen sink benchmarking". He's illustrating how numbers can be used to selectively represent reality.

Sent from my Pixel using Tapatalk
 

Expand  

 

Yes the GPU Benchmark for example must be taken with doubts, it shows a result from ~255fps with glmark2 and --off-screen rendering switch with fbdev drivers on XU4 and 285fps on N1 and 305fps on N2, but the XU4 was able to do the same test with X11 drivers and 1200fps, but we do not have X11 drivers for N2, but from my experience the N1 and XU4 was relatively equal when it comes to real live gaming / emulation tests, maybe slightly better and not so stressed out like the XU4 with the same tasks in terms of temperature, also the Mali-G52 GPU is very new (Q1 2018) and maybe not fully supported from software side, so it's too early to make a clear statement.

Posted
  On 2/22/2019 at 3:21 PM, tkaiser said:

Blender test. Checking the relevance of SETTINGS instead of focusing on irrelevant hardware details like DDR3 vs. DDR4:

  • RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:15:31
  • RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:06:59

That's 8:30 minutes difference due to switching from CONFIG_HZ=1000 to CONFIG_HZ=250. Check %sys vs. %user below (iostat 60 output):

  Reveal hidden contents
Expand  

Thank you for checking.
That indeed also makes a big difference. How do I know if it's CONFIG_HZ=250 or not? I imagine all my tests were with CONFIG_HZ=1000 since it's a new setting.
Could you explain to the ignorant (me) what exactly this does?

Posted
  On 2/22/2019 at 2:59 PM, rooted said:

I ran the BMW benchmark at stock frequencies, I haven't ran it overclocked and my AC was off. 

Expand  

Yes, I remember that you did this test with the default CPU frequency settings. :)

I am interested in the technical capability of the S922 to operate at higher frequencies than is limited in the default settings (I assume that the default values are the result of the limitation from AML). In RK, with an increase in the frequency of the processor (ceteris paribus), the result changes significantly, indicating a real increase in frequency (system performance). I fear that The s922 has the same situation as it did with the S905, limiting the limiting frequencies is not related to limiting the thermal state. Perhaps this architecture is not able to work at higher frequencies with all the cores and will require them to be disabled (ie 2000\1900 will not work 4 cores, but only 1-2 cores).

Posted
  On 2/22/2019 at 4:30 PM, balbes150 said:

In RK, with an increase in the frequency of the processor (ceteris paribus), the result changes significantly, indicating a real increase in frequency (system performance)

Expand  

Same with S922X, simply check @rooted's sbc-bench results (where even the cpufreq OPP are checked and confirmed). The problem is not thermal but reliability/undervoltage due to the firmware running on the Cortex-M3 controlling DVFS which can be clearly seen by running the most heavy load called cpuminer. 

 

Quoting myself: ''Overclocked' executions with both CPU clusters set to 2.0 GHz showed reliability issues most probably due to DVFS undervoltage (cpuminer quit almost immediately here while it ran only 50 seconds there -- this tool since being a load generator checking for data corruption can also be used for reliability testing but I would prefer our StabilityTester instead)'

 

I already suggested to Justin and Dongjin to take our StabilityTester approach to provide N2 users with an easy way to check for undervoltage/instabilities since a lot of those users will activate the 'overclocking' settings (most probably these are undervolting settings at the same time based on results) and then end up with silent data corruption and/or crashes (all the great results of trying to get laughable 7%-8% performance gain almost nobody is able to notice).

 

  On 2/22/2019 at 4:03 PM, NicoD said:

Could you explain to the ignorant (me) what exactly this does?

Expand  

 

Simply do a web search for CONFIG_HZ :)

Posted

 

 

 

  On 2/22/2019 at 4:03 PM, NicoD said:

That indeed also makes a big difference. How do I know if it's CONFIG_HZ=250 or not? I imagine all my tests were with CONFIG_HZ=1000 since it's a new setting.

 

The stock N2 kernel is running at CONFIG_HZ=250 so this is what my BMW benchmark was using.

 

It's not a new setting, it's been around for a long time.

Posted
  On 2/22/2019 at 5:08 PM, rooted said:

The stock N2 kernel is running at CONFIG_HZ=250 so this is what my BMW benchmark was using.

 

It's not a new setting, it's been around for a long time

Expand  

 

'Around for a long time'? The point is not since when settings exist but what the defaults are and why people interested in maximum performance should care about such settings. This was the context:

 

  On 2/21/2019 at 9:48 AM, tkaiser said:

All the RK3399 'benchmark' results collected so far were done with CONFIG_HZ=1000 (good for a responsive UI in Android and Linux, not so good for normal computing or server tasks) while everywhere around CONFIG_HZ=250 is the default. Is this affecting Blender or not? Anyone tested so far (latest ayufan images switched to CONFIG_HZ=250)?

Expand  

 

And by looking at both tinymembench and Blender scores it should be pretty obvious that this makes a difference for workloads that utilize CPU cores fully... just to explain why comparing RK3399 and S922X benchmark scores doesn't make that much sense as long as such essential stuff is not also considered.

 

@NicoD: I explained in my former post how to check for such settings, simply check the spoiler.

Posted
  On 2/22/2019 at 5:26 PM, tkaiser said:

@NicoD: I explained in my former post how to check for such settings, simply check the spoiler.

Expand  

Thanks, I'm already doing a test with my M4 with Armbian Bionic. It seems to be set to 250. I'll do some tests with different settings.
I didn't know about this, it indeed seems very important.  That's another thing I have to add to "reasons why benchmarks can be misleading". Spoiler
I received the OPi3 today, that's another vat of issue's it seems. unable to keep it cool with heatsink+fan and underclocks at 60°C to seemingly 1.5Ghz. It seems to underperform to the Rock64, OdroidC2. But that's another topic for another day.

  Reveal hidden contents


 

Posted
  On 2/22/2019 at 5:40 PM, NicoD said:

That's another thing I have to add to "reasons why benchmarks can be misleading".

Expand  

 

There's a lot more. I just updated my post above with GCC 8.2 results. When Blender is built with a more recent compiler rendering gets faster (this is one of the many reasons why this Phoronix stuff is so bad -- Michael Larabel doesn't educate his users about such basics but throws a bunch of meaningless numbers and graphs at them to create the impression benchmarking would be something magic).

 

And if you build Blender from source with appropriate compiler flags (not those ultra conservative distro defaults, especially not with 'stable' distros like Debian and Ubuntu) then it will be even faster.

 

  On 2/22/2019 at 5:40 PM, NicoD said:

OPi3 ... it seems to underperform to the Rock64, OdroidC2

Expand  

 

Very unlikely. And performance is already known, check sbc-results for PineH64 (board vendors don't matter, it's only about the SoC in question). And of course settings matter. If you use one of those crappy Xunlong images there's no need to test further since they're known for using crappy Allwinner defaults that suck.

Posted
  On 2/22/2019 at 6:10 PM, tkaiser said:

 

There's a lot more. I just updated my post above with GCC 8.2 results. When Blender is built with a more recent compiler rendering gets faster (this is one of the many reasons why this Phoronix stuff is so bad -- Michael Larabel doesn't educate his users about such basics but throws a bunch of meaningless numbers and graphs at them to create the impression benchmarking would be something magic).

 

And if you build Blender from source with appropriate compiler flags (not those ultra conservative distro defaults, especially not with 'stable' distros like Debian and Ubuntu) then it will be even faster.

 

 

Very unlikely. And performance is already known, check sbc-results for PineH64 (board vendors don't matter, it's only about the SoC in question). And of course settings matter. If you use one of those crappy Xunlong images there's no need to test further since they're known for using crappy Allwinner defaults that suck.

Expand  

Indeed there's a lot more. I also showed that xenial vs bionic performed worse on blender(now the same with bionic -> cosmic) and that different versions of Blender perform different.
I tried to keep all that in mind, and only compare bionic -> bionic and the same versions. But even all that isn't sufficient to have a good comparison. I also said there's no perfect benchmark tool. I still stand behind that.
We did see that the N2 seems to perform well. Even if it's not as much as first thought. A day not learned is a day not lived.
 

  On 2/22/2019 at 6:10 PM, tkaiser said:

Very unlikely. And performance is already known, check sbc-results for PineH64 (board vendors don't matter, it's only about the SoC in question). And of course settings matter. If you use one of those crappy Xunlong images there's no need to test further since they're known for using crappy Allwinner defaults that suck.

Expand  

Indeed, the images are aweful. It's a quest for a good image. I've also got the Pine H64 model b. Both companies that really can use a good Armbian.
 

I just done my first Blender test with the M4. 1h08m23s. +5 minutes faster than my previous tests. In the margines of your test. (1 minute difference is no difference...) You were right. (I don't hate to say it)

Posted
  On 2/22/2019 at 6:42 PM, NicoD said:

just done my first Blender test with the M4. 1h08m23s. +5 minutes faster than my previous tests. In the margines of your test. (1 minute difference is no difference...)

Expand  

 

Another thing to consider: SoC temperature. Even if no throttling happens higher temperature will result in both slightly lower performance and slightly higher consumption (details). With a task running over an hour SoC temperature 20°C vs. 70°C might result in ~1 minute difference (maybe even more -- at least that's another reason why sbc-bench always does temperature monitoring and this also explains why N2 benchmarks currently running on "ODROID bench" in a container but with active cooling seem to be faster compared to running bare metal with passive cooling only at higher temperatures).

 

Posted
  On 2/22/2019 at 6:58 PM, rooted said:

 

No this was the context to my post @tkaiser

Expand  

 

  On 2/22/2019 at 5:26 PM, tkaiser said:

'Around for a long time'? The point is not since when settings exist but what the defaults are

Expand  

I did all my tests with the M4 4 months ago for a video Benchmarking single board computers Video
 

I did my tests with the RockPi4B 2 months later. I'm now redoing all the tests with both. The better result of the RockPi4B could be because of this. So then the default CONFIG_HZ could have changed between 2 - 4 months for Bionic(guess).
I'll know when I'm finished with the RockPi4B.
@rooted So what he said is also correct (as is what you said) The difference was too big to be true. I'm happy to know this. It's still 15minutes difference.
 

  On 2/22/2019 at 7:06 PM, tkaiser said:

 

Another thing to consider: SoC temperature. Even if no throttling happens higher temperature will result in both slightly lower performance and slightly higher consumption (details). With a task running over an hour SoC temperature 20°C vs. 70°C might result in ~1 minute difference (maybe even more

 

Expand  

 

I am noticing this. A difference of 6°C lower, while it's faster. But that's compared to my previous test of 4 months ago, now SBC-bench @ HZ=250 with M4. Then everything again with HZ=1000.... RockPi4B ...
I know what to do this weekend.
 

  On 2/22/2019 at 7:06 PM, tkaiser said:

at least that's another reason why sbc-bench always does temperature monitoring and this also explains why N2 benchmarks currently running on "ODROID bench" with active cooling seem to be faster compared to running bare metal with passive cooling only at higher temperatures).

 

Expand  

@rooted Are you willing/able  to do some more tests with a fan?(sbc-bench/blender) I also wonder if Blender would work at 2Ghz, it crashes quickly on unstable systems.
We've seen on your 2Ghz SBC-Bench that CPUMiner didn't work either.

 

Posted



  On 2/22/2019 at 7:27 PM, NicoD said:
[mention=8966]rooted[/mention] Are you willing/able  to do some more tests with a fan?(sbc-bench/blender) I also wonder if Blender would work at 2Ghz, it crashes quickly on unstable systems.
We've seen on your 2Ghz SBC-Bench that CPUMiner didn't work either.
 


My son is in the hospital right now so I won't be able to test anything.
Posted
  On 2/22/2019 at 7:41 PM, rooted said:


 

 


My son is in the hospital right now so I won't be able to test anything.

 

Expand  

Take good care of him. More important than an sbc. Greetings.

Posted
  On 2/22/2019 at 12:14 PM, TonyMac32 said:

He's illustrating how numbers can be used to selectively represent reality.

Expand  

The concept of 'alternative facts' isn't it? :D

 

Sine I'm barely interested in benchmarks I didn't even read the whole benchmarking tests from odroid (https://www.hardkernel.com/blog-2/odroid-n2/).. But a few things which are for sure misleading.

 

figure-5.png

a graphical trick which is quite often used by populists to make a thing bigger than it is.. There are only a very few cases where a graph shouldn't go thorough zero as a nomination.. If we do so here:

Capture.PNG.954908b246d4ed4758a7180f4f1b1306.PNG

the whole story looks different.. doesn't it? (using 3d-ish plots for 2d data is also useless, but different story.)

 

Besides the benchmarking which is somehow boring for me.. I still think about use-case for this board.. Lightweight desktop replacement? TV-box on linux? Or 'cluster like' replacement? For the first I would prefer to have at least on USB on the other side, for the second I don't care cause not my field and for the third others may comment on this. Btw. the same counts for this graph:

figure-11.png

which would better be summarized in a table not a graph..

 

Posted
  On 2/22/2019 at 7:27 PM, NicoD said:

The better result of the RockPi4B could be because of this. So then the default CONFIG_HZ could have changed between 2 - 4 months for Bionic(guess).
I'll know when I'm finished with the RockPi4B

Expand  

 

No, you do NOT know when you're finished firing up your next round of passive benchmarking tests spitting out some numbers. You need to know what's going on to generate answers to the question 'why is A faster than B' and also 'what is the limiter on A and why can't A not be twice as fast'? You missed this change and attributed faster RockPi scores to DDR4 vs. DDR3 memory while in reality you compared old Armbian kernel config with newer one. This whole passive benchmarking approach (and IMO 'technical' Youtube videos in general) always only contributes to confusion and generates zero insights.

 

The general rule of passive benchmarking and what in fact happened here is: you benchmark A, but actually measure B, and conclude you've measured C (what's needed instead is Active Benchmarking)

 

What matters are insights, settings and software. And this not only applies to RK3399 but to N2/S922X too of course. So with your use case that utilizes all CPU cores in parallel you surely should go with S922X (since being definitely faster than RK3399 with everything that's multi-threaded), then use a kernel with CONFIG_HZ=100 and an optimized software stack. With Gentoo for example, most recent GCC, optimal/aggressive compiler flags for the A53/A73 combo and a CONFIG_HZ=100 kernel on the N2 your Blender job might finish in less than 40 minutes (maybe just 20 or even less if a programmer equipped with knowledge starts to look into Blender code and adds NEON optimizations to the performance critical code parts -- see here for a great example of Active Benchmarking and adding NEON optimizations on ARM to a software that utilizes SIMD Extensions only on x86 so far just like SSE2 optimized Blender does)

 

The price you'll pay is a few weeks of your life needed to become a Linux expert since there's nothing ready. Choosing Armbian is usually a matter of convenience but if you want software that performs as fast as possible it's a problematic choice since the software stack is not meant to be performant but to be compatible and stable (funny joke since Armbian's own approach with kernel/bootloader updates is the opposite). The two distro variants Armbian provides are

  • Ubuntu also known as 'everything outdated'
  • Debian also known as 'everything outdated as hell'

(there are areas where Armbian is in fact faster than other Ubuntu/Debian based images or distros using a more modern software stack like Gentoo or Fedora but this is due to what separates Armbian from the stock Debian/Ubuntu stuff: settings like CPU/IRQ affinity, thermal/DVFS tuning, zram, log2ram and such stuff. But two guys who mostly took care of this contributed nothing to Armbian for a longer time now and no idea how Armbian evolves here in the future. On EoL platforms like Allwinner H5 IRQ affinity is already broken but nobody cares)

 

Posted
  On 2/23/2019 at 7:23 AM, tkaiser said:

You missed this change and attributed faster RockPi scores to DDR4 vs. DDR3 memory while

Expand  

Yes Thomas, you are right. I had no idea. But I do know why you knew. Evidence No 1;)
Not many others will have known this changed.

  On 2/23/2019 at 7:23 AM, tkaiser said:

The price you'll pay is a few weeks of your life needed to become a Linux expert

Expand  

If only it was this simple. I've got the weeks, but that wont make me an expert at anything.

It's strange, I don't believe in benchmaking, I've showed most/all benchmarking is off. But still I need to do it, and I hate it.
 

  On 2/23/2019 at 7:23 AM, tkaiser said:

But two guys who mostly took care of this contributed nothing to Armbian for a longer time now and no idea how Armbian evolves here in the future. On EoL platforms like Allwinner H5 IRQ affinity is already broken but nobody cares) 

Expand  

That's why Armbian could do well with more well informed and knowledgeable people like yourself. You know your place is here. It's too hard for me searching the whole web for all the posts you've scattered over all other forums.
Cheers.

Posted
  On 2/23/2019 at 11:34 AM, NicoD said:

I don't believe in benchmaking, I've showed most/all benchmarking is off. But still I need to do it, and I hate it.

Expand  

 

Benchmarking is great! You simply fire up a bunch of tests in uncontrolled manner without monitoring what's happening, then generate bar charts out of the generated numbers and can then show that your devices are the fastest SBC around (even faster than those expensive NVIDIA Jetson thingies!!1!1!!):

 

PTS-ODROID-N1-vs-N2.png

 

(full results)

 

Only problem: these bar charts above create the impression ODROID-N1 would be faster than N2 which is something Hardkernel clearly wants to avoid. Last year they cancelled their RK3399 based N1 for three reasons (two of them their customers don't want to hear) so they need to choose a bunch of benchmarks where N2 looks like an improvement compared to N1 and this pretty much explains why they only chose multi-threaded CPU benchmarks since with single-core stuff RK3399 usually is as fast as S922X or even slightly faster (this whole CPU benchmarking crap boils down to exactly this: S922X wins over RK3399 with multi-threaded loads while single-core stuff is usually faster on RK3399. Whether this is important or not always depends on the use case only. Majority of users staring at CPU benchmark charts simply don't understand that the relevant stuff happens somewhere else than stupid multi-core '100% CPU utilization' tests)

 

So how to benchmark properly: if you're coming from the developer/researcher perspective then you need Active Benchmarking. All this kitchen-sink stuff is pretty useless. And then you do not benchmark to show how product A is faster than B but to get B as fast as A or even faster.

 

From a user or consumer point of view it's always 'use case first'. Let's take your 'Blender' test here since this is also a real use case you're interested in (rendering stuff on slow SBC for whatever reasons). I mentioned Blender using SIMD Extensions only on x86 for a reason: to illustrate that if you're not a developer able to code and familiar with NEON2 on ARM you might be better off looking at x86 instead. The Gemini Lake thing on ODROID H2 for example is not equipped with Intel's latest and greatest extensions like AVX but at least fully supports SSE2 so maybe @rooted is so kind providing you with Blender numbers from H2? Maybe then your excitement for N2 is already gone and you're a future H2 buyer?

 

Your stuff (a rather special application making use of special instructions) is an exception so how should benchmarks that test entirely different stuff show what's going on? You can't appropriately benchmark without knowledge and without being focused on the use case. Otherwise you're all the time just collecting numbers without meaning.

 

Here I won't comment on why further contributing to Armbian (or using this forum) doesn't make that much sense for me but you should be aware that https://www.cnx-software.com is a great source of knowledge (especially in the comments section where insiders share details and experts explain so much stuff like how/why A72 and A73 differ and so on)

 

 

 

Posted
  On 2/23/2019 at 1:31 PM, tkaiser said:

you might be better off looking at x86 instead.

Expand  

I've got a very special use case that I don't think anybody else uses it for. I use them (among many other things) to edit and render videos on location(while traveling, while visiting interesting stuff...)
I must do this with power banks charged with solar panels. So for that reason ARM is the best choice(power efficient). I've got my "render bread box" for a long time with the C2(fastest while best passive cooled) for editing and rendering. The RPi2B with small display to record sound(because it's most power efficient). And I also just love sbc's...


I'm now putting together something better with a 13" 5V display and probably with the RockPi4B for this summer. The case will need to be the cooling for it. I've got more solar panels, so I can power more.
And I hope in the future to be able to make a new more powerful handheld sbc with the Pine H64.(if it's ever released, I've seen TL Lim on FOSDEM and he said it would be out first half of Februari) Video Pine64@FOSDEM

Since I couldn't get the right info for my use case(because nobody else uses it like this) I filmed some tests when I bought the Tinker Board and put it on Youtube. That video got a few thousand views. So I just kept making videos about new boards.
I try to do this a good as I can. That's why I love it when somebody comes with information I didn't know about what you often do.

Many people get their info on Youtube, it is mostly another public then we meet here in the Armbian forum, that uses SBC's for other tasks(gaming, desktop use). So that's what I show.
 

  On 2/23/2019 at 1:31 PM, tkaiser said:

you should be aware that https://www.cnx-software.com is a great source of knowledge

Expand  

I'll follow it. Thanks for the info.
 

  On 2/23/2019 at 1:31 PM, tkaiser said:

and then you do not benchmark to show how product A is faster than B but to get B as fast as A or even faster.

Expand  

But that's what people want. They want to see them lined up in an order. So it's the reviewer his choice what order he shows to the viewers/readers. That's why I try to find the better tools for the job, but none of those I've used are even near to perfect.(as I've shown in my videos) So I try to minimise the variables as much as possible by using the same distro, no throttlin, the same software version, the same settings, only ARMV7->V7 - V8->V8 ... But it's never possible to do this well. Most Youtubers don't even care about that, and show very weird results of sysbench or so.
I'll keep using 7zip, CPUMiner and blender for now because even with their faults. They still manage to make the most sensible line up. And I'll keep saying it only shows how well it does that task only. And there's a lot more to SBC's than how powerful the CPU is.

  On 2/23/2019 at 1:31 PM, tkaiser said:

Here I won't comment on why further contributing to Armbian (or using this forum)

Expand  

Well the last 2 days we've managed to keep you bussy here. And I enjoyed it, and I bet many readers will have too. So it's only a tiny step to stay and end the self-censorship. ;)
Have a great day. Read you soon.

Posted

I will do the BMW benchmark on an H2 if you are interested in that, it may be a few days as my son is still in the hospital.

I would like to know how it stacks up against the N2 anyway.

Posted
  On 2/23/2019 at 4:50 PM, NicoD said:

But that's what people want. They want to see them lined up in an order. So it's the reviewer his choice what order he shows to the viewers/readers.

Expand  

I'm not sure on this one.. I seriously don't read many benchmarks.. I bought my notebook without looking at any benchmark.. I decided against another one cause it's a known not properly working one on linux.. I had a few other requirements which a bunch of them didn't fulfill (e.g. browsers are ram hungry and no way in heaven I go back to a 8gb ram one).. Whats about showing how a random board with the current available OS's perform on the use-cases you and others might have? E.g. lightweight desktop scenarios etc.

Personally, I think benchmarking makes sense to compare the same board with various distros or find bottlenecks.. e.g. is my SBC capable of saturating a GbE link with encrypted storage.. and when not why? Can this part be improved or not? If I've a desktop usage in mind should I use a Debian, an Ubuntu or might something like a Arch or Gentoo be the better option. For sure this is difficult due to Linux on Arm (especially in desktop scenarios, except android) is nowhere compared to x86 world.

 

This approach doesn't benchmark the max. performance a *random SoC* is capable to deliver (and should also not be sold as such) but instead it represents the current state of what you can expect by buying random product. The majority of users don't spend as much time as some of us do to tweak settings. Such 'benchmarks' have then to be repeated over time to see if *random distribution for random board* improves or not. A board might be badly supported in the beginnings but hopefully they get it and pick up such tweaks so that it gets better over time. As a customer of a board I'm interested that the board producer makes my board better over time (e.g. as a beginner I would never buy a xunlong board, cause they're notoriously known to provide no support at all for their SBCs - the whole story differs as a more experienced guy, I can fix some flaws of a OPi as long as they're only software wise). But you target audience is more the end-user side right? So it might be interesting for them what they can expect by buying *random board* from *random boardmaker*.

 

This approach differs from @tkaiser benchmarking. He tries to benchmark the hardware and tweaks settings to ensure that the software impact is as marginal as it can be. Don't get me wrong, this is important as well, but as long as I don't have a distro which delivers those tweaks in their stock Images, I can't benefit from a powerful SoC (e.g. if thermal settings are badly on every distro for a random board I can't benefit from better settings as a end-user). Somehow the same counts for the blender benchmark:

  On 2/22/2019 at 3:21 PM, tkaiser said:

Blender test. Checking the relevance of SETTINGS instead of focusing on irrelevant hardware details like DDR3 vs. DDR4:

  • RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:15:31
  • RockPro64, Ubuntu Bionic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:06:59
  • RockPro64, Ubuntu Cosmic with LXDE, 2.0/1.5GHz, CONFIG_HZ=250: 1:01:11
Expand  

How often did you run those tests? Roughly 25% looks indeed like a real difference but I would prefer to have at least a triplicate to get a clue how much error margin I expect for such a test. I'm quite sure you'll still see a difference but the dataset locks not complete (e.g. 'RockPro64, Ubuntu Cosmic with LXDE, 2.0/1.5GHz, CONFIG_HZ=1000: 1:01:11' is missing). So you could twice see a difference between config_hz and twice a difference between cosmic and bionic. By a triplicate with an error margin you could also show that those numbers are somehow reliable (my current work outside SBCs has error margins of ~30-100%, means I run a lot of triplicates to ensure that there's a real difference before I get excited about something).

 

This isn't grass root benchmarking, it's more a treetops benchmarking and it should also be sold as such. Means that you've to be honest to your viewers that such numbers can change if distributions tweaks their settings and you can elaborate why a board-distribution combination might perform worse compared to another combination.

Posted
  On 2/23/2019 at 9:34 PM, rooted said:

I will do the BMW benchmark on an H2 if you are interested in that, it may be a few days as my son is still in the hospital.

Expand  

I wish you and your family the best that your son recovers soon. :) Just ignore all wishes from other users as long as you've more important issues.

Posted
  On 2/24/2019 at 2:10 PM, chwe said:
I wish you and your family the best that your son recovers soon. Just ignore all wishes from other users as long as you've more important issues.
Thanks@chwe it's appreciated. I just read the forum to keep my mind busy instead of worrying, anyone that has spent much time in a hospital likely knows what I mean.
Posted
  On 2/24/2019 at 2:35 PM, rooted said:
Thanks[mention=5623]chwe[/mention] it's appreciated. I just read the forum to keep my mind busy instead of worrying, anyone that has spent much time in a hospital likely knows what I mean.
Yes. Best wishes from my side as well.

Sent from my Pixel using Tapatalk

Posted
  On 2/24/2019 at 2:35 PM, rooted said:

I just read the forum to keep my mind busy instead of worrying, anyone that has spent much time in a hospital likely knows what I mean. 

Expand  

We create our own future, if you firmly believe that you will be all right-then everything will be fine.

Guest
This topic is now closed to further replies.
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines