NicoD

Hi all. 
I again had the pleasure of working with an amazing server. This time the AMD Threadripper 3990X, 64-cores and 128 threads.
After last week working on a 32-core ARM server I thought I had seen performance.

This is again not comparable with anything before.
 
I again got private SSH access. So I opened 3 terminals. One with HTop, another to check sensors. And the 3th to execute my benchmarks.
First thing I saw were the 128-threads. Being used to seeing 6, this was almost unbelievable.

With light loads it turbo's up to 4.3Ghz. All cores maxed out @ 3Ghz while consuming 400W.
Reaching a single core 7zip decompression score of 4545MIPS @ 4.3Ghz.
The Ampere 32-core ARM server at 3.3Ghz reached 2763.
This again shows the Ampere server doesn't use high performance cores. It doesn't perform great per clock.

Coming soon is a benchmark of an AWS server. This uses high performance cores based on the ARM N1 cores. A derivative of the A76.
This reaches 3393. This clocked at only 2.5Ghz. So this does perform better per clock. Do know this is comparing peers with bananas(don't want to confuse with apples).

And scoring 391809MIPS with 7zip multi-core decompression with default settings.

Then with an overclock to 3.9Ghz all cores it consumed +600W. With a 7zip decompression score of 433702MIPS
This is again so many levels better than the Ampere 32-core ARM server which got 85975MIPS. 32-cores of the AWS graviton2 does 110628.
So this AMD server is up to 5 x more powerful when overclocked, than the Ampere 32-core server. Consuming 6 x as much. 
With normal configuration they both perform almost as well in performance/watt.

In idle the Threadripper sonsumed 100W, what is a lot for doing nothing.
The 32-core ARM server only consumed a bit more than 100W maxed out. And about 20W in idle.

The BMW Blender benchmark, which takes 29m23s on the fastest ARM SBC the Odroid N2+. The Ampere ARM server did it in 8m27s.
For the Threadripper this was a way too light load, it did it in 30s. 

Even when doing this render 10 x after each other it didn't raise the temperatures much. The maximum I've seen was 50C.

To try a heavier load I downloaded the Barber Shop Blender render. This was 6912 tiles to render. But again the Threadripper wasn't impressed by this load. 2m18s79. The AWS with 32-cores (of 64) done this in 8m28s. So this ARM server does compete well per clock for a floating point task with TR.

ARM may be great, but AMD is mighty. Intel does not have anything to compete with this. Certainly not performance/watt. 
It was a pleasure benchmarking this server. 
I learned a lot, like that I need to find better tools for these amazing machines.  
 
The specs of this monster :
 
ASRock Rack TRX40D8-2N2T AMD Ryzen Threadripper 3990x 256GB memory (8 x 32Gb) ECC 2 x 1TB PCI 4.0 Nvme SSD Water Cooling  
The specs of the Threadripper 3990x
 
64-cores 128-threads AMD64 Zen2 Matisse 2.9Ghz - 4.3Ghz 4-channel DDR4-3200 MHz 256GB RAM 88 lanes PCIe4 TSMC's 7nm process node 280W - +400W 32 KB L1 per core (64x) 64 x 512 KB L2 256 MB L3 cache shared
You can see my full review video here, greetings.
NicoD

 

I was just kidding. 
 
 
I had to look mine up (I have a Kill-A-Watt and do periodic power testing on things) and the KGPE-D16 idles about 165 W and 372 at full power.  But it's older tech, much less powerful than this brand new Threadripper of course.  I trade efficiency for freedom (and happy to do so). 
 
Until NSA launch their back doors, and then I am the only one left standing with a running computer.  Muhahahahah!    Hello, this is John Connor. 
 
But I don't need it most of the time, for personal use.  In fact just talking to Nico about his videos making me think very hard about using SBC for daily driver desktop and save some electricity.

Link to donate for the server 
May we know more about the ARM64 server? What cores? How many? What frequency? What cooling? Just any info  

I've got my RK3399's with the M4 and M4V2. Plenty enough. It will not perform any differnt.
Why don't you write a review about it on the forum? And if you want, you can film a few things and I can put it in a video for you.
Shipping in Europe is also pretty expensive. But thanks for the offer.

P.S.: Amazing gaming Armbian image almost available for RK3399 made by @Salvador Liébana. I'm now making a preview video about it.
Box86 installed. So x86 linux games, and Windows games/programs are possible on RK3399.

I will but split it up into at least three posts to fulfill the requirements
 
Would be 14€ from DE to BE.

I've got my RK3399's with the M4 and M4V2. Plenty enough. It will not perform any differnt.
Why don't you write a review about it on the forum? And if you want, you can film a few things and I can put it in a video for you.
Shipping in Europe is also pretty expensive. But thanks for the offer.

P.S.: Amazing gaming Armbian image almost available for RK3399 made by @Salvador Liébana. I'm now making a preview video about it.
Box86 installed. So x86 linux games, and Windows games/programs are possible on RK3399.

I've got my RK3399's with the M4 and M4V2. Plenty enough. It will not perform any differnt.
Why don't you write a review about it on the forum? And if you want, you can film a few things and I can put it in a video for you.
Shipping in Europe is also pretty expensive. But thanks for the offer.

P.S.: Amazing gaming Armbian image almost available for RK3399 made by @Salvador Liébana. I'm now making a preview video about it.
Box86 installed. So x86 linux games, and Windows games/programs are possible on RK3399.

Yes I am. From Belgium.
I sent them a mail but didn't get an answer.
I couldn't put a message on the forum.

That is 4.4 x the 32-core ARM server.
25 x my I5 2500K @ 3.7Ghz
32.4 x Odroid N2+
45.1 x RK3399

That would be amazing. Another video coming then, and another round to ask for funding
I bet people would love to see this. I'm baffled by that result.
Now if that will not be powerful enough to build images... I could never have imagined 20 years ago something like this would be possible.
I now know it's time to look for a better pc

I had many difficulties with a setup, hw and sw wise + daily workload - was totally exhausted. It was insane intense day  Fear and fatigue mixed with excitement. 
 
I did quick 7zip benchmark with slight load in the back so numbers are not very accurate:
 
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,128 CPUs AMD Ryzen Threadripper 3990X 64-Core Processor (830F10),ASM,AES-NI) AMD Ryzen Threadripper 3990X 64-Core Processor (830F10) CPU Freq: - - - - - - 512000000 - - RAM size: 257677 MB, # CPU hardware threads: 128 RAM usage: 28240 MB, # Benchmark threads: 128 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 265590 11420 2263 258367 | 4449269 12095 3139 379390 23: 158167 10527 1531 161153 | 4467139 12343 3133 386528 24: 120144 11304 1143 129179 | 4297484 12140 3108 377129 25: 104345 11827 1008 119137 | 4206512 12129 3088 374316 ---------------------------------- | ------------------------------ Avr: 11270 1486 166959 | 12177 3117 379341
What we need it for is awesome - it can rebuild kernels in no time. Only thing i recently measured was that it can assemble 194 images in 32 minutes (sadly i don't have exact number from previous server, but it was measured in hours) and it has more efficient cooling. It doesn't heat up much but it consumes around 100W more power than the previous one  In term of raw cpu power Passmark went from 24k -> 80k + a lot better IO due to raid 0 PCI4 nvme drives and faster memory.
 
@Werner @NicoD
 
I will give you access for a few days, then its erased and goes into the real function, where access under certain conditions will also be possible - we are discussing terms here https://armbian.atlassian.net/browse/AR-538

I don't think the OPi3 can boot from USB directly. So you'll always need an sd-card for initial boot.

Too bad you don't have eMMC. It is a great feature to have on these OPi's. 

Assembled - first boot of Ubuntu 20.04 looks very very nice  But it was tense ... "what if it will not work?"
 
Also NVME and SATA DOMs are recognized but 10Gb NIC. Which probably need more recent kernel ... and water cooling doesn't fit into the case. Currently mounted outside, which might even not be that bad.
 

what a shame,
no i did not :-(
 

"1. Application Date: 2020.10.16 - 2020.11.14"

Date today 2020.11.15  

Hi all.
Here my newest video about old boards that still do their job well.
OrangePi+ and OPi+2.
My favorite NAS.


Here all my gathered data.
 

Greetings, NicoD 

You can still build on Bionic if you issue code { font-family: Consolas,"courier new"; color: crimson; background-color: rgba(0, 0, 0, 0.2); padding: 2px; font-size: 105%; } NO_HOST_RELEASE_CHECK=yes
Whatever may or may not get built will not receive any support whatsoever.

I love this picture. Between porn and art. 
Looks like a miniature city I'd want to live in. Great how the memory is so in focus. You could have become a pro fotographer instead  

Boot SATA DOM installed. Let someone double check if polarity is correct even its hard to make it wrong  
 

Today I had the pleasure of benchmarking an ARM64 server.
This server has been made available for Armbian to test native ARM64 image building.
I knew nothing about the server. Nobody told me any details.
So everything was an adventure for me to find out. I got SSH access, so my research began.

A lscpu informed me it had 32-cores all clocked at 3.3Ghz. 
 
cat /proc/cpuinfo confirmed these 32-cores
 
Checking on what kernel we're on. Ubuntu Focal 5.4.0-52-generic. 
And how much memory. 128GB RAM.

So first thing I wanted to know, how does one core perform with 7-zip benchmark?
The record I had seen until now was from the A73 cores from the Odroid N2+ clocked at 2.4Ghz. 2504MIPS decompression.
So :
taskset -c 31 7z b
 
This beats the Odroid N2+ its A73 cores clocked at 2.4Ghz. 2763 vs 2504MIPS decompression. 
This also tells me these cores do not perform as good per clock as a high performance core. 
While doing the single core benchmark I checked the sensors to know the wattage and temperature.
 
CPU power is about 20W for a single core tasks. 
Without a load the CPU consumes between 10W-15W. So in total it consumes a bit more than 20W in idle.
Temperature never went under 49C even after +5 minutes in idle. 
 
Of course, the next thing to do is an all-core 7zip benchmark. 
This gives an amazing result. Way higher than anything I had ever seen on ARM.
 
85975MIPS decompression. This is amazing.
Best I had seen was 11000MIPS of the Odroid N2+. So this server does 8 x better than the N2+. 
Tho, I must say. 7zip does bad with unequal clusers. The N2+ has a great difference in cluster frequencies. So it performs worse then expected here. 

The wattage went a lot higher, up to 110W. And the temperature rose quickly up to 75C in seconds.
 
To test the internet connection I downloaded an Armbian image multiple times. Sometimes it was as low as 3MB/s. 
Highest average speed I've seen was 12.5MB/s
 
Next test. BMW Blender render benchmark. 
Here the fastest I had ever seen was by the Khadas VIM3. That did it in 42m51s.
I haven't done this yet with the N2+ in Armbian. In Odroid's Ubuntu it was a little slower. I expect it to be a little faster than the VIM3 in Armbian Bionic. 
This is a tile based test. So every core gets its own task, until all tiles are done. 

Well, this ARM64 server did this in 8m27s. 
5 x faster compared to the Khadas VIM3. 

For this the wattage didn't go over 85W. But the temperature did rise to 83C. So it started to throttle. 

@lanefu already had done SBC-Bench on it when it was free. So this I didn't have to do myself.
http://ix.io/2Dcc
Here we see a lot. For example the CPUMiner did : 81.0kH/s 
The Odroid N2+                                                         : 14 kH/s         5.7 x less 
RK3399 does a maximum of                                     : 10.23kH/s     8 x less
Odroid C2 clocked at 1.75Ghz                                   : 4.65kH/s       17 x less

So this server clearly can move a lot of bits around. 
Now, what is this server? Ask google if nobody else tells me. "32 core ARM server 3.3Ghz"
First answer : https://www.theregister.com/2018/09/18/ampere_shipping/
That looks like it is this CPU. But still I can't find the exact name. 
2nd answer : https://www.servethehome.com/ampere-32-core-64-bit-arm-chip-x-gene-3-ip/
So this is the Ampere 32-core 64-bit from X-Gene 3 IP.

Here the wikichip : https://en.wikichip.org/wiki/apm/x-gene/apm883832-x3?fbclid=IwAR0ljCQ61DY8Zwh_VyZd0fQH43dmPUTJA-CGLiQKYqU2fWwszFm1CPjH6Zo

This supports up to 1TB RAM. 8 channels @ 2666Mhz. With a maximum memory bandwidth of 158.95 GiB/s.
42 lanes of PCIe Gen 3, with 8 controllers
– x16 or two x8/x4
– x16 or two x8/x4
– x8 or two x4
– Two x1

4 x SATA Gen 3 ports, 2 x USB2. And a TDP of 125W TDP.

For me this is just an awesome thing to behold. I use ARM for almost everything.
The NanoPi M4V2 is my main desktop computer.
It isn't as powerful as my PC, but does the task for 10 x less power consumption, while being completely silent.

But when I need a big CPU, it isn't enough.
Even the more powerful Odroid N2+ isn't powerful enough to render long, +20minutes 1440p video's for example for my Youtube channel.
So then i need to use my x86/amd64 PC. 

Today I have seen and tasted the future. 
While this doesn't use the most modern Cortex/clusters. And it is only 16nm.
So there is still a lot of room for improvements in performance and lower power consumption. 

ARM for desktop is possible, and ARM servers for big datacenters is possible(AWS). I have seen the future, I loved every second of it. 

Here benchmarks compared to my SBCs

 

Greetings, NicoD

Memory installed.
 

Today I had the pleasure of benchmarking an ARM64 server.
This server has been made available for Armbian to test native ARM64 image building.
I knew nothing about the server. Nobody told me any details.
So everything was an adventure for me to find out. I got SSH access, so my research began.

A lscpu informed me it had 32-cores all clocked at 3.3Ghz. 
 
cat /proc/cpuinfo confirmed these 32-cores
 
Checking on what kernel we're on. Ubuntu Focal 5.4.0-52-generic. 
And how much memory. 128GB RAM.

So first thing I wanted to know, how does one core perform with 7-zip benchmark?
The record I had seen until now was from the A73 cores from the Odroid N2+ clocked at 2.4Ghz. 2504MIPS decompression.
So :
taskset -c 31 7z b
 
This beats the Odroid N2+ its A73 cores clocked at 2.4Ghz. 2763 vs 2504MIPS decompression. 
This also tells me these cores do not perform as good per clock as a high performance core. 
While doing the single core benchmark I checked the sensors to know the wattage and temperature.
 
CPU power is about 20W for a single core tasks. 
Without a load the CPU consumes between 10W-15W. So in total it consumes a bit more than 20W in idle.
Temperature never went under 49C even after +5 minutes in idle. 
 
Of course, the next thing to do is an all-core 7zip benchmark. 
This gives an amazing result. Way higher than anything I had ever seen on ARM.
 
85975MIPS decompression. This is amazing.
Best I had seen was 11000MIPS of the Odroid N2+. So this server does 8 x better than the N2+. 
Tho, I must say. 7zip does bad with unequal clusers. The N2+ has a great difference in cluster frequencies. So it performs worse then expected here. 

The wattage went a lot higher, up to 110W. And the temperature rose quickly up to 75C in seconds.
 
To test the internet connection I downloaded an Armbian image multiple times. Sometimes it was as low as 3MB/s. 
Highest average speed I've seen was 12.5MB/s
 
Next test. BMW Blender render benchmark. 
Here the fastest I had ever seen was by the Khadas VIM3. That did it in 42m51s.
I haven't done this yet with the N2+ in Armbian. In Odroid's Ubuntu it was a little slower. I expect it to be a little faster than the VIM3 in Armbian Bionic. 
This is a tile based test. So every core gets its own task, until all tiles are done. 

Well, this ARM64 server did this in 8m27s. 
5 x faster compared to the Khadas VIM3. 

For this the wattage didn't go over 85W. But the temperature did rise to 83C. So it started to throttle. 

@lanefu already had done SBC-Bench on it when it was free. So this I didn't have to do myself.
http://ix.io/2Dcc
Here we see a lot. For example the CPUMiner did : 81.0kH/s 
The Odroid N2+                                                         : 14 kH/s         5.7 x less 
RK3399 does a maximum of                                     : 10.23kH/s     8 x less
Odroid C2 clocked at 1.75Ghz                                   : 4.65kH/s       17 x less

So this server clearly can move a lot of bits around. 
Now, what is this server? Ask google if nobody else tells me. "32 core ARM server 3.3Ghz"
First answer : https://www.theregister.com/2018/09/18/ampere_shipping/
That looks like it is this CPU. But still I can't find the exact name. 
2nd answer : https://www.servethehome.com/ampere-32-core-64-bit-arm-chip-x-gene-3-ip/
So this is the Ampere 32-core 64-bit from X-Gene 3 IP.

Here the wikichip : https://en.wikichip.org/wiki/apm/x-gene/apm883832-x3?fbclid=IwAR0ljCQ61DY8Zwh_VyZd0fQH43dmPUTJA-CGLiQKYqU2fWwszFm1CPjH6Zo

This supports up to 1TB RAM. 8 channels @ 2666Mhz. With a maximum memory bandwidth of 158.95 GiB/s.
42 lanes of PCIe Gen 3, with 8 controllers
– x16 or two x8/x4
– x16 or two x8/x4
– x8 or two x4
– Two x1

4 x SATA Gen 3 ports, 2 x USB2. And a TDP of 125W TDP.

For me this is just an awesome thing to behold. I use ARM for almost everything.
The NanoPi M4V2 is my main desktop computer.
It isn't as powerful as my PC, but does the task for 10 x less power consumption, while being completely silent.

But when I need a big CPU, it isn't enough.
Even the more powerful Odroid N2+ isn't powerful enough to render long, +20minutes 1440p video's for example for my Youtube channel.
So then i need to use my x86/amd64 PC. 

Today I have seen and tasted the future. 
While this doesn't use the most modern Cortex/clusters. And it is only 16nm.
So there is still a lot of room for improvements in performance and lower power consumption. 

ARM for desktop is possible, and ARM servers for big datacenters is possible(AWS). I have seen the future, I loved every second of it. 

Here benchmarks compared to my SBCs

 

Greetings, NicoD

I fixed it
 

Here my review video about the server.
 

Today I had the pleasure of benchmarking an ARM64 server.
This server has been made available for Armbian to test native ARM64 image building.
I knew nothing about the server. Nobody told me any details.
So everything was an adventure for me to find out. I got SSH access, so my research began.

A lscpu informed me it had 32-cores all clocked at 3.3Ghz. 
 
cat /proc/cpuinfo confirmed these 32-cores
 
Checking on what kernel we're on. Ubuntu Focal 5.4.0-52-generic. 
And how much memory. 128GB RAM.

So first thing I wanted to know, how does one core perform with 7-zip benchmark?
The record I had seen until now was from the A73 cores from the Odroid N2+ clocked at 2.4Ghz. 2504MIPS decompression.
So :
taskset -c 31 7z b
 
This beats the Odroid N2+ its A73 cores clocked at 2.4Ghz. 2763 vs 2504MIPS decompression. 
This also tells me these cores do not perform as good per clock as a high performance core. 
While doing the single core benchmark I checked the sensors to know the wattage and temperature.
 
CPU power is about 20W for a single core tasks. 
Without a load the CPU consumes between 10W-15W. So in total it consumes a bit more than 20W in idle.
Temperature never went under 49C even after +5 minutes in idle. 
 
Of course, the next thing to do is an all-core 7zip benchmark. 
This gives an amazing result. Way higher than anything I had ever seen on ARM.
 
85975MIPS decompression. This is amazing.
Best I had seen was 11000MIPS of the Odroid N2+. So this server does 8 x better than the N2+. 
Tho, I must say. 7zip does bad with unequal clusers. The N2+ has a great difference in cluster frequencies. So it performs worse then expected here. 

The wattage went a lot higher, up to 110W. And the temperature rose quickly up to 75C in seconds.
 
To test the internet connection I downloaded an Armbian image multiple times. Sometimes it was as low as 3MB/s. 
Highest average speed I've seen was 12.5MB/s
 
Next test. BMW Blender render benchmark. 
Here the fastest I had ever seen was by the Khadas VIM3. That did it in 42m51s.
I haven't done this yet with the N2+ in Armbian. In Odroid's Ubuntu it was a little slower. I expect it to be a little faster than the VIM3 in Armbian Bionic. 
This is a tile based test. So every core gets its own task, until all tiles are done. 

Well, this ARM64 server did this in 8m27s. 
5 x faster compared to the Khadas VIM3. 

For this the wattage didn't go over 85W. But the temperature did rise to 83C. So it started to throttle. 

@lanefu already had done SBC-Bench on it when it was free. So this I didn't have to do myself.
http://ix.io/2Dcc
Here we see a lot. For example the CPUMiner did : 81.0kH/s 
The Odroid N2+                                                         : 14 kH/s         5.7 x less 
RK3399 does a maximum of                                     : 10.23kH/s     8 x less
Odroid C2 clocked at 1.75Ghz                                   : 4.65kH/s       17 x less

So this server clearly can move a lot of bits around. 
Now, what is this server? Ask google if nobody else tells me. "32 core ARM server 3.3Ghz"
First answer : https://www.theregister.com/2018/09/18/ampere_shipping/
That looks like it is this CPU. But still I can't find the exact name. 
2nd answer : https://www.servethehome.com/ampere-32-core-64-bit-arm-chip-x-gene-3-ip/
So this is the Ampere 32-core 64-bit from X-Gene 3 IP.

Here the wikichip : https://en.wikichip.org/wiki/apm/x-gene/apm883832-x3?fbclid=IwAR0ljCQ61DY8Zwh_VyZd0fQH43dmPUTJA-CGLiQKYqU2fWwszFm1CPjH6Zo

This supports up to 1TB RAM. 8 channels @ 2666Mhz. With a maximum memory bandwidth of 158.95 GiB/s.
42 lanes of PCIe Gen 3, with 8 controllers
– x16 or two x8/x4
– x16 or two x8/x4
– x8 or two x4
– Two x1

4 x SATA Gen 3 ports, 2 x USB2. And a TDP of 125W TDP.

For me this is just an awesome thing to behold. I use ARM for almost everything.
The NanoPi M4V2 is my main desktop computer.
It isn't as powerful as my PC, but does the task for 10 x less power consumption, while being completely silent.

But when I need a big CPU, it isn't enough.
Even the more powerful Odroid N2+ isn't powerful enough to render long, +20minutes 1440p video's for example for my Youtube channel.
So then i need to use my x86/amd64 PC. 

Today I have seen and tasted the future. 
While this doesn't use the most modern Cortex/clusters. And it is only 16nm.
So there is still a lot of room for improvements in performance and lower power consumption. 

ARM for desktop is possible, and ARM servers for big datacenters is possible(AWS). I have seen the future, I loved every second of it. 

Here benchmarks compared to my SBCs

 

Greetings, NicoD

Today I had the pleasure of benchmarking an ARM64 server.
This server has been made available for Armbian to test native ARM64 image building.
I knew nothing about the server. Nobody told me any details.
So everything was an adventure for me to find out. I got SSH access, so my research began.

A lscpu informed me it had 32-cores all clocked at 3.3Ghz. 
 
cat /proc/cpuinfo confirmed these 32-cores
 
Checking on what kernel we're on. Ubuntu Focal 5.4.0-52-generic. 
And how much memory. 128GB RAM.

So first thing I wanted to know, how does one core perform with 7-zip benchmark?
The record I had seen until now was from the A73 cores from the Odroid N2+ clocked at 2.4Ghz. 2504MIPS decompression.
So :
taskset -c 31 7z b
 
This beats the Odroid N2+ its A73 cores clocked at 2.4Ghz. 2763 vs 2504MIPS decompression. 
This also tells me these cores do not perform as good per clock as a high performance core. 
While doing the single core benchmark I checked the sensors to know the wattage and temperature.
 
CPU power is about 20W for a single core tasks. 
Without a load the CPU consumes between 10W-15W. So in total it consumes a bit more than 20W in idle.
Temperature never went under 49C even after +5 minutes in idle. 
 
Of course, the next thing to do is an all-core 7zip benchmark. 
This gives an amazing result. Way higher than anything I had ever seen on ARM.
 
85975MIPS decompression. This is amazing.
Best I had seen was 11000MIPS of the Odroid N2+. So this server does 8 x better than the N2+. 
Tho, I must say. 7zip does bad with unequal clusers. The N2+ has a great difference in cluster frequencies. So it performs worse then expected here. 

The wattage went a lot higher, up to 110W. And the temperature rose quickly up to 75C in seconds.
 
To test the internet connection I downloaded an Armbian image multiple times. Sometimes it was as low as 3MB/s. 
Highest average speed I've seen was 12.5MB/s
 
Next test. BMW Blender render benchmark. 
Here the fastest I had ever seen was by the Khadas VIM3. That did it in 42m51s.
I haven't done this yet with the N2+ in Armbian. In Odroid's Ubuntu it was a little slower. I expect it to be a little faster than the VIM3 in Armbian Bionic. 
This is a tile based test. So every core gets its own task, until all tiles are done. 

Well, this ARM64 server did this in 8m27s. 
5 x faster compared to the Khadas VIM3. 

For this the wattage didn't go over 85W. But the temperature did rise to 83C. So it started to throttle. 

@lanefu already had done SBC-Bench on it when it was free. So this I didn't have to do myself.
http://ix.io/2Dcc
Here we see a lot. For example the CPUMiner did : 81.0kH/s 
The Odroid N2+                                                         : 14 kH/s         5.7 x less 
RK3399 does a maximum of                                     : 10.23kH/s     8 x less
Odroid C2 clocked at 1.75Ghz                                   : 4.65kH/s       17 x less

So this server clearly can move a lot of bits around. 
Now, what is this server? Ask google if nobody else tells me. "32 core ARM server 3.3Ghz"
First answer : https://www.theregister.com/2018/09/18/ampere_shipping/
That looks like it is this CPU. But still I can't find the exact name. 
2nd answer : https://www.servethehome.com/ampere-32-core-64-bit-arm-chip-x-gene-3-ip/
So this is the Ampere 32-core 64-bit from X-Gene 3 IP.

Here the wikichip : https://en.wikichip.org/wiki/apm/x-gene/apm883832-x3?fbclid=IwAR0ljCQ61DY8Zwh_VyZd0fQH43dmPUTJA-CGLiQKYqU2fWwszFm1CPjH6Zo

This supports up to 1TB RAM. 8 channels @ 2666Mhz. With a maximum memory bandwidth of 158.95 GiB/s.
42 lanes of PCIe Gen 3, with 8 controllers
– x16 or two x8/x4
– x16 or two x8/x4
– x8 or two x4
– Two x1

4 x SATA Gen 3 ports, 2 x USB2. And a TDP of 125W TDP.

For me this is just an awesome thing to behold. I use ARM for almost everything.
The NanoPi M4V2 is my main desktop computer.
It isn't as powerful as my PC, but does the task for 10 x less power consumption, while being completely silent.

But when I need a big CPU, it isn't enough.
Even the more powerful Odroid N2+ isn't powerful enough to render long, +20minutes 1440p video's for example for my Youtube channel.
So then i need to use my x86/amd64 PC. 

Today I have seen and tasted the future. 
While this doesn't use the most modern Cortex/clusters. And it is only 16nm.
So there is still a lot of room for improvements in performance and lower power consumption. 

ARM for desktop is possible, and ARM servers for big datacenters is possible(AWS). I have seen the future, I loved every second of it. 

Here benchmarks compared to my SBCs

 

Greetings, NicoD