0
esbeeb

NanoPi Neo 2 LTS: net I/O speed tests, etc.

Recommended Posts

Hello,

There seems to be scant details about the NanoPi Neo 2 LTS, so I thought I'd contribute some.  My testing is much less formal than how some would measure it.  What I especially like to do in the way of network speed testing are the "worst-case scenarios". 

 

My way of invoking a sort of "torture test" is to mostly test upload speeds (as they tend to be slower than download speeds), both large files (much easier), as well as uploading many tiny files (much more gruelling to store, so is therefore a major "torture test").

 

Having said that, I'll usually favor all-GbE networking (noting which I'm using, GbE or wifi), because I know GbE will be quite a bit faster than WiFi.

 

FWIW, here are my findings.

 

- I got the "1-bay NAS Dock v1.2 for NanoPi NEO/NEO2", allowing me to hook up a 2.5" SSD, I got a Samsung 860 EVO (500GB).

 

For the OS, I used the current Armbian Stretch.  Once booted up, and logged in over ssh, I created a btrfs partition on the Samsung EVO, then mounted it, then installed "pure-ftpd". 

 

When I did an upload test to the Samsung SSD entirely through Gigabit ethernet (using a Gigabit ethernet dongle on my laptop, which I know can do 40MB/sec easily), here's what I found:

 

- When you upload larger files, like say a 2 GB .zip file, the average FTP upload speed was 29.9 MB/sec

- When you upload a whole bunch of smaller files, like 2 GB worth (2029 files), the average FTP upload speed was 18.7 MB/sec

 

Note: I'm using a lacklustre GbE dongle on my laptop, a TP-Link UE300 (which is better than jack squat, because it's supported in Linux whatsoever), and the fastest I've ever been able to get it to go is 105MB/sec, sustained (when connected do a different server, not this NanoPi). This dongle is definately fast enough for my tests.

Share this post


Link to post
Share on other sites

I'd like to say a big Thanks for the nice "armbian-config" utility, especially the "Softy" submenu (where I'm delighted to find a Nextcloud installer!).  I'm using Armbian for the first time, and I'm learning the ropes (having said that, I have lots of Debian sysadmin experience).  I also love how the root filesystem is ext4, not ext2!  Yay, journalling.

Share this post


Link to post
Share on other sites

Here's some iperf3 output (Edit: ...and this was was when my laptop was using WiFi):

 

Connecting to host nanopi, port 5201
[  4] local 192.168.0.129 port 48934 connected to 192.168.0.137 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  31.2 MBytes   261 Mbits/sec   47    313 KBytes       
[  4]   1.00-2.00   sec  33.7 MBytes   283 Mbits/sec    0    376 KBytes       
[  4]   2.00-3.00   sec  35.0 MBytes   293 Mbits/sec   32    317 KBytes       
[  4]   3.00-4.00   sec  35.3 MBytes   296 Mbits/sec   26    272 KBytes       
[  4]   4.00-5.00   sec  35.2 MBytes   295 Mbits/sec    0    346 KBytes       
[  4]   5.00-6.00   sec  34.6 MBytes   290 Mbits/sec   10    315 KBytes       
[  4]   6.00-7.00   sec  35.2 MBytes   295 Mbits/sec   26    269 KBytes       
[  4]   7.00-8.00   sec  32.6 MBytes   273 Mbits/sec    2    283 KBytes       
[  4]   8.00-9.00   sec  34.1 MBytes   286 Mbits/sec    0    354 KBytes       
[  4]   9.00-10.00  sec  34.8 MBytes   292 Mbits/sec   23    328 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   342 MBytes   287 Mbits/sec  166             sender
[  4]   0.00-10.00  sec   340 MBytes   285 Mbits/sec                  receiver

iperf Done.

 

Share this post


Link to post
Share on other sites

I installed Nextcloud 14 on my NanoPi Neo 2 LTS, with the Nas kit, and a 2.5" SATA Samsung SSD (and I used "ncp-config" to move the Nextcloud database, and datadir onto that SSD, not the MicroSD card).  I was curious to see the performance on copying files into Nextcloud (over SSL-encrypted WebDAV, using my Linux Mint’s Nemo File Browser).

 

For testing, I had a 2GB .zip file, which uploaded in 3:03, or 10.9 MB/sec average (Note: If that same 2GB file were uploading through FTP into the very same hardware, it would instead get 29.9 MB/sec, see above).  The NanoPi's 4 cores all worked about 15% average, each, and the load average stayed at about 1.3 the whole transfer.  The CPU was about 45 deg C max, in a room which had an ambient temp. of about 30 deg C.

 

I also had a folder containing 2GB of many very small files (2,785 files of varying sizes).  Copying the folder in took 34:11, or 1.0 MB/sec average (Note: If that same 2GB folder were uploading through FTP into the very same hardware, it would instead get 18.7 MB/sec, see above).  The NanoPi's 4 cores all worked about 25% average, each, and the load average peaked out at a max of about 1.5, but was usually like 1.2 the whole transfer.  The CPU slowly climbed up to 62 deg C, in a room which had an ambient temp. of about 30 deg C.

 

Pictured just below: on the left is Nemo file Browser copying in the 2GB folder over WebDAV, and on the right, you see an htop running on the NanoPi, running Nextcloud 14:

 

upload_webdav.thumb.png.9843f16239f8e71ad812992aa8cd0be5.png

 

I would suggest that the NanoPi Neo 2 LTS, with the NAS kit and an SSD is sort of like a bare minimum for an office with like 1 or 2 people in it, who are just storing documents, etc.  IMHO, You are just sort of torturing yourself far too much, just to save a very small amount of money, if you go any lower than the NanoPi Neo 2 LTS (with an SSD).

Share this post


Link to post
Share on other sites
On 10/25/2018 at 10:35 AM, esbeeb said:

Here's some iperf3 output

 

There are unusual lots of retries, speed should be better. Now this can be a 3rd party hardware issues, your cabling or router ... I only have a board without dock and can't recreate.

 

Try change cables and you could test new (stable to be) kernel - but I can't give you warranty that it works since we are still testing/debugging it: https://forum.armbian.com/topic/8795-next-lts-kernel-419y-allwinner-a10-a20-a64-h2-h3-h5-h6-debugging-party/

 

Share this post


Link to post
Share on other sites
1 hour ago, Igor said:

 

There are unusual lots of retries, speed should be better. Now this can be a 3rd party hardware issues, your cabling or router ... I only have a board without dock and can't recreate.

 

Try change cables and you could test new (stable to be) kernel - but I can't give you warranty that it works since we are still testing/debugging it: https://forum.armbian.com/topic/8795-next-lts-kernel-419y-allwinner-a10-a20-a64-h2-h3-h5-h6-debugging-party/

 

I changed the ethernet cable to a brand new, store-bought cable.  But the iperf3 output didn't really improve:

root@chicklet:~# iperf3 -c nextcloudpi
Connecting to host nextcloudpi, port 5201
[  4] local 192.168.0.129 port 44196 connected to 192.168.0.137 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  36.2 MBytes   303 Mbits/sec   69    317 KBytes       
[  4]   1.00-2.00   sec  32.3 MBytes   271 Mbits/sec    7    288 KBytes       
[  4]   2.00-3.00   sec  32.2 MBytes   270 Mbits/sec    9    272 KBytes       
[  4]   3.00-4.00   sec  32.5 MBytes   273 Mbits/sec    2    264 KBytes       
[  4]   4.00-5.00   sec  32.3 MBytes   270 Mbits/sec    1    255 KBytes       
[  4]   5.00-6.00   sec  32.6 MBytes   274 Mbits/sec    0    338 KBytes       
[  4]   6.00-7.00   sec  32.0 MBytes   268 Mbits/sec    4    259 KBytes       
[  4]   7.00-8.00   sec  32.4 MBytes   272 Mbits/sec    1    255 KBytes       
[  4]   8.00-9.00   sec  32.1 MBytes   269 Mbits/sec    2    243 KBytes       
[  4]   9.00-10.00  sec  30.3 MBytes   254 Mbits/sec   13    260 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   325 MBytes   272 Mbits/sec  108             sender
[  4]   0.00-10.00  sec   322 MBytes   270 Mbits/sec                  receiver

iperf Done.

Furthermore, when I do an "ifconfig" both on my client and server machines (for iperf3), neither shows any dropped packets (Note the "errors 0 dropped 0" for both RX and TX in all cases):

root@chicklet:~# /sbin/ifconfig 
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 371  bytes 32291 (32.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 371  bytes 32291 (32.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.129  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::9ec0:39e5:56a0:9721  prefixlen 64  scopeid 0x20<link>
        ether f8:bb:aa:zz:yy:xx  txqueuelen 1000  (Ethernet)
        RX packets 198902  bytes 18974787 (18.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 458133  bytes 701689760 (701.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
root@nextcloudpi:~# ifconfig 
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.137  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::dcb7:78de:d32f:64c4  prefixlen 64  scopeid 0x20<link>
        ether 02:bb:aa:zz:yy:xx  txqueuelen 1000  (Ethernet)
        RX packets 454893  bytes 688170496 (656.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 193446  bytes 12843866 (12.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 24  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Note that my client laptop, "chicklet", is connected over wifi to the router.  So there's only one ethernet cable to possibly blame, here.

Edited by esbeeb
obscured MAC addresses.

Share this post


Link to post
Share on other sites

When I go all-Ethernet (and don't have wifi between my laptop, chicklet, and the wifi router), then iperf3 gets much better:

root@chicklet:~# iperf3 -c nextcloudpi
Connecting to host nextcloudpi, port 5201
[  4] local 192.168.0.115 port 33706 connected to 192.168.0.137 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   114 MBytes   952 Mbits/sec    0    307 KBytes       
[  4]   1.00-2.00   sec   112 MBytes   941 Mbits/sec    0    307 KBytes       
[  4]   2.00-3.00   sec   112 MBytes   942 Mbits/sec    0    324 KBytes       
[  4]   3.00-4.00   sec   112 MBytes   939 Mbits/sec    0    348 KBytes       
[  4]   4.00-5.00   sec   112 MBytes   942 Mbits/sec    0    348 KBytes       
[  4]   5.00-6.00   sec   112 MBytes   941 Mbits/sec    0    348 KBytes       
[  4]   6.00-7.00   sec   112 MBytes   942 Mbits/sec    0    348 KBytes       
[  4]   7.00-8.00   sec   112 MBytes   940 Mbits/sec    0    348 KBytes       
[  4]   8.00-9.00   sec   112 MBytes   941 Mbits/sec    0    348 KBytes       
[  4]   9.00-10.00  sec   112 MBytes   942 Mbits/sec    0    348 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.10 GBytes   942 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec                  receiver

iperf Done.

Note: I'm using a lacklustre GbE dongle on my laptop, a TP-Link UE300 (which is better than jack squat, because it's supported in Linux whatsoever), and the fastest I've ever been able to get it to go is 40MB/sec, sustained (when connected do a different server, not this NanoPi). This dongle should be fast enough for my tests, as I've never hit that 40 MB/sec (and topped out).

Edited by esbeeb
stated GbE dongle type

Share this post


Link to post
Share on other sites

OK, I re-ran my Nextcloud tests, shown above, using all-Ethernet. 

 

When uploading the 2GB .zip file, the speed jumped from 10.9 MB/sec (with wifi-from-laptop) up to 17.2 MB/sec (all-ethernet), average.  That's considerably closer to my FTP test (which I'm almost certain I did as all-Ethernet), which got me 29.9 MB/sec@Igor, thanks so much for the great tip.

 

When uploading the 2GB folder of many tiny files, the speed very slightly improved from 1.0 MB/sec (with wifi-from-laptop) up to 1.1 MB/sec (all-ethernet), average.  That's still a far cry from the FTP test (which I'm almost certain I did as all-Ethernet), which got me 18.7 MB/sec:

 

ul2.thumb.png.c8230870be55158dae7d43ad8fd1eaff.png

 

Note that I used BTRFS on the SSD being uploaded into, for both the FTP test, and the Nextcloud test.

 

 

Share this post


Link to post
Share on other sites

I wanted to get some speed tests over SMB (ver 3.11, the default in OMV) protocol, using OpenMediaVault.  I used the pre-built image for this board available here The quick summary is that FTP is way faster than SMB, like 50ish % faster.

 

Here were some Samba speeds, uploading the same 2GB .zip file, then 2GB worth of tiny files, to the 2.5" Samsung SSD:

  • Over all-GbE (as in, no wifi between my laptop and the wifi router), 18.3 MB/sec for the 2GB .zip file (would have been 29.9 MB/sec over FTP), and 11.9 MB/sec for the 2GB of tiny files (would have been 18.7 MB/sec over FTP).
  • Over wifi (as in, there was wifi between my laptop and the wifi router), 14.0 MB/sec for the 2GB .zip file, and 7.8 MB/sec for the 2GB of tiny files.

Here were some Samba speeds, uploading the same 2GB .zip file, then 2GB worth of tiny files, to an external 4TB 3.5" Hitachi hard drive, in a USB 3.0 enclosure which surely has a JMS 578 controller chip (connected to the USB port on the mainboard of the NanoPi, not the USB port on the NAS accesory board):

  • Over all-GbE (as in, no wifi between my laptop and the wifi router), 18.7 MB/sec for the 2GB .zip file, and 12.1 MB/sec for the 2GB of tiny files.
  • Over wifi (as in, there was wifi between my laptop and the wifi router), 13.4 MB/sec for the 2GB .zip file, and 8.0 MB/sec for the 2GB of tiny files.

 

So a plain old 3.5" spinning hard drive, in a good USB enclosure, with a JMicron 578 controller chip, slightly beats an SSD!!

 

DSCF8312.JPG

Share this post


Link to post
Share on other sites

I did an FTP upload test, using all-GbE (no wifi), comparing how the 4TB 3.5" Hitachi (in the JMS 578 enclosure) would compare to the SSD.  The 3.5" Hitachi won, slightly: 32.8 MB/sec for the 2GB .zip file (compare to the SSD, which got 29.9 MB/sec), and 18.7 MB/sec for the 2GB worth of tiny files (compare to the SSD, which got the very same 18.7 MB/sec).

 

Note: SSL was used for FTP.  So FTP also kicks SMB's butt for speed, even when used with SSL.  OpenMediaVault made it easy to generate and then use a self-signed SSL cert (for use within my LAN).

Share this post


Link to post
Share on other sites

I hooked a second external 3.5" Hitachi up to the additional USB port provided by the "NAS kit" add-on board.  This one is a 6TB.  I was wanting to set this up as a backupdrive of my other 2 drives, and serve it out read-only over the network with FTP and SMB. When I formatted it with BTRFS, I got error messages about failed checksums when I tried to FTP my sample 2GB of tiny files out of it!  Kernel error messages began appearing during the FTP, such as the following:

 

Dec 11 15:47:43 filesrv1 kernel: [ 1564.078380] BTRFS warning (device sdc1): csum failed root 257 ino 967 off 41418752 csum 0x394f3c35 expected csum 0x9fc46317 mirror 1
Dec 11 15:47:43 filesrv1 kernel: [ 1564.152892] BTRFS warning (device sdc1): csum failed root 257 ino 967 off 41746432 csum 0xf2ebac76 expected csum 0xa8e57e86 mirror 1
Dec 11 15:47:43 filesrv1 kernel: [ 1564.180123] BTRFS warning (device sdc1): csum failed root 257 ino 967 off 42795008 csum 0x9b4b400b expected csum 0x3b97019a mirror 1
Dec 11 15:47:43 filesrv1 kernel: [ 1564.277908] BTRFS warning (device sdc1): csum failed root 257 ino 967 off 41418752 csum 0x394f3c35 expected csum 0x9fc46317 mirror 1

So I reformatted the drive as EXT4 instead, then tried the same test again.  No error messages that time.  Strange...  BTW: both BTRFS and EXT4 performed the same, when copying data into the drive, and the FTP download speed I achieved (once formatted with EXT4), was 37 MB/sec.

 

Edit: I changed the USB 3.0 cable.  I'm pretty sure the cable that came with the enclosure was flawed.  No more error messages now, and the blue light on the enclosure no longer flashes in a warning-like way, when there's no disk activity (and I observed this warning-like flashing, as well as other kernel error messages, when I used a too-long 3-meter-long USB cable previously).

Edited by esbeeb
changed cable, and Oops, not SMB1, but SMB v3.11 (default in OMV)

Share this post


Link to post
Share on other sites

The NAS-case has an JMS567 - but this USB 3.0 SATA controller is internally connected via USB 2.0 to the Neo2.

 

How did you connect the Neo Core 2 LTS to the NAS-Case for the Neo/Neo2 - or do you mean the NanoPi Neo2 LTS

 

I couldnt find a information that the Neo Core 2 LTS will work with the NAS-Case

https://www.friendlyarm.com/index.php?route=product/product&amp;product_id=222

Share this post


Link to post
Share on other sites

Aargh.  Even after changing the USB cable, on the drive connected to the USB port of the NAS kit board, I'm getting UAS-related kernel error messages, when I download many small files over SMB:

Dec 12 14:00:48 filesrv1 kernel: [ 5957.276611] sd 2:0:0:0: [sdb] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: IN 
Dec 12 14:00:48 filesrv1 kernel: [ 5957.276630] sd 2:0:0:0: [sdb] tag#1 CDB: opcode=0x88 88 00 00 00 00 00 00 11 b1 10 00 00 03 00 00 00
Dec 12 14:00:48 filesrv1 kernel: [ 5957.287599] scsi host2: uas_eh_device_reset_handler start
Dec 12 14:00:48 filesrv1 kernel: [ 5957.401615] usb 3-1: reset high-speed USB device number 2 using ehci-platform
Dec 12 14:00:48 filesrv1 kernel: [ 5957.608831] scsi host2: uas_eh_device_reset_handler success
Dec 12 14:01:21 filesrv1 kernel: [ 5990.045561] sd 2:0:0:0: [sdb] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: IN 
Dec 12 14:01:21 filesrv1 kernel: [ 5990.045580] sd 2:0:0:0: [sdb] tag#1 CDB: opcode=0x88 88 00 00 00 00 00 00 11 84 c8 00 00 03 c0 00 00
Dec 12 14:01:21 filesrv1 kernel: [ 5990.056539] scsi host2: uas_eh_device_reset_handler start
Dec 12 14:01:21 filesrv1 kernel: [ 5990.170553] usb 3-1: reset high-speed USB device number 2 using ehci-platform
Dec 12 14:01:21 filesrv1 kernel: [ 5990.338706] scsi host2: uas_eh_device_reset_handler success
Dec 12 14:01:55 filesrv1 kernel: [ 6024.350540] sd 2:0:0:0: [sdb] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: IN 
Dec 12 14:01:55 filesrv1 kernel: [ 6024.350559] sd 2:0:0:0: [sdb] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 92 88 ee b0 00 00 00 88 00 00
Dec 12 14:01:55 filesrv1 kernel: [ 6024.361523] scsi host2: uas_eh_device_reset_handler start
Dec 12 14:01:55 filesrv1 kernel: [ 6024.475536] usb 3-1: reset high-speed USB device number 2 using ehci-platform
Dec 12 14:01:55 filesrv1 kernel: [ 6024.604705] scsi host2: uas_eh_device_reset_handler success

 

Share this post


Link to post
Share on other sites

Here's what I think the problem is.  The USB port on the NanoPi Neo 2 mainboard itself works well with the default UAS driver from the kernel.  But the second USB port, which seems to be an afterthought on the NAS kit accessory board, does NOT work well with the UAS driver from the kernel.  Why would I conclude this?  Because when I put the "problematic" (seeming) drive onto the other USB port (on the mainboard of the NanoPi) the problem went away.

 

Edit: I found the fix I needed, detailed here.

Share this post


Link to post
Share on other sites

Just to clarify.  Drives connected to both the USB port on the NanoPi Neo 2, and the SATA port of the NAS kit work well.  It's just a third drive hooked up to the USB port on the NAS kit accessory board where things get sketchy, for that that third drive only.

 

Hindsight being 20/20, if you want to have a NAS with a backup disk, you would do best to get a NON-SSD 2.5" laptop hard drive (with the longest warranty you can find, like 3yr., and paying a little extra is worth it) to hook up to the NAS kit SATA port (where you'll get, say 4TB max these days), then hook up a 4ishTB 3.5" hard drive (in a good enclosure, like the one I linked to above, with a JMS578), to the USB port on the NanoPi Neo 2 mainboard, NOT the USB port on the NAS kit accessory board.  Don't listen to advice about the importance of using an SSD (which is of course going to be way smaller in capacity due to far higher price-per-TB) to get the best random IO speed over UAS.  It turns out to be no faster, even on tiny files, when there is only one network transfer at a timeFor this board, which only has USB 2, (not USB 3) that SSD-related advice is a red herring (when it comes to speed).

Share this post


Link to post
Share on other sites
3 hours ago, esbeeb said:

Here's what I bought:

https://www.friendlyarm.com/index.php?route=product/product&amp;path=69&amp;product_id=211

I tell you solemnly, there's only one way to connect it to the Nas Kit board.

 

OK, the pinout of NAS-case seems also to be useable with the Core2 (but FrriendlyARM doenst wrote it on their page), but it seems that you
- with that configuration - not directly could use ethernet (there isnt a PHY) nor the USB-Port 3 nor I2C2 nor the additionally SPI1.

Did you additionally install a PHY (Ethernet port) on your Core2?

 

Maybe - if you install a header (or did you already) - you could USB-Port 3 for better results?

PinOut_Neo2_Core2.jpg

Share this post


Link to post
Share on other sites

Thanks for that diagram.  Oops, actually, I have the NanoPi Neo 2, not the NanoPI Neo Core 2 (and I've changed the title of this topic accordingly).  And I downloaded the wrong thing (although it booted with no complaints!).  I'll try again...

 

BTW: I've been trying the OMV pre-built image (correctly for the NanoPi Neo 2, not Core 2), from here by tkaiser, where I had these glitches.  In an attempt to get that USB port on the NAS kit board working more reliably, my plan now is to use the stock Armbian for the NanoPi Neo 2, then use the "softy" menu to install OMV.  That way I'll get a newer kernel (which maybe will use that 2nd USB port, on the NAS kit board, more reliably).  This will let me jump from an older kernel to a newer one:

 

Current OMV 4.1 image pre-built by tkaiser:
Linux hostname 4.14.70-sunxi64 #274 SMP Wed Sep 19 12:09:30 CEST 2018 aarch64 GNU/Linux

 

Current stock Armbian:
Linux hostname 4.14.78-sunxi64 #416 SMP Fri Oct 26 15:54:03 CEST 2018 aarch64 GNU/Linux

 

I found the USB-related hardware problem, detailed here.

Edited by esbeeb
title changed. my fix mentioned

Share this post


Link to post
Share on other sites
On 12/5/2018 at 5:50 PM, esbeeb said:

Although these enclosures have a good controller chip, the USB3.0 cables provided suck.  I would get UAS-related "inflight" error messages as seen above, when using them, for any prolonged period of time. 

 

These error messages went away when using plain old USB 2.0 USB-A-to-USB-B male-male cables (as in, USB "printer cables"), which had the same overall thickness of wire when you compared them side-to-side to the junky USB 3.0 cables, but the USB 2.0 cables have like half the metal wires inside them, which were probably much thicker guage than the junky USB 3.0 cables.

 

Boo, Orico!  Mr. Scissors was not impressed:

 

IMG_20181221_055721.thumb.jpg.4457ea8c4012ae65ddd60dc229f1d0b4.jpg

 

We also bought a couple of USB 3.0 cables on AliExpress (don't ask which ones, one was 1.5m, one was 3m), and they too similarly sucked.  If Mr. Scissors could speak, he would say, "Off with their heads!":

 

IMG_20181221_055739.thumb.jpg.283943d9df0a8e0bc5e3ef6d93b999e6.jpg

 

PS: the reference to Mr. Scissors was a little shout out to Chris Barnatt of explainingcomputers.com.  :)

Share this post


Link to post
Share on other sites

Here's a picture of the NAS I've set up for our small office here:

 

 

IMG_20181219_171037.jpg

 

Inside the NAS kit aluminum case on the top is a 500GB SSD, and there are two more 3.5" drives on the bottom, in enclosures, attached with USB 2.0 cables.  OpenMediaVault uses all 3 drives.

Share this post


Link to post
Share on other sites

As I rsync'd data around between the drives (all formatted with BTRFS), while doing no other disk operations to speak of, I would at first get 40MB/sec (shown with iotop), then after about 10 seconds, it would simmer down to 20MB/sec and stay there.  So you can expect a solid 20MB/sec through those USB 2.0 cables (and the USB 3.0 cables were never any faster, even when no "UAS inflight" errors were getting generated), with quick initial surges of 40ish MB/sec.  These speeds (when doing just one data transfer at a time) held true wether or not you wrote to SATA or a USB disk, and wether or not you used a spinning 3.5" drive, or an SSD.

 

So in conclusion, IMHO, this rig was born to run OpenMediaVault (in a small office of a few people, where it's usually/often the case that only one user at a time is hitting this server), but not Nextcloud! 

 

(Edit: ...granted you only hook up one 2.5" drive to the SATA port, as I could never manage to get rock-solid stably-working drives connected over USB.)

 

And those users should probably use Filezilla to transfer files (not Windows Explorer, ie. SMB3), if they are transferring any more than about 300MB (at which time I say it's worth one's time to spend the 30 seconds or so starting Filezilla, and logging in, and navigating to the source/destination folders.)  Furthermore, you get a very noticeable boost in file transfer speeds, if you are willing to use a Gigabit Ethernet dongle on your laptop, even a cheap one like I bought (see above), not wifi.

Share this post


Link to post
Share on other sites

After @tkaiserexplained that GVFS (used by Linux Mint's default File Browser, Nemo) isn't as fast as it could possibly be these days, for transferring files over SMB, I wondered if KDE's way of transferring files over SMB (called KIO) might be any faster.

 

So I did a comparison of uploading my same folder of 2GB of ebooks, containing many tiny files, (same as used above) using 3 different methods in Linux Mint:

- Nemo 3.8.6/GVFS

- KDE Dolphin File Browser (17.12.3)/KIO

- SMB4K (2.1.0), a userspace SMB share mounting utility/KIO

 

All 3 methods used the SMB3 protocol, over the very same GbE connection.

 

Nemo kicked the butts of the two KIO-based methods: 1:26 (minutes:seconds) = 23 MB/sec average

Dolphin: 1:56 = 17.2MB/sec average, 75% as fast as GVFS.

SMB4K: 2:27 = 13.6MB/sec average, 59% as fast as GVFS.
 

 

Share this post


Link to post
Share on other sites

Word to the wise: use brand-new USB 2.0 A-to-B cables (printer cables) within those cylindrical inline magnets on the wire (which clean up electrical interference).  I had "UAS inflight" errors (and checksums were failing on BTRFS scrubs) until I changed these USB cables accordingly.

 

In the future, if I ever make any other NAS, I'll never use USB to attach any "permanent" drive (which will have either the OS on it, or have a file share served from it).  USB might be OK for some temporarily-attached drive or stick, where a one-off backup is made (say, for making periodic, "off-site" backups).

 

The slightly higher cost to have proper SATA connections is so well worth it, I say (even if on an 8-year old, used PC, that's still in good running condition.)  Cheaping out, and depriving yourself of enough SATA ports is a very bad idea, to save a small amount of money, IMHO.

Share this post


Link to post
Share on other sites

I've been suspicious that my TP-Link UE300 GbE dongle is sort of slow.  I've got a new GbE dongle, to compare with, an Anker AH212. loading 2GB THe Anker goes a bit faster than the TP-Link, for FTP and SMB, but not by much:

 

Uploading 2GB of tiny ebooks (2029 files in 756 folders):

  - TP-Link:

    - FTP: 22.3 MB/sec.

    - SMB: 12.0 MB/sec

  - Anker:

    - FTP: 25.0 MB/sec

    - SMB: 14.6 MB/sec

 

So basically, all my tests above are quite trustworthy, I think.  With a more carefully chosen GbE dongle, you only get a couple more MB/sec.

Share this post


Link to post
Share on other sites

Am I ever glad I used BTRFS on all the file shares.  I have "snapper" set up nicely to do automated snapshots on all 3, and combined, these proved to be invaluable in recovering from the less-than-excellent USB cables I used.

 

Let me once again stress that you absolutely cannot afford to cut corners on the USB 2.0 cables you choose.  They must be brand spanking new, and have the inline cylindrical magnets to cleanse the signal.

 

Also, I implore anyone using Armbian to make periodic backups of the MicroSD card (or wherever you install the OS), with, say, "dd", or some other disk imaging tool.  I had an " apt-get update && apt-get upgrade" blow up in my face badly recently: I couldn't log into the web management interface of OMV, nor could I even log in with SSH any more! 

 

Not being able to log in with SSH (which is the normal way I would rescue such a problem) left me shocked and horrified, as there is no video port whatsoever on this rig to log in at the console.  I have no equipment for a serial connection either.  The backup SD card I had saved my bacon.

 

I'm now losing my interest in using this ARM board for OMV.  There have been just too many surprises and pitfalls, try as I might to do my best, and I only have enough spare time to slay just a few of the smaller dragons. 

 

I plan to use OMV in a PC instead (where I would have a video port for rescuing), and use this NanoPi just for SyncThing, which is less ambitious (than OMV), yet still useful to my organization.  By using OMV in a PC, I'll also be able to install some Docker containers that my organization needs, this "sweetens the pot" enough for me to want to rebuild the NAS on that equipment.

 

I hereby retract my bold statement above, that "This rig was born to run Openmediavault".  I hope my small contributions above save others time and stress.

 

Share this post


Link to post
Share on other sites

I kept getting disk IO problems on my 2 external USB drives.  I could never narrow down just what to blame: the USB cables (which I changed like 5 or 6 times), the USB enclosures I bought, or the two USB ports on the NanoPi itself.  Even though these enclosures have JMS578 controller chips in them, and those are pretty ideal controller chips, I don't really trust the other ORICO components around those chips. 

 

I wouldn't recommend buying these ORICO 7688U3 enclosures ever again, even if just for the reason that they secure the drives poorly (which is why I had to hack together a nylon strap to prevent the drives from vibrating inside the enclosures).  There's no "rails" on either side of the drives while they are in the enclosure.  The only thing stopping the drives from wobbling side to side in the enclosures is the SATA port being connected itself, plus there is a little block of rubbery foam that the front of the drive pushes up against (and that only holds in the center, not at the corners).

 

The SATA-connected SSD drive (connected to the NanoPi's "NAS kit" accessory board) always behaved nicely.  No disk IO errors.  So therefore, I suggest using the NanoPi Neo2 only for a project where you need 1 disk, and that 1 disk should be a 2.5" SATA attached to the NanoPi's "NAS kit" accessory board.  Beware the USB ports, and you would be well advised that you are likely to run into Disk IO errors if you still dare.  Watch your /var/log/syslog like a hawk, and maybe you'll see disk IO errors appearing there.

Share this post


Link to post
Share on other sites

So I built a new OpenMediaVault server from a new PC, and used the same 3 SATA disks mentioned above (two Toshiba 3.5", and one Samsung 2.5" SSD). This PC has an Asus mobo (Asus Tuf B450 Gaming) with GbE and a sweet B450 disk controller.  The two 3.5" SATA drives now get about 200 MB/sec with hdparm, and the 2.5" SSD gets about 500 MB/sec, which of course is way faster than when these were hooked up to the NanoPi Neo2.

 

Here are the results of the network transfer speed tests which mattered to me most, for sake of comparison:

 

Using two different USB 3.0 GbE dongles on my laptop to upload (an Anker AH212, and a smaller, cheaper and slightly slower TP-Link UE300), I uploaded 2 GB of eBooks into this new Asus PC OMV server in different ways:

 

Anker AH212:

  • SMB upload of a 2GB .zip file: 54.1 MB/sec
  • SMB upload of a 2GB folder with many tiny files (2029) in many folders: 24.7 MB/sec
  • FTP upload of a 2GB .zip file: 105 MB/sec
  • FTP upload of a 2GB folder with many tiny files (2029) in many folders: 40.8 MB/sec

 

TP-Link UE300:

  • SMB upload of a 2GB .zip file: 47.6 MB/sec
  • SMB upload of a 2GB folder with many tiny files (2029) in many folders: 19.2 MB/sec
  • FTP upload of a 2GB .zip file: 105 MB/sec
  • FTP upload of a 2GB folder with many tiny files (2029) in many folders: 37.0 MB/sec

 

Here are some of those test results again (gathering from my results above), when I uploaded using the same GbE dongles into the NanoPi Neo2, into the same disks (and let's set aside the disk IO error problems I had with the two 3.5" SATA drives in crappy Orico USB enclosures):

 

Anker AH212:

  • SMB upload of a 2GB folder with many tiny files (2029) in many folders: 14.6 MB/sec
  • FTP upload of a 2GB folder with many tiny files (2029) in many folders: 25.0 MB/sec

 

TP-Link UE300:

  • SMB upload of a 2GB folder with many tiny files (2029) in many folders: 12.0 MB/sec
  • FTP upload of a 2GB .zip file: 29.9 MB/sec
  • FTP upload of a 2GB folder with many tiny files (2029) in many folders: 22.3 MB/sec

 

Share this post


Link to post
Share on other sites

Dear moderator, could this topic be moved to the H5 subforum?  It's misfiled under the H2 & H3 subforum.  Thanks in advance.

@lanefu, @Igor

Sorry, I don't know who the other mods are.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
0