esbeeb Posted October 24, 2018 Posted October 24, 2018 Hello, There seems to be scant details about the NanoPi Neo 2 LTS, so I thought I'd contribute some. My testing is much less formal than how some would measure it. What I especially like to do in the way of network speed testing are the "worst-case scenarios". My way of invoking a sort of "torture test" is to mostly test upload speeds (as they tend to be slower than download speeds), both large files (much easier), as well as uploading many tiny files (much more gruelling to store, so is therefore a major "torture test"). Having said that, I'll usually favor all-GbE networking (noting which I'm using, GbE or wifi), because I know GbE will be quite a bit faster than WiFi. FWIW, here are my findings. - I got the "1-bay NAS Dock v1.2 for NanoPi NEO/NEO2", allowing me to hook up a 2.5" SSD, I got a Samsung 860 EVO (500GB). For the OS, I used the current Armbian Stretch. Once booted up, and logged in over ssh, I created a btrfs partition on the Samsung EVO, then mounted it, then installed "pure-ftpd". When I did an upload test to the Samsung SSD entirely through Gigabit ethernet (using a Gigabit ethernet dongle on my laptop, which I know can do 40MB/sec easily), here's what I found: - When you upload larger files, like say a 2 GB .zip file, the average FTP upload speed was 29.9 MB/sec - When you upload a whole bunch of smaller files, like 2 GB worth (2029 files), the average FTP upload speed was 18.7 MB/sec Note: I'm using a lacklustre GbE dongle on my laptop, a TP-Link UE300 (which is better than jack squat, because it's supported in Linux whatsoever), and the fastest I've ever been able to get it to go is 105MB/sec, sustained (when connected do a different server, not this NanoPi). This dongle is definately fast enough for my tests.
esbeeb Posted October 25, 2018 Author Posted October 25, 2018 I'd like to say a big Thanks for the nice "armbian-config" utility, especially the "Softy" submenu (where I'm delighted to find a Nextcloud installer!). I'm using Armbian for the first time, and I'm learning the ropes (having said that, I have lots of Debian sysadmin experience). I also love how the root filesystem is ext4, not ext2! Yay, journalling.
esbeeb Posted October 25, 2018 Author Posted October 25, 2018 Here is diagnostic info, although I have no problems on this board. But perhaps juicy things can be seen there none the less. http://ix.io/1pYF
esbeeb Posted October 25, 2018 Author Posted October 25, 2018 Here's some iperf3 output (Edit: ...and this was was when my laptop was using WiFi): Connecting to host nanopi, port 5201 [ 4] local 192.168.0.129 port 48934 connected to 192.168.0.137 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 31.2 MBytes 261 Mbits/sec 47 313 KBytes [ 4] 1.00-2.00 sec 33.7 MBytes 283 Mbits/sec 0 376 KBytes [ 4] 2.00-3.00 sec 35.0 MBytes 293 Mbits/sec 32 317 KBytes [ 4] 3.00-4.00 sec 35.3 MBytes 296 Mbits/sec 26 272 KBytes [ 4] 4.00-5.00 sec 35.2 MBytes 295 Mbits/sec 0 346 KBytes [ 4] 5.00-6.00 sec 34.6 MBytes 290 Mbits/sec 10 315 KBytes [ 4] 6.00-7.00 sec 35.2 MBytes 295 Mbits/sec 26 269 KBytes [ 4] 7.00-8.00 sec 32.6 MBytes 273 Mbits/sec 2 283 KBytes [ 4] 8.00-9.00 sec 34.1 MBytes 286 Mbits/sec 0 354 KBytes [ 4] 9.00-10.00 sec 34.8 MBytes 292 Mbits/sec 23 328 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 342 MBytes 287 Mbits/sec 166 sender [ 4] 0.00-10.00 sec 340 MBytes 285 Mbits/sec receiver iperf Done.
esbeeb Posted November 18, 2018 Author Posted November 18, 2018 I installed Nextcloud 14 on my NanoPi Neo 2 LTS, with the Nas kit, and a 2.5" SATA Samsung SSD (and I used "ncp-config" to move the Nextcloud database, and datadir onto that SSD, not the MicroSD card). I was curious to see the performance on copying files into Nextcloud (over SSL-encrypted WebDAV, using my Linux Mint’s Nemo File Browser). For testing, I had a 2GB .zip file, which uploaded in 3:03, or 10.9 MB/sec average (Note: If that same 2GB file were uploading through FTP into the very same hardware, it would instead get 29.9 MB/sec, see above). The NanoPi's 4 cores all worked about 15% average, each, and the load average stayed at about 1.3 the whole transfer. The CPU was about 45 deg C max, in a room which had an ambient temp. of about 30 deg C. I also had a folder containing 2GB of many very small files (2,785 files of varying sizes). Copying the folder in took 34:11, or 1.0 MB/sec average (Note: If that same 2GB folder were uploading through FTP into the very same hardware, it would instead get 18.7 MB/sec, see above). The NanoPi's 4 cores all worked about 25% average, each, and the load average peaked out at a max of about 1.5, but was usually like 1.2 the whole transfer. The CPU slowly climbed up to 62 deg C, in a room which had an ambient temp. of about 30 deg C. Pictured just below: on the left is Nemo file Browser copying in the 2GB folder over WebDAV, and on the right, you see an htop running on the NanoPi, running Nextcloud 14: I would suggest that the NanoPi Neo 2 LTS, with the NAS kit and an SSD is sort of like a bare minimum for an office with like 1 or 2 people in it, who are just storing documents, etc. IMHO, You are just sort of torturing yourself far too much, just to save a very small amount of money, if you go any lower than the NanoPi Neo 2 LTS (with an SSD).
Igor Posted November 18, 2018 Posted November 18, 2018 On 10/25/2018 at 10:35 AM, esbeeb said: Here's some iperf3 output There are unusual lots of retries, speed should be better. Now this can be a 3rd party hardware issues, your cabling or router ... I only have a board without dock and can't recreate. Try change cables and you could test new (stable to be) kernel - but I can't give you warranty that it works since we are still testing/debugging it: https://forum.armbian.com/topic/8795-next-lts-kernel-419y-allwinner-a10-a20-a64-h2-h3-h5-h6-debugging-party/
esbeeb Posted November 18, 2018 Author Posted November 18, 2018 (edited) 1 hour ago, Igor said: There are unusual lots of retries, speed should be better. Now this can be a 3rd party hardware issues, your cabling or router ... I only have a board without dock and can't recreate. Try change cables and you could test new (stable to be) kernel - but I can't give you warranty that it works since we are still testing/debugging it: https://forum.armbian.com/topic/8795-next-lts-kernel-419y-allwinner-a10-a20-a64-h2-h3-h5-h6-debugging-party/ I changed the ethernet cable to a brand new, store-bought cable. But the iperf3 output didn't really improve: root@chicklet:~# iperf3 -c nextcloudpi Connecting to host nextcloudpi, port 5201 [ 4] local 192.168.0.129 port 44196 connected to 192.168.0.137 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 36.2 MBytes 303 Mbits/sec 69 317 KBytes [ 4] 1.00-2.00 sec 32.3 MBytes 271 Mbits/sec 7 288 KBytes [ 4] 2.00-3.00 sec 32.2 MBytes 270 Mbits/sec 9 272 KBytes [ 4] 3.00-4.00 sec 32.5 MBytes 273 Mbits/sec 2 264 KBytes [ 4] 4.00-5.00 sec 32.3 MBytes 270 Mbits/sec 1 255 KBytes [ 4] 5.00-6.00 sec 32.6 MBytes 274 Mbits/sec 0 338 KBytes [ 4] 6.00-7.00 sec 32.0 MBytes 268 Mbits/sec 4 259 KBytes [ 4] 7.00-8.00 sec 32.4 MBytes 272 Mbits/sec 1 255 KBytes [ 4] 8.00-9.00 sec 32.1 MBytes 269 Mbits/sec 2 243 KBytes [ 4] 9.00-10.00 sec 30.3 MBytes 254 Mbits/sec 13 260 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 325 MBytes 272 Mbits/sec 108 sender [ 4] 0.00-10.00 sec 322 MBytes 270 Mbits/sec receiver iperf Done. Furthermore, when I do an "ifconfig" both on my client and server machines (for iperf3), neither shows any dropped packets (Note the "errors 0 dropped 0" for both RX and TX in all cases): root@chicklet:~# /sbin/ifconfig lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 371 bytes 32291 (32.2 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 371 bytes 32291 (32.2 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 wlp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.0.129 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 fe80::9ec0:39e5:56a0:9721 prefixlen 64 scopeid 0x20<link> ether f8:bb:aa:zz:yy:xx txqueuelen 1000 (Ethernet) RX packets 198902 bytes 18974787 (18.9 MB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 458133 bytes 701689760 (701.6 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 root@nextcloudpi:~# ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.0.137 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 fe80::dcb7:78de:d32f:64c4 prefixlen 64 scopeid 0x20<link> ether 02:bb:aa:zz:yy:xx txqueuelen 1000 (Ethernet) RX packets 454893 bytes 688170496 (656.2 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 193446 bytes 12843866 (12.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 24 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 Note that my client laptop, "chicklet", is connected over wifi to the router. So there's only one ethernet cable to possibly blame, here. Edited November 18, 2018 by esbeeb obscured MAC addresses.
esbeeb Posted November 18, 2018 Author Posted November 18, 2018 (edited) When I go all-Ethernet (and don't have wifi between my laptop, chicklet, and the wifi router), then iperf3 gets much better: root@chicklet:~# iperf3 -c nextcloudpi Connecting to host nextcloudpi, port 5201 [ 4] local 192.168.0.115 port 33706 connected to 192.168.0.137 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 114 MBytes 952 Mbits/sec 0 307 KBytes [ 4] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 0 307 KBytes [ 4] 2.00-3.00 sec 112 MBytes 942 Mbits/sec 0 324 KBytes [ 4] 3.00-4.00 sec 112 MBytes 939 Mbits/sec 0 348 KBytes [ 4] 4.00-5.00 sec 112 MBytes 942 Mbits/sec 0 348 KBytes [ 4] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 0 348 KBytes [ 4] 6.00-7.00 sec 112 MBytes 942 Mbits/sec 0 348 KBytes [ 4] 7.00-8.00 sec 112 MBytes 940 Mbits/sec 0 348 KBytes [ 4] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 0 348 KBytes [ 4] 9.00-10.00 sec 112 MBytes 942 Mbits/sec 0 348 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 1.10 GBytes 942 Mbits/sec 0 sender [ 4] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver iperf Done. Note: I'm using a lacklustre GbE dongle on my laptop, a TP-Link UE300 (which is better than jack squat, because it's supported in Linux whatsoever), and the fastest I've ever been able to get it to go is 40MB/sec, sustained (when connected do a different server, not this NanoPi). This dongle should be fast enough for my tests, as I've never hit that 40 MB/sec (and topped out). Edited December 13, 2018 by esbeeb stated GbE dongle type
esbeeb Posted November 18, 2018 Author Posted November 18, 2018 OK, I re-ran my Nextcloud tests, shown above, using all-Ethernet. When uploading the 2GB .zip file, the speed jumped from 10.9 MB/sec (with wifi-from-laptop) up to 17.2 MB/sec (all-ethernet), average. That's considerably closer to my FTP test (which I'm almost certain I did as all-Ethernet), which got me 29.9 MB/sec. @Igor, thanks so much for the great tip. When uploading the 2GB folder of many tiny files, the speed very slightly improved from 1.0 MB/sec (with wifi-from-laptop) up to 1.1 MB/sec (all-ethernet), average. That's still a far cry from the FTP test (which I'm almost certain I did as all-Ethernet), which got me 18.7 MB/sec: Note that I used BTRFS on the SSD being uploaded into, for both the FTP test, and the Nextcloud test.
esbeeb Posted December 5, 2018 Author Posted December 5, 2018 I wanted to get some speed tests over SMB (ver 3.11, the default in OMV) protocol, using OpenMediaVault. I used the pre-built image for this board available here. The quick summary is that FTP is way faster than SMB, like 50ish % faster. Here were some Samba speeds, uploading the same 2GB .zip file, then 2GB worth of tiny files, to the 2.5" Samsung SSD: Over all-GbE (as in, no wifi between my laptop and the wifi router), 18.3 MB/sec for the 2GB .zip file (would have been 29.9 MB/sec over FTP), and 11.9 MB/sec for the 2GB of tiny files (would have been 18.7 MB/sec over FTP). Over wifi (as in, there was wifi between my laptop and the wifi router), 14.0 MB/sec for the 2GB .zip file, and 7.8 MB/sec for the 2GB of tiny files. Here were some Samba speeds, uploading the same 2GB .zip file, then 2GB worth of tiny files, to an external 4TB 3.5" Hitachi hard drive, in a USB 3.0 enclosure which surely has a JMS 578 controller chip (connected to the USB port on the mainboard of the NanoPi, not the USB port on the NAS accesory board): Over all-GbE (as in, no wifi between my laptop and the wifi router), 18.7 MB/sec for the 2GB .zip file, and 12.1 MB/sec for the 2GB of tiny files. Over wifi (as in, there was wifi between my laptop and the wifi router), 13.4 MB/sec for the 2GB .zip file, and 8.0 MB/sec for the 2GB of tiny files. So a plain old 3.5" spinning hard drive, in a good USB enclosure, with a JMicron 578 controller chip, slightly beats an SSD!!
esbeeb Posted December 5, 2018 Author Posted December 5, 2018 I did an FTP upload test, using all-GbE (no wifi), comparing how the 4TB 3.5" Hitachi (in the JMS 578 enclosure) would compare to the SSD. The 3.5" Hitachi won, slightly: 32.8 MB/sec for the 2GB .zip file (compare to the SSD, which got 29.9 MB/sec), and 18.7 MB/sec for the 2GB worth of tiny files (compare to the SSD, which got the very same 18.7 MB/sec). Note: SSL was used for FTP. So FTP also kicks SMB's butt for speed, even when used with SSL. OpenMediaVault made it easy to generate and then use a self-signed SSL cert (for use within my LAN).
esbeeb Posted December 11, 2018 Author Posted December 11, 2018 (edited) I hooked a second external 3.5" Hitachi up to the additional USB port provided by the "NAS kit" add-on board. This one is a 6TB. I was wanting to set this up as a backupdrive of my other 2 drives, and serve it out read-only over the network with FTP and SMB. When I formatted it with BTRFS, I got error messages about failed checksums when I tried to FTP my sample 2GB of tiny files out of it! Kernel error messages began appearing during the FTP, such as the following: Dec 11 15:47:43 filesrv1 kernel: [ 1564.078380] BTRFS warning (device sdc1): csum failed root 257 ino 967 off 41418752 csum 0x394f3c35 expected csum 0x9fc46317 mirror 1 Dec 11 15:47:43 filesrv1 kernel: [ 1564.152892] BTRFS warning (device sdc1): csum failed root 257 ino 967 off 41746432 csum 0xf2ebac76 expected csum 0xa8e57e86 mirror 1 Dec 11 15:47:43 filesrv1 kernel: [ 1564.180123] BTRFS warning (device sdc1): csum failed root 257 ino 967 off 42795008 csum 0x9b4b400b expected csum 0x3b97019a mirror 1 Dec 11 15:47:43 filesrv1 kernel: [ 1564.277908] BTRFS warning (device sdc1): csum failed root 257 ino 967 off 41418752 csum 0x394f3c35 expected csum 0x9fc46317 mirror 1 So I reformatted the drive as EXT4 instead, then tried the same test again. No error messages that time. Strange... BTW: both BTRFS and EXT4 performed the same, when copying data into the drive, and the FTP download speed I achieved (once formatted with EXT4), was 37 MB/sec. Edit: I changed the USB 3.0 cable. I'm pretty sure the cable that came with the enclosure was flawed. No more error messages now, and the blue light on the enclosure no longer flashes in a warning-like way, when there's no disk activity (and I observed this warning-like flashing, as well as other kernel error messages, when I used a too-long 3-meter-long USB cable previously). Edited December 13, 2018 by esbeeb changed cable, and Oops, not SMB1, but SMB v3.11 (default in OMV)
guidol Posted December 11, 2018 Posted December 11, 2018 The NAS-case has an JMS567 - but this USB 3.0 SATA controller is internally connected via USB 2.0 to the Neo2. How did you connect the Neo Core 2 LTS to the NAS-Case for the Neo/Neo2 - or do you mean the NanoPi Neo2 LTS I couldnt find a information that the Neo Core 2 LTS will work with the NAS-Case https://www.friendlyarm.com/index.php?route=product/product&product_id=222
esbeeb Posted December 12, 2018 Author Posted December 12, 2018 (edited) On 12/11/2018 at 7:32 PM, guidol said: How did you connect the Neo Core 2 LTS to the NAS-Case Here's what I bought: https://www.friendlyarm.com/index.php?route=product/product&path=69&product_id=180 I tell you solemnly, there's only one way to connect it to the Nas Kit board. Edited December 13, 2018 by esbeeb Oops, Not Core 2
esbeeb Posted December 12, 2018 Author Posted December 12, 2018 Aargh. Even after changing the USB cable, on the drive connected to the USB port of the NAS kit board, I'm getting UAS-related kernel error messages, when I download many small files over SMB: Dec 12 14:00:48 filesrv1 kernel: [ 5957.276611] sd 2:0:0:0: [sdb] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: IN Dec 12 14:00:48 filesrv1 kernel: [ 5957.276630] sd 2:0:0:0: [sdb] tag#1 CDB: opcode=0x88 88 00 00 00 00 00 00 11 b1 10 00 00 03 00 00 00 Dec 12 14:00:48 filesrv1 kernel: [ 5957.287599] scsi host2: uas_eh_device_reset_handler start Dec 12 14:00:48 filesrv1 kernel: [ 5957.401615] usb 3-1: reset high-speed USB device number 2 using ehci-platform Dec 12 14:00:48 filesrv1 kernel: [ 5957.608831] scsi host2: uas_eh_device_reset_handler success Dec 12 14:01:21 filesrv1 kernel: [ 5990.045561] sd 2:0:0:0: [sdb] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: IN Dec 12 14:01:21 filesrv1 kernel: [ 5990.045580] sd 2:0:0:0: [sdb] tag#1 CDB: opcode=0x88 88 00 00 00 00 00 00 11 84 c8 00 00 03 c0 00 00 Dec 12 14:01:21 filesrv1 kernel: [ 5990.056539] scsi host2: uas_eh_device_reset_handler start Dec 12 14:01:21 filesrv1 kernel: [ 5990.170553] usb 3-1: reset high-speed USB device number 2 using ehci-platform Dec 12 14:01:21 filesrv1 kernel: [ 5990.338706] scsi host2: uas_eh_device_reset_handler success Dec 12 14:01:55 filesrv1 kernel: [ 6024.350540] sd 2:0:0:0: [sdb] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: IN Dec 12 14:01:55 filesrv1 kernel: [ 6024.350559] sd 2:0:0:0: [sdb] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 92 88 ee b0 00 00 00 88 00 00 Dec 12 14:01:55 filesrv1 kernel: [ 6024.361523] scsi host2: uas_eh_device_reset_handler start Dec 12 14:01:55 filesrv1 kernel: [ 6024.475536] usb 3-1: reset high-speed USB device number 2 using ehci-platform Dec 12 14:01:55 filesrv1 kernel: [ 6024.604705] scsi host2: uas_eh_device_reset_handler success
esbeeb Posted December 12, 2018 Author Posted December 12, 2018 Here's what I think the problem is. The USB port on the NanoPi Neo 2 mainboard itself works well with the default UAS driver from the kernel. But the second USB port, which seems to be an afterthought on the NAS kit accessory board, does NOT work well with the UAS driver from the kernel. Why would I conclude this? Because when I put the "problematic" (seeming) drive onto the other USB port (on the mainboard of the NanoPi) the problem went away. Edit: I found the fix I needed, detailed here. 1
esbeeb Posted December 12, 2018 Author Posted December 12, 2018 Just to clarify. Drives connected to both the USB port on the NanoPi Neo 2, and the SATA port of the NAS kit work well. It's just a third drive hooked up to the USB port on the NAS kit accessory board where things get sketchy, for that that third drive only. Hindsight being 20/20, if you want to have a NAS with a backup disk, you would do best to get a NON-SSD 2.5" laptop hard drive (with the longest warranty you can find, like 3yr., and paying a little extra is worth it) to hook up to the NAS kit SATA port (where you'll get, say 4TB max these days), then hook up a 4ishTB 3.5" hard drive (in a good enclosure, like the one I linked to above, with a JMS578), to the USB port on the NanoPi Neo 2 mainboard, NOT the USB port on the NAS kit accessory board. Don't listen to advice about the importance of using an SSD (which is of course going to be way smaller in capacity due to far higher price-per-TB) to get the best random IO speed over UAS. It turns out to be no faster, even on tiny files, when there is only one network transfer at a time. For this board, which only has USB 2, (not USB 3) that SSD-related advice is a red herring (when it comes to speed).
guidol Posted December 12, 2018 Posted December 12, 2018 3 hours ago, esbeeb said: Here's what I bought: https://www.friendlyarm.com/index.php?route=product/product&path=69&product_id=211 I tell you solemnly, there's only one way to connect it to the Nas Kit board. OK, the pinout of NAS-case seems also to be useable with the Core2 (but FrriendlyARM doenst wrote it on their page), but it seems that you - with that configuration - not directly could use ethernet (there isnt a PHY) nor the USB-Port 3 nor I2C2 nor the additionally SPI1. Did you additionally install a PHY (Ethernet port) on your Core2? Maybe - if you install a header (or did you already) - you could USB-Port 3 for better results?
guidol Posted December 12, 2018 Posted December 12, 2018 (edited) -deleted- Edited December 12, 2018 by guidol
esbeeb Posted December 13, 2018 Author Posted December 13, 2018 (edited) Thanks for that diagram. Oops, actually, I have the NanoPi Neo 2, not the NanoPI Neo Core 2 (and I've changed the title of this topic accordingly). And I downloaded the wrong thing (although it booted with no complaints!). I'll try again... BTW: I've been trying the OMV pre-built image (correctly for the NanoPi Neo 2, not Core 2), from here by tkaiser, where I had these glitches. In an attempt to get that USB port on the NAS kit board working more reliably, my plan now is to use the stock Armbian for the NanoPi Neo 2, then use the "softy" menu to install OMV. That way I'll get a newer kernel (which maybe will use that 2nd USB port, on the NAS kit board, more reliably). This will let me jump from an older kernel to a newer one: Current OMV 4.1 image pre-built by tkaiser: Linux hostname 4.14.70-sunxi64 #274 SMP Wed Sep 19 12:09:30 CEST 2018 aarch64 GNU/Linux Current stock Armbian: Linux hostname 4.14.78-sunxi64 #416 SMP Fri Oct 26 15:54:03 CEST 2018 aarch64 GNU/Linux I found the USB-related hardware problem, detailed here. Edited December 14, 2018 by esbeeb title changed. my fix mentioned
esbeeb Posted December 19, 2018 Author Posted December 19, 2018 On 12/5/2018 at 5:50 PM, esbeeb said: , in a USB 3.0 enclosure which surely has a JMS 578 controller chip Although these enclosures have a good controller chip, the USB3.0 cables provided suck. I would get UAS-related "inflight" error messages as seen above, when using them, for any prolonged period of time. These error messages went away when using plain old USB 2.0 USB-A-to-USB-B male-male cables (as in, USB "printer cables"), which had the same overall thickness of wire when you compared them side-to-side to the junky USB 3.0 cables, but the USB 2.0 cables have like half the metal wires inside them, which were probably much thicker guage than the junky USB 3.0 cables. Boo, Orico! Mr. Scissors was not impressed: We also bought a couple of USB 3.0 cables on AliExpress (don't ask which ones, one was 1.5m, one was 3m), and they too similarly sucked. If Mr. Scissors could speak, he would say, "Off with their heads!": PS: the reference to Mr. Scissors was a little shout out to Chris Barnatt of explainingcomputers.com.
esbeeb Posted December 19, 2018 Author Posted December 19, 2018 Here's a picture of the NAS I've set up for our small office here: Inside the NAS kit aluminum case on the top is a 500GB SSD, and there are two more 3.5" drives on the bottom, in enclosures, attached with USB 2.0 cables. OpenMediaVault uses all 3 drives. 1
esbeeb Posted December 21, 2018 Author Posted December 21, 2018 As I rsync'd data around between the drives (all formatted with BTRFS), while doing no other disk operations to speak of, I would at first get 40MB/sec (shown with iotop), then after about 10 seconds, it would simmer down to 20MB/sec and stay there. So you can expect a solid 20MB/sec through those USB 2.0 cables (and the USB 3.0 cables were never any faster, even when no "UAS inflight" errors were getting generated), with quick initial surges of 40ish MB/sec. These speeds (when doing just one data transfer at a time) held true wether or not you wrote to SATA or a USB disk, and wether or not you used a spinning 3.5" drive, or an SSD. So in conclusion, IMHO, this rig was born to run OpenMediaVault (in a small office of a few people, where it's usually/often the case that only one user at a time is hitting this server), but not Nextcloud! (Edit: ...granted you only hook up one 2.5" drive to the SATA port, as I could never manage to get rock-solid stably-working drives connected over USB.) And those users should probably use Filezilla to transfer files (not Windows Explorer, ie. SMB3), if they are transferring any more than about 300MB (at which time I say it's worth one's time to spend the 30 seconds or so starting Filezilla, and logging in, and navigating to the source/destination folders.) Furthermore, you get a very noticeable boost in file transfer speeds, if you are willing to use a Gigabit Ethernet dongle on your laptop, even a cheap one like I bought (see above), not wifi.
esbeeb Posted February 6, 2019 Author Posted February 6, 2019 After @tkaiserexplained that GVFS (used by Linux Mint's default File Browser, Nemo) isn't as fast as it could possibly be these days, for transferring files over SMB, I wondered if KDE's way of transferring files over SMB (called KIO) might be any faster. So I did a comparison of uploading my same folder of 2GB of ebooks, containing many tiny files, (same as used above) using 3 different methods in Linux Mint: - Nemo 3.8.6/GVFS - KDE Dolphin File Browser (17.12.3)/KIO - SMB4K (2.1.0), a userspace SMB share mounting utility/KIO All 3 methods used the SMB3 protocol, over the very same GbE connection. Nemo kicked the butts of the two KIO-based methods: 1:26 (minutes:seconds) = 23 MB/sec average Dolphin: 1:56 = 17.2MB/sec average, 75% as fast as GVFS. SMB4K: 2:27 = 13.6MB/sec average, 59% as fast as GVFS.
esbeeb Posted March 13, 2019 Author Posted March 13, 2019 Word to the wise: use brand-new USB 2.0 A-to-B cables (printer cables) within those cylindrical inline magnets on the wire (which clean up electrical interference). I had "UAS inflight" errors (and checksums were failing on BTRFS scrubs) until I changed these USB cables accordingly. In the future, if I ever make any other NAS, I'll never use USB to attach any "permanent" drive (which will have either the OS on it, or have a file share served from it). USB might be OK for some temporarily-attached drive or stick, where a one-off backup is made (say, for making periodic, "off-site" backups). The slightly higher cost to have proper SATA connections is so well worth it, I say (even if on an 8-year old, used PC, that's still in good running condition.) Cheaping out, and depriving yourself of enough SATA ports is a very bad idea, to save a small amount of money, IMHO.
esbeeb Posted March 14, 2019 Author Posted March 14, 2019 I've been suspicious that my TP-Link UE300 GbE dongle is sort of slow. I've got a new GbE dongle, to compare with, an Anker AH212. loading 2GB THe Anker goes a bit faster than the TP-Link, for FTP and SMB, but not by much: Uploading 2GB of tiny ebooks (2029 files in 756 folders): - TP-Link: - FTP: 22.3 MB/sec. - SMB: 12.0 MB/sec - Anker: - FTP: 25.0 MB/sec - SMB: 14.6 MB/sec So basically, all my tests above are quite trustworthy, I think. With a more carefully chosen GbE dongle, you only get a couple more MB/sec.
esbeeb Posted April 6, 2019 Author Posted April 6, 2019 Am I ever glad I used BTRFS on all the file shares. I have "snapper" set up nicely to do automated snapshots on all 3, and combined, these proved to be invaluable in recovering from the less-than-excellent USB cables I used. Let me once again stress that you absolutely cannot afford to cut corners on the USB 2.0 cables you choose. They must be brand spanking new, and have the inline cylindrical magnets to cleanse the signal. Also, I implore anyone using Armbian to make periodic backups of the MicroSD card (or wherever you install the OS), with, say, "dd", or some other disk imaging tool. I had an " apt-get update && apt-get upgrade" blow up in my face badly recently: I couldn't log into the web management interface of OMV, nor could I even log in with SSH any more! Not being able to log in with SSH (which is the normal way I would rescue such a problem) left me shocked and horrified, as there is no video port whatsoever on this rig to log in at the console. I have no equipment for a serial connection either. The backup SD card I had saved my bacon. I'm now losing my interest in using this ARM board for OMV. There have been just too many surprises and pitfalls, try as I might to do my best, and I only have enough spare time to slay just a few of the smaller dragons. I plan to use OMV in a PC instead (where I would have a video port for rescuing), and use this NanoPi just for SyncThing, which is less ambitious (than OMV), yet still useful to my organization. By using OMV in a PC, I'll also be able to install some Docker containers that my organization needs, this "sweetens the pot" enough for me to want to rebuild the NAS on that equipment. I hereby retract my bold statement above, that "This rig was born to run Openmediavault". I hope my small contributions above save others time and stress.
esbeeb Posted May 2, 2019 Author Posted May 2, 2019 I kept getting disk IO problems on my 2 external USB drives. I could never narrow down just what to blame: the USB cables (which I changed like 5 or 6 times), the USB enclosures I bought, or the two USB ports on the NanoPi itself. Even though these enclosures have JMS578 controller chips in them, and those are pretty ideal controller chips, I don't really trust the other ORICO components around those chips. I wouldn't recommend buying these ORICO 7688U3 enclosures ever again, even if just for the reason that they secure the drives poorly (which is why I had to hack together a nylon strap to prevent the drives from vibrating inside the enclosures). There's no "rails" on either side of the drives while they are in the enclosure. The only thing stopping the drives from wobbling side to side in the enclosures is the SATA port being connected itself, plus there is a little block of rubbery foam that the front of the drive pushes up against (and that only holds in the center, not at the corners). The SATA-connected SSD drive (connected to the NanoPi's "NAS kit" accessory board) always behaved nicely. No disk IO errors. So therefore, I suggest using the NanoPi Neo2 only for a project where you need 1 disk, and that 1 disk should be a 2.5" SATA attached to the NanoPi's "NAS kit" accessory board. Beware the USB ports, and you would be well advised that you are likely to run into Disk IO errors if you still dare. Watch your /var/log/syslog like a hawk, and maybe you'll see disk IO errors appearing there.
esbeeb Posted May 7, 2019 Author Posted May 7, 2019 So I built a new OpenMediaVault server from a new PC, and used the same 3 SATA disks mentioned above (two Toshiba 3.5", and one Samsung 2.5" SSD). This PC has an Asus mobo (Asus Tuf B450 Gaming) with GbE and a sweet B450 disk controller. The two 3.5" SATA drives now get about 200 MB/sec with hdparm, and the 2.5" SSD gets about 500 MB/sec, which of course is way faster than when these were hooked up to the NanoPi Neo2. Here are the results of the network transfer speed tests which mattered to me most, for sake of comparison: Using two different USB 3.0 GbE dongles on my laptop to upload (an Anker AH212, and a smaller, cheaper and slightly slower TP-Link UE300), I uploaded 2 GB of eBooks into this new Asus PC OMV server in different ways: Anker AH212: SMB upload of a 2GB .zip file: 54.1 MB/sec SMB upload of a 2GB folder with many tiny files (2029) in many folders: 24.7 MB/sec FTP upload of a 2GB .zip file: 105 MB/sec FTP upload of a 2GB folder with many tiny files (2029) in many folders: 40.8 MB/sec TP-Link UE300: SMB upload of a 2GB .zip file: 47.6 MB/sec SMB upload of a 2GB folder with many tiny files (2029) in many folders: 19.2 MB/sec FTP upload of a 2GB .zip file: 105 MB/sec FTP upload of a 2GB folder with many tiny files (2029) in many folders: 37.0 MB/sec Here are some of those test results again (gathering from my results above), when I uploaded using the same GbE dongles into the NanoPi Neo2, into the same disks (and let's set aside the disk IO error problems I had with the two 3.5" SATA drives in crappy Orico USB enclosures): Anker AH212: SMB upload of a 2GB folder with many tiny files (2029) in many folders: 14.6 MB/sec FTP upload of a 2GB folder with many tiny files (2029) in many folders: 25.0 MB/sec TP-Link UE300: SMB upload of a 2GB folder with many tiny files (2029) in many folders: 12.0 MB/sec FTP upload of a 2GB .zip file: 29.9 MB/sec FTP upload of a 2GB folder with many tiny files (2029) in many folders: 22.3 MB/sec
esbeeb Posted May 7, 2019 Author Posted May 7, 2019 Dear moderator, could this topic be moved to the H5 subforum? It's misfiled under the H2 & H3 subforum. Thanks in advance. @lanefu, @Igor Sorry, I don't know who the other mods are. 1
Recommended Posts