Jump to content


Photo

H3 devices as NAS


  • Please log in to reply
44 replies to this topic

#31 mdel

mdel

    Advanced Member

  • Senior Members
  • PipPipPip
  • 100 posts

Posted 10 August 2016 - 10:48 PM

Yeah you'll have to be "careful" with usb sata bridges on those boards i guess.

 

I wanted to try something on my pcduino3 nano (A20) but the two usb host ports don't seem to be able to power any of my 2.5 hdd enclosures, i can only get one of my ssd to be powered from there.

 

I'm using a bench power supply to avoid any problems, and tried from the default usb power plug, and gpio +5 DCin pin but no luck.

Using an usb hub with external power works but that's a lot of things i don't want to add.

 

I'm a bit surprised, although i haven't used that board a lot and only with its sata connector and an external hdd power supply so i didn't really test the usb ports with power devices..

 

Anyways those linksprite boards never inspired me much confidence, and they are a bit pricy, so i think i'll just ditch them and turn to H3 boxes or boards.

I have no problem powering those hdd enclosures with my beelink x2 or opi pc with a stock 2A supply..



#32 mdel

mdel

    Advanced Member

  • Senior Members
  • PipPipPip
  • 100 posts

Posted 12 August 2016 - 09:25 AM

well i've ditched my pc3duino3 nano which is simply a complete mess when it comes to USB.

 

Using xenial vanilla kernel, i'm trying to use my Orange Pi PC with an usb3 Gbe adapter which is that one :

Bus 003 Device 002: ID 0b95:1790 ASIX Electronics Corp. AX88179 Gigabit Ethernet

and its driver should be built as module according to /boot/config-3.4.112-sun8i :

CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
 
but that module is nowhere to be found in /lib/modules, am i missing something ?
thx
 
--
 
humm that 3.4.112 kernel is missing CONFIG_USB_NET_AX88179_178A
 
--
 
okay the official "android" sources seem to work out of the box : 
 
That USB2-Gbe link gives me a steady bi directional 340Mbps (iperf) which is not bad i guess.
 
tests with performance or interactive governors gave the same result, performance gives a steady 340Mbps.
 
i didn't expect to get the full 480Mps but i'm not sure how high you can aim. tkaiser got 270-300Mbps on his opi lite from the otg port so it'll probaby be the range of speeds you can expect.
 
here's where i bought that adapter (sunsky-online), a bit expensive :

Edited by mdel, 12 August 2016 - 10:17 AM.


#33 mdel

mdel

    Advanced Member

  • Senior Members
  • PipPipPip
  • 100 posts

Posted 12 August 2016 - 05:36 PM

quick followup 

 

For some reason my AX88179_178A shows dramatically different performances depending on the way you do the iperf / nfs tests.

When the boards acts as the client then the speed drops to 95Mbps on the opi pc and 104Mbps on the pcduino nano. I tested that adapter on another device to make sure it was not related to the driver i downloaded, the pcduino has the armbian vailla 4.6 kernel with built in driver.

 

If it's related to my local network, in both cases the board is connected to my desktop through a manageable BGe switch.

But i can do 980Mbps in bot directions from that pc to another local server, through that same switch.

 

i now use the same JMicron enclosure as tkaiser used in it's UAS test, although i'm not doing UAS test yet and i've populated that enclosure with a 250G evo750 SSD.

 

So, local performances on that usb-ssd, using an ext4 partition are pretty close to what tkaiser had in his test, speeds around 35MB/s (although dd claims it's 45MB/s) and iozone tests are pretty close (a little higher considering i'm using a SSD).

 

i then setup nfs4 and tested transfering numerous large files >2GB, i'm using the Gbe USB3 on the otg port in order to avoid sharing usb hosts as i'm not sure what's populated on the opi pc (3 usb + otg).

 

Writing files to the SSD on the opi worked fine, i'm getting the "full" 35MB/s write speed and with some caching in the background, the client sees a speed around 42MB/s.

Unfortunately due the that Tx low speed identified above with iperf, reading from the opi is always capped at around 95Mbps.

 

Using a ramfs partition, i can max out Rx speed at 340Mbps, but Tx is still not working properly.

Not sure what's going on with that Gbe adapter.

 

One quick note though, i started doing network read/write tests using the -F "file" option of iperf3 but the disk access is extremely slow and shows speeds around 17MB/s, but as it's only happening with iperf i assume it's due to its way of writing data, so don't use that..



#34 mdel

mdel

    Advanced Member

  • Senior Members
  • PipPipPip
  • 100 posts

Posted 15 August 2016 - 08:35 AM

how do you read lsusb outputs to know if your devices are plugged on different physical usb hosts or on a single host through an integrated usb hub ?

thx



#35 tkaiser

tkaiser

    Advanced Member

  • Moderators
  • 2842 posts

Posted 15 August 2016 - 05:37 PM

So, local performances on that usb-ssd, using an ext4 partition are pretty close to what tkaiser had in his test, speeds around 35MB/s (although dd claims it's 45MB/s) and iozone tests are pretty close (a little higher considering i'm using a SSD)

 

That's the reason why dd should be avoided when doing any IO measurements. And you shouldn't be too surprised that USB-Ethernet adapters show unbalanced performance depending on the direction. If you get 300 Mbits/sec in either direction you're a lucky guy.

 

how do you read lsusb outputs to know if your devices are plugged on different physical usb hosts or on a single host through an integrated usb hub ?

 

Simply check bus numbers. Or linux-sunxi wiki where device details for nearly every H3 device are listed (internal USB hub or not)


Please don't send personal messages! Use the forum so others can participate and benefit!

 

Before you report any problem please be aware that crappy SD cards and insufficient power supply are reason N° 1 why things are failing. Try to rule this out first please, check 'getting started' recommendations and check/provide 'sudo armbianmonitor -u' output first!

 

Did you check out custom google powered forum search already (before opening new threads or asking questions)?


#36 mdel

mdel

    Advanced Member

  • Senior Members
  • PipPipPip
  • 100 posts

Posted 18 August 2016 - 12:36 PM

@tkaiser

 

did you try ipv6 on your gbe adapter bandwidth tests ?

 

Out of curiosity (and frustration) i tested iperf3 ipv6 and the results were quite catastrophic, i can barely get above 100Mbps in both directions..

I'm wondering if i screwed something up along the way, didn't test other devices yet.

 

I'm still wondering why we get so different performances using the same gbe adapter on various hosts / systems (laptops, win 7 10), is it related to drivers or usb2/3 performance of the host ?

 

And basically is the mention of "gigabit" only for the sticker or should some of those adapters actually provide Gbe ?

thx



#37 tkaiser

tkaiser

    Advanced Member

  • Moderators
  • 2842 posts

Posted 18 August 2016 - 01:17 PM

did you try ipv6 on your gbe adapter bandwidth tests ?

 

Never. I have two other USB-Ethernet dongles, one with 'Gigabit Lan' claim from SMSC and Apples pricy Fast Ethernet dongle. Guess what? Apple's Fast Ethernet adapter is faster in iperf' --dualtest mode :)

 

IMO it's better to have zero expectations when buying these dongles... prevents frustration. I bought the two RTL8153 based dongles since @Rodolfo recommended them and at least they do not suck too much. But I wouldn't have been surprised if they were a fail since as we all know there are market places where fakes (be it whole products or 'just' the used main IC) can be considered the norm.

 

As an example: you can get Fast Ethernet adapters that claim to contain an ASIX AX88772A but in reality it's just a cheap clone: https://projectgus.c...hernet-adapter/-- why shouldn't happen with your more expensive AX88179 what already happened with its little/cheaper sibling? ;)


Please don't send personal messages! Use the forum so others can participate and benefit!

 

Before you report any problem please be aware that crappy SD cards and insufficient power supply are reason N° 1 why things are failing. Try to rule this out first please, check 'getting started' recommendations and check/provide 'sudo armbianmonitor -u' output first!

 

Did you check out custom google powered forum search already (before opening new threads or asking questions)?


#38 rodolfo

rodolfo

    Advanced Member

  • Senior Members
  • PipPipPip
  • 182 posts

Posted 18 August 2016 - 03:14 PM

IMO it's better to have zero expectations when buying these dongles... prevents frustration

 

Most USB2-based add-ons ( SATA adapters, LAN sticks, flash drives ) have vastly inferior performance compared to the newer USB3 stuff as the bandwidth limitation of USB2 does not really merit faster components. On the other hand USB3-based gear uses faster components and the limitation will be USB2 speed.

 

USB3 Gigabit dongle + USB3 SATA adapter ( or 2x USB3 nano flash drives for low power use ) are combinations working very well with OPI LITE.



#39 CampGareth

CampGareth

    Member

  • Senior Members
  • PipPip
  • 12 posts

Posted 20 August 2016 - 03:33 PM

Well I've just hit a blocker for my usage, no XFS or BTRFS support using the kernel included in the legacy images for the OPi+ 2E. Could use ext4 but it's not recommended because of how Ceph relies on xattrs. I guess I want to upgrade to mainline, any problems with that?

Separate note some USB 3.0 to eSATA adapters I ordered got delivered, crazy good speeds, like 450MB/s sequential access. JMS567s at their hearts. Definitely agreed on USB 3 stuff being a good bet.

#40 tkaiser

tkaiser

    Advanced Member

  • Moderators
  • 2842 posts

Posted 20 August 2016 - 04:46 PM

I guess I want to upgrade to mainline, any problems with that?

Separate note some USB 3.0 to eSATA adapters I ordered got delivered, crazy good speeds, like 450MB/s sequential access. JMS567s at their hearts. Definitely agreed on USB 3 stuff being a good bet.

 

Well, 450MB/s is a clear sign of testing DRAM and not disks since H3 features only USB 2.0 ports and with legacy kernel you get ~35MB/s max (maybe 37 under best conditions). If I were you I would use exactly the three iozone calls from here to test difference between legacy and mainline kernel with your disk setup: http://linux-sunxi.org/USB/UAS

 

Regarding mainline kernel: you would've to build an image yourself or read through post #8 of this thread to recycle an older preliminary OS image for BPi M2+ (will work out of the box on OPi Plus 2E since BPi M2+ is just a lousy clone but one of the USB host ports is not enabled since not available on BPi M2+). If you apply a heatsink to H3 on OPi Plus 2E you're pretty safe regarding overheating (still no throttling implemented in mainline kernel for H3, but thermal design of Orange Pis is pretty good).

 

BTW: I personally would never ever use Ceph on any node lacking ECC RAM. But that's just a personal opinion, I really hate data corruption and have to deal with such cases maybe a little bit too often :)


Please don't send personal messages! Use the forum so others can participate and benefit!

 

Before you report any problem please be aware that crappy SD cards and insufficient power supply are reason N° 1 why things are failing. Try to rule this out first please, check 'getting started' recommendations and check/provide 'sudo armbianmonitor -u' output first!

 

Did you check out custom google powered forum search already (before opening new threads or asking questions)?


#41 CampGareth

CampGareth

    Member

  • Senior Members
  • PipPip
  • 12 posts

Posted 20 August 2016 - 08:16 PM

Well, 450MB/s is a clear sign of testing DRAM and not disks since H3 features only USB 2.0 ports and with legacy kernel you get ~35MB/s max (maybe 37 under best conditions). If I were you I would use exactly the three iozone calls from here to test difference between legacy and mainline kernel with your disk setup: http://linux-sunxi.org/USB/UAS
 
Regarding mainline kernel: you would've to build an image yourself or read through post #8 of this thread to recycle an older preliminary OS image for BPi M2+ (will work out of the box on OPi Plus 2E since BPi M2+ is just a lousy clone but one of the USB host ports is not enabled since not available on BPi M2+). If you apply a heatsink to H3 on OPi Plus 2E you're pretty safe regarding overheating (still no throttling implemented in mainline kernel for H3, but thermal design of Orange Pis is pretty good).
 
BTW: I personally would never ever use Ceph on any node lacking ECC RAM. But that's just a personal opinion, I really hate data corruption and have to deal with such cases maybe a little bit too often :)


Yeah that speed comes from attaching one and an ssd to my desktop, it's more of a proof that there's spare capacity in these things and if I ever wanted an ssd attached to a USB 3 capable ARM board they'd be a good choice :)

Thanks, I'll give that image a test on one of my 2Es. It's been encoding videos for the past few days (24x slower than my desktop but 3x the efficiency) and it's only at 65C with the help of a small heatsink and fan so no worries there.

As for ECC agreed it's a risk but my cluster isn't in production, it's just replacing a HP microserver with something cheaper to run and grow. Here's hoping scrubs pick up any small problems, nodes get kicked out for big problems and off-site incremental backups leave me a good copy somewhere.

#42 Zammy

Zammy

    Member

  • Senior Members
  • PipPip
  • 15 posts

Posted 22 August 2016 - 07:16 PM

@CampGareth,

 

could you please share some links for fast USB3 <-> SATA/eSATA adapters?


Spoiler

#43 CampGareth

CampGareth

    Member

  • Senior Members
  • PipPip
  • 12 posts

Posted 22 August 2016 - 10:06 PM

@CampGareth,

 

could you please share some links for fast USB3 <-> SATA/eSATA adapters?

https://www.amazon.c...uct/B00PAFDFCU/
 

£3.92 each, shipping's super slow though. I don't doubt you can find equivalent devices on ebay etc.

The SATA adapters I'm finding try to power the drive over USB which isn't ideal for me, but if you find some with the same JMS567 chip and a USB 3.0 interface you should be good.



#44 mdel

mdel

    Advanced Member

  • Senior Members
  • PipPipPip
  • 100 posts

Posted 26 August 2016 - 10:12 AM

still doing some more tests.

 

i've switched to an s905 tv box Nexbox A95X, in order to get an idea of possible limitations of the H3 soc.

It's running a custom desktop armbian image (legacy kernel) from bable150, from which i removed all the desktop components. I'll try to use the official armbian ordoid C2 image later (once i get the trick to get rid of that boot.ini).

 

So that box has only two usb ports, two root hosts, i'm using one for the Gbe adapter and a second for an USB3 HDD (not SSD).

For the Gbe i'm getting this time 215Mbps upstream and 245Mbps downstream, which is not too bad i guess.

 

So i did tests running iperf3 up / down, along with 4GB dd read / writes to the HDD.

The good thing is that in any combination the results are very consistent, i never got a test that would deviate of 5Mbps on the network or 2MB/s on the HDD.

 

The network speeds are always as mentioned above, and for the HDD speeds i get a steady 38-40MB/s.

That would pretty much match up any of the tests done on the H3 on this thread.

 

The ondemand governor almost never went up to the full 2GHz and happily stayed at the 1.5/1.75GHz frequencies, with a cpu load barely going over 20% with some spikes at 60% on one core but most probably caused by the monitoring scripts (htop, glances..).

 

So as seen with the H3, the s905 cpu is clearly not a bottleneck and the same kind of nas setup should be achievable.

 

USB2 still feels really weak compared to current day bandwidths (or even poor native sata2), but as long as you don't really expect Gbe speeds to your storage device it should suffice.

 

 

I'm quite interested to see how the Allwinner R40 will perform, it could shuffle the cards again.

My current setup is still using an old A20 running 1 sata HDD, and moved the vpn work to a second device (old s805 tv box) with real Gbe performance..

 

Doing laptop repairs i also got the idea to use old eeepc motherboards that can be found at ridiculous prices (~35e), they are basically the same thing as an arm dev board (with sata), the problem is identifying the ones that do have Gbe. I haven't done much test on those weak atoms cpus so i'm expecting performances issues when it comes to high speed vpn, but for regular storage it should really work fine, we'll see..



#45 earth08

earth08

    Advanced Member

  • Senior Members
  • PipPipPip
  • 33 posts

Posted 29 August 2016 - 08:12 AM

Dear tkaiser,
What is for NAS do you suggest for orange pi plus 2e?
Right now I am using armbian xenial server with nextcloud.

Please guide,
Thanks in adv.