Jump to content

CampGareth

Members
  • Posts

    12
  • Joined

  • Last visited

Posts posted by CampGareth

  1. Tried just the kernel but for whatever reason the NIC didn't start. Anyway I made a new image and am testing it now, results in a few minutes. It's stayed up for 150 seconds so far which bodes well.

    Check out that stability, 600 seconds! http://pastebin.com/GLzMQFs2 

    Now I'll move iperf off core 3: http://pastebin.com/fmj8gL4F

     

    Still not perfect but good enough for my purposes. Let's try the other way: http://pastebin.com/95mFBs0V

     

    Hmm 718mbit vs 835 without the patch. Next up let's mess with the RPS values, the default is 0 for both:
     

    root@orangepiplus:~# cat /proc/sys/net/core/rps_sock_flow_entries
    0
    root@orangepiplus:~# cat /sys/class/net/eth0/queues/rx-0/rps_flow_cnt
    0
    
    

    First up, 32768 for both. Results? OPi to thinkpad sees 720 for a while, rising to 870 before dropping back down. That makes sense as the inbound queue is the only one changing. I'll only mention it if it gets interesting from here: http://pastebin.com/k7sEZC48

    Inbound is where the interesting numbers should be at and indeed they are, 937mbit average! That's with iperf on cores 0-2 btw: http://pastebin.com/6HZGDYqz

    So the question is how do we improve outbound?  *edit* I reran the outbound test and got 870 continuously. 

    Separate note, I'm still trying to get booting from eMMC working. nand-sata-install is the latest version, boot.cmd uses UUIDs so the advice here doesn't work: http://forum.armbian.com/index.php/topic/2082-banana-pi-m2-with-system-on-emmc-data-on-sd-card/

     

    Observed behaviour is that with just running nand-sata-install then trying to boot from eMMC I get no green light on the board. If I then follow the uboot copying advice found in the following post I get a green light but... that's it, the NIC never fires up, I can't tell whether anything else happens: http://forum.armbian.com/index.php/topic/2046-vanilla-kernel-on-opi-pc-install-to-emmc/?p=15685

     

    *edit* Found it! So the nand-sata-install script, it writes to /mnt/bootfs/etc/fstab and adds a line saying mount this UUID at / and that works. However, the one in /mnt/rootfs/etc/fstab still uses names, mmcblk2. Change it to use UUIDs too and it works, basically just copy /mnt/bootfs/etc/fstab over /mnt/rootfs/etc/fstab. 

     

  2. Did you apply montjoie's latest patch? Since without kernel panics / reboot are what's to be expected (BTW: consistent MAC address can be both assigned in .dts and /etc/network/interfaces -- if you're already there you could assign static IP addresses also)

     

    Regarding the 'strange' variation in throughout that's most likely caused by iperf3 running on the same CPU core as the Ethernet IRQ handler. Using taskset to keep iperf3 away from cpu3 is key to more constant iperf performance: https://irclog.whitequark.org/linux-sunxi/2016-09-19#17605514;

    Nope, didn't apply that patch as I couldn't find it! Figured it'd be at http://sunxi.montjoie.ovh/ethernet/ but I get a 403 trying to load that. 

  3. Well I can give you numbers but take them with a large pinch of salt, for the baseline tests pfsense is running on esxi though it does have high priority and a NIC dedicated to it with pcie passthrough. 

    Baseline Thinkpad T60 to pfsense: http://pastebin.com/zGsyZ4xN
    Baseline pfsense to Thinkpad T60: http://pastebin.com/raLKzdRw
    So network itself good for 900-940mbit/s.

    OPi2E running kernel 4.8 to Thinkpad: http://pastebin.com/k7sEZC48
    OPi2E running kernel 3.4 to Thinkpad: http://pastebin.com/nmbkMqf1
    Both devices get 830-840mbit/s but the one running 4.8 takes longer to get there, spending a while at 730 first. 

    Thinkpad to OPi2E running kernel 4.8 caused the OPi to drop off the network after 50 seconds, it was all over the place though, 700-900mbit.
    Thinkpad to OPi2E running kernel 3.4: http://pastebin.com/n5ENN9v7
    The board running 3.4 is all over the place too, guess that's just normal behaviour.

    I went looking and found the one that dropped off the network had rebooted and changed MAC/IP in the process. I ran iperf against it again, this time it got to 21 seconds before rebooting. http://pastebin.com/SqQ74ZaX

    That rebooting is a problem meant to be solved by the v4 branch right?
     

  4. root@orangepiplus:~# uname -a
    Linux orangepiplus 4.8.0-sun8i #1 SMP Mon Sep 19 18:07:30 BST 2016 armv7l armv7l armv7l GNU/Linux
    

    Wahey! So to recap, switching to montjoie's repo and running that script was enough. Next up installing to eMMC, then setting up ceph. One thing I've noticed is that /sys/class/thermal has no devices so definitely need a heatsink on these as they won't know they're on fire. 

    *edit* aaaand installing to eMMC failed, or rather booting from it failed.

  5. Those lazy Armbian devs ;)

     

    To be honest I got really to lazy to keep up with all the device tree stuff you would've to have to adjust with every new kernel version. Currently we use megi's github repo as source (at 4.7, containing cpufreq/dvfs settings that would need some love/attention but only outdated Ethernet stuff and lack support for GbE enabled boards).

     

    I'm currently testing for montjoie his Ethernet v4 driver, you can have a look here what to change in the build system (click on the spoiler thingie): http://forum.armbian.com/index.php/topic/2044-some-discovery-while-trying-520-builds/?p=15717 (choose then Orange Pi Plus, will run on OPi Plus 2E but not all USB ports will work).

     

    You should have at least a heatsink on H3 (1296 MHz, no throttling implemented) and then you most probably want to try out this patch: https://irclog.whitequark.org/linux-sunxi/2016-09-19#17604648;

    Image building now, I don't like the retries inbound but eh, it's probably fine, not gonna be under too much load when disk/cpu's a bottleneck  :)

     

  6. So I know you said not to ask about gigabit ethernet on say the 2E buuuuuut.... :P

    What's blocking that? Looks like support for EMACs isn't in 4.7, how's about 4.8?

    Amusingly I just got around to using your tools to build my own armbian image with 4.7.4, hit the same gigE roadblock, figured 4.8 would probably help then upon checking here I see you're already working on it :P

  7. @CampGareth,

     

    could you please share some links for fast USB3 <-> SATA/eSATA adapters?

    https://www.amazon.co.uk/gp/product/B00PAFDFCU/

     

    £3.92 each, shipping's super slow though. I don't doubt you can find equivalent devices on ebay etc.

     

    The SATA adapters I'm finding try to power the drive over USB which isn't ideal for me, but if you find some with the same JMS567 chip and a USB 3.0 interface you should be good.

  8. Well, 450MB/s is a clear sign of testing DRAM and not disks since H3 features only USB 2.0 ports and with legacy kernel you get ~35MB/s max (maybe 37 under best conditions). If I were you I would use exactly the three iozone calls from here to test difference between legacy and mainline kernel with your disk setup: http://linux-sunxi.org/USB/UAS

     

    Regarding mainline kernel: you would've to build an image yourself or read through post #8 of this thread to recycle an older preliminary OS image for BPi M2+ (will work out of the box on OPi Plus 2E since BPi M2+ is just a lousy clone but one of the USB host ports is not enabled since not available on BPi M2+). If you apply a heatsink to H3 on OPi Plus 2E you're pretty safe regarding overheating (still no throttling implemented in mainline kernel for H3, but thermal design of Orange Pis is pretty good).

     

    BTW: I personally would never ever use Ceph on any node lacking ECC RAM. But that's just a personal opinion, I really hate data corruption and have to deal with such cases maybe a little bit too often :)

    Yeah that speed comes from attaching one and an ssd to my desktop, it's more of a proof that there's spare capacity in these things and if I ever wanted an ssd attached to a USB 3 capable ARM board they'd be a good choice :)

     

    Thanks, I'll give that image a test on one of my 2Es. It's been encoding videos for the past few days (24x slower than my desktop but 3x the efficiency) and it's only at 65C with the help of a small heatsink and fan so no worries there.

     

    As for ECC agreed it's a risk but my cluster isn't in production, it's just replacing a HP microserver with something cheaper to run and grow. Here's hoping scrubs pick up any small problems, nodes get kicked out for big problems and off-site incremental backups leave me a good copy somewhere.

  9. Well I've just hit a blocker for my usage, no XFS or BTRFS support using the kernel included in the legacy images for the OPi+ 2E. Could use ext4 but it's not recommended because of how Ceph relies on xattrs. I guess I want to upgrade to mainline, any problems with that?

     

    Separate note some USB 3.0 to eSATA adapters I ordered got delivered, crazy good speeds, like 450MB/s sequential access. JMS567s at their hearts. Definitely agreed on USB 3 stuff being a good bet.

  10. Hi,

    is my Cubieboard is damaged?

     

    Couple days ago my server was suddenly turned off. When I checked, it turned out, that the Allwinner is very hot. So I added a radiator and run the Cubie with serial console. Seems like everything is ok, but temperature... almost 50*C.

     

    attachicon.giflog_15082016_2.txt

    Don't know where in the world you are but it's summer here and some of my electronics have been overheating as a result. The projector runs louder, I'd stuffed 6 drives in a 4 drive NAS which was fine for months but suddenly the drives were overheating and throttling reducing performance to about 10% of normal.

     

    Just a thought.

  11. I'm planning to go down this route too with a couple of OPi Plus 2Es en-route from china now. Since I'm powering both the boards and the drives with a desktop PC PSU I didn't want to use HDD enclosures. Thankfully I've found USB 3.0 to eSATA adapters that're pretty much perfect for the job. 

    The NAS aspect is a little different though, I use Ceph at home which is a distributed storage system so individual node performance doesn't matter much so long as there are a lot of them. I've been having problems with it crashing on my previous A20 powered board but I'll figure that out soon.

×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines