Igor Posted August 1, 2017 Posted August 1, 2017 Clearfog PRO with Linux 4.12.4-mvebu Samsung 840 PRO utilization: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 random random kB reclen write rewrite read reread read write 102400 4 62501 101691 98761 94430 34522 93050 102400 16 183325 215091 231439 232948 101667 171875 102400 512 301985 307789 341261 344567 328849 309560 102400 1024 294818 309177 345099 347791 339763 309775 102400 16384 271066 344426 384226 388587 387169 346825 2 x Samsung 840 PRO in RAID0, ext4 utilization: random random kB reclen write rewrite read reread read write 102400 4 67614 95508 94561 94984 32561 74157 102400 16 174075 200947 222430 223840 109654 199578 102400 512 307905 294689 317652 308007 307057 268841 102400 1024 307821 317784 327007 330213 322851 317024 102400 16384 286193 383463 394221 398474 397951 374333 Logs: http://sprunge.us/CfRS Controller temperature is stable at 64°C, network utilization is also at top speed. Card was donated via Amazan wish list. Thank you! Hummingboard 2 with Linux 3.14.79-cubox (mainline not working out of the box) Not that impressive as Clearfog, but works. random random kB reclen write rewrite read reread read write 102400 4 28502 39271 37895 37983 22325 38821 102400 16 68494 80988 69940 70511 53264 80047 102400 512 116594 118132 93789 94280 92828 118884 102400 1024 138185 140277 122610 123717 120853 139393 102400 16384 183976 183169 151745 153813 151890 179652 Logs: http://sprunge.us/OeMY Espressobin 2 x Samsung PRO 840, RAID 0, BTRFShttp://sprunge.us/iBFS random random kB reclen write rewrite read reread read write 102400 4 21547 20493 37509 37024 21747 17728 102400 16 61955 56230 93068 93795 64349 45724 102400 512 188611 186828 175688 176961 172098 185109 102400 1024 215817 216559 189477 191846 188481 214727 102400 16384 226764 228502 229162 238883 222826 229894 Intel N4200 / Up2board Square Spoiler 1 x Samsung 840 PRO random random kB reclen write rewrite read reread read write 102400 4 65003 76031 93147 92507 32024 73029 102400 16 146880 175968 187734 224409 106025 163916 102400 512 244782 225736 348401 357949 343145 221037 102400 1024 239203 247778 313476 313372 309170 248540 102400 16384 254396 266530 357708 368483 370731 276458 2 x Samsung 840 PRO in RAID 0 random random kB reclen write rewrite read reread read write 102400 4 61680 70018 67650 66635 18221 67984 102400 16 145869 164784 147884 143877 46355 160169 102400 512 230140 219870 213069 199087 213587 210607 102400 1024 273150 271782 257206 279397 275632 271896 102400 16384 314895 311385 365117 376228 375081 317790 Native onboard SATA 1 x Samsung 840 PRO random random kB reclen write rewrite read reread read write 102400 4 70611 80842 100647 97276 30072 77666 102400 16 194707 208422 239401 224119 102211 218393 102400 512 346735 410867 401536 425167 396013 412711 102400 1024 374032 366490 378981 394321 382352 368195 102400 16384 441298 430380 475063 486540 482151 414518 Reference: A20's native SATA Spoiler random random kB reclen write rewrite read reread read write 102400 4 18190 21041 44757 45359 24223 21639 102400 16 27700 30195 101335 105895 72086 29925 102400 512 34835 35088 182999 184586 179342 34970 102400 1024 34851 35237 188542 189265 187278 35235 102400 16384 34571 36150 236663 238700 238065 35947 This document is work in progress ... 2
tkaiser Posted August 19, 2017 Posted August 19, 2017 Mine arrived today 7 SATA ports in total now but most probably I'll focus on dual disk performance comparing ASM1061 / ASM1061 88SE9215 / 88SE9215 ASM1061 / 88SE9215 Armada / 88SE9215 Armada / ASM1061 Armada / Armada (using some u-boot Voodoo to transform one of the mPCIe slots into mSATA and using the other adapter to do mSATA --> SATA 'conversion') BTW: Is there a reason why you did not publish Espressobin numbers so far? I made another attempt with this board today but stopped already again...
Igor Posted August 19, 2017 Author Posted August 19, 2017 30 minutes ago, tkaiser said: BTW: Is there a reason why you did not publish Espressobin numbers so far? I made another attempt with this board today but stopped already again... I haven't seen any mPCI card on this board yet at least on stock kernel while I could not boot mainline kernel - from SD card.
tkaiser Posted August 19, 2017 Posted August 19, 2017 5 hours ago, Igor said: I haven't seen any mPCI card on this board yet at least on stock kernel Worked out of the box for me: https://pastebin.com/8Hcttvft (though no idea about CPU clockspeeds since this stuff is currently not working on Espressobin legacy kernel)
Igor Posted August 19, 2017 Author Posted August 19, 2017 What about some wireless card? You have some Atheros, right? Does it work for you?
tkaiser Posted August 19, 2017 Posted August 19, 2017 Since I had the EspressoBin booted, the Marvell 88SE9215 SATA card in the mPCIe slot and 2 SSDs provided with external power I did some more testing comparing single SSD performance with an ext4 partition on the onboard SATA port and the 88SE9215 btrfs RAID-0 performance with both SSD attached to the 88SE9215 compared to one on each controller My test setup sucks since I've no idea how fast Armada 3720 CPU cores clock (I let 7-zip benchmark run and '7z b' reported a total of just 1414) and the 2 SSDs are cheap consumer crap way too slow (see smartctl output here -- write performance with both SSDs sucks as soon as amount of data written exceeds more than 1 or 2 TB, especially the Intel 540 then gets dog slow at around ~60MB/s, we can see this at the end of the numbers when I tested btrfs RAID-0 with 2 GB filesize): Intel 540 / Armada 3720 SATA random random kB reclen write rewrite read reread read write 102400 4 47255 70977 87288 81713 29007 71294 102400 16 108707 103552 209717 215060 75508 71568 102400 512 127331 195726 399994 455759 374609 82367 102400 1024 143364 339287 468693 472660 423549 339132 102400 16384 292534 328546 404603 407138 406887 336762 Intel 540 / Marvell 88SE9215 SATA random random kB reclen write rewrite read reread read write 102400 4 34074 45336 41414 41787 18210 38243 102400 16 54685 48116 108386 102390 50574 109665 102400 512 92229 68978 184259 197881 185550 74988 102400 1024 108717 144664 261007 261002 230533 108468 102400 16384 164837 191212 291178 292019 285631 252738 Samsung EVO840 / Armada 3720 SATA random random kB reclen write rewrite read reread read write 102400 4 44342 74243 88564 87235 40085 68688 102400 16 138468 204412 223066 224952 127620 189047 102400 512 392681 384072 445833 452317 425046 401434 102400 1024 391877 383661 456881 464721 450389 399017 102400 16384 375014 381085 434583 440510 437606 377176 Samsung EVO840 / Marvell 88SE9215 SATA random random kB reclen write rewrite read reread read write 102400 4 34457 50728 54088 52026 28858 51844 102400 16 102471 131968 144037 143400 87486 131722 102400 512 246405 252458 249820 244493 246951 251780 102400 1024 251542 258106 261483 249819 255789 261397 102400 16384 244643 272618 279133 292771 291875 266423 btrfs raid-0 / 88SE9215 + 3720 SATA random random kB reclen write rewrite read reread read write 102400 4 28214 27320 49198 49484 26154 23649 102400 16 70382 66525 126629 127612 72779 57479 102400 512 349803 345833 302641 308043 285616 345608 102400 1024 399405 392532 320462 331041 314945 409883 102400 16384 433521 426136 407682 416113 407861 410509 2048000 16384 271999 126972 412175 412226 403249 126939 btrfs raid-0 / 2 x Marvell 88SE9215 SATA random random kB reclen write rewrite read reread read write 102400 4 25292 23843 42802 42662 23992 21229 102400 16 62994 56114 105396 105729 65521 52447 102400 512 225543 225750 193574 195734 188411 217733 102400 1024 232385 224001 202101 207403 201948 231705 102400 16384 270143 272697 247752 254730 260208 271869 To me it looks obvious that we're suffering currently from CPU cores being clocked too slow on Espressobin. But no idea how to adjust this (does cpufreq scaling work with https://dl.armbian.com/espressobin/nightly/Armbian_5.32.170626_Espressobin_Ubuntu_xenial_default_4.4.73.7z?) 11 minutes ago, Igor said: What about some wireless card? You have some Atheros, right? Does it work for you? Not tried yet. I've one AR9380. Will give it a try tomorrow.
Igor Posted August 19, 2017 Author Posted August 19, 2017 It works for me too, but it's not very impressive.
tkaiser Posted August 19, 2017 Posted August 19, 2017 4 minutes ago, Igor said: It works for me too, but it's not very impressive. Well, by looking at the scores above to me it seems obvious we're bottlenecked by CPU on Espressobin currently (EVO840 directly connected to 3720 SATA port maxes out at 440 MB/s while the RAID-0 using two separate controllers/buses does not exceed 412 MB/s -- I would guess that's due to RAID/checksumming overhead here and if CPU wouldn't be the bottleneck we would see with a RAID-0 combining onboard SATA and 88SE9215 +700MB/s or even more. I tried to use 7-zip numbers to get an idea at which clockspeed Armada 3720 is running but realized that I'm trying to compare Debian Stretch package (7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21) with numbers generated by Jessie/Xenial: '7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18'. Crap, need to retest with Jessie on Espressobin or test another 64-bit platform with Jessie.
tkaiser Posted August 20, 2017 Posted August 20, 2017 Testing on a Clearfog Pro with performance cpufreq governor (1600 MHz). Only interested in multi disk performance and comparing onboard SATA (labeled 'Armada' below), ASM1061 and Marvell 88SE9215 which this topic is about. The two 128GB SSDs I test with are a Samsung EVO 840 and an Intel 540 (both are not the best candidates for such a test since too slow especially wrt writes when amount of data written exceeds ~1 TB). For convenience reasons and to be able to detect possible data corruption I relied on btrfs RAID-0 (striped data but mirrored metadata/checksums): 'mkfs.btrfs -m raid1 -d raid0 /dev/sda /dev/sdb'. Mounted all the time as '/mnt/raid-0 btrfs rw,relatime,ssd,space_cache,subvolid=5,subvol=/ 0 0'. ATA hotplug works with both ASM1061 and 88SE9215 (externally powered the SSDs so I was able to switch them on/off) but I doubt it's wise to try out hotswap with the standard SATA connectors here so I chose to shutdown the board between all tests. Below the combinations listed, the left one is always the EVO 840 and the right one the Intel 540, the sprunge links are just to document whether something strange happened (dmesg output included) and controller features are as expected (eg. NCQ / queue depth 31 all the time): ASM1061 / ASM1061 http://sprunge.us/KFJE random random kB reclen write rewrite read reread read write 102400 4 41581 40231 66934 65931 30858 36496 102400 16 78795 85162 129029 132381 75406 78708 102400 512 235104 230833 224359 235073 230958 233557 102400 1024 237243 237869 231138 233893 231208 236350 102400 16384 278629 280890 346025 366094 368530 144826 88SE9215 / 88SE9215 http://sprunge.us/GiOD random random kB reclen write rewrite read reread read write 102400 4 42725 40902 67686 67951 30666 37689 102400 16 88221 86106 133593 134186 75724 79272 102400 512 230455 230557 236345 239362 234916 228943 102400 1024 232845 228149 231002 236846 232255 228675 102400 16384 251719 144849 342737 350257 349299 174308 88SE9215 / ASM1061 http://sprunge.us/VcaN random random kB reclen write rewrite read reread read write 102400 4 42550 41349 67662 67702 30974 37499 102400 16 87533 87006 127737 133436 75295 79513 102400 512 321095 317909 324035 329646 318073 315473 102400 1024 319864 324307 320939 325636 320102 322490 102400 16384 381197 349591 538466 580906 588403 357470 ASM1061 / 88SE9215 http://sprunge.us/KfCX random random kB reclen write rewrite read reread read write 102400 4 42143 41064 66865 67056 30924 37030 102400 16 87302 77005 132274 132805 75252 79618 102400 512 317596 315885 300093 328767 315876 315025 102400 1024 322192 320242 319936 325295 316761 318602 102400 16384 347208 277583 556563 592881 600894 117150 Armada / 88SE9215 http://sprunge.us/gAeb random random kB reclen write rewrite read reread read write 102400 4 43562 42551 71538 72146 31692 38201 102400 16 91960 90462 142261 142811 78192 83157 102400 512 320566 317511 364218 370690 351141 315569 102400 1024 269086 262615 348363 363624 355324 319879 102400 16384 349914 376723 531844 561922 587068 221096 Armada / ASM1061 http://sprunge.us/EOYJ random random kB reclen write rewrite read reread read write 102400 4 43630 41769 70908 71520 31611 37924 102400 16 89669 88133 137684 125599 77745 82730 102400 512 316515 314883 347953 360841 344984 220572 102400 1024 144820 324249 349778 355869 349079 326118 102400 16384 380195 417907 583988 580550 604784 376562 Armada / Armada random random kB reclen write rewrite read reread read write 102400 4 45984 43445 75553 75726 32493 37967 102400 16 96203 92794 150890 151756 81030 86489 102400 512 357109 357089 363720 369983 354740 354042 102400 1024 357485 369719 354278 358965 353952 367930 102400 16384 406932 402512 532718 559035 554006 361749 Armada / Armada 2nd run http://sprunge.us/iIhN random random kB reclen write rewrite read reread read write 102400 4 38889 43882 75523 75631 32143 34014 102400 16 90735 89593 149764 150979 81072 82520 102400 512 204282 114150 352789 365723 352634 126669 102400 1024 122663 117834 325485 360243 353983 123490 102400 16384 126510 120139 530389 561534 558192 125412 First conclusions: The above numbers unfortunately do NOT indicate SATA controller performance but are the results of an insufficient test setup (wrong SSDs used, both my SSDs start to slow down especially write performance after a small amount of data is written, Samsung calls this TurboWrite, Intel does the same on cheap consumer SSDs and my 540 drops then down to ~60 MB/s). The effect of this can be seen with the last 2 tests executed one after the other with otherwise identical test setup. Write performance ~400 MB/s on first run (limited by the slower Intel 540 writing with ~200 MB/s), then dropping down to ~120 MB/s (now the Intel's cache is full and it slows down to 60 MB/s -- with a dual disk RAID-0 overall bandwidth is the one of the slowest RAID member * 2). To get really an idea how the 3 SATA controller setups vary I would've needed at least 2 Samsung 850 Pro with 256GB (or any other SSD that reads/writes with +500 MB/s all the time). If I would do the test a second time I would ensure that the SSDs remain powered on after each test for some minutes to empty their caches to recover full performance. I skipped this above (quick&dirty mode) which resulted in a lot of write performance numbers that are lower than necessary. When both disks are attached to the same PCIe controller performance of ASM1061 and 88SE9215 is identical (I would believe in this mode they're bottlenecked by the single PCIe 2.x lane). I had zero issues with our kernel and NCQ (tested also with the ASM1061 and 'echo 1 > /sys/block/sd?/device/queue_depth' but no differences in performance which is an indication that we need to look a little bit closer here) It also made no difference which SSD was connected to which controller when I tested the '1 SSD per PCIe controller' modes. The numbers above when one or two SSDs were connected to Armada's native SATA ports are IMO all questionable. Too much influence due to insufficient SSDs, these numbers should be dropped and another round of tests with really performant SSDs is needed For that what most people are most likely interested in (attaching HDDs) the numbers are sufficient since 'fast enough'. In case anyone is asking about RAID5: At least I won't test this since I consider RAID a waste of money and resources at home and even dangerous since home/SOHO RAID afficionados never test what's important (failure scenarios and rebuilds) Final remarks: My numbers above can not be compared with Igor's since RAID-0 vs. single SSD, btrfs (CoW, checksumming, read/modify/write cycle with data smaller/equal default blocksize) vs. ext4 (so different amount of overhead involved) I found it somewhat surprising getting identical performance with NCQ queue depths of 1 and 31. This is something that needs investigation with next rounds of tests (some NCQ details here) The Marvell 88SE9215 on all those mPCIe cards wears a heatsink for a reason. It gets really hot during operation and I would assume consumption is also significantly higher than ASM1061 (not tested/measured yet).
Igor Posted August 20, 2017 Author Posted August 20, 2017 16 minutes ago, tkaiser said: My numbers above can not be compared with Igor's since RAID-0 vs. single SSD, btrfs (CoW, checksumming, read/modify/write cycle with data smaller/equal default blocksize) vs. ext4 (so different amount of overhead involved) I can add a test with 2 x 840 PRO 256G into RAID0 and BTRFS with Clearfog. If it makes a value?
tkaiser Posted August 20, 2017 Posted August 20, 2017 On 20.8.2017 at 11:49 AM, Igor said: I can add a test with 2 x 840 PRO 256G into RAID0 and BTRFS with Clearfog. If it makes a value? Definitely. But we must really take care to not mix these numbers since RAID/btrfs/dual disk overhead massively influences performance. To get an idea I repeated single SSD tests with my EVO840 and a single ext4 partition on the Clearfog to check whether then we see some differences between the 3 controllers: Armada EVO840 ext4 http://sprunge.us/MLGY random random kB reclen write rewrite read reread read write 102400 4 81488 117728 132076 135454 41432 121492 102400 16 215389 267562 278121 283023 127460 266775 102400 512 372032 368574 411084 423599 378666 383032 102400 1024 362836 369646 415503 394737 411496 366720 102400 16384 327837 379992 468341 469976 481556 387493 88SE9215 EVO840 ext4 http://sprunge.us/SiEG random random kB reclen write rewrite read reread read write 102400 4 80772 104881 113867 115767 45327 104451 102400 16 183112 214327 235989 237563 131997 213925 102400 512 310924 316593 351351 354992 342506 310179 102400 1024 313374 313495 356670 360649 354205 307503 102400 16384 296566 340321 372968 394449 393854 338588 ASM1061 EVO840 ext4 http://sprunge.us/dSAN random random kB reclen write rewrite read reread read write 102400 4 77791 101160 113403 113886 45548 98541 102400 16 Error writing block 5446, fd= 3 write: Input/output error iozone: interrupted Edit: Stopped the board, re-inserted SATA cable (the connectors on my ASM1061 do not provide cables fitting tightly), tested on both SATA ports various times. It now looks like this with ext4/EVO840/ASM1061: ASM1061 EVO840 ext4 http://sprunge.us/VOON random random kB reclen write rewrite read reread read write 102400 4 78708 101923 108780 111426 46000 95101 102400 16 178421 214891 233270 234523 133188 214486 102400 512 336295 332287 349769 353435 341314 340677 102400 1024 334856 336743 344395 356114 350605 335704 102400 16384 310091 377760 385584 390347 389789 366494 Edit 2: SMART CRC error counter attribute was 88 before (see smartctl output from yesterday). Now it's 89: 199 CRC_Error_Count 0x003e 099 099 000 Old_age Always - 89 In other words: The issue I ran into before was SATA connector crappiness: The 2 SATA connectors on my el cheapo ASM1061 card aren't that reliable. With such cheap SATA cards after connecting any device it's advisable to check smartctl output for attribute 199 and then immediately run a heavy disk benchmark and check dmesg output and SMART attribute 199 again. This SMART counter is there for exactly this reason: to identify cabling/contact problems!
tkaiser Posted August 30, 2017 Posted August 30, 2017 (edited) Today this little guy arrived. It's a Transcend TS120GMTS420 M.2 2242 SSD I chose for good random and sequential IO performance. Full smartctl output after mounting it on my Clearfog here (this thing is pretty new so not already supported by smartmontools -- time to submit a patch open a ticket). Since we have a few btrfs RAID-0 numbers above I chose to repeat such tests, comparing single drive performance with possible RAID-0 modes (btrfs vs. mdraid -- the whole testing is somewhat useless and this is just to illustrate that the computation overhead with such modes on rather weak ARM boards is something to have in mind): TS120GMTS420 in M.2 slot random random kB reclen write rewrite read reread read write 102400 4 86865 124684 140973 140224 32522 101844 102400 16 206105 251838 293776 282729 84623 202503 102400 512 363707 377155 409380 401946 358966 372971 102400 1024 364578 358095 402539 404249 387812 376067 102400 16384 310808 407947 500231 497107 470879 398703 3072000 16384 420958 403649 509368 507120 469027 405752 EVO840 behind 88SE9215 random random kB reclen write rewrite read reread read write 102400 4 77592 98082 105627 110948 42939 80088 102400 16 161754 167814 224142 231793 131453 194794 102400 512 254022 251896 321636 324871 314115 286720 102400 1024 288675 293282 325360 328868 323345 294681 102400 16384 278020 331917 371216 393551 390127 338349 3072000 16384 335804 338734 384926 384487 383777 224859 TS120GMTS420 and EVO840 as btrfs RAID-0 random random kB reclen write rewrite read reread read write 102400 4 42327 42858 73490 73437 30527 37809 102400 16 91631 89645 145469 146531 80406 65167 102400 512 336147 329055 364294 372163 336223 342325 102400 1024 338846 344515 361248 365292 348673 331471 102400 16384 391086 399913 619237 574991 416424 359991 3072000 16384 415433 418134 587887 626527 620356 308949 TS120GMTS420 and EVO840 as mdraid RAID-0 random random kB reclen write rewrite read reread read write 102400 4 83308 109760 113192 120491 38131 108471 102400 16 191221 226954 241912 247788 105236 200482 102400 512 314386 324797 357034 361831 338066 335216 102400 1024 476716 500618 542100 550804 581167 549686 102400 16384 503050 659126 745353 751381 749971 655232 3072000 16384 677670 660505 734458 746781 739883 655215 (bash history here to check test methodology) Off-topic: I bought the SSD as companion for this little USB3 M.2 enclosure (JMS578 so excellent performance expected). But since SMART attribute 194 (disk temperature) after benchmarks reads already 65°C I'm a bit concerned Edited August 30, 2017 by tkaiser Added smartmontools ticket link for SSD
Igor Posted August 30, 2017 Author Posted August 30, 2017 1 hour ago, tkaiser said: Off-topic: I bought the SSD as companion for this little USB3 M.2 enclosure (JMS578 so excellent performance expected). But since SMART attribute 194 (disk temperature) after benchmarks reads already 65°C I'm a bit concerned Got similar one, also hot like hell ... chip was covered with some nail polish or something. Spoiler 1st post updated with 2 x 840PRO in RAID0.
Technicavolous Posted November 3, 2017 Posted November 3, 2017 I'm not able to see clearly exactly which units you got and from where can I purchase one in the US? My application is a classroom where students have numerous boards, we keep images for various systems backed to SSD as well as run local repositories so students can have their systems up faster than over the internet, and share cool setups they've made. Various instructors keep their fav's on their own SSDs. We tried to make a system using PM cards and failed miserably ... this looks a viable solution. Hotswappable would be a plus but not necessary.
Igor Posted November 3, 2017 Author Posted November 3, 2017 9 minutes ago, Technicavolous said: I'm not able to see clearly exactly which units you got and from where can I purchase one in the US? You mean this USB3 M2 key? It looks like it's JMS578 and I don't recommend it. It gets really hot. I bought it on eBay, China origin.
Technicavolous Posted November 3, 2017 Posted November 3, 2017 I guess I should have asked my question more thoroughly ;] The first photo in Mr. Kaisers Aug. 19 post - I assume clearfog pro - without enclosure but what are the sata boards and where do I get those? I was fairly sure RAID wan't the best idea on any ARM board but we have no need for that anyway, this is JBOD for convenience. I have a little 4 bay removable array that will be nice once I get a decent system running. Thanks for pointing out this board, it reduces several other pieces of hardware and introduces a nice way to package the whole thing. BTW, we always introduce our students to Armbian, because it behaves pretty much the same on any board so they have more in common right off the bat :]
tkaiser Posted November 4, 2017 Posted November 4, 2017 12 hours ago, Technicavolous said: but what are the sata boards and where do I get those? The 4 port things are based on Marvell 88SE9215 (this is a 4 port SATA to single PCIe 2.x lane controller, so suitable for mPCIe where only one lane is available). I got mine from Aliexpress, Igor got his for twice the price from Amazon, you also find them sometimes at local resellers, just search for '88SE9215'. (there's also a x2 variant called 88SE9235 -- just like there's ASM1062 vs. ASM1061 -- but at least behind mPCIe there's no advantage in choosing this over 88SE9215, it would need M.2 or a real PCIe port to make use of the second available lane) 1
Technicavolous Posted November 5, 2017 Posted November 5, 2017 22 hours ago, tkaiser said: The 4 port things are based on Marvell 88SE9215 ... there's also a x2 variant called 88SE9235 I found the four port units thanks. Many questions :] Did either of you get yours with any of the options such as 2GB RAM or eMMC? I don't see any way of choosing the extra RAM or eMMC options on the Solid-Run webstore ... did you get yours through a distributor or ... ? In the Armbian Download notes for the Clearfog Pro it refers to the eMMC, confirm we are talking about the eMMC on the SOM unit? Solid-Run's sales data seems lacking, what is the port on the back and what adapter did you use there? Can I assume this port would support something like a USB3 adapter? I see there is a USB3 port, thought it was a slot for an adapter ... I assume then this port would be for a SSD or ... ? Armbian support something like this for wifi? Intel 3160 or AC 7260 And for USB3? Renesas NEC The renesas one looks easier to run power - but is it supported? Assuming you are running legacy? Anything I should have asked and didn't? ;] This board is looking like the perfect machine for our classroom server but I want to research well before diving in. Don't need Bluetooth but WiFi and USB3 would be incredibly useful.
Igor Posted November 5, 2017 Author Posted November 5, 2017 I think it's possible to get 2GB version ... Contact solid-run. The same goes for eMMC ... I think now it is on by default, while you can order without. You have 2 x PCI and 1 x M2 2242 slot for OS and main storage. USB3 is here in place for external SSD or similar if you need that. 3 hours ago, Technicavolous said: Armbian support something like this for wifi? Intel 3160 or AC 7260 Yes, but if you want to run AP on those, you will be disappointed. 2.4G only. You need Compex WLE600VX at the minimum. I recommend exactly this one since older revisions had many troubles. If you don't need AC speed, but 300/450Mbit at 5G you can make use of these: Atheros AR9380 AR5BXB112 (put this to your local ebay.com). I add a hack to the kernel, that you can make use of them as AP at 5G. This used to be locked down since cards came from some older notebooks. 3 hours ago, Technicavolous said: Assuming you are running legacy? No. (our) Mainline is in better shape.
Technicavolous Posted November 6, 2017 Posted November 6, 2017 21 hours ago, Igor said: No. (our) Mainline is in better shape. Glad I asked! 21 hours ago, Igor said: You need Compex WLE600VX at the minimum. I recommend exactly this one OK thanks that makes a big help. 'at the minimum' - The WLE600VX looks great. Just happened across the WLE900VX, 3xMIMO any sense messing with that? Have you found any NVMe M.2 drive smaller than 2280? Would a riser / extender such as this be a problem? This would allow one to mount a 2280 somewhere else in the cabinet. SolidRun states 'some click boards supported' - do you know if something like this board or this one would be supported to display basic info like IP, temp, # users ... ? Back in the day we used LCD4Linux, I'm sure there is a mechanism in Armbian for that? On 8/20/2017 at 5:29 AM, tkaiser said: ATA hotplug works with both ASM1062 and 88SE9215 (externally powered the SSDs so I was able to switch them on/off) but I doubt it's wise to try out hotswap with the standard SATA connectors here so I chose to shutdown the board between all tests. I have this 4 bay SAS/SATA cage. I assume this should be safe for adding and removing student drives on the fly? Thanks for tolerating all my questions, this is a significant project and I want to get the right stuff the first time. I appreciate the link from the other forum to the data above. Oh, and by the way while looking for SSDs I found this, not marvel but thought it was interesting if you wanted to add yet 3 more drives ;]
Igor Posted November 6, 2017 Author Posted November 6, 2017 9 minutes ago, Technicavolous said: The WLE600VX looks great. Just happened across the WLE900VX, 3xMIMO any sense messing with that? If you have a classroom with a lot of clients, yes. I have this 512GB 2242 in my server for about a year. Still working flawlessly. Its probably the best what you can get in this form factor but it's not NvME. I am not sure that such exists ATM. I don't know if there will be any gain with NvME drive. Don't have any spare at the moment to see. I attached some generic I2C display without a problem. Same should go for SPI. No need to use exactly clickboard products. Don't have any experiences with them but they should work. It's not rocket science. Kernel has build in drivers for many small displays https://github.com/notro/fbtft/ 17 minutes ago, Technicavolous said: I have this 4 bay SAS/SATA cage. I assume this should be safe for adding and removing student drives on the fly? I would say it should work. 18 minutes ago, Technicavolous said: Oh, and by the way while looking for SSDs I found this, not marvel but thought it was interesting if you wanted to add yet 3 more drives ;] Rather use M2 slot for fast SSD
Technicavolous Posted November 6, 2017 Posted November 6, 2017 35 minutes ago, Igor said: Rather use M2 slot for fast SSD Me too thanks. I'll post when I receive the toys.
tkaiser Posted November 6, 2017 Posted November 6, 2017 8 hours ago, Technicavolous said: Oh, and by the way while looking for SSDs I found this, not marvel but thought it was interesting if you wanted to add yet 3 more drives ;] This is Marvell (since 'Hyper Duo' being mentioned) but you can use this thing only in PCIe M.2 slots. The Clearfogs have SATA by default routed to the M.2 slot there but I don't know whether the SERDES voodoo that's possible with the mPCI slots (switching between mPCIe and mSATA). But even if it would be possible to switch from SATA to PCIe you would get only one PCIe lane so really better use the M.2 slot for SATA (M.2 SSD or with a cheap mechanical adapter any affordable SATA device you can imagine). If you see M.2 slots you always have to check what's available at the connector (can be PCIe, SATA or even just USB only)
Technicavolous Posted November 6, 2017 Posted November 6, 2017 From Ilya Viten at Clearfog - "Hi Tracy, Thanks for the email, the 2GB version is available for a MOQ of 60pcs at least. By default we provide 1GB version. " 1GB will have to do for me ;] @tkaiserThe 960 EVO seems to have a good price / performance ratio, don't see any advantage to the PRO over the big price, is the EVO overkill for this application? You had specifically mentioned the 950 in an above post ... I looked at the SSD @Igor posted above but here in US that is almost as expensive. 256 is actually large enough I just want speed for this one. Again thanks, I've been pouring though all the suppliers looking for the components of this and I think, thanks to your help, I'm on my way.
tkaiser Posted November 6, 2017 Posted November 6, 2017 5 minutes ago, Technicavolous said: 960 EVO Is NVMe exposing a PCIe 3.0 x4 interface. On SBC currently -- with some RK3399 expections (soon) -- you have either SATA or PCIe 2.x x1 (that's just 10 times slower). As already said I doubt the M.2 slot on Clearfogs can be switched to PCIe so you need SATA anyway (and you don't need to buy M.2 SSDs, the cheap M.2-SATA adapters cost less than 2 bucks) and for boot support you need SATA anyway (or a bleeding edge u-boot with PCIe storage support -- most probably not existing anytime soon or ever). Any Samsung 840/850 EVO/PRO should be ok. To be able to really make use of NVMe you need a very powerful multi-core host otherwise PCIe/AHCI or 'good old' SATA will do the job...
Technicavolous Posted November 6, 2017 Posted November 6, 2017 1 minute ago, tkaiser said: To be able to really make use of NVMe you need a very powerful multi-core host That's exactly what I needed thanks.
Technicavolous Posted January 2, 2018 Posted January 2, 2018 Received an email from Solid-Run this morning, stating they had a leak in a pipe from a neighbor that shut them down for a while. They state the Clearfog Pro will be shipping end of January. I've been bugging them since November hihi. They said that Arrow has some in stock, but it appears they only have the SD version, not the eMMC. Although, looks like the SD version might be a better dev unit anyway. Just thought I'd report, I've gathered all the goodies to go with it and can't wait to build my classroom server.
Technicavolous Posted February 16, 2018 Posted February 16, 2018 On 8/30/2017 at 9:07 AM, tkaiser said: (bash history here to check test methodology) @tkaiser where do I find instructions on setting up the tests to get the results you post here? The link above gives a 500 error ...
tkaiser Posted February 16, 2018 Posted February 16, 2018 15 minutes ago, Technicavolous said: On 30.8.2017 at 3:07 PM, tkaiser said: (bash history here to check test methodology) @tkaiser where do I find instructions on setting up the tests to get the results you post here? The link above gives a 500 error ... Well, since it's not here https://web.archive.org/web/*/http://sprunge.us/WbOK/* it's gone... sprunge.us stopped working some time ago...
![m][sko](https://forum.armbian.com/uploads/n2a/n2a-180904eb9612afa23a4c3358385bd86f384613e3.avatars_letters_gd.png){kind=avatar}
Recommended Posts