tkaiser

Can you please provide output from 'armbianmonitor -u' first?

Huh? Your 'observations' are based on a SinoVoip product (a board maker giving a sh*t about heat dissipation and settings -- the 'start to throttle at 47"C settings' are not 'conservative' but simply silly or most probably as everything relating to this vendor 'copy&paste gone wrong'). Why should this be of any relevance for any other board around? With USB (even with UAS) the amount of interrupts to be processed is higher compared to SATA, see my RK3399 tests with RK's 4.4 kernel some while ago: https://forum.armbian.com/topic/6496-odroid-n1-not-a-review-yet/?do=findComment&comment=49627 This results in slightly higher CPU utilization which results in slightly more heat generated. I tested with a fast SSD and here it's about boring/slow HDDs. Why should this little extra efforts with USB3 matter as long as there are no cable/connection issues? Still I wonder why nobody is 'designing a RK3399 'NAS board' with a JMS561 attached to each USB3 port to provide 4 SATA ports for spinning rust and a M.2 key M slot for a fast NVMe SSD or alternatively something like this to provide another 4 SATA ports'

Sorry, MT7623 is a bit off-topic here. And I really don't know whether you're talking about temperatures or silly DT settings (probably leftovers from here). Anyway: with USB attached SATA the amount of interrupts to be processed is higher compared to native or PCIe attached SATA with same performance. So CPU utilization is higher too. But when throttling occurs caused by a simple disk benchmark then there's something wrong (e.g. defining 47°C as first thermal trip point)

Usually you do not want an USB hub between host and disk, especially if disk accesses happen in parallel. Since all RPi just have one single USB2 port there's always one (internal) hub involved if you're not using an RPi Zero. The RPi Trading 'geniuses' when mixing ingredients for their latest update of the platform chose a combined Ethernet/hub thingy (LAN7515) that does not only suck when it's about networking but also contains not just one but two internal hubs in a cascaded fashion. So with your external hub you might already have 3 cascaded hubs between host port and disks. Terrible. Also RPi 3 and 3+ are one of the few 64-bit capable SBC that do not implement ARMv8 Crypto Extensions and therefore suck horribly when it's about AES performance: https://github.com/ThomasKaiser/sbc-bench/blob/master/Results.md While I personally consider mdraid-1 just a weird waste of disks obviously you want to access your both disks concurrently and fast. As already mentioned: combining any cheap USB3 equipped SBC like Rock64 with the JMS561 thing will work fine with HDDs or you could use an EspressoBin with one disk connected to the SATA and the other to the USB3 port. Or using an PCIe attached SATA controller on those PCIe capable boards. Just a matter of money you want to spend on the setup. Since all HDD suck horribly at random IO I personally would also consider USB3 combined with good UAS capable SATA bridges since... fast enough. But USB3 when you have to use the USB3-A connector can be a real sh*t show since connection issues are quite common with this crappy connector. There was a RK3399 board announced a while ago called Rock960 Pro with an JMS561 on the PCB providing 2 SATA ports suitable for HDDs but no idea what has happened (@hipboi asked me whether I want a developer sample some time ago but we did not succeed for whatever reasons)

Not the best description of your use case. We have servers based on Olimex Lime2 with one 2.5" disk connected to the SATA port and another connected to one USB2 port (and powered by both). The latter is the backup disk and since Olimex boards are designed well we can power it off completely when not used (using sunxi-pio utility) USB3 attached SATA is often faster than 'native' SATA or even PCIe attached SATA: https://forum.armbian.com/topic/1925-some-storage-benchmarks-on-sbcs/?do=findComment&comment=51350 An alternative to more than one SATA port can be port multipliers. Cheap SATA PMs like JMB321 have the disadvantage that they only support CBS but not FIS-based switching (slowing down concurrent accesses), better ones like JMB575 are usually pretty expensive. But if you search the forum for these two strings (JMB321/JMB575 you get some insights, interesting to combine those with EspressoBin). And then there also exist such 2-port SATA PMs integrated in an UAS capable USB-to-SATA bridge, e.g. JMS561: https://forum.openmediavault.org/index.php/Thread/19871-Which-energy-efficient-ARM-platform-to-choose/?postID=169303#post169303 Such a thing attached to an USB3 port is fast enough for two 2.5" HDD but almost all SSDs will be bottlenecked. Maybe you elaborate a bit on your use case first...

Hmm... IMO all you need is VDD_5V and GND (available at various locations) and cabling/contacts that are sufficient for 1A peak consumption. I've never seen a 2.5" HDD needing 3.3V (only some server disks also wanting 12V) so you should be fine using any standard SATA power cable connected to 5V/GND only.

For anyone stumbling accross these numbers: They're useless. It's about the SoC and how CPU and memory clockspeeds are controlled. An RPi 3 B+ might reach 3600 7-zip MIPS (cpufreq/DVFS is controlled by a closed source BLOB so we can't exceed these numbers). An RK3399 device with default settings (1.8GHz big cores, 1.4GHz little cores), LPDDR with default DRAM initialization BLOB will score ~6300 7-zip MIPS with approriate cooling. Since RK3399 is not a closed source platform like the RPi we can also increase clockspeeds and then obviously performance increases too (of course then even more appropriate cooling being important). https://github.com/ThomasKaiser/sbc-bench/blob/master/Results.md (or see here for a nicely overclocked Firefly RK3399) Useless since pv uses a buffer size too small. I would assume with large blocksizes the 16 GB eMMC is capable of up to 300 MB/s read speeds as the other RK3399 boards using same Samsung eMMC. Sysbench is totally useless since not able to benchmark hardware. It's only a compiler benchmark:

There's no 5V on the mSATA connector (schematics) so you need to get them from somewhere else anyway. See EspressoBin and Helios4 (or do a search for 'male molex site:forum.armbian.com')

Such a SATA controller like this doesn't handle powering at all. I would most probably choose a 12V PSU to be combined with a buck converter (or plain 5V if it's about 2.5" HDDs).

Sure: Cheap SSDs are the same crap as cheap eMMC. If you care about your data you need SSDs that expose their health via SMART (so don't buy cheap SSD crap but only from those vendors providing a SMART attribute you can use as wear out indicator). I would choose an JMS578 thingy combined with a good SSD (be it mSATA, M.2 or normal SATA). More info: https://forum.openmediavault.org/index.php/Thread/23724-Run-OS-and-database-on-SD-card-to-prevent-HDD-spin-up/

Are broken? So you tested them all? If it's about Stretch next: problem known:

I updated my EsbressoBin with flash-image-1g-2cs-1000_800_boot_sd_and_usb.bin from https://dl.armbian.com/espressobin/u-boot/ (with flash-image-1g-2cs-1200_750_boot_sd_and_usb.bin I got ext4 errors). Quick sbc-bench test revealed that the CPU cores aren't running at 1 GHz but just 800 MHz and that thermal readouts are broken: http://ix.io/1kt2 Storage performance test with a SATA connected EVO 840 and performance governor while running at 800 MHz: random random kB reclen write rewrite read reread read write 102400 4 29006 67436 34313 35602 33972 64807 102400 16 94899 157206 111199 112478 112152 112260 102400 512 355983 392732 344499 347130 346680 343875 102400 1024 370437 386847 358432 353261 360746 365497 102400 16384 282429 379475 434285 439253 439033 384619 1024000 1024 401822 402875 360320 360639 1024000 16384 388697 402818 439262 439727 Edit: This morning ext4 errors are back: [32076.089783] EXT4-fs error (device mmcblk1p1): ext4_find_entry:1437: inode #3042: comm gmain: reading directory lblock 0 [32080.090124] EXT4-fs error (device mmcblk1p1): ext4_find_entry:1437: inode #3042: comm gmain: reading directory lblock 0 [32084.089476] EXT4-fs error (device mmcblk1p1): ext4_find_entry:1437: inode #3042: comm gmain: reading directory lblock 0 [32088.089836] EXT4-fs error (device mmcblk1p1): ext4_find_entry:1437: inode #3042: comm gmain: reading directory lblock 0 [32092.089191] EXT4-fs error (device mmcblk1p1): ext4_find_entry:1437: inode #3042: comm gmain: reading directory lblock 0 [32096.089549] EXT4-fs error (device mmcblk1p1): ext4_find_entry:1437: inode #3042: comm gmain: reading directory lblock 0 [32100.088931] EXT4-fs error (device mmcblk1p1): ext4_find_entry:1437: inode #3042: comm gmain: reading directory lblock 0 @Igor: Downloading Debian_stretch_next.7z from https://dl.armbian.com/espressobin/ the contents are as follows (obviously something missing?): macbookpro-tk:~ tk$ ls -la /Users/tk/Downloads/Debian_stretch_next/ total 2261040 drwxr-xr-x 5 tk staff 170 17 Aug 22:14 . drwxr-xr-x 5 tk staff 170 17 Aug 22:14 .. -rw-r--r--@ 1 tk staff 1157627904 24 Jul 20:24 Armbian_5.54_Espressobin_Debian_stretch_next_4.17.9.img -rw-r--r--@ 1 tk staff 18577 24 Jul 20:24 armbian.txt -rw-r--r--@ 1 tk staff 122 24 Jul 20:24 sha256sum.sha

I downloaded the tarball and extracted the relevant stuff: http://kaiser-edv.de/tmp/jk4NM5/rk3399-orangepi.dts.tgz Back in March Xunlong said they have no RK3399 dev samples to send out. No idea whether this has changed. But whoever is interested in getting Armbian on this board needs at least the contained 3 files: -rw-r--r--@ 1 tk staff 5777 19 Dez 2017 rk3399-orangepi.dts -rw-r--r--@ 1 tk staff 19811 2 Nov 2017 rk3399-sdram-lpddr3-4GB-1600.dtsi -rw-r--r--@ 1 tk staff 53378 2 Nov 2017 rk3399.dtsi

So in the end we discussed a lot and then @Igor decided to use another kernel branch: https://github.com/armbian/build/blob/6d82a8974872ee261006d2cdf6726b7d13df5032/config/sources/rk3399.conf#L34

Your board crashed, log2ram was not able to write /var/log contents back to disk and an 'mkdir -p /var/log/samba ; reboot' as root will most probably fix it.

To create awareness that the amount of data to test with is important and to outline that initialization overhead is something to consider. The whole approach is not to generate graphs and numbers to compare without using the brain but to generate insights instead.

A better question is why this motd stuff needs to be broken? If the goal of Igor's motd is to display information then simply reading a random file /etc/armbian-release not containing information but just a bogus number is obviously not the correct way. A more sane way might be to parse 'dpkg -l | grep "^armbian"' output and report the highest package number. But why such a waste of time at every login? I don't like this motd stuff delaying logins anyway, I also don't like creating confusion and misinformation but weighing both I gave up dealing with this motd stuff after trying to fix most severe performance issues some time ago.

Did you prefix the iozone call with 'taskset -c 4-5 '?

Buying a fast board to operate it slow is an option? Really? I would cap max frequency of the big cores from 2.0 GHz to 1.8 GHz or maybe 1.6GHz (/etc/default/cpufrequtils) and then check again. The upper DVFS operating points have to use a much higher voltage to get CPU cores working stable so the amount of heat the SoC dissipates at the top clockspeeds is much higher than slightly below. I would prefer a 5% or 10% performance drop instead of running slow all the time. And yeah, the fansink is a joke since not sufficient to provide appropriate cooling. With demanding workloads throttling will always occur (you can't run anything really heavy at 2.0 GHz with the fansink) and the sound is annoying. BTW: To diagnose and optimize behaviour running 'armbianmonitor -m' in a shell is always a good idea. In case the CPU cores run at high clockspeeds with ondemand even when idle this might be another occurence of 'sampling_rate' being inapppropriate with recent kernels.

Don't think so and irqbalanced is broken on ARM anyway

That's only Hi-Speed but not SuperSpeed: Bus 007 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 006 Device 004: ID 174c:1153 ASMedia Technology Inc. ASM2115 SATA 6Gb/s bridge ... /: Bus 07.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M /: Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M |__ Port 1: Dev 4, If 0, Class=Mass Storage, Driver=usb-storage, 480M But nice to see that we get +41MB/s with UAS Mass Storage protocol and "USB2" (correctly called Hi-Speed) Edit: It's not even UAS that gets negotiated.

So you can neither power through GPIO any more nor use their own 'special' PSU since this thing is a 5.0V PSU and not a 5.25V as required?! What do these guys think? The problem is known since day one! Skip the entire review and just look at 2:19: https://www.youtube.com/watch?v=y4EIhh0VT5g#t=2m19s Edit: @TonyMac32 updated his results and so still powering through GPIO header is the way to go with Tinkerboard:

Next week. Now on a business trip. Thank you for all your support!

I benchmarked the PineH64 now at 1800 MHz: https://github.com/ThomasKaiser/sbc-bench/commit/2f4dfe6a21db48bc130fc3801b78caf4412bb048 Are you also running Allwinner's BSP on H6 boards? Would be pretty interesting to get tinymembench results with BSP DRAM initialization and kernel (see end of the thread)

Doesn't do the trick -- still below 80MB/s: root@pineh64:/mnt/sda# /usr/local/src/devmemX/devmem2 0x03001510 w 0x03000002 root@pineh64:/mnt/sda# iozone -e -I -a -s 1000M -r 16384k -i 0 -i 1 | grep 102400 File size set to 1024000 kB 1024000 16384 77356 77853 348247 349195