tkaiser

Yes, RK3328, RK3399, Exynos 5244. Error messages on RK once the enclosures appear on the bus: [ 1503.960555] xhci-hcd xhci-hcd.2.auto: ERROR Transfer event for disabled endpoint or incorrect stream ring [ 1503.960574] xhci-hcd xhci-hcd.2.auto: @00000000b684e310 00000000 00000000 04000000 03038001 I wonder whether the tinymembench numbers are ok (especially memcpy) and whether we can compare with Allwinner BSP?

No FEL setup here currently. Is there an dev2mem alternative? Anyway: Just added PineH64 with 1800 MHz to the list: https://github.com/ThomasKaiser/sbc-bench/blob/master/Results.md Quick look at USB3: When attaching my EVO840 SSD in UAS capable disk enclosures (JMS567 and then ASM1153) I get these messages. [ 1890.116983] xhci-hcd xhci-hcd.0.auto: ERROR Transfer event for unknown stream ring slot 1 ep 6 [ 1890.116987] xhci-hcd xhci-hcd.0.auto: @00000000b8048580 00000000 00000000 1b000000 01078001 ... [ 2444.903488] xhci-hcd xhci-hcd.0.auto: ERROR Transfer event for unknown stream ring slot 1 ep 2 [ 2444.903493] xhci-hcd xhci-hcd.0.auto: @00000000b8048b20 00000000 00000000 1b000000 01038000 UAS storage performance as follows (forget about the ASM1153 numbers, this enclosure can not be powered externally, doesn't get enough juice my PineH64's USB3 port -- I have an early developer sample 'Model A' -- and therefore chooses a strategy to deal with underpowering --> lowering performance) EVO 840 / JMS567 random random kB reclen write rewrite read reread read write 102400 4 6090 6479 6300 6358 6403 6508 102400 16 24386 24449 24493 24570 24570 24445 102400 512 72744 72826 253654 261201 260663 72693 102400 1024 73473 73695 302101 314144 313409 74325 102400 16384 73443 77008 336140 350131 348980 76653 102400 16384 73768 77008 336151 350292 EVO 840 / ASM1153 (under)powered by PineH64 random random kB reclen write rewrite read reread read write 102400 4 6464 6907 6427 6424 6493 6828 102400 16 24434 24532 24956 25071 25039 24482 102400 512 70552 70668 87827 88720 88691 70667 102400 1024 70528 70713 92226 93154 93183 70684 102400 16384 69263 72237 96030 96996 97009 72169 102400 16384 69044 71879 96031 97134 Read performance isn't bad but write is too low. I remember @Xalius reported the same when playing with H6 already months ago. I also tested with the RTL8153 dongle that is known to saturate link speed (940 Mbits/sec). Too early since only ~200 Mbits/sec in both directions. Onboard GbE looks promising (though there's a few Mbits/sec missing in TX direction): bash-3.2# iperf3 -c 192.168.83.65 Connecting to host 192.168.83.65, port 5201 [ 4] local 192.168.83.64 port 60382 connected to 192.168.83.65 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 113 MBytes 948 Mbits/sec [ 4] 1.00-2.00 sec 112 MBytes 940 Mbits/sec [ 4] 2.00-3.00 sec 112 MBytes 940 Mbits/sec [ 4] 3.00-4.00 sec 112 MBytes 940 Mbits/sec [ 4] 4.00-5.00 sec 112 MBytes 940 Mbits/sec [ 4] 5.00-6.00 sec 112 MBytes 940 Mbits/sec [ 4] 6.00-7.00 sec 112 MBytes 940 Mbits/sec [ 4] 7.00-8.00 sec 112 MBytes 940 Mbits/sec [ 4] 8.00-9.00 sec 112 MBytes 940 Mbits/sec [ 4] 9.00-10.00 sec 112 MBytes 940 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec sender [ 4] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver iperf Done. bash-3.2# iperf3 -R -c 192.168.83.65 Connecting to host 192.168.83.65, port 5201 Reverse mode, remote host 192.168.83.65 is sending [ 4] local 192.168.83.64 port 60379 connected to 192.168.83.65 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 108 MBytes 902 Mbits/sec [ 4] 1.00-2.00 sec 108 MBytes 905 Mbits/sec [ 4] 2.00-3.00 sec 108 MBytes 905 Mbits/sec [ 4] 3.00-4.00 sec 108 MBytes 905 Mbits/sec [ 4] 4.00-5.00 sec 108 MBytes 905 Mbits/sec [ 4] 5.00-6.00 sec 108 MBytes 905 Mbits/sec [ 4] 6.00-7.00 sec 108 MBytes 905 Mbits/sec [ 4] 7.00-8.00 sec 108 MBytes 905 Mbits/sec [ 4] 8.00-9.00 sec 108 MBytes 905 Mbits/sec [ 4] 9.00-10.00 sec 108 MBytes 905 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 1.06 GBytes 906 Mbits/sec 1 sender [ 4] 0.00-10.00 sec 1.05 GBytes 905 Mbits/sec receiver iperf Done. Full debug output (including dmesg output for USB events): http://ix.io/1jEy

Already patched and benchmarking. My board seems to be fine with 1800 MHz @ 1080mV

The AX88179 isn't the best choice (driver hassles and poor performance in many situations). RTL8153 is clearly the better choice. Even the Raspberry Pi 'experts' found that out! https://lb.raspberrypi.org/forums/viewtopic.php?t=208512&start=100#p1297857 When I'm done benchmarking PineH64 I might provide some numbers made with RTL8153... Unfortunately not for the reality Armbian faces (with older kernels like RK's 4.4 that do not get such stuff backported).

Great! Just for fun: shall we throw in this patch already? Situation will change only for people who know what they do since https://github.com/armbian/build/blob/master/config/sources/sun50iw6.conf#L8 will still prevent cpufreq exceeding 900 MHz by default.

BTW: in case you re-test can you please test also with a much larger test data size? iozone -e -I -a -s 1000M -r 1024k -r 16384k -i 0 -i 1 See https://forum.armbian.com/topic/1925-some-storage-benchmarks-on-sbcs/?do=findComment&comment=51350 -- with your SSD combined with RK3399 and mainline kernel when using the 'right' benchmark even 1.6GB/s can be measured. Here ayufan's scores from RockPro64 also using the same Samsung SSD: https://gist.github.com/ayufan/30c46381c5e4e5c5264a834a752946db

At least 'sbc-bench m' (background monitoring) should always work. And yeah, on other platforms like Armada 38x cryptodev makes a huge difference wrt %usr vs. %sys. This is without cryptodev on Armada 385: Time CPU load %cpu %sys %usr %nice %io %irq Temp 12:25:41: 1332MHz 1.00 50% 0% 49% 0% 0% 0% 74.2°C 12:25:51: 1332MHz 1.00 50% 0% 50% 0% 0% 0% 75.1°C 12:26:01: 1332MHz 1.00 50% 0% 50% 0% 0% 0% 76.5°C 12:26:11: 1332MHz 1.00 50% 0% 50% 0% 0% 0% 74.2°C 12:26:21: 1332MHz 1.00 50% 0% 50% 0% 0% 0% 74.6°C 12:26:31: 1332MHz 1.00 50% 0% 50% 0% 0% 0% 78.0°C 12:26:41: 1332MHz 1.00 50% 0% 50% 0% 0% 0% 77.0°C 12:26:51: 1332MHz 1.00 50% 0% 49% 0% 0% 0% 74.6°C 12:27:01: 1332MHz 1.00 50% 0% 49% 0% 0% 0% 78.5°C 12:27:11: 1332MHz 1.00 50% 0% 50% 0% 0% 0% 77.5°C Vs. with CESA/Cryptodev: Time CPU load %cpu %sys %usr %nice %io %irq Temp 08:17:40: 1600MHz 1.07 33% 33% 0% 0% 0% 0% 54.0°C 08:17:50: 1600MHz 1.14 17% 16% 0% 0% 0% 0% 54.0°C 08:18:00: 1600MHz 1.12 31% 30% 0% 0% 0% 0% 54.0°C 08:18:10: 1600MHz 1.10 22% 21% 0% 0% 0% 0% 54.0°C 08:18:20: 1600MHz 1.16 24% 23% 0% 0% 0% 0% 54.0°C 08:18:30: 1600MHz 1.13 28% 27% 0% 0% 0% 0% 54.0°C 08:18:40: 1600MHz 1.11 20% 19% 0% 0% 0% 0% 54.0°C 08:18:50: 1600MHz 1.18 31% 30% 0% 0% 0% 0% 54.0°C 08:19:00: 1600MHz 1.15 15% 15% 0% 0% 0% 0% 54.0°C 08:19:10: 1600MHz 1.12 31% 30% 0% 0% 0% 0% 53.0°C Edit: Oh man, 3rd time in a few days that the forum software ate text when inserting code blocks. I really hate the forum engine.

As expected initialization overhead negatively affecting tiny data chunks. What would be interesting is how CPU utilization looked while executing the test. IMO the simpelst idea is to run latest version of sbc-bench. See also https://forum.armbian.com/topic/7763-benchmarking-cpus/?page=4&tab=comments#comment-59576 BTW: Why are you using an older OpenSSL version?

Thank you, latest sbc-bench version should log this better Also interesting: it's not only twice as much crypto performance but also half as much CPU utilization at the same time. When I tested on the Clearfog I had constant 50% CPU utilization (%usr) with this single threaded test while with cryptodev on your Helios4 it looks like this (all %sys): Time CPU load %cpu %sys %usr %nice %io %irq Temp 08:17:40: 1600MHz 1.07 33% 33% 0% 0% 0% 0% 54.0°C 08:17:50: 1600MHz 1.14 17% 16% 0% 0% 0% 0% 54.0°C 08:18:00: 1600MHz 1.12 31% 30% 0% 0% 0% 0% 54.0°C 08:18:10: 1600MHz 1.10 22% 21% 0% 0% 0% 0% 54.0°C 08:18:20: 1600MHz 1.16 24% 23% 0% 0% 0% 0% 54.0°C 08:18:30: 1600MHz 1.13 28% 27% 0% 0% 0% 0% 54.0°C 08:18:40: 1600MHz 1.11 20% 19% 0% 0% 0% 0% 54.0°C 08:18:50: 1600MHz 1.18 31% 30% 0% 0% 0% 0% 54.0°C 08:19:00: 1600MHz 1.15 15% 15% 0% 0% 0% 0% 54.0°C 08:19:10: 1600MHz 1.12 31% 30% 0% 0% 0% 0% 53.0°C

So you simply replaced a lib somewhere with openssl binary still being 1.1.0f? Interesting the initialization overhead with data chunks below 1KB. These were my numbers with same SoC at same clockspeed on the Clearfog Pro: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes aes-128-cbc 44566.57k 54236.33k 58027.01k 58917.21k 59400.19k 59408.38k aes-128-cbc 44378.63k 54520.66k 58114.47k 59056.47k 59225.43k 59331.93k aes-192-cbc 39354.78k 46780.01k 49606.14k 50319.02k 50328.92k 50435.41k aes-192-cbc 39281.18k 46864.04k 49414.40k 50279.08k 50536.45k 50369.88k aes-256-cbc 35510.68k 41226.05k 43100.59k 43753.13k 43950.08k 43816.28k aes-256-cbc 35009.43k 41104.00k 43218.77k 43696.81k 43909.12k 43958.27k

Since https://forum.armbian.com/topic/7310-rockpro64/?do=findComment&comment=56184 There's PCIe link training but if DT defines a conservative link speed then no higher speeds will be negotiated. Checking lspci output is mandatory in such cases (and that's the reason why I added this to our hardware logging. But I don't remember whether lspci is present on Armbian images by default or not) Edit: the kernel version and settings matter too of course. With RK's 4.4 there's /sys/module/pcie_aspm/parameters/policy which defaults to powersave.

Are you kidding? https://www.google.com/search?rls=en&q=Micro-USB_1_01.pdf This is the relevant specification for WHAT YOUR USERS HAVE AT HOME. The connector is rated for 1.8A max, 99.99% of cables are made for 500mA max. That is the REALITY OUT THERE if you throw an electronics device on the market equipped with this crappy Micro USB connector. It encourages users to use common Micro USB gear which simply WILL FAIL with the Tinkerboard since not made for higher currents than a few hundred mA. USB PD (power delivery) specs in the meantime defined NEW specifications so with both NEW receptacles and NEW cables 3A are possible with Micro USB. The USB PD specs (see here for an overview) while now defining NEW Micro USB connections (needing NEW receptacles, jacks and cables!) do of course not define any PD profile that uses 3A at 5V. Why? Since VOLTAGE DROP. Ohm's law still exists. The lower the voltage, the higher the drop. The USB PD specs define Micro USB carrying 3A with NEW Micro USB connectors and NEW cables only at 12V (Profile 3) or 20V (Profile 4). And the reason is once again called VOLTAGE DROP. If you put a Micro USB receptacle on your product to power your board you do two things at the same time: Encourage users to use the Micro USB gear that's already lying around (crappy phone chargers, average cables not suitable for more than 500mA). You introduce underpowering hassles Encourage users to make the wrong calculation since they think they won't need a new and special PSU if they already have a charger or an USB PSU lying around. You try to create the impression your product would be cheaper than it is (since always needing a SPECIAL Micro USB PSU which is additional costs) By using Micro USB board makers trick their customers into believing they could re-use existing PSUs so the board appears to be a less expensive buy compared to honestly stating that while the board uses a pretty common connector it's incompatible to 99.99% of all Micro USB gear that consumers already have. And the whole sh*t show started over at the RPi. If they would've started with a sane barrel plug this whole mess wouldn't exist. Since then those other board makers who copied the crappy Micro USB connector would now copy this barrel plug instead to advertise their '1:1 replacement for the RPi'. Since having the same moronic discussion with the RPi fanclub some while ago: https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=208192&start=50#p1292499 (just a quick check how many different Micro USB connectors at which amperage rating a company like RS sells. If you read through the whole thread please keep in mind that RPi forum moderators act as censors. They partially edited my posts and banned my account as a result of frequent mentions of competitor's products. But that's normal over there)

Well, the seller is somewhat honest, isn't he? That's from 'our' affiliate link: But I second that. I never understood why @Igor added affiliate links to crappy hardware on the download pages.

I would never buy a S912 device. Pretty underwhelming 'performance' especially with single threaded applications that land on the wrong cores and the usual Amlogic cheating: https://github.com/ThomasKaiser/sbc-bench/blob/master/Results.md#the-bigger-picture And if you buy cheap TV boxes it shouldn't be surprising that components inside are also cheap. Same with SSDs BTW

Rock64 (but their eMMC is neither super fast nor would I trust too much into it). Based on your other post the obvious choice is still Rock64 + their SATA cable + a great SSD.

mSATA defines two sizes and both fit. Each JMS578 chip on the expansion board is just like an 'external USB disk' so all the information available talking about USB disks also applies to the situation here. If your Zero has SPI NOR flash and you manage to burn u-boot into it you can load everything else from USB. But this requires you doing some research and is not automated by nand-sata-install (what a stupid name these days). If there's something I would put my swap on it would be the fastest storage available (especially wrt random IO). On those USB2 boards without superfast eMMC that's always an UAS attached SSD. Compare here with there: Close to 3,000 4K IOPS with a good SSD behind a good USB-to-SATA bridge (like JMS578) behind an Allwinner USB2 port A SanDisk Extreme A1 SD card behind a standard (bottlenecked) DDR50 SDIO controller only scores ~1000/~2500 4K IOPS. It would need a better SDIO interface mode for better performance but that's nothing any of those cheap Allwinner boards provides On boards with super fast eMMC this is also an option. Recent Samsung 5.1 eMMC provides +7000 4K IOPS (see ODROID-N1 numbers -- you have to divide numbers through block size to translate from KB/s to IOPS) If you avoid cheap crap SSDs it's 100% save to use them since unlike SD cards or USB pendrives you can query them about their health and replace them before they finally die. The magic word is SMART: https://forum.openmediavault.org/index.php/Thread/23724-Run-OS-and-database-on-SD-card-to-prevent-HDD-spin-up/?postID=180003#post180003

If you boot off the stick already then it's as easy as cd $HOME cpufreq-set -g performance iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 I bought 3 different SanDisk pendrives last year (all USB3) and they showed nice sequential performance that dropped after few minutes of use since the things started to throttle down. And with random IO (which is the important metrics on SBC or computers in general) they all sucked horribly after some time. I then tested on macOS putting some virtual machines on the fastest stick and attached it to an USB3 port of a MacBook Pro. Unusable since all my 3 sticks are simply overwhelmed by increased random IO workloads.

I did a quick check with some other sysfs node containing a colon and '/bin/bash -x armbianmonitor -m' execution mode. Seems like removing the single quotes should do the trick?

Seems so. And it seems like it's a common convention to locate those ADC sysfs nodes at this path. See 'Command to read ADC values using sysfs' in https://www.cnx-software.com/2018/08/06/aio-3399j-development-board-review-ubuntu-16-04/

Maybe needs to be used as '/sys/bus/iio/devices/iio\:device0/in_voltage2_raw' in bash scripts? [ -f /sys/bus/iio/devices/iio\:device0/in_voltage2_raw ] && echo "Exists"

I just checked also against RK3399 with A72 cores killed. Below 7-zip MIPS and cpuminer kH/s (all binaries built with GCC 6.3): RK3328: Renegade @ 1380 MHz: 3710, 3.92 kH/s (faster memory than Rock64) Rock64 @ 1380 MHz: 3610, 3.85 kH/s RK3399 (with A72 cores killed): NanoPC T4 @ 1415 MHz: 3920, 4.54 kH/s (way faster memory than the other boards) S905: NanoPi K2 @ 1480 MHz: 3850, 4.61 kH/s (slightly higher cpufreq) ODROID-C2 @ 1530 MHz: 3870, 4.63 kH/s (slightly higher cpufreq) S905X: Le Potato @ 1410 MHz: 3780, 3.92 kH/s (faster memory than Rock64) Unfortunately testing on NanoPi Fire3 with just 4 CPU cores is not possible since when trying to kill CPU cores the system deadlocks. Might need changed kernel config? root@nanopifire3:~# zgrep -i hotplug /proc/config.gz CONFIG_HOTPLUG_CPU=y # CONFIG_ARM_DYNAMIC_CLUSTER_HOTPLUG is not set # CONFIG_CPU_HOTPLUG_STATE_CONTROL is not set cpuminer allows to set CPU affinity but this doesn't work since still firing up 8 threads and then results are lower as expected (3.24kH/s): taskset -c 0-3 ./cpuminer --benchmark --cpu-priority=2 --cpu-affinity=0x15

Hmm... S905: L1: '32KB instruction cache and 32KB data cache' + '512KB Unified L2 cache' S905X: L1: 32KB I/D-Cache + 'large unified L2 cache' Well, 32KB for each or for both and 512KB vs. 'large'. If something that has been called '512KB' in the past now is called 'large' it's safe to assume that 'large' is less than 512KB? Just as a reference: A64: L1: '32KB L1 Instruction cache and 32KB L1 Data cache' + '512KB L2 cache' RK3328: L1 I cache size: 32K, L1 D cache size: 32K, L2 cache size: 256K RK3399: 'Integrated 48KB L1 instruction cache, 32KB L1 data cache for each Cortex-A72', 'Integrated 32KB L1 instruction cache, 32KB L1 data cache for each Cortex-A53' and '1024KB unified L2 Cache for big cluster, 512KB unified L2 Cache for little cluster' (Rockchip_RK3399TRM_V1.4_Part1-20170408.pdf is even more verbose) S5P6818: 'L1: '32 Kbyte I-Cache, 32 Kbyte D-Cache' + L2: '1 Mbyte Shared Cache'

@TonyMac32 can you please check whether this commit (+ fix) is sufficient? (Still wondering why it's not just 'val2_raw / 154.31567328918322295376 + 0.1'?

Care to provide numbers from a quick iozone benchmark as described here: https://forum.armbian.com/topic/954-sd-card-performance/? I tested some 'quality' pendrives last year and was highly disappointed about the performance after some time or usage. Nice performance when starting to use them but after some time or amounts of write performance really started to suck. The only 'pendrive' working flawlessly so far looks like this for me: A real M.2 SATA SSD with heatsinks to prevent overheating/throttling on a JMS578 adapter. Without heatsink and with enclosure closed also unusable since the SSD overheats too much (sequential transfers then drop from 400 MB/s to ~30 MB/s) Why should everybody have the same needs? Users might want to attach a HDD to the USB3 port then 'booting from USB3' is not an option any more as long as you want the HDD to spin down. Putting an USB hub between host and disk is always a great idea to introduce problems.

ASUS is not here. You might ask them where they are (no idea) and most probably they tell you to use a PSU with higher amperage ratings since it's so much fun to use this Micro USB crap in the first place and then fool all the unfortunate users suffering from the usual voltage drop problems afterwards (at least that's what the RPi clowns do who invented the 'great' idea to use this shitty connector for DC-IN).