Jump to content

tkaiser

Members
  • Posts

    5455
  • Joined

Profile Information

  • Gender
    Not Telling

Recent Profile Visitors

28523 profile views
  1. Why? What are the benefits over the Radxa provided OS images?
  2. Today I had the chance to retest again. The good news: there's no real difference whether ASPM is set to default or performance. The bad news is that the idle consumption difference is quite higher. With dmc governor enabled (and therefore idling at 528 MHz DRAM clock) the difference between powersave and default/performance is almost 250 mW, at the highest DRAM clock it's around 150mW (back then when measuring the consumption difference there was no dmc governor and most probably the 100mW I was talking about were the result of rounding down). More details here: https://forum.radxa.com/t/rock-5b-debug-party-invitation/10483/508?u=tkaiser From an energy efficiency point of view a script line in armbian-hardware-optimization checking for PCIe devices being present at boot, then setting either powersave or performance would be perfect but as I recently learned Armbian isn't about such optimisations any more
  3. https://github.com/ThomasKaiser/Knowledge/blob/master/articles/Quick_Preview_of_ROCK_5B.md#important-insights-and-suggested-optimisations BTW: The device is called Rock 5B as such congratulations on the subforum name...
  4. All of this (maybe except USB PD negotiations) applies to M3 as well: https://github.com/ThomasKaiser/Knowledge/blob/master/articles/Quick_Preview_of_ROCK_5B.md#important-insights-and-suggested-optimisations
  5. https://forum.radxa.com/t/rock-5b-debug-party-invitation/10483/472?u=tkaiser Latest sbc-bench version 0.9.9 has also an additional service for ignorant people: reporting probably performance relevant governors...
  6. Well, I was talking about 3 different issues: switching with ASPM from powersave to default (you chose performance now, no idea what this does wrt consumption, switching from powersave to defailt is 100mW) io_is_busy (that's addressed now though in a redundant way but doesn't matter since only delaying the armbian-hardware-optimization service by a few ms). Both these changes should help with NVMe (and PCIe device) performance in general setting the dmc_governor to performance instead of dmc_ondemand (results in 600mW higher consumption though). This is reponsible for better overall performance since the dmc_ondemand governor often does not ramp up memory clock fast enough. A comparison between clocking LPDDR4 with only 528 MHz vs. the current maximum of 2112 MHz is here: https://browser.geekbench.com/v5/cpu/compare/17009700?baseline=17009078 The latter isn't addressed at all but I posted multiple times how this can be changed from userspace: echo performance >/sys/class/devfreq/dmc/governor @balbes150 did (only) two changes and the io_is_busy tweak will help with every storage device (USB and SATA too). No need for a new image since you can simply edit /usr/lib/armbian/armbian-hardware-optimization. Or overwrite it with the contests of http://ix.io/4aFe
  7. It's not about mainline kernel vs. BSP kernel but the latter having settings that might fit for Android (use cases like watching video, playing games) but not for Linux. You seem to be using these defaults without questioning them. Rockchip's BSP kernel defaults to powersave for ASPM (Active State Power Management) which of course negatively affects NVMe performance. As such you need to either eliminate CONFIG_PCIEASPM_POWERSAVE=y from kernel config or need to execute somewhere after booting: echo default >/sys/module/pcie_aspm/parameters/policy Also the BSP kernel when the dmc/dfi device-tree nodes are enabled (seems to be the case with your RK3588 kernel fork since the 7-zip scores you and @blondu are sharing are below 14800 while they could be around 16500) defaults to dmc_ondemand governor which can be changed by doing this: echo performance >/sys/class/devfreq/dmc/governor (similar way as I've done this for RK3399 years ago: https://github.com/armbian/build/blob/fdf73a025ba56124523baefaf705792b74170fb8/packages/bsp/common/usr/lib/armbian/armbian-hardware-optimization#L241-L244 ) And this here: prefix="/sys/devices/system/cpu" CPUFreqPolicies=($(ls -d ${prefix}/cpufreq/policy? | sed 's/freq\/policy//')) if [ ${#CPUFreqPolicies[@]} -eq 1 -a -d "${prefix}/cpufreq" ]; then # if there's just a single cpufreq policy ondemand sysfs entries differ CPUFreqPolicies=${prefix} fi for i in ${CPUFreqPolicies[@]}; do affected_cpu=$(tr -d -c '[:digit:]' <<< ${i}) echo ondemand >${prefix}/cpu${affected_cpu:-0}/cpufreq/scaling_governor echo 1 >${i}/cpufreq/ondemand/io_is_busy echo 25 >${i}/cpufreq/ondemand/up_threshold echo 10 >${i}/cpufreq/ondemand/sampling_down_factor echo 200000 >${i}/cpufreq/ondemand/sampling_rate done is the exact replacement for lines 81-89 in armbian-hardware-optimization: https://github.com/armbian/build/blob/fdf73a025ba56124523baefaf705792b74170fb8/packages/bsp/common/usr/lib/armbian/armbian-hardware-optimization#L81-L89 Without this I/O performance with Armbian sucks on a variety of boards for example ODROID N2/N2+, VIM3 or now the RK3588/RK3588S based boards. Unfortunately @lanefu seems to be way too biased or limited in his thinking to understand this when he creates bizarre tickets that are rotting around somewhere: https://armbian.atlassian.net/browse/AR-1262 I really don't care whether these fixes will be incorporated into Armbian. But if you benchmark stuff the settings should be adjusted accordingly. And we (we as a broader community – not this place here) already know how ASPM settings negatively affect performance of PCIe devices (like for example NVMe SSDs): https://forum.radxa.com/t/rock-5b-debug-party-invitation/10483/86?u=tkaiser, we know which role io_is_busy has and what the benefits and drawbacks of the chosen dmc governor are. A quality NVMe SSD even when just connected with a single Gen2 lane should always outperform any SATA SSD if it's about what really matters: random I/O. If the SSD is cheap garbage or the settings are garbage it might look differently.
  8. But your image or lets better say the kernel you're using is made for Android and lacks optimisations. As for NVMe what about echo default >/sys/module/pcie_aspm/parameters/policy or removing CONFIG_PCIEASPM_POWERSAVE=y from kernel config? And for I/O performance in general this would be needed since with my code fragments from half a decade ago that are still part of Armbian the 3rd CPU cluster isn't adjusted properly: prefix="/sys/devices/system/cpu" CPUFreqPolicies=($(ls -d ${prefix}/cpufreq/policy? | sed 's/freq\/policy//')) if [ ${#CPUFreqPolicies[@]} -eq 1 -a -d "${prefix}/cpufreq" ]; then # if there's just a single cpufreq policy ondemand sysfs entries differ CPUFreqPolicies=${prefix} fi for i in ${CPUFreqPolicies[@]}; do affected_cpu=$(tr -d -c '[:digit:]' <<< ${i}) echo ondemand >${prefix}/cpu${affected_cpu:-0}/cpufreq/scaling_governor echo 1 >${i}/cpufreq/ondemand/io_is_busy echo 25 >${i}/cpufreq/ondemand/up_threshold echo 10 >${i}/cpufreq/ondemand/sampling_down_factor echo 200000 >${i}/cpufreq/ondemand/sampling_rate done And based on sbc-bench results shared here the dmc/dfi nodes are enabled in device-tree (defaulting to dmc_ondemand) and as such this would restore 'full performance': echo performance >/sys/class/devfreq/dmc/governor
  9. How did you perform this comparison? Looking only at sequential transfer speeds? Or checking random I/O (which matters way more on an OS drive or when building software)? What does /sys/module/pcie_aspm/parameters/policy look like? powersave, right? And for a quick comparison of schedutil, performance, ondemand and ondemand with io_is_busy see the comments below https://github.com/radxa/kernel/commit/55f540ce97a3d19330abea8a0afc0052ab2644ef
  10. Really low numbers. 1) hdparm uses 128K block size to test which was a lot last century when whoever hardcoded this in hdparm but today it's just a joke. Use iozone or fio with larger block sizes 2) Armbian doesn't care since years about low-level optimizations. Better search for 'io_is_busy' and 'dmc/governor' to get full speed (both storage and CPU performance or at least a 2000 higher 7-zip score): https://github.com/ThomasKaiser/Knowledge/blob/master/articles/Quick_Preview_of_ROCK_5B.md 3) check 'cat /sys/devices/system/cpu/cpufreq/policy?/ondemand/io_is_busy' – if I/O is processed by cpu6 or cpu7 it might be a lot slower compared even to the little cores 4) A PCIe Gen2 lane allows for 5 GT/s, SATA 6Gbps has 120% the data rate and both use 8b10b coding. As such SATA should be faster with sequential transfer speeds 5) Sequential transfer speeds are BS if it's about an OS drive since here random I/O matters. And due to SATA relying on AHCI made for spinning rust last century NVMe should outperform SATA even in a single Gen2 lane config and even if silly 'benchmarks' like hdparm draw a different picture. 6) What is 'cat /sys/module/pcie_aspm/parameters/policy' telling?
  11. OMFG. You're so busy 'reviewing' stuff that you have no time becoming familiar with the basics you babble about.
  12. Who's talking about system administrators? It's about Armbian users now encouraged to rely on ZFS based on proposals at the top post in this thread and eg. support for ZFS in Armbian's bloated motd output. Are these users aware of the two main problems putting their ZFS pools or the data inside at risk? Are they aware of the 'correct write barrier semantics' issue especially with crappy storage hardware as it's definitely more rule than exception with SBC (some random USB-to-SATA bridge combined with freezes or power losses might be recipe for disaster with both ZFS and btrfs)? Are they aware that if they install ZFS via DKMS they might end up with something like this (a kernel update will be rushed out for certain boards, building zfs or spl modules will fail in certain combinations and affected users will realize only after next reboot for the kernel update to take effect that their pools are unavailable). How much testing does each and every kernel update receive? You know the answer just like me. And it's easy to understand the difference with Ubuntu or Debian on amd64 where there are only just a few kernel variants that can be tested through with reasonable efforts) And then there's another issue called 'confusing redundancy with backup' which affects an awful lot of home users. They think if they do some sort of RAID their data would be save. In situations where crappy storage hardware with old filesystems will just result in some unnoticed silent data corruption using advanced attempts like ZFS or btrfs can result in the whole pool/filesystem becoming inaccessible. While professionals then restore from backup the average home users will just cry. If users are encouraged to rely on a certain FS implementation based on FUD (data losses with btrfs) or theoretical superiority '(the guys at Lawrence Livermore National Laboratory' for example) that's a bit unfair since they're affected by practical limitations of the hardware Armbian is mostly used on and the practical limitations of another group of guys. Btrfs on the other hand is built right into the kernel and receives a lot more testing than the ZFS situation at Armbian allows for (dealing already with +30 kernel branches and complaining about having no resources constantly since years). BTW: thank you, I edited my post above and exchanged ARM with Armbian.
  13. Guess why you hear so often about data loss with btrfs? Three main reasons: 1) Shooting the messenger: Putting a btrfs on top of hardware raid, mdraid or lvm is as 'great' as doing the same with ZFS but 'Linux experts' and NAS vendors did it and still do it. Data corruption at the mdraid layer resulting in a broken btrfs on top and guess who's blamed then? https://github.com/openmediavault/openmediavault/issues/101#issuecomment-473920806 2) btrfs lives inside the kernel and as such you need a recent kernel version to escape old and well known bugs. Now look at kernel versions in popular distros like Debian and you get the problem. With ZFS that's different, most recent ZoL versions still run well with horribly outdated kernels. Hobbyists look into btrfs wiki, see that a certain issue has been fixed recently so they're gonna use it forgetting that the 'everything outdated as hell' distro (AKA Debian) they're using is still on an ancient 4.9 kernel or even worse. 3) Choice of hardware. ZFS is most of the time used on more reliable hardware than btrfs and that's mostly due to some BS spread by selected FreeNAS/TrueNAS forum members who created the two urban myths that 'running ZFS without ECC RAM will kill your data' (nope) and 'you'll need at least 8GB of RAM for ZFS' (also BS, I'm running lots of VMs with ZFS with as less as 1GB). While both myths are technically irrelevant a lot of people believe into and therefore invest in better hardware. Check which mainboards feature ECC memory and you'll realize that exactly those mainboards are the ones with proper onboard storage controllers. If you believe you need at least 8 GB of RAM this also rules out a lot of crappy hardware (like eg. vast majority of SBCs). The main technical challenge for majority of modern filesystem attempts (even for journaled ext4) is correct write-barrier semantics (more on this in the referenced github issue above). Without those both ZFS and btrfs will fail in the same way. So using an SBC with flaky USB host controller combined with crappy USB-to-SATA controller in external enclosure is recipe for desaster with both ZFS and btrfs. ZFS is more mature than btrfs but guess what has happened when ZFS was rather new over a decade ago: people lost their pools due to broken write-barrier semantics caused by erratic HBAs and drive cache behavior (reporting back 'yeah, I've written the sector to spinning rust' while keeping data in some internal cache and after a power loss the pool was gone). The last time I wasted time with ZFS on ARM it was a true desaster. Over a year ago I tried to run several ODROID HC2 as Znapzend targets for a bunch of fileservers but always ran into stability issues (freezes) after a couple of days/weeks. Switched to btrfs just to check whether all the HC2 might have a hardware problem: a year later realized by accident that the HC2's in question all had an uptime of 300 days or above (I stopped doing kernel updates on isolated Armbian machines since Igor bricked already way too much of my boards over the past years). Do I prefer btrfs over ZFS? Nope. Being a real ZFS fanboi almost all the storage implementations here and at customers are based on ZFS. On x86 hardware due to not only relying on OpenZFS team but also the guys at Canonical who care about 'ZFS correctly built and fully working with the kernel in question' (on Debian installations pulling in the Proxmox kernel which is latest Ubuntu kernel with additional tweaks/fixes). And that's the main difference compared to the situation with Armbian. There is no such team that takes care about you being able to access your ZFS pools flawlessly after the next kernel update. Which is the reason why the choice of filesystem (btrfs or ZFS) at my place depends on x86 (ZFS) or ARM (btrfs). And that's not because of the CPU architecture but solely due to with x86 being able to rely on professionals who take care about the updates they roll out vs. situation on ARM with a team of hobbyists who still try harder and harder to provide OS images for as much different SBC as possible so it's absolutely impossible to properly test/support all of them. Playground mode.
  14. The BPi M5 is just an ODROID C4 clone (check also the serial console output in the wall of text above) so simply look there. They even took the same VL817 hub (a way better choice than the Genesis Logic thingy on the N2+) but of course decided to change the power circuitry so that their users suffer from the usual Banana undervoltage drama (unlike the C4 which is up to the task powering a number of USB3 consumers with stable 5V unlike the M5 here).
  15. https://github.com/armbian/build/commit/1a04b50674626cf0165c84ef463c2b9e3df07061#commitcomment-40258499
×
×
  • Create New...