jobenvil Posted June 5, 2015 Posted June 5, 2015 used your script to make rootfs on SATA device but it doesn't boot up. I just modified your script: if [[ $KERNEL == *"3.4"* ]]; then sed -e 's,root=\/dev\/mmcblk0p1,root=/dev/'"$DESTPART"',g' -i /boot/boot.cmd mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr else sed -e 's,root=\/dev\/mmcblk0p1,root=/dev/'"$DESTPART"',g' -i /boot/uEnv.txt sed -e 's,root=\/dev\/mmcblk0p1,root=/dev/'"$DESTPART"',g' -i /boot/boot.cmd if [ -f /boot/boot.cmd ]; then mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr fi fi # change fstab sed -e 's/mmcblk0p1/sda1/g' -i /mnt/etc/fstab sed -i "s/data=writeback,//" /mnt/etc/fstab mkdir -p /mnt/media/mmc echo "/dev/mmcblk0p1 /media/mmc ext4 defaults 0 0" >> /mnt/etc/fstab echo "/media/mmc/boot /boot none bind 0 0" >> /mnt/etc/fstab fi to Kernel 4.0.4 but it doesn't help as well. It is not possible for R4.0.4 to boot from sata at all?, or making more changes to the script could be possible? Thank you very much Igor. I appreciate much your work. BTW I want to share with you/other that using kernel 4.0.4 the ethernet speed (696Mbits/s) increased a lot, compared with the 3.4.107 (412Mbits/s) acting the bananapi pro like as server (-s) testing with iperf. This kernel has "something" which it works more effizient. I checked the dmseg for 4.0.4 and is free of errors. All what I need works (SATA/Eth) and it works, until now, very stable. Regards Jobenvil
Igor Posted June 5, 2015 Posted June 5, 2015 I had no problems booting even from USB drive on kernel 4.x ... with latest kernel configuration updates of course. My pre-build images are not fixed yet! (I think only OrangePi)
tkaiser Posted June 5, 2015 Posted June 5, 2015 Just as a small addendum: Many things one might attribute to "new kernel" are just side effects of different settings. Given the hardware implementation of the GMAC ethernet (everything has to be done by the CPU) clock speeds and cpufreq settings matter. So you should check what you had when running Igor's image with kernel 3.4 (I would assume interactive governor and max. 912 MHz?) and what you've now. If you didn't change anything then kernel 4.0.x with the new cpufreq stuff comes with these defaults: /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor = ondemand /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq = 960000 /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq = 144000 /sys/devices/system/cpu/cpufreq/ondemand/io_is_busy = 0 /sys/devices/system/cpu/cpufreq/ondemand/up_threshold = 95 /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor = 1 And while this might be nice on a tablet or on a machine that's mostly idling around it leads to bad throughput especially when it comes down to NAS useage. By tweaking mostly these settings as well as TCP/IP and CPU affinity settings you can double throughput in one direction and let it max out in the other: http://forum.lemaker.org/forum.php?mod=viewthread&tid=15543&fromuid=33332 Which results do you get when you try it with the following settings? echo ondemand >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor echo 960000 >/sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq echo 528000 >/sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq echo 1 > /sys/devices/system/cpu/cpufreq/ondemand/io_is_busy echo 25 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold echo 10 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor sysctl -w net/core/rmem_max=8738000 sysctl -w net/core/wmem_max=6553600 sysctl -w net/ipv4/tcp_rmem="8192 873800 8738000" sysctl -w net/ipv4/tcp_wmem="4096 655360 6553600" sysctl -w vm/min_free_kbytes=65536 sysctl -w net.ipv4.tcp_window_scaling=1 sysctl -w net.ipv4.tcp_timestamps=1 sysctl -w net.ipv4.tcp_sack=1 sysctl -w net.ipv4.tcp_no_metrics_save=1 BTW: Solely relying on iperf is wrong judging from my experiences. Real-world performance might differ. So I always double-check with a combined benchmark that checks a couple of different things including disk access (unfortunetaly we have a SATA write limitation where network throughput is faster in this direction and vice versa. When writing from client to Banana Pi disk I/O is the bottleneck and in the other direction it's network when using a SATA disk) 1
jobenvil Posted June 5, 2015 Author Posted June 5, 2015 Just as a small addendum: Many things one might attribute to "new kernel" are just side effects of different settings. echo ondemand >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor echo 960000 >/sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq echo 528000 >/sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq echo 1 > /sys/devices/system/cpu/cpufreq/ondemand/io_is_busy echo 25 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold echo 10 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor sysctl -w net/core/rmem_max=8738000 sysctl -w net/core/wmem_max=6553600 sysctl -w net/ipv4/tcp_rmem="8192 873800 8738000" sysctl -w net/ipv4/tcp_wmem="4096 655360 6553600" sysctl -w vm/min_free_kbytes=65536 sysctl -w net.ipv4.tcp_window_scaling=1 sysctl -w net.ipv4.tcp_timestamps=1 sysctl -w net.ipv4.tcp_sack=1 sysctl -w net.ipv4.tcp_no_metrics_save=1BTW: Solely relying on iperf is wrong judging from my experiences. Real-world performance might differ. So I always double-check with a combined benchmark that checks a couple of different things including disk access (unfortunetaly we have a SATA write limitation where network throughput is faster in this direction and vice versa. When writing from client to Banana Pi disk I/O is the bottleneck and in the other direction it's network when using a SATA disk) What soll I tell you more? you are absolutely right; regarding to the SATA and bottlenecks as well. I tested your settings; they are really good for iperf. TX and RX are almost the same. The values are: from Laptop-to-> Banana Pro 686Mbits/s (Bananae Server) = 85MB/s from Banana Pro-to->Laptop 568Mbits/s (Banana Client) = 71 MB/s (somehow I'm not more able to upload the screenshoot I took) May I ask you, because your many experience in testing, if it makes really/"auf jdem fall" sense (why? CPU/GMAC/acces time to SATA) to have rootfs in SATA and not in MicroSD UHS-1 (classs 10) in a banana Pi Pro? Basically should run Seafile Server/SAMBA and on the background typical LEMP Stack. I had no problems booting even from USB drive on kernel 4.x ... with latest kernel configuration updates of course. My pre-build images are not fixed yet! (I think only OrangePi) What do you mean with pre-build images? Which are already to download 3.0.107/4.0.4 (M)® in your internet page? Do you mean to apply the last Github changes (config/linux-sunxi-next.config ) and compile the kernel? Because I supposse that your (SATA) changes are for compiling, isn't it? Many thanks in advance
tkaiser Posted June 6, 2015 Posted June 6, 2015 May I ask you, because your many experience in testing, if it makes really/"auf jdem fall" sense (why? CPU/GMAC/acces time to SATA) to have rootfs in SATA and not in MicroSD UHS-1 (classs 10) in a banana Pi Pro? Basically should run Seafile Server/SAMBA and on the background typical LEMP Stack. All A20 based boards seem to be limited to approx. 16,x MB/s when reading/writing sequentially from/to SD card. What really makes a difference when you're frequently accessing files on the SD card will be random I/O. And there some SD cards outperforman others easily and sequential writes (or class xy) do not count any more. In my eyes it depends whether you use an SSD or a HDD on the SATA port. In case it's a spinning disk there might be many situations where it's better to keep the rootfs on SD card (and using Wheezy and Igor's ramlog configuration to minimize writes to SD card) to let the HDD spin down when idle after longer periods. When you're using an SSD IMO it's a no-brainer to move the rootfs to SSD. On my TODO list are experiments with F2FS on SD card (only possible using a different partition scheme than the one Igor's using now since u-boot can not directly read from F2FS at the moment). That might both decrease wear-out of the SD card and increase (random) performance. But I won't have the time to test this the next weeks/months. But if I'm done I'll report back. 1
jobenvil Posted June 6, 2015 Author Posted June 6, 2015 All A20 based boards seem to be limited to approx. 16,x MB/s when reading/writing sequentially from/to SD card. What really makes a difference when you're frequently accessing files on the SD card will be random I/O. And there some SD cards outperforman others easily and sequential writes (or class xy) do not count any more. In my eyes it depends whether you use an SSD or a HDD on the SATA port. In case it's a spinning disk there might be many situations where it's better to keep the rootfs on SD card (and using Wheezy and Igor's ramlog configuration to minimize writes to SD card) to let the HDD spin down when idle after longer periods. When you're using an SSD IMO it's a no-brainer to move the rootfs to SSD. On my TODO list are experiments with F2FS on SD card (only possible using a different partition scheme than the one Igor's using now since u-boot can not directly read from F2FS at the moment). That might both decrease wear-out of the SD card and increase (random) performance. But I won't have the time to test this the next weeks/months. But if I'm done I'll report back. Thanks for the really good technical explanation! I will post the dd and hdparm tests of microSD Kingston UHS-1 8GB that I made.
jobenvil Posted June 6, 2015 Author Posted June 6, 2015 The values differ regarding the size of the testfile. Like you wrote before, may assume that they are sequencial reading/writing.
tkaiser Posted June 7, 2015 Posted June 7, 2015 Sorry, but both dd as well as hdparm aren't benchmark tools. They're abused in this way but the results are simply crap. When you use dd with filesizes too small and without the appropriate flags ('direct' parameter) it always tests partially caches/buffers and sequential throughput seems to be higher than it is in reality. Same applies to 'hdparm -tT'. One parameter just shows RAM throughput and when used together with the other parameter a so called 'correction factor' will be applied to the results shown. In other words: The results given do NOT show what the disk in question is able to deliver. And both pseudo benchmarks only test sequential throughput. Use bonnie++ or iozone (and use always at least twice as much as RAM available for test filesizes) to get a clue how the SD card or disk in question really performs. And both tools are able to show the performance of random I/O with different block/sector sizes. 1
jobenvil Posted June 7, 2015 Author Posted June 7, 2015 Sorry, but both dd as well as hdparm aren't benchmark tools. They're abused in this way but the results are simply crap. When you use dd with filesizes too small and without the appropriate flags ('direct' parameter) it always tests partially caches/buffers and sequential throughput seems to be higher than it is in reality. Same applies to 'hdparm -tT'. One parameter just shows RAM throughput and when used together with the other parameter a so called 'correction factor' will be applied to the results shown. In other words: The results given do NOT show what the disk in question is able to deliver. And both pseudo benchmarks only test sequential throughput. Use bonnie++ or iozone (and use always at least twice as much as RAM available for test filesizes) to get a clue how the SD card or disk in question really performs. And both tools are able to show the performance of random I/O with different block/sector sizes. Thanks Kaiser, I research in Internet/Wiki for this; how accurate are the results or how the fit to the reality I don't know. I will check both programms you wrote. Thanks again. BTW, I installed the orange pi of Igor and runned the sata script. Now I have rootfs on SATA HDD, but the **wa** value of top is higher than I imagined; around 7 % even in EXT4 file system. I will perform some SAMBA /Seafile tests and probably think I will come back to the rootfs on SD Card. It is important for me to let save energy when there is no activity on SATA HDD -which should be the normal case-. I realized that the RAM (if less than 1GB) plays a nice efficient role and doesn't carry about HDD/SD.
(Phone).thumb.jpg.f60309bb32d95d32f028a6642a35c6ea.jpg)
Recommended Posts