pfeerick Posted September 6, 2016 Posted September 6, 2016 Many thanks for that info tkasier... I'll certainly put that to the test after I've done the others, as it will be interesting to seek how much more I can elk out of the board. As things stand, I can at least evidence for newbies that you can't complain about performance if you switch over to a sanely configured distro... (shameless Armbian plug)... it will improve significantly. And this is the out of the box experience, so it ain't rocket science! So, next thing too do is to push it to the max... and see if we can get this puppy purring like a kitten... or some spontaneous combustion perhaps? Sweet... so it is theoretically possible to get the pine64 into some decent throughput speeds... assuming we don't find some other bottleneck to bash into! 0 Quote
tkaiser Posted September 7, 2016 Author Posted September 7, 2016 As things stand, I can at least evidence for newbies that you can't complain about performance if you switch over to a sanely configured distro... Please keep in mind what I've written in the first post: "User experience will not be much different compared to longsleep's minimal Ubuntu image. If you prefer Debian then at least you can be assured that our images do not contain bad settings and silly bugs like the one's from official Pine64 downloads section" and in post #28: "Pine64 folks decided for whatever reasons to not feature good OS images but to use community's work and convert it to crap. The OS images they put online did not contain the already available bug fixes or new features, they used wrong settings and added even more bugs" So you might want to test also using longsleep's minimal Ubuntu image for a basic performance test too (applying u-boot/kernel updates but without any tweaks and stock Samba config but maybe ext4 instead of any of the 'foreign' FS like FAT32 or exFAT? Or is the average Pine64 user doing stuff like that?). I would expect performance numbers to be on a par since the most important choice is the cpufreq governor which is the same for obvious reasons: interactive. 0 Quote
tkaiser Posted September 8, 2016 Author Posted September 8, 2016 @androsch's Pine64 arrived. iperf3 1st take: tk@pine64:~$ iperf3 -c 192.168.83.245 Connecting to host 192.168.83.245, port 5201 [ 4] local 192.168.83.57 port 51790 connected to 192.168.83.245 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 15.9 MBytes 134 Mbits/sec 38 12.7 KBytes [ 4] 1.00-2.00 sec 15.2 MBytes 128 Mbits/sec 36 12.7 KBytes [ 4] 2.00-3.00 sec 30.2 MBytes 253 Mbits/sec 74 21.2 KBytes [ 4] 3.00-4.00 sec 19.3 MBytes 163 Mbits/sec 53 14.1 KBytes [ 4] 4.00-5.00 sec 18.2 MBytes 153 Mbits/sec 57 9.90 KBytes [ 4] 5.00-6.00 sec 22.2 MBytes 187 Mbits/sec 55 8.48 KBytes [ 4] 6.00-7.00 sec 26.3 MBytes 221 Mbits/sec 58 15.6 KBytes [ 4] 7.00-8.00 sec 21.7 MBytes 182 Mbits/sec 54 8.48 KBytes [ 4] 8.00-9.00 sec 21.6 MBytes 181 Mbits/sec 51 7.07 KBytes [ 4] 9.00-10.00 sec 20.7 MBytes 173 Mbits/sec 52 12.7 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 211 MBytes 177 Mbits/sec 528 sender [ 4] 0.00-10.00 sec 211 MBytes 177 Mbits/sec receiver Since I made a mistake (please note what's written on our download page and what is known since a long time) I then used the only way to reliably power the Pine64: iperf3 2nd take: tk@pine64:~$ iperf3 -c 192.168.83.245 Connecting to host 192.168.83.245, port 5201 [ 4] local 192.168.83.57 port 55237 connected to 192.168.83.245 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.01 sec 109 MBytes 900 Mbits/sec 7 60.8 KBytes [ 4] 1.01-2.00 sec 106 MBytes 902 Mbits/sec 5 99.0 KBytes [ 4] 2.00-3.01 sec 108 MBytes 894 Mbits/sec 8 67.9 KBytes [ 4] 3.01-4.01 sec 107 MBytes 902 Mbits/sec 8 59.4 KBytes [ 4] 4.01-5.00 sec 104 MBytes 875 Mbits/sec 4 123 KBytes [ 4] 5.00-6.00 sec 110 MBytes 922 Mbits/sec 4 93.3 KBytes [ 4] 6.00-7.00 sec 105 MBytes 882 Mbits/sec 7 66.5 KBytes [ 4] 7.00-8.01 sec 105 MBytes 872 Mbits/sec 8 86.3 KBytes [ 4] 8.01-9.00 sec 100 MBytes 845 Mbits/sec 27 74.9 KBytes [ 4] 9.00-10.00 sec 103 MBytes 864 Mbits/sec 7 52.3 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 1.03 GBytes 886 Mbits/sec 85 sender [ 4] 0.00-10.00 sec 1.03 GBytes 886 Mbits/sec receiver Only difference: instead of the crap connector I used pins 4 and 6 of the Euler connector. Not that surprising since it's long known that the crap connector (also called Micro USB) is exactly that: crap. I started my first try also with a usual (that means crappy) USB cable between a well known good USB PSU and the board and measured only ~4.4V on Pine64's USB ports (@androsch's Pine64 features the DC5V/BAT jumper and I used the DC5V setting). With a good USB cable (also 1 m long but 20AWG rating) voltage on Pine64's USB port was between 4.75 and 4.85V, when using the only reliable powering method it's now at 4.6V. So there clearly is something wrong with the power traces on the board. The numbers above were in idle mode. When powering through the Euler pins voltage on Pine64's USB port dropped to 4.5V (stress -c 2) and 4.4V with 'stress -c 4'. That's bad news since I wanted to do some NAS tests later (USB high power devices require 4.75V minimum) With the jumper in BAT setting voltage on the USB ports is provided through PMIC and remains pretty stable ~5V regardless of the load. But in this mode maximum current provided is limited so it's of no use either. 0 Quote
tkaiser Posted September 8, 2016 Author Posted September 8, 2016 Next update: I managed to attach two 128GB Samsung SSDs to Pain64 with jumper in BAT setting (stable 5V on USB ports), one in a bus-powered JMS567 enclosure (UASP capable which is only important when using mainline kernel later) on the lower USB port and the other in an 3.5" enclosure with an own PSU. I created a RAID-0 and performance is ok-ish: 70 MB/s sequential transfer speeds: root@pine64:~# mkfs.btrfs -f -m raid0 -d raid0 /dev/sda /dev/sdb btrfs-progs v4.4 See http://btrfs.wiki.kernel.org for more information. Label: (null) UUID: 7425ecc4-494a-488f-bb8c-0f9e51e07267 Node size: 16384 Sector size: 4096 Filesystem size: 231.03GiB Block group profiles: Data: RAID0 2.01GiB Metadata: RAID0 2.01GiB System: RAID0 20.00MiB SSD detected: no Incompat features: extref, skinny-metadata Number of devices: 2 Devices: ID SIZE PATH 1 119.24GiB /dev/sda 2 111.79GiB /dev/sdb root@pine64:~# mkdir /mnt/raid-0 root@pine64:~# mount -o noatime /dev/sda /mnt/raid-0 root@pine64:~# cd /mnt/raid-0/ root@pine64:/mnt/raid-0# iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 ; iozone -a -g 6000m -s 6000m -i 0 -i 1 -r 4K -r 1024K Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 64 bit mode. Build: linux Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Thu Sep 8 12:25:54 2016 Include fsync in write timing O_DIRECT feature enabled Auto Mode File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 7946 8786 9140 9140 8071 8542 102400 16 17344 18445 21823 21843 20680 17639 102400 512 64905 62626 66159 66436 65781 61350 102400 1024 55950 59729 67007 67047 67055 50464 102400 16384 67375 67490 72136 71918 71976 66829 iozone test complete. Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 64 bit mode. Build: linux Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Thu Sep 8 12:28:08 2016 Auto Mode Using maximum file size of 6144000 kilobytes. File size set to 6144000 kB Record Size 4 kB Record Size 1024 kB Command line used: iozone -a -g 6000m -s 6000m -i 0 -i 1 -r 4K -r 1024K Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 6144000 4 71929 70699 69798 70205 6144000 1024 71906 71430 68917 69258 iozone test complete. But any Samba tests are postponed since @androsch's board shows also a real problem in RX direction. This is with BSP kernel: tk@armbian:~$ iperf3 -c 192.168.83.57 Connecting to host 192.168.83.57, port 5201 [ 4] local 192.168.83.115 port 51536 connected to 192.168.83.57 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 77.8 KBytes 636 Kbits/sec 17 1.41 KBytes [ 4] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 3 2.83 KBytes [ 4] 2.00-3.00 sec 52.3 KBytes 429 Kbits/sec 10 1.41 KBytes [ 4] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec 8 1.41 KBytes [ 4] 4.00-5.00 sec 83.4 KBytes 684 Kbits/sec 21 5.66 KBytes [ 4] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec 5 1.41 KBytes [ 4] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes [ 4] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes [ 4] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes [ 4] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 214 KBytes 175 Kbits/sec 67 sender [ 4] 0.00-10.00 sec 137 KBytes 112 Kbits/sec receiver And this with mainline kernel (limited there by a fixed CPU clockspeed of just 408 MHz, with my board now being @androsch's in this direction +600 Mbits/sec are possible with 4.7): root@armbian:~# iperf3 -c 192.168.83.67 Connecting to host 192.168.83.67, port 5201 [ 4] local 192.168.83.115 port 43444 connected to 192.168.83.67 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 27.8 MBytes 233 Mbits/sec 751 17.0 KBytes [ 4] 1.00-2.00 sec 19.5 MBytes 163 Mbits/sec 537 4.24 KBytes [ 4] 2.00-3.00 sec 19.8 MBytes 166 Mbits/sec 518 17.0 KBytes [ 4] 3.00-4.00 sec 25.3 MBytes 212 Mbits/sec 737 11.3 KBytes [ 4] 4.00-5.00 sec 30.3 MBytes 254 Mbits/sec 935 21.2 KBytes [ 4] 5.00-6.00 sec 41.3 MBytes 346 Mbits/sec 1304 4.24 KBytes [ 4] 6.00-7.00 sec 13.1 MBytes 110 Mbits/sec 486 7.07 KBytes [ 4] 7.00-8.00 sec 23.8 MBytes 200 Mbits/sec 726 5.66 KBytes [ 4] 8.00-9.00 sec 30.6 MBytes 257 Mbits/sec 848 15.6 KBytes [ 4] 9.00-10.00 sec 17.2 MBytes 144 Mbits/sec 549 19.8 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 249 MBytes 208 Mbits/sec 7391 sender [ 4] 0.00-10.00 sec 248 MBytes 208 Mbits/sec receiver So what do we know now? Settings matter: most if not all OS images available on Pine64 wiki or pine64.pro suffer from wrong cpufreq governor and show way too low network performance numbers (both in benchmarks and in real-world situations since single-threaded tasks get bottlenecked by CPU) Avoiding the crap connector (known as Micro USB) helps on boards that show bad GbE performance. In case of @androsch's board TX performance increased from 177 Mbits/sec (with 528 retries) to 886 Mbits/sec (with 'just' 85 retries) Mainline kernel deals somewhat better with this even if currently limited to only 408 MHz CPU clockspeeds Obviously there's something wrong with 'power'. AFAIK we still have no schematics for the production batches so it's useless to continue So even if we manage to find software work-arounds the average user of these boards will still run in troubles due to power issues we most probably can do nothing against. 0 Quote
tkaiser Posted September 8, 2016 Author Posted September 8, 2016 Update: I suggested measuring voltage on the USB ports with jumper in 5VDC position when board is powered through Euler pins and then compare between boards that work fine with GbE and those that do not: https://irclog.whitequark.org/linux-sunxi/2016-09-08#17497310; Maybe it's just a simple (under)power issue that affects some boards (since we already know that it's not related to the PHY chips used, at least boards made with the same batch of PHY chips differ, some show that strange behaviour, some not). Since I can not contribute to pine64 forums (have been banned a few days ago but more importantly my posts there get constantly censored by one specific moderator) maybe others might encourage users to do these measurements and compare? To me (being an electrics NOOB ) a correlation with voltage seems obvious when I'm able to increase TX throughput from 170 Mbits/sec by avoiding Micro USB (responsible for undervoltage) to almost the possible maximum (while also reducing retransmissions). Maybe someone with a bench PSU and an affected board can test whether increasing input voltage to 5.3V or above 'cures' the problem. But such a test would require avoiding the crappy DC-IN connector and crappy OS images (tampering network throughput numbers by wrong cpufreq governor). And it would be even better if we would know whether/where test points to check PHY voltages are available. 0 Quote
androsch Posted September 8, 2016 Posted September 8, 2016 Hi tkaiser, really interesting, so it mainly really has to do with power, even using the same PSU with your developer board (also being a 1GB board) will have 'normal' GbE. Sorry for my maybe dumb question: Its not absolutely clear to me which one is client and which one is server? You know i also had about 200mbps as client, but my server-speed was very poor. Can you just explain, which ip/board from above is mine? 0 Quote
tkaiser Posted September 8, 2016 Author Posted September 8, 2016 Update on 'environmental' conditions. Fortunately I found my 5.5/2.1 barrel to crap connector adapters again (the crap connector is also called Micro USB by most people). Now testing with 2 PSUs that differ regarding the voltage they provide. I measure the voltage in a 2nd step with a Banana Pro since there AXP209 PMIC provides access to measured voltage through sysfs): PSU N° 1 (providing 5.2V to a Banana Pro where voltage can be read out). tk@pine64:~$ iperf3 -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.83.115, port 51612 [ 5] local 192.168.83.57 port 5201 connected to 192.168.83.115 port 51614 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 14.1 MBytes 118 Mbits/sec [ 5] 1.00-2.00 sec 20.2 MBytes 170 Mbits/sec [ 5] 2.00-3.00 sec 28.6 MBytes 239 Mbits/sec [ 5] 3.00-4.00 sec 22.8 MBytes 192 Mbits/sec [ 5] 4.00-5.00 sec 25.6 MBytes 214 Mbits/sec [ 5] 5.00-6.00 sec 31.1 MBytes 261 Mbits/sec [ 5] 6.00-7.00 sec 39.8 MBytes 334 Mbits/sec [ 5] 7.00-8.00 sec 29.9 MBytes 250 Mbits/sec [ 5] 8.00-9.00 sec 35.0 MBytes 293 Mbits/sec [ 5] 9.00-10.00 sec 31.8 MBytes 267 Mbits/sec [ 5] 10.00-10.02 sec 619 KBytes 286 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 5] 0.00-10.02 sec 280 MBytes 234 Mbits/sec 9192 sender [ 5] 0.00-10.02 sec 279 MBytes 234 Mbits/sec receiver PSU N° 2 (providing 5.0V to a Banana Pro where voltage can be read out). tk@pine64:~$ iperf3 -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.83.115, port 51616 [ 5] local 192.168.83.57 port 5201 connected to 192.168.83.115 port 51618 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 49.5 KBytes 405 Kbits/sec [ 5] 1.00-2.00 sec 89.1 KBytes 730 Kbits/sec [ 5] 2.00-3.00 sec 25.5 KBytes 209 Kbits/sec [ 5] 3.00-4.00 sec 17.0 KBytes 139 Kbits/sec [ 5] 4.00-5.00 sec 58.0 KBytes 475 Kbits/sec [ 5] 5.00-6.00 sec 83.4 KBytes 683 Kbits/sec [ 5] 6.00-7.00 sec 80.6 KBytes 660 Kbits/sec [ 5] 7.00-8.00 sec 49.5 KBytes 406 Kbits/sec [ 5] 8.00-9.00 sec 69.3 KBytes 568 Kbits/sec [ 5] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 10.00-10.02 sec 0.00 Bytes 0.00 bits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 5] 0.00-10.02 sec 594 KBytes 486 Kbits/sec 118 sender [ 5] 0.00-10.02 sec 522 KBytes 427 Kbits/sec receiver So the 0.2V more or less make a 'slight' difference of below 500 Kbits/sec vs. +230 Mbits/sec on this board. But hey, that's just ~500 times slower with the 5.0V PSU (please keep in mind that I used the crap connector responsible for further voltage drops) 0 Quote
tkaiser Posted September 8, 2016 Author Posted September 8, 2016 even using the same PSU with your developer board (also being a 1GB board) will have 'normal' GbE. 'My' board is a developer sample made in Dec 2015. Your board is from the production batches. And Pine64 folks changed powering in between (the jumper to choose how USB ports will be powered the most obvious difference). AFAIK we have no schematics for the production batches so it's useless to speculate what's going on/wrong here. But the correlation with DC-IN voltage is IMO somewhat obvious So when talking about GbE performance of Pine64 it's not only software (the crappy OS images from pine64 wiki and pine64.pro preventing GbE being useful due to bad settings) but also environment (how one powers the board, the PSU ratings are totally irrelevant it's all about cable and contact resistance). On a couple of boards something with power traces seems to be wrong (disclaimer: I really have no idea what I'm talking here about, I'm not an expert at all in this area). If we would know whether/where test points are to check PHY voltage then maybe the whole issue could've been resolved way earlier (today I drove into city to buy a new multimeter... just in case). All numbers on this page of the thread are made with your board testing against a real Linux host or a virtualized host ('worst case' conditions since this VM is behind 3 physical and one virtual switch) My (non expert) guess is that on your board I'm testing with some PCB traces are too tiny and lead to a severe voltage drop undervolting the PHY. Willmore in linux-sunxi IRC just confirmed that on his board voltage fed and being available on USB ports is identical (and his board behaves fine regarding GbE). So it seems we're getting closer. BTW: It would be sooooo easy to nail the problem down if one has both a non-working and a working board since it's totally simple to compare if you have access to both. 0 Quote
androsch Posted September 8, 2016 Posted September 8, 2016 We have somebody who has two boards, one functional and the other one not, but he is not really willing to prove anything beside his own stubborn head, so i'm out there also, it really makes no sense. http://forum.pine64.org/showthread.php?tid=597&pid=19374#pid19374 If there is something else i can support from my side, would be glad to help, but seems to be outside my limited resource and brain :-) Thanks for testing, maybe we get also closer with some other guys willing to do deeper tests. BTW: Is there a PSU for the Euler bus to be known as reliable and sufficient for the Pine? Which PSU are you using for the bus? 0 Quote
zador.blood.stained Posted September 8, 2016 Posted September 8, 2016 At least dev samples seems to be not affected by input voltage: I tested 'mine' Pine64 with regulated power supply and legacy Xenial image, and even at 4.2v provided to Euler connector it still shows 700Mbit/s and no retransmits. Edit: Mbit, not MB of course 0 Quote
tkaiser Posted September 8, 2016 Author Posted September 8, 2016 BTW: Is there a PSU for the Euler bus to be known as reliable and sufficient for the Pine? Which PSU are you using for the bus? Still the same as in March when we improved thermal/dvfs settings for Pine64: https://github.com/longsleep/build-pine64-image/pull/3#issuecomment-195927887 (I followed simply Igor's recommendation for a dual voltage PSU with screw terminal and then used simple jumper wires). As a result I wrote an email to TL Lim back then emphasising on how important it is to replace the crap connector with a sane barrel jack/plug solution. To no avail. BTW: Please accept that I don't follow links to pine64 forum. The stuff happening there is just weird (moderators providing broken OS images that will 'prove' GbE Ethernet has no advantages over Fast Ethernet, moderators all the time giving wrong advices, moderators generating a moronic hype about 'the Mali' and now also moderators censoring other posts -- it has been bad there almost from the beginning but since the pine64 folks now enabled a Dunning-Kruger guy to censor everything he does not understand it got even more worse) At least dev samples seems to be not affected by input voltage: I tested 'mine' Pine64 with regulated power supply and legacy Xenial image, and even at 4.2v provided to Euler connector it still shows 700MB/s and no retransmits. Thanks for the confirmation. Seems the changes made to power traces are responsible for the malfunction of some boards. Any further investigation should be done by Pine64 folks themselves. They should ask 3 to 5 affected users for their boards and test through. It's a hardware issue, it can't be resolved by software or powering through the Euler connector (helped only in TX direction with @androsch's board but not in the other), the environmental condition is DC-IN voltage being provided (see the 5.0V vs. 5.2V difference above and keep in mind that the average USB cable used is responsible for another whopping voltage drop) and of course one needs a good and not a crappy OS image to realize the difference at all). Until this issue is resolved from an Armbian position it's just a simple 'don't buy' recommendation. 0 Quote
androsch Posted September 8, 2016 Posted September 8, 2016 BTW: Please accept that I don't follow links to pine64 forum. Any further investigation should be done by Pine64 folks themselves. They should ask 3 to 5 affected users for their boards and test through. It's a hardware issue, it can't be resolved by software or powering through the Euler connector (helped only in TX direction with @androsch's board but not in the other), the environmental condition is DC-IN voltage being provided (see the 5.0V vs. 5.2V difference above and keep in mind that the average USB cable used is responsible for another whopping voltage drop) and of course one needs a good and not a crappy OS image to realize the difference at all). Until this issue is resolved from an Armbian position it's just a simple 'don't buy' recommendation. Fully understand your position. Will try to get some response from tllim about checking my board by Pine and try to get some response from other users for testing their boards at least with higher voltages. Thanks for your help! 0 Quote
tkaiser Posted September 8, 2016 Author Posted September 8, 2016 Thanks for your help! Thanks for your help! By linux-sunxi community standards it took quite long (almost 8 hours between receiving your board and nailing the problem down) but when I took the bike to buy a new multimeter I met a friend and we ended up temporarely in Augustiner Keller drinking a Maß BTW: Unfortunately I did the Samba tests not with 'my' board before so I can now only provide sequential performance numbers in direction Pine64 --> client (since even when powered through Euler connector your board behaves badly in RX direction, I have no bench PSU here so I can't try out whether providing 5.3V or 5.4V already cures the GbE issue): That's a samba share on a RAID-0 made out of 2 Samsung 120GB SSDs (now part of a burn-in test since both SSDs are for a customer's server working there in RAID-1 mode): One EVO 750 and one PM851: root@pine64:~# smartctl -a /dev/sda ; smartctl -a /dev/sdb smartctl 6.5 2016-01-24 r4214 [aarch64-linux-3.10.102-pine64] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Device Model: SAMSUNG MZ7TE128HMGR-00004 Serial Number: S1RLNSAF805588 LU WWN Device Id: 5 002538 8a061c4d9 Firmware Version: EXT0200Q User Capacity: 128,035,676,160 bytes [128 GB] Sector Size: 512 bytes logical/physical Rotation Rate: Solid State Device Device is: Not in smartctl database [for details use: -P showall] ATA Version is: ACS-2, ATA8-ACS T13/1699-D revision 4c SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Thu Sep 8 20:09:28 2016 UTC SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 0) seconds. Offline data collection capabilities: (0x53) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. No Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 70) minutes. SCT capabilities: (0x003d) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 1 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 5 Reallocated_Sector_Ct 0x0033 100 100 000 Pre-fail Always - 0 9 Power_On_Hours 0x0032 099 099 000 Old_age Always - 5 12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 3 177 Wear_Leveling_Count 0x0013 100 100 000 Pre-fail Always - 100 179 Used_Rsvd_Blk_Cnt_Tot 0x0013 100 100 010 Pre-fail Always - 100 181 Program_Fail_Cnt_Total 0x0032 100 100 000 Old_age Always - 0 182 Erase_Fail_Count_Total 0x0032 100 100 000 Old_age Always - 0 183 Runtime_Bad_Block 0x0013 100 100 000 Pre-fail Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0032 068 067 000 Old_age Always - 32 195 Hardware_ECC_Recovered 0x001a 200 200 000 Old_age Always - 0 199 UDMA_CRC_Error_Count 0x003e 100 100 000 Old_age Always - 0 235 Unknown_Attribute 0x0012 100 100 000 Old_age Always - 0 241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 30004069 242 Total_LBAs_Read 0x0032 099 099 000 Old_age Always - 29379222 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short captive Completed without error 00% 0 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. smartctl 6.5 2016-01-24 r4214 [aarch64-linux-3.10.102-pine64] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Device Model: Samsung SSD 750 EVO 120GB Serial Number: S33MNB0H570161T LU WWN Device Id: 5 002538 d40e27d62 Firmware Version: MAT01B6Q User Capacity: 120,034,123,776 bytes [120 GB] Sector Size: 512 bytes logical/physical Rotation Rate: Solid State Device Form Factor: 2.5 inches Device is: Not in smartctl database [for details use: -P showall] ATA Version is: ACS-2, ATA8-ACS T13/1699-D revision 4c SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Thu Sep 8 20:09:28 2016 UTC SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART Status not supported: Incomplete response, ATA output registers missing SMART overall-health self-assessment test result: PASSED Warning: This result is based on an Attribute check. General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 0) seconds. Offline data collection capabilities: (0x53) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. No Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 64) minutes. SCT capabilities: (0x003d) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 1 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 9 Power_On_Hours 0x0032 099 099 000 Old_age Always - 5 12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 34 177 Wear_Leveling_Count 0x0013 099 099 000 Pre-fail Always - 1 179 Used_Rsvd_Blk_Cnt_Tot 0x0013 100 100 010 Pre-fail Always - 0 181 Program_Fail_Cnt_Total 0x0032 100 100 010 Old_age Always - 0 182 Erase_Fail_Count_Total 0x0032 100 100 010 Old_age Always - 0 183 Runtime_Bad_Block 0x0013 100 100 010 Pre-fail Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0032 071 056 000 Old_age Always - 29 195 Hardware_ECC_Recovered 0x001a 200 200 000 Old_age Always - 0 199 UDMA_CRC_Error_Count 0x003e 100 100 000 Old_age Always - 0 235 Unknown_Attribute 0x0012 099 099 000 Old_age Always - 25 241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 313838401 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing 255 0 65535 Read_scanning was never started Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. I simply measured time to complete copying a 1GB file from the USB RAID-0 via SMB from Pine64 to my MacBook: macbookpro-tk:~ tk$ du -sh /Volumes/raid-0/1gb_zeroes.file 1,0G /Volumes/raid-0/1gb_zeroes.file macbookpro-tk:~ tk$ time cp /Volumes/raid-0/1gb_zeroes.file /tmp/ real 0m14.152s user 0m0.003s sys 0m1.804s So the bottleneck is storage (the RAID-0 maxing out at 73MB/s -- OS X Finder dynamically adjusts the read block size to the maximum, something you have to take into account since it affects performance of the first test run being made!) which is good news if we keep in mind that mainlining progress for A64 is progressing nicely with dvfs / cpufreq scaling support network performance with mainline kernel will be way better than now with mainline kernel we can make use of UASP with A64 (which will enable us to get close to 40MB/s per disk) we can configure the upper USB port (now OTG in host mode) to be a full USB host port by fiddling around with magical bits So in case one uses a Pine64+ that's not affected by the GbE hardware flaw (or the voltage drops can be avoided by providing higher DC-IN voltage) with mainline kernel we're soon able to get single disk NAS performance close to 40MB/s or 80MB/s when using 2 disks and relying on RAID-0. With a somewhat stupid RAID-0 setup (using two powered USB hubs on each USB port and two disks behind each hub) it should even be possible to get close to 100MB/s (the USB2.0 limitation of hitting a 40MB/s barrier is a per device limit. But whether performance improves when adding USB hubs to the scenario or not depends on the hub in question -- some add too much overhead and then performance decreases. And of course it's better to choose any other ARM board that is more suited for NAS usage). Yes, that's MB/s above and not Mb/s. On a board affected by the GbE issue and still showing PHY problems (but not that much compared to powering through crap connector). Pine64+ performs pretty well with GbE and can handle that even if some moderators constantly tell the opposite in pine64 forum. 1 Quote
-L0thar- Posted September 8, 2016 Posted September 8, 2016 Hi, @tkaiser, been following your posts with interest and appreciation with regard to pine64 and arm sbc's in general. Thanks for the education! I have two red-led 2GB pine64 boards. One shows <1Mb/s, the other ~900Mb/s with iperf3. That's with a recent armbian legacy image, fat microUSB cables, and a dual port USB charger spec'd at 4.2A (2.1A/port). ethtool at 100Mb/s yields reliable fast ethernet on the bad board. uname -a: Linux pine64 3.10.102-pine64 #1 SMP Sun Aug 28 20:00:30 CEST 2016 aarch64 aarch64 aarch64 GNU/Linux Anyway, I'll look into getting a higher rated pwr supply and some quality leads for eular. Thanks. 0 Quote
tkaiser Posted September 8, 2016 Author Posted September 8, 2016 I'll look into getting a higher rated pwr supply and some quality leads for eular. Thanks for the feedback. I don't think 'higher ratings' will help that much but it would be interesting if you could measure voltage provided on Pine64's USB ports when the jumper is in 5VDC position (just to get the idea whether a voltage drop here correlates with the GbE issue). And it would also be interesting if you've a bench PSU able to provide more than 5.2V to fed to either the Micro USB jack or Euler pins (the behaviour on @androsch's board is somewhat strange since when powering through Euler pins the voltage on the USB ports drops down more than using Micro USB). Anyway: 4.6V on the USB ports dropping down to 4.4V with some load on androsch's board is a sign for me that's something wrong (disclaimer: I'm an absolute NOOB regarding electronics, I just had to learn the hard way that insufficient power supply can cause many issues) so even if higher DC-IN voltage might 'resolve' the issue I would still have doubts regarding reliability (in other words: the reason for these voltage drops has to be found) 1 Quote
@lex Posted September 9, 2016 Posted September 9, 2016 @tkaiser, I found your explanation very interesting and always enjoy reading your comments, specially when making comments precise. My Pine64 does not suffer from the GbE issue, it is a relatively new board, i think production batch and i use a PSU rated 5v 2A (micro Usb) and i do a lot power on/off with a USB hub, keyboard and mouse attached and some times, not very often it does not boot, actually it boots but drops to u-boot prompt, making occasional users to think it does not boot. I can still load the kernel from the u-boot command, this is possibly a power source issue. (that's what i think). I have also RTC backup battery attached. Sorry, i have to ask, if possible can you please provide a picture where i should put the leads to measure the volts? I may get interested in providing some input if you think this could help. I have PSU rated 5v 2A (microUSB) and 4.5v ~ 5v 2.5A (this is how it is printed on PSU, may be a warning that the volts will drop?). I will not torture the board since it is the only sample i have. 0 Quote
-L0thar- Posted September 9, 2016 Posted September 9, 2016 Thanks for the feedback. I don't think 'higher ratings' will help that much but it would be interesting if you could measure voltage provided on Pine64's USB ports when the jumper is in 5VDC position (just to get the idea whether a voltage drop here correlates with the GbE issue). And it would also be interesting if you've a bench PSU able to provide more than 5.2V to fed to either the Micro USB jack or Euler pins (the behaviour on @androsch's board is somewhat strange since when powering through Euler pins the voltage on the USB ports drops down more than using Micro USB). Anyway: 4.6V on the USB ports dropping down to 4.4V with some load on androsch's board is a sign for me that's something wrong (disclaimer: I'm an absolute NOOB regarding electronics, I just had to learn the hard way that insufficient power supply can cause many issues) so even if higher DC-IN voltage might 'resolve' the issue I would still have doubts regarding reliability (in other words: the reason for these voltage drops has to be found) Using a $20 analog meter (calibrated ??) I read 4.8vdc across pins 1 and 4 of either usb port on the bad (GBE) board. The CPU is idle and power is jumpered for 5vdc. (Off hand I don't know what would load a quad core.. flac, zip /usr.) Negative on bench psu. I'll look around.. Thanks. 0 Quote
tkaiser Posted September 9, 2016 Author Posted September 9, 2016 can you please provide a picture where i should put the leads to measure the volts? If you provide power through Micro USB you can measure at Euler pins 4 and 6 (5V/GND) or also at the USB ports (the outer contacts carry power, the inner data) with jumper set to 5VDC (since otherwise the AXP803 PMIC provides stable 5V there). The reason I ask for this measurements is that I really don't understand how it's possible on my (@androsch's) board that I feed 5.1V through Euler pins but get then only 4.6V on the USB ports while I get there a higher voltage when using Micro USB for DC-IN (and Micro USB is known for severe voltage drops caused by cable and contact resistance, it's also limiting maximum current so any advises to use a PSU capable of more than 2A are BS). Here's a picture of 'my' 1st Pine64+ dev sample (now at Zador's place): http://linux-sunxi.org/File:Pine64_Powered_through_Euler_Connector.jpg (red is 5V, black is GND) Using a $20 analog meter (calibrated ??) I read 4.8vdc across pins 1 and 4 of either usb port on the bad (GBE) board. A comparison with the good board would help. Using same PSU and measuring there too. Both at USB ports and Euler pins 4/6. Just to get the idea whether voltage drop behaviour differs between good and bad board and we find an easy correlation. But even if not: the board @androsch sent to me clearly has a power problem if voltage between Euler pins and USB port drops by 0.5V (idle) and just 0.7V with 'stress -c 4'. To measure the influence of voltage drops you could use 'stress' (contained in Armbian for exactly this reason). A 'stress -c 2' is pretty lightweight, a 'stress -c 4 -m 2' for example will lead to higher voltage drops (that's the problem with these drops, they depend on the amperage needed by the board so they increase under load or when bus-powered USB peripherals are connected). BTW: These measurements in 'stab in the dark mode' are of course somewhat stupid since if we would have a test point to check voltage provided to RTL8211E this would make much more sense. But at least on androsch's board behaviour of GbE performance depends on input voltage (see the 5.0V vs. 5.2V experiment) and the voltage readout on the USB ports simply feels wrong. So IMO it's a good idea to check these voltages to help pine64 folks identify the root cause (now that hopefully other reasons for bad network performance -- the OS images using ondemand cpufreq governor for example -- are identified and able to be ruled out by switching to good OS images or better settings). In the meantime TL Lim promised releasing schematics of the 2GB Pine64+ this weekend. Let's wait and see. 1 Quote
tkaiser Posted September 9, 2016 Author Posted September 9, 2016 not very often it does not boot, actually it boots but drops to u-boot prompt, making occasional users to think it does not boot. I can still load the kernel from the u-boot command, this is possibly a power source issue. Thanks for mentioning that. It never occured with Pine64 to me but a few weeks back when we added NanoPi NEO support (or to be more precise tried to find perfect consumption / thermal settings for these boards) this happened very often. When the issue occured it was enough to connect serial console and then type 'boot' to let u-boot continue. Funnily this never occured when serial console was already connected and finally I realized that the PSU was responsible (noise on DC-IN led to noise on serial RX line that u-boot interpreted as keystrokes and stopped autoboot). The PSU died in the meantime. 0 Quote
tkaiser Posted September 9, 2016 Author Posted September 9, 2016 And another update for those playing around with the vanilla Armbian images. By adding the following as first line to /boot/boot.cmd mw.l 0x1c20000 0x80001110 and then compiling boot.cmd with the usual: mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr the CPU cores run with 864 MHz instead of 408 MHz Higher clockspeeds are risky since currently we have no PMIC support in Linux mainline kernel and therefore the CPU cores are fed with the default 1.1V VDD_CPUX so even 864 MHz can already be considered overclocking/undervolting (816 MHz is considered safe). Reference: https://irclog.whitequark.org/linux-sunxi/2016-09-09#17504981 You might also add 'mw.l 0x1c2005c 1' in another line to speed up Ethernet and USB with mainline kernel (for reasons see irc log above). 0 Quote
-L0thar- Posted September 10, 2016 Posted September 10, 2016 @tkaiser So I went shopping for some test gear and ordered a linear psu, fluke multimeter, and 22 awg leads for euler. The fluke is factory calibrated. The psu can vary current or voltage. ETA on the 15th. A comparison with the good board would help. Using same PSU and measuring there too. Both at USB ports and Euler pins 4/6. Just to get the idea whether voltage drop behaviour differs between good and bad board and we find an easy correlation. But even if not: the board @androsch sent to me clearly has a power problem if voltage between Euler pins and USB port drops by 0.5V (idle) and just 0.7V with 'stress -c 4'. To measure the influence of voltage drops you could use 'stress' (contained in Armbian for exactly this reason). A 'stress -c 2' is pretty lightweight, a 'stress -c 4 -m 2' for example will lead to higher voltage drops (that's the problem with these drops, they depend on the amperage needed by the board so they increase under load or when bus-powered USB peripherals are connected). BTW: These measurements in 'stab in the dark mode' are of course somewhat stupid since if we would have a test point to check voltage provided to RTL8211E this would make much more sense. But at least on androsch's board behaviour of GbE performance depends on input voltage (see the 5.0V vs. 5.2V experiment) and the voltage readout on the USB ports simply feels wrong. So IMO it's a good idea to check these voltages to help pine64 folks identify the root cause (now that hopefully other reasons for bad network performance -- the OS images using ondemand cpufreq governor for example -- are identified and able to be ruled out by switching to good OS images or better settings). In the meantime TL Lim promised releasing schematics of the 2GB Pine64+ this weekend. Let's wait and see. For a noobie you sure sound like you know what you're doing. I'm good with all above.. good idea to compare boards. I hate to suggest this but I think it would be prudent to wait for the euler wires to arrive before proceeding. Its usually ESD that worries me but trying to place probes on a loose board then observe the meter is way tricky. But, I did receive a playbox enclosure, that may stabilize the board enough. I did a test run on the slow gbe board with 'stress -c 4 -m 2'. Idle voltage was 5V, busy was 4.8. I know I said idle was 4.8V in my previous post.. Maybe I wasn't facing the analog as face on as I thought. I don't know. On another note, before I checked voltages I monitored with 'armbian -m' while 'stress -c 4 -m 2' was running. The temp remained at 30.0C with the A64 clking at 1152 or 1008Mhz then 408Mhz when I stopped the stress test. The test duration was only about a minute but the A64 was quite hot to the touch. That 30.0 temp should have increased me thinks. Neither of these boards have heat sinks. Thanks. 0 Quote
tkaiser Posted September 10, 2016 Author Posted September 10, 2016 The temp remained at 30.0C with the A64 Yeah, we're currently using the wrong sysfs node on Pine64+: https://github.com/igorpecovnik/lib/issues/457 Apart from that I also bought a bench PSU yesterday (just to realize they're sold without cables -- even those that cost +100 bucks -- WTF?), then bought a set of cables and now built an universal cable set providing 2 jumper wire pairs, 1 x 5.5/2.1, 1 x 4.0/1.7 and 1 x 'ODROID' so I'm prepared to destroy a lot of boards the next time since I might forget to adjust voltage before switching on (currently the PSU provides 12V to a Clearfog Pro). Apart from that when testing with different voltages yesterday I had to realize that my observations before using two different PSUs (one with 5.0V, the other providing 5.2V and showing way better GbE behaviour) are not solely related to the voltage. So different behaviour is caused by other PSU characteristics (noise, ripple?). Anyway: I'm out, no further time to waste. IMO @androsch's board is defective and it's now up to the Pine64 folks to come up with a solution. I hope this thread and our observations might help Pine64 users to understand that there are a couple of reasons why network performance sucks (wrong cpufreq governor used in most OS images, benchmark tools invalidating the results since they're CPU bound, environmental conditions like voltage fed or PSU used in case one owns a 'slow' board) Ah, just as a list the various Ethernet troubles Pine64 users had to suffer from or are still suffering from: Android/RemixOS not booting when Ethernet cable is connected on first boot. Status: no idea / don't care Ethernet unusable on 2GB boards. Reason: Integer overflow in BSP kernel Ethernet driver. Status: long ago fixed by longsleep, no idea whether fix has been applied to all available OS images Random MAC address on each boot. Fixed for BSP kernel by longsleep in Feb or March by using an entry in /boot/uEnv.txt that contains the address. Took ages to adopt this change with the OS images available in Pine64 wiki or on pine64.pro. Currently with Armbian vanilla OS images the same happens: Work-around is to add an 'hwaddress ether' entry to /etc/network/interfaces or a 'mac_address' entry to the ethernet@1c30000 node in .dts. Real fix will be then MAC address based on A64's serial number (SID) as with most/all other Allwinner SoCs currently (u-boot generates and sets the address, the kernel just uses this later) All Pine64 having the same MAC address. This is only a problem with the OS images available in Pine64 wiki or on pine64.pro since they contain already an MAC address in /boot/uEnv.txt (for whatever reasons, this is just weird) No real difference between GbE and Fast Ethernet speed. This is only a problem with the OS images available in Pine64 wiki or on pine64.pro since they use the wrong cpufreq governor which bottlenecks all networking (see @pfeerick's investigations and the reasons explained in post #2 of this thread) On a couple of boards problems with GbE negotiation happen. Symptoms vary depending on environmental conditions (PSU and powering method used -- at least on @androsch's board severe and hard to explain voltage drops happen) Talking about 'Ethernet problems' without differentiating between those 6 different problems is useless. 0 Quote
pfeerick Posted September 10, 2016 Posted September 10, 2016 so any advises to use a PSU capable of more than 2A are BS). BTW: These measurements in 'stab in the dark mode' are of course somewhat stupid since if we would have a test point to check voltage provided to RTL8211E this would make much more sense. Just on the PSU more than 2A - the only caveat I would add to that point is that with a power supply capable of say 5A as opposed to 2A, the PSU itself probably won't start to drop it's voltage below 5v on Pine64-ish loads, so may be one less source of voltage drop. Having said that, from the various USB leads I've tried over the years, that is still the sole biggest contributor, followed closely by the connector used. And the lack of test points would also explain why this may turn out to be the product of a hardware / construction issue, as they don't have the test points to be able to be able to put the pine64 on an automated test jig in order to do basic electrical tests as part of the QC process. Duh! Pretty much mandatory when making a product of this complexity and volume... as you don't want a massive number of returns that you then have to replace and reship, and the bad press! Even a small kickstarter ('only' 3173 backers) like this one worked that out pretty quickly! Admittedly he has been doing this for a while... but it isn't rocket science! 0 Quote
tkaiser Posted September 10, 2016 Author Posted September 10, 2016 Just on the PSU more than 2A - the only caveat I would add to that point is that with a power supply capable of say 5A as opposed to 2A, the PSU itself probably won't start to drop it's voltage below 5v on Pine64-ish loads Emphasis by me. This is just an assumption and hoping that more expensive equipment cures a problem that needs detailed attention. As an example: when @androsch's board arrived the day before yesterday I used an 5V/2A USB PSU with a well known 'crap cable' (tiny wires, resistance high, the only reason I have this cable in the drawer is to demonstrate how shitty Micro USB is). Crappy cable + crappy connector --> resistance way too high --> voltage way below 5V: I replaced this with a dual voltage PSU connecting the 5V rail to Euler pins and GbE problems in TX direction disappeared almost immediately (that's the difference the powering method made). But this stable PSU provides only constant 4.97V on idle and 4.95 under load (and only voltage on the 12V rail can be adjusted). So this stable PSU didn't help in RX direction even if it's stable. Since I found my barrel plug to Micro USB adapters I could try two different PSUs now connecting to the Micro USB port using the adapter. With the one providing 5.0V RX throughput was in the range of Kbits/sec, with the one providing 5.2V it was at Mbits/sec (nearly 500 times more 'performance' -- but since I was not able to repeat this with the new bench PSU I bought yesterday there must be different other influences also). So the assumption that a higher rated PSU will behave better is just that: an assumption not backed by anything. I did also some tests yesterday with a USB disk connected (higher consumption --> voltage drops). Without USB disk I measured 5.23V on the RPi header, with disk connected 5.17V. Iperf3 numbers varied reproduceable between 300 and just 100 Mbits/sec in RX direction. So these minimal differences are important. By believing in a 5V/5A brick you don't get even close to this since maybe this brick starts at 5.05V idle and then drops down to just 5.02V under load (both values being too low at least for @androsch's affected board). BTW: Software matters. With mainline kernel on @androsch's board when using the 5.0V PSU the kernel driver did not even find the RTL8211E PHY. Shutting then the board down, replacing the PSU with the 5.2V one and I 'got the PHY back': https://irclog.whitequark.org/linux-sunxi/2016-09-09#17505156; And now the most funny part: the AXP803 PMIC present on each A64 device is measuring voltages and current all the time since it takes decisions based on that (eg compensating from undervoltage on mains by using a connected battery). So instead of relying on assumptions a responsible vendor would have added readouts to sysfs so every user can easily monitor whether he uses an insufficient powering method or not. And I write about powering method and not PSU since this is important. The average user has not the slightest idea that the main problem is the cable between PSU and the crap connector (called Micro USB). The average USB cable is crap and responsible for insane voltage drops! It makes absolutely no difference whether you use a 5V/10A PSU or a 5V/2A PSU if the cable between PSU and board is crap since here the more severe voltage drop happens since the cable (and the tiny contacts inside the crap connector) act like a resistor. Regarding test points and QC: No idea, there are a bunch of test points on the lower PCB side... but without any description they're at least useless for us users. At least TL Lim promised to release schematics for the 2GB model soon. 1 Quote
pfeerick Posted September 11, 2016 Posted September 11, 2016 Emphasis by me. This is just an assumption and hoping that more expensive equipment cures a problem that needs detailed attention. . So the assumption that a higher rated PSU will behave better is just that: an assumption not backed by anything. Regarding test points and QC: No idea, there are a bunch of test points on the lower PCB side... but without any description they're at least useless for us users. At least TL Lim promised to release schematics for the 2GB model soon. lol... I don't disagree with your emphasis... but notice I didn't mention the crappy connector at all (for that exact reason)... as it doesn't matter if you have a 2A 5v PSU or a 200A 5v PSU... you can't avoid the voltage sag on the cable or connector, but then you get hit with connector loss/resistance. And since the 5v 5A PSUs I had in mind are deliberately tweaked by the supplier (Adafruit) to start at 5.25v, and sag down to 4.9-5.0 at 5A, they don't follow the usual rules... but then again... even the cheapie I bought the other day for another project sat at 5.03v when I load tested it at half (3A) load. I do wonder what is going on with PMIC... since the input voltage range for 'AC' and 'VBUS' is as wide as 3.5-7v... if it is really responsible for powering everything else (when the jumper is in the BATT position, anyway)... surely input power is essentially a non-issue!? The bigger problem seems to be power losses on the board itself, for no apparent reason. I suppose we'll know better what's going on when we get those schematics (thanks for to the link to the IRC chat... I'll post something about that in the schematic thread on the pine64 forum). 0 Quote
zador.blood.stained Posted September 12, 2016 Posted September 12, 2016 New Pine64 schematic released: http://files.pine64.org/doc/Pine%20A64%20Schematic/Pine%20A64plus%202GB%20Rev%20C-20160113_Release.pdf Currently I don't see any significant differences in PHY power part, but I think I found a point where the voltage can be measured, at least on my board, and running iperf3 test shows that voltage drops by 0.02V Edit: I mean PHY 3.3V voltage (PHY_VDD33) 1 Quote
tkaiser Posted September 12, 2016 Author Posted September 12, 2016 Edit: I mean PHY 3.3V voltage (PHY_VDD33) Thanks for looking into (I also opened the new schematics an hour ago but am too NOOB to understand anything). Could you mark the specific test point on a picture please (maybe using those in our wiki)? The defective Pine64+ is on its way back to @androsch so I can not measure myself right now (wrong timing again). 0 Quote
zador.blood.stained Posted September 12, 2016 Posted September 12, 2016 The defective Pine64+ is on its way back to @androsch so I can not measure myself right now (wrong timing again). That's bad, so we won't get any results anytime soon. Thanks for looking into (I also opened the new schematics an hour ago but am too NOOB to understand anything). Could you mark the specific test point on a picture please (maybe using those in our wiki)? My guess is that this is a place for optional 3.3/2.5V RGMII regulator, mentioned in the schematic (Page 18, top right corner). It can't be LP3986 since LP3986 should have 8 pins and additional components, but schematic shows an optional 3-pin component and some 0-ohm resistors around it anyway. 1 Quote
tkaiser Posted September 13, 2016 Author Posted September 13, 2016 The fun over in pine64 forums continues: @Davey Darko also posted his experiences to the LAN/NIC thread (getting a warning via PM from this one moderator for giving him a bad reputation), the moderator deleted the post and closed the thread. So he perfectly ensures again that no progress can be made. I just wanted to post one final answer to him but since he closed the thread (his usual behaviour according to others) I document here my answer to this: No, of course I don't have a solution, I'm just interested in finding the culprit. By following sources outside of this forum (here contribution is useless since censorship happens all the time -- please be aware that your actions are documented publicly so think first before deleting/censoring again!) you could really be able to understand that at least on one board there is a strong correlation between DC-IN (PSU used, method used, voltage fed) and GbE behaviour. Therefore it would be really nice if people affected by the issue could measure PHY_VDD33 on their boards: [inserting Zador's picture from above] To get the idea whether there is a correlation on more than one board (if it helps you to understand: this is some sort of a poll -- you like polls don't you). Possible results: No, there is none (search has to continue) or there is one. Of course it's important that NO WORK-AROUND is used when these measurements happen (no destroyed cables, no ethtool 'fix'). And of course it doesn't play a role what you measured in your lab since it's about getting results from the field. And all you do is constantly censoring everything that is beyond your imagination and discourage users to try this out ('I have proven that the VDD33 voltage is not the problem'). How did you do that and when? Isn't your board on its way to TL Lim: https://irclog.whitequark.org/linux-sunxi/2016-09-09#17502646;(or is he speaking of someone else?). Anyway: I'm pretty confident that you don't get why it could be important to accept other issues than those your 'expert soldering eyes' are able to see. And that's the reason you should resign from your moderator role rather sooner than later. Well, again: It seems every product gets the 'supporters' it deserves -- it's fascinating that the Pine64 folks don't care at all what happens there 1 Quote
Davey Darko Posted September 13, 2016 Posted September 13, 2016 The fun over in pine64 forums continues: @Davey Darko also posted his experiences to the LAN/NIC thread (getting a warning via PM from this one moderator for giving him a bad reputation), the moderator deleted the post and closed the thread. So he perfectly ensures again that no progress can be made. I just wanted to post one final answer to him but since he closed the thread (his usual behaviour according to others) I document here my answer to this: Well, again: It seems every product gets the 'supporters' it deserves -- it's fascinating that the Pine64 folks don't care at all what happens there The out of control moderator just banned me. I PM that tllm or whatever the guys moniker is about the mods temper tantrum before I was banned. He first sent me a warning just because I gave neg rep on 3 of his posts, I hadn't even posted in that thread at that point. He does not have the temper to be a moderator on an Internet forum in my humble opinion. I get better information here anyway, just like with OrangePis. Thanks Armbian! 1 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.