Anders Posted May 19, 2019 Posted May 19, 2019 (edited) Armbianmonitor: http://ix.io/1Jsj Hi All! I am having a lot of trouble making my ESPRESSObin v7 1GB DDR4 run without crashes. I have tried both Bionic (5.75 4.19.20) and Stretch (5.75 4.19.20) images, and neither are stable. I have tried with a lot of different PSUs (all switch-mode though) and booting from both microSD and USB sticks. My U-Boot is "flash-image-ddr4-1g-1cs-1000_800.bin". It seems like there is about 50% chance that it will boot without a crash. The type of crashes I get seems random. Ex http://ix.io/1JGY And then it just freezes. Or this one which includes a proper kernel Oops: http://ix.io/1JGZ While running, I get errors like this one: [ 71.136812] BUG: Bad page state in process khugepaged pfn:31d99 [ 71.140206] page:ffffffbf00c76640 count:0 mapcount:0 mapping:0000000000002000 index:0x0 [ 71.148428] flags: 0x0() [ 71.151049] raw: 0000000000000000 0000000000000000 ffffffbf00c76648 0000000000002000 [ 71.159015] raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000 [ 71.166975] page dumped because: non-NULL mapping [ 71.172329] BUG: Bad page state in process khugepaged pfn:3261c [ 71.178001] page:ffffffbf00c98700 count:0 mapcount:0 mapping:0000000000002000 index:0x0 [ 71.186236] flags: 0x0() [ 71.188853] raw: 0000000000000000 0000000000000000 ffffffbf00c98708 0000000000002000 [ 71.196825] raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000 [ 71.204782] page dumped because: non-NULL mapping Once successfully booted, setting the CPUfreq governor to "performance" seems to make it more stable. Am I the only one having problems getting a stable boot on V7? Has anyone of you any tips on how to make it stable? I am willing to try compiling a new kernel if some experimental patches exists. Edited May 21, 2019 by Anders 0 Quote
Anders Posted May 19, 2019 Author Posted May 19, 2019 Here is an update: I just tried replacing the kernel with 4.4.8-armada-17.02-espressobin downloaded from http://espressobin.net/tech-spec/ I put the "Image" file at "/boot/Image", and the "armada-3720-community.dtb" at "/boot/dtb/marvell/armada-3720-community.dtb". Now everything seems fine. I haven't seen any crashes at all so far, and rebooting works fine. root@espressobin:~# uname -a Linux espressobin 4.4.8-armada-17.02.2-g8148be9 #8 SMP PREEMPT Wed Jul 4 11:59:53 UTC 2018 aarch64 GNU/Linux root@espressobin:~# 7za b 7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs LE) LE CPU Freq: 997 997 998 998 997 998 995 996 RAM size: 930 MB, # CPU hardware threads: 2 RAM usage: 441 MB, # Benchmark threads: 2 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 868 152 555 844 | 22465 199 962 1918 23: 859 153 574 876 | 22076 199 958 1911 24: 853 153 599 918 | 21731 200 955 1908 25: 848 153 632 968 | 21340 199 953 1899 ---------------------------------- | ------------------------------ Avr: 153 590 902 | 199 957 1909 This is probably a stupid question, but why don't we just apply the patches from https://github.com/MarvellEmbeddedProcessors/linux-marvell/tree/linux-4.4.8-armada-17.02-espressobin to the 4.19.20 kernel used by Armbian? What is the current state of that work? Can I help out? 0 Quote
Anders Posted May 19, 2019 Author Posted May 19, 2019 (edited) I just tried building and booting Linux 5.1.0-g72cf0b074 directly from torvalds/linux.git. It boots fine, but after some time, it crashes with the same kind of errors as the Armbian kernel. Here is one of them: http://ix.io/1JH1 Edited May 21, 2019 by Anders 0 Quote
Anders Posted May 19, 2019 Author Posted May 19, 2019 (edited) The situation is the same with the kernel from archlinuxarm, even though it only runs at 800 MHz: http://ix.io/1JH2 Edited May 21, 2019 by Anders 0 Quote
chrisf Posted May 19, 2019 Posted May 19, 2019 I had kernel panic issues with my Espressobin, when it was functional. It worked best with the 1200/750 firmware. It's probably something to do with memory timing/board layout. 0 Quote
Igor Posted May 20, 2019 Posted May 20, 2019 14 hours ago, Anders said: This is probably a stupid question, but why don't we just apply the patches from https://github.com/MarvellEmbeddedProcessors/linux-marvell/tree/linux-4.4.8-armada-17.02-espressobin to the 4.19.20 kernel used by Armbian? This is exactly what is behind, but keeping it updated with resources that works on Armbian is close to impossible. 14 hours ago, Anders said: What is the current state of that work? Can I help out? It is summed/documented/resulted in: patches on top of mainline: https://github.com/armbian/build/tree/master/patch/kernel/mvebu64-next kernel config: https://github.com/armbian/build/blob/master/config/kernel/linux-mvebu64-next.config u-boot / ATF building process: https://github.com/armbian/build/blob/master/config/sources/mvebu64.conf (if it's a kernel problem, this can be fully skipped) ... and forum will also provide some clues about that state. 0 Quote
ebin-dev Posted May 20, 2019 Posted May 20, 2019 13 hours ago, Anders said: I just tried building and booting Linux 5.1.0-g72cf0b074 directly from torvalds/linux.git. It boots fine, but after some time, it crashes with the same kind of errors as the Armbian kernel. Here is one of them: Could you flash your bootloader to 600_600 and try again ? If no errors occur in that case we could assume that the bootloader needs to be adapted at least for V7 boards ... 0 Quote
Anders Posted May 20, 2019 Author Posted May 20, 2019 @Igor Thanks! 8 hours ago, ebin-dev said: Could you flash your bootloader to 600_600 and try again ? If no errors occur in that case we could assume that the bootloader needs to be adapted at least for V7 boards ... That seems to work fine. I only ran it for about 30 minutes though. root@espressobin:~# uname -a Linux espressobin 5.1.0-g72cf0b074 #1 SMP PREEMPT Sun May 19 18:21:04 CEST 2019 aarch64 GNU/Linux root@espressobin:~# 7za b 7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs LE) LE CPU Freq: 439 450 368 589 591 505 585 487 RAM size: 984 MB, # CPU hardware threads: 2 RAM usage: 441 MB, # Benchmark threads: 2 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 438 122 349 427 | 13482 198 581 1151 23: 524 146 365 534 | 13276 198 580 1149 24: 520 147 381 560 | 12839 195 577 1127 25: 517 147 402 591 | 12862 198 580 1145 ---------------------------------- | ------------------------------ Avr: 141 374 528 | 197 580 1143 Tot: 169 477 835 I'm confused. I do not understand what this has to do with the U-Boot bootloader? 1000_800 worked fine with 4.4.8-armada-17.02-espressobin kernel. My board came with "U-Boot 2017.03-armada-17.10.2-g14aeedc" preinstalled, which on boot reported: CPU @ 1000 [MHz] L2 @ 800 [MHz] TClock @ 200 [MHz] DDR @ 800 [MHz] Which is why I choose to flash "flash-image-ddr4-1g-1cs-1000_800.bin". Espressobin v7 should run at 1.2 GHz, so I just tried flashing "flash-image-ddr4-1g-1cs-1200_750.bin", but now my board does not boot: Marvell>> bubt flash-image-ddr4-1g-1cs-1200_750.bin spi usb Burning U-BOOT image "flash-image-ddr4-1g-1cs-1200_750.bin" from "usb" to "spi" USB0: Register 2000104 NbrPorts 2 Starting the controller USB XHCI 1.00 USB1: USB EHCI 1.00 scanning bus 0 for devices... 1 USB Device(s) found scanning bus 1 for devices... 2 USB Device(s) found Image checksum...OK! SF: Detected mx25u3235f with page size 256 Bytes, erase size 64 KiB, total 4 MiB Erasing 917504 bytes (14 blocks) at offset 0 ...Done! Writing 889028 bytes from 0x8000000 to offset 0 ...Done! Marvell>> Marvell>> Marvell>> reset resetting ... TIM-1.0 WTMI-devel-18.12.0-a0a1cb8 WTMI: system early-init SVC REV: 5, CPU VDD voltage: 0.898V It never gets any further than "SVC REV: 5, CPU VDD voltage: 0.898V" - any ideas? 0 Quote
ebin-dev Posted May 21, 2019 Posted May 21, 2019 18 hours ago, Anders said: That seems to work fine. I only ran it for about 30 minutes though. We are desperately waiting for the next release of u-boot / atf / A3700-utils by Marvell (enhanced DDR tuning for all EspressoBins is expected). Together with the pending patch performance may increase by about 20 % and stability of V7 EspressoBins will hopefully be established too ... 0 Quote
kostap Posted May 21, 2019 Posted May 21, 2019 I have updated the 18.12 branch with patches from current development stream. Please check if it improves the stability. https://github.com/MarvellEmbeddedProcessors/A3700-utils-marvell/commits/A3700_utils-armada-18.12 1 Quote
ebin-dev Posted May 21, 2019 Posted May 21, 2019 @kostap Thank you very much ! @Anders Please find the updated bootloader here (u-boot is unchanged; WTMI is updated to 18.12.1) - it works fine on my V5_0_1 EspressoBin (https://pastebin.com/xJCtLXsH). It should solve the stability issues ... Edit: the recovery images are also updated: sata-images and uart-images Edit: links changed ; the images are now available through Armbian servers 0 Quote
barish Posted May 21, 2019 Posted May 21, 2019 I got my EspressoBin-V7 yesterday and spent most of today to get it "stable" with Armbian. I haven't succeeded so far, although with your patch @ebin-dev, I got an uptime of almost 10 mins, which is an absolute record for now... I tried 1000-800 and 800-800-Mhz so far. I am running the Armbian provided Debian kernel on it (4.19.20). I tried what @Anders describes above and replace the kernel and dts file, but for me, it doesn't work: The kernel gets stuck at the very beginning [ 0.000000] bootconsole [uart0] enabled was the last message of boot process. Update: I just tried the flash-image-ddr4-1g-1cs-1200_750.bin and this broke u-boot completely. I will have to do a rescue with WtpDownload_linux tomorrow... 0 Quote
ManoftheSea Posted May 22, 2019 Posted May 22, 2019 @barish, I was recently trying to debug a similar thing. Do you have "verbosity=1" in /boot/armbianEnv.txt? If so, you're telling the kernel not to log. I set mine to 7 based on a very poor understanding of things I read, and it gives me more log. If you already knew this, then maybe my message will help someone else sometime. 0 Quote
ebin-dev Posted May 22, 2019 Posted May 22, 2019 @barish Please try the new 600_600 flash image. 0 Quote
barish Posted May 22, 2019 Posted May 22, 2019 (edited) Thanks for your replies. I spent many hours today to upload a recovery image onto the EspressoBin, but no success. Usually, after a few seconds, the upload stops (no error message), longest was about 30s till stop. I am not sure if this has to do with the fact, that I am using a Virtual Box Debian guest on a MacOS host, so that the serial connection might not be as perfectly timed as necessary? I have never had any problems with this and a serial connection at 115200 baud is not rocket science really (I tried lower baud rates too...). If I can't find a solution for this, the EspressoBin is going to take its way back to GlobalScale. Update: Ok VirtualBox is one part of the problem. I installed the Prolific drivers for MacOS, I compiled WtpDownload for MacOS (which went really smoothly), and *most* of the time, I can now accomplish a recovery upload. The 600_600 from 18.12v2 got me back to the Marvell>> prompt, which I really welcomed when I saw it. So I "bubt" the 600_600 u-boot and started setting the env parameters, but in the middle of that the board crashed and rebooted...! I am about to give it up. I will try once more setting the env and booting from SD, but that's the last chance I am willing to give to the V7 board... Edited May 22, 2019 by barish update 0 Quote
barish Posted May 22, 2019 Posted May 22, 2019 Ok, that's it. Twice more the board crashed when I tried to set the env parameters. BTW I tried three different 12V power supplies, all have operated reliably in the past, one delivers about 11,5V, one 12,1V, one 12,5V, all 2A or more. The board is sitting on the bottom part of the original housing, which includes a large heat sink, so it has never become hot (I'd say below 30°C). I am not sure if I should try another specimen or get a refund... or try a V5, they are supposed to be more reliable, are they. 0 Quote
ebin-dev Posted May 23, 2019 Posted May 23, 2019 On 5/22/2019 at 10:27 PM, barish said: I am not sure if I should try another specimen or get a refund... or try a V5, they are supposed to be more reliable, are they. You could ask for a replacement V7 board AND for a V5 board as some kind of compensation for all the trouble you had so far. My V5_0_1 boards are running 24/7 (with Armbian Stretch, mainline-stable, kernel 4.19.20) ... 0 Quote
Anders Posted May 27, 2019 Author Posted May 27, 2019 @barish I managed to fix mine by following those two guides: https://dl.armbian.com/espressobin/u-boot/rescue/uart-rescue-espressobin.pdf and http://wiki.espressobin.net/tiki-index.php?page=Bootloader+recovery+via+UART+-+v7&highlight=recovery I used the files from https://dl.armbian.com/espressobin/u-boot/rescue/uart-images-18.12-v2/uart-images-ddr4-1g-1cs-1000_800.tgz @ebin-dev Using flash-image-ddr4-1g-1cs-600_600.bin from 2019-05-21 seems fine so far: root@espressobin:~# uname -a Linux espressobin 4.19.20-mvebu64 #5.75 SMP PREEMPT Fri Feb 8 09:54:18 CET 2019 aarch64 aarch64 aarch64 GNU/Linux root@espressobin:~# 7za b 7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs LE) LE CPU Freq: 296 296 296 296 296 296 294 RAM size: 990 MB, # CPU hardware threads: 2 RAM usage: 441 MB, # Benchmark threads: 2 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 289 143 197 282 | 6691 199 287 571 23: 286 142 205 292 | 6616 199 287 573 24: 284 143 214 306 | 6553 199 289 575 25: 283 143 227 324 | 6476 199 290 576 ---------------------------------- | ------------------------------ Avr: 143 211 301 | 199 288 574 Tot: 171 249 437 root@espressobin:~# uname -a Linux espressobin 5.1.0-g72cf0b074 #1 SMP PREEMPT Sun May 19 18:21:04 CEST 2019 aarch64 GNU/Linux root@espressobin:~# 7za b 7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs LE) LE CPU Freq: 594 594 594 594 595 595 594 595 RAM size: 984 MB, # CPU hardware threads: 2 RAM usage: 441 MB, # Benchmark threads: 2 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 532 147 353 518 | 13527 198 582 1155 23: 525 147 365 536 | 13303 198 581 1152 24: 521 147 381 561 | 13050 198 579 1146 25: 518 147 402 593 | 12934 198 580 1151 ---------------------------------- | ------------------------------ Avr: 147 375 552 | 198 580 1151 Tot: 173 478 851 0 Quote
ebin-dev Posted May 27, 2019 Posted May 27, 2019 22 minutes ago, Anders said: Using flash-image-ddr4-1g-1cs-600_600.bin from 2019-05-21 seems fine so far: Thank you for your feedback. Do you realise that the CPU clock speed is different with kernel 4.19.20-mvebu64 (about 300MHz) and with 5.1.0-g72cf0b074 (about 600MHz) ? (see my comment in the other thread) 0 Quote
Anders Posted May 28, 2019 Author Posted May 28, 2019 (edited) @ebin-dev Yes. I do not know why though. Using flash-image-ddr4-1g-1cs-800_800-2019-05-21.bin also seems to work fine: root@espressobin:~# uname -a Linux espressobin 4.19.20-mvebu64 #5.75 SMP PREEMPT Fri Feb 8 09:54:18 CET 2019 aarch64 aarch64 aarch64 GNU/Linux root@espressobin:~# 7za b 7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs LE) LE CPU Freq: 793 793 794 794 794 794 794 794 RAM size: 990 MB, # CPU hardware threads: 2 RAM usage: 441 MB, # Benchmark threads: 2 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 726 149 473 706 | 17714 200 758 1512 23: 721 150 488 735 | 17480 200 757 1513 24: 715 151 510 770 | 17197 200 755 1510 25: 711 151 538 812 | 16950 200 755 1509 ---------------------------------- | ------------------------------ Avr: 151 502 756 | 200 756 1511 Tot: 175 629 1133 root@espressobin:~# uname -a Linux espressobin 5.1.0-g72cf0b074 #1 SMP PREEMPT Sun May 19 18:21:04 CEST 2019 aarch64 GNU/Linux root@espressobin:~# 7za b 7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs LE) LE CPU Freq: 647 794 794 794 794 794 794 794 RAM size: 984 MB, # CPU hardware threads: 2 RAM usage: 441 MB, # Benchmark threads: 2 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 715 147 474 696 | 17894 196 778 1528 23: 707 147 490 721 | 17845 198 778 1545 24: 702 148 512 755 | 17546 198 776 1540 25: 698 148 541 798 | 17261 198 774 1536 ---------------------------------- | ------------------------------ Avr: 147 504 742 | 198 777 1537 Tot: 173 640 1140 flash-image-ddr4-1g-1cs-1000_800-2019-05-21.bin also works fine so far, but only runs at 800 mhz?: root@espressobin:~# uname -a Linux espressobin 4.19.20-mvebu64 #5.75 SMP PREEMPT Fri Feb 8 09:54:18 CET 2019 aarch64 aarch64 aarch64 GNU/Linux root@espressobin:~# 7za b 7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs LE) LE CPU Freq: 793 794 794 794 794 794 794 794 RAM size: 990 MB, # CPU hardware threads: 2 RAM usage: 441 MB, # Benchmark threads: 2 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 730 150 472 711 | 17719 200 758 1513 23: 720 150 488 734 | 17447 199 757 1510 24: 716 151 510 770 | 17170 199 756 1507 25: 711 151 538 813 | 16964 200 756 1510 ---------------------------------- | ------------------------------ Avr: 151 502 757 | 199 757 1510 Tot: 175 630 1133 root@espressobin:~# uname -a Linux espressobin 5.1.0-g72cf0b074 #1 SMP PREEMPT Sun May 19 18:21:04 CEST 2019 aarch64 GNU/Linux root@espressobin:~# 7za b 7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs LE) LE CPU Freq: 794 794 793 794 794 794 794 794 RAM size: 984 MB, # CPU hardware threads: 2 RAM usage: 441 MB, # Benchmark threads: 2 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 714 147 473 695 | 18084 198 778 1544 23: 707 147 490 720 | 17813 198 777 1542 24: 702 148 512 756 | 17451 198 775 1532 25: 699 148 541 798 | 17261 198 775 1536 ---------------------------------- | ------------------------------ Avr: 147 504 742 | 198 776 1539 Tot: 173 640 1140 But sometimes it freezes very early in the boot process. This happened with all versions: Marvell>> reset resetting ... TIM-1.0 WTMI-devel-18.12.1-e6bb176 WTMI: system early-init SVC REV: 5, CPU VDD voltage: 1.073V (no more output) flash-image-ddr4-1g-1cs-1200_750-2019-05-21.bin killed my board again. Same as above, but last line is: "SVC REV: 5, CPU VDD voltage: 0.898V". Also, I haven't tested with the patches from https://github.com/MarvellEmbeddedProcessors/linux-marvell/pull/19 yet. Will do that once I recover my board. Edited May 28, 2019 by Anders Add tests for other images. 0 Quote
ebin-dev Posted May 28, 2019 Posted May 28, 2019 1 hour ago, Anders said: Also, I haven't tested with the patches from https://github.com/MarvellEmbeddedProcessors/linux-marvell/pull/19 yet. @Anders The reason for the different CPU clock speeds with kernel 4.19.20 and 5.1.0 is that the above patches already made it into the kernel (since 4.19.4x). Therefore with newer kernels the CPU is clocked as expected. In kernel 4.19.20 (and with earlier kernels) the CPU was clocked with lower speeds (800MHz instead of 1000MHz, 300MHz instead of 600MHz) as you have seen yourself. Now the new boot loader has to (ddr) tune the system such that it is stable under these new circumstances. 1 hour ago, Anders said: flash-image-ddr4-1g-1cs-1000_800-2019-05-21.bin also works fine so far, but only runs at 800 mhz?: Would you please test the 1000_800 image again (from here) ? I am surprised that the CPU only runs at 800 MHz in this case. Please press the reset button once you have flashed the 1000_800 image ... (You need to adapt the values in /etc/default/cpufrequtils to available values shown by cpufreq-info and to reboot) 0 Quote
barish Posted May 28, 2019 Posted May 28, 2019 @Anders I am happy for you that the u-boot and kernel work for you. I did exactly what you did with all different u-boots and never got a stable board, not even a stable pre-kernel prompt (marvell>>). The only thing that was (mostly) stable was the pre-u-boot prompt (>)... but even then I had crashes during upload with WtpDownload, which may also be due to the fact that Prolific PL2303 drivers for MacOS (and for Windows) are quite buggy and lead to system crashes of the host (I had twice a detached process of MacOS with 100% cpu on one core, completely un-killable, I had to reboot the machine). I can't really understand why the EspressoBin uses this very cheap USB-serial-chip. OT: For me, it's not a good sign that the very only and first GlobalScale-board I ordered is so completely unreliable (I was thinking of using it for a series, this idea is more than dead now). 0 Quote
Anders Posted May 31, 2019 Author Posted May 31, 2019 (edited) On 5/28/2019 at 10:17 AM, ebin-dev said: @Anders The reason for the different CPU clock speeds with kernel 4.19.20 and 5.1.0 is that the above patches already made it into the kernel (since 4.19.4x). Therefore with newer kernels the CPU is clocked as expected. In kernel 4.19.20 (and with earlier kernels) the CPU was clocked with lower speeds (800MHz instead of 1000MHz, 300MHz instead of 600MHz) as you have seen yourself. Now the new boot loader has to (ddr) tune the system such that it is stable under these new circumstances. Would you please test the 1000_800 image again (from here) ? I am surprised that the CPU only runs at 800 MHz in this case. Please press the reset button once you have flashed the 1000_800 image ... (You need to adapt the values in /etc/default/cpufrequtils to available values shown by cpufreq-info and to reboot) @ebin-dev Using flash-image-ddr4-1g-1cs-1000_800-2019-05-21.bin (md5: 1b0cc3d743be6e5881d0f9010df79f4f), and mainline Linux with the config from https://github.com/armbian/build/blob/master/config/kernel/linux-mvebu64-next.config this is what I get: Marvell>> version U-Boot 2018.03-devel-18.12.3-gc9aa92c-armbian (Feb 20 2019 - 09:45:04 +0100) aarch64-linux-gnu-gcc (Linaro GCC 7.4-2019.02) 7.4.1 20181213 [linaro-7.4-2019.02 revision 56ec6f6b99cc167ff0c2f8e1a2eed33b1edc85d4] GNU ld (Linaro_Binutils-2019.02) 2.28.2.20170706 root@espressobin:~# uname -a Linux espressobin 5.2.0-rc2+ #3 SMP PREEMPT Fri May 31 18:28:10 CEST 2019 aarch64 GNU/Linux root@espressobin:~# cat /etc/default/cpufrequtils # WARNING: this file will be replaced on board support package (linux-root-...) upgrade ENABLE=true MIN_SPEED=200000 MAX_SPEED=1300000 GOVERNOR=ondemand root@espressobin:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 0.97 ms. hardware limits: 200 MHz - 1000 MHz available frequency steps: 200 MHz, 250 MHz, 500 MHz, 1000 MHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance, schedutil current policy: frequency should be within 200 MHz and 1000 MHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 200 MHz (asserted by call to hardware). cpufreq stats: 200 MHz:81.44%, 250 MHz:1.75%, 500 MHz:0.79%, 1000 MHz:16.02% (193) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 0.97 ms. hardware limits: 200 MHz - 1000 MHz available frequency steps: 200 MHz, 250 MHz, 500 MHz, 1000 MHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance, schedutil current policy: frequency should be within 200 MHz and 1000 MHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 200 MHz (asserted by call to hardware). cpufreq stats: 200 MHz:81.44%, 250 MHz:1.75%, 500 MHz:0.79%, 1000 MHz:16.02% (193) root@espressobin:~# 7za b 7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs LE) LE CPU Freq: 500 794 794 794 794 794 794 794 RAM size: 987 MB, # CPU hardware threads: 2 RAM usage: 441 MB, # Benchmark threads: 2 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 717 149 469 698 | 18046 200 772 1541 23: 708 149 485 722 | 17712 199 770 1533 24: 703 149 506 757 | 17468 200 769 1534 25: 699 150 534 799 | 17211 199 769 1532 ---------------------------------- | ------------------------------ Avr: 149 498 744 | 199 770 1535 Tot: 174 634 1139 If I do "cpufreq-set -g performance", "cpufreq-info" reports: "current CPU frequency is 1000 MHz (asserted by call to hardware).", though 7zip continues to report 794. No immediate crashes so far. Got a crash on boot: starting version 232 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done. Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done. [ 6.189319] Internal error: Oops: 86000005 [#1] PREEMPT SMP [ 6.192204] Modules linked in: [ 6.195346] CPU: 0 PID: 211 Comm: spi0 Not tainted 5.2.0-rc2+ #3 [ 6.201522] Hardware name: Globalscale Marvell ESPRESSOBin Board (DT) [ 6.208154] pstate: 80000005 (Nzcv daif -PAN -UAO) [ 6.213085] pc : 0x100000 [ 6.215778] lr : spi_res_release+0x5c/0x98 [ 6.219978] sp : ffffff801173bd00 [ 6.223382] x29: ffffff801173bd00 x28: 0000000000000000 [ 6.228848] x27: ffffff8011121b30 x26: ffffff8011029000 [ 6.234314] x25: ffffffc03fb5bba0 x24: dead000000000100 [ 6.239780] x23: dead000000000200 x22: ffffff801005b818 [ 6.245245] x21: ffffffc03fb5b800 x20: ffffff801005b7c0 [ 6.250710] x19: ffffffc03fbed040 x18: 0000000000000000 [ 6.256176] x17: 0000000000000000 x16: 0000000000000000 [ 6.261642] x15: 0000000000000000 x14: 0000000000000000 [ 6.267107] x13: 0000000000000001 x12: 0000000000000000 [ 6.272573] x11: 0000000000000001 x10: 0000000000000910 [ 6.278038] x9 : ffffff801173bbe0 x8 : ffffffc03c86e630 [ 6.283504] x7 : ffffffc03c86dd80 x6 : 0000000018f873c1 [ 6.288970] x5 : 0000000000000323 x4 : 0000000000000000 [ 6.294435] x3 : 0000000000100000 x2 : ffffffc03fbed058 [ 6.299901] x1 : ffffff801005b7c0 x0 : ffffffc03fb5b800 [ 6.305368] Call trace: [ 6.307877] 0x100000 [ 6.310210] spi_transfer_one_message+0x25c/0x3c8 [ 6.315047] __spi_pump_messages+0x32c/0x520 [ 6.319439] spi_pump_messages+0x14/0x20 [ 6.323474] kthread_worker_fn+0xac/0x198 [ 6.327591] kthread+0x124/0x128 [ 6.330908] ret_from_fork+0x10/0x1c [ 6.334584] Code: bad PC value [ 6.337717] ---[ end trace c792078f48e6ceda ]--- Edited May 31, 2019 by Anders cpufreq-set -g performance, crash 0 Quote
ebin-dev Posted May 31, 2019 Posted May 31, 2019 1 hour ago, Anders said: Got a crash on boot: If you want to learn something you need to test with kernel 4.19.46 (please select next-nightly in armbian-config) with the new boot loader 1000_800 and 800_800. /etc/default/cpufrequtils should be adapted to: cat /etc/default/cpufrequtils #1000_800 ENABLE=true MIN_SPEED=250000 MAX_SPEED=1000000 GOVERNOR=ondemand Otherwise It is difficult to say if the crashes on boot are caused by kernel 5.2 or by the new boot loader. 0 Quote
Anders Posted May 31, 2019 Author Posted May 31, 2019 (edited) 3 hours ago, ebin-dev said: If you want to learn something you need to test with kernel 4.19.46 (please select next-nightly in armbian-config) Done. Result: http://ix.io/1KAv No crashes yet in Linux yet, but about 5% of the time U-boot comes no further than: TIM-1.0 WTMI-devel-18.12.1-e6bb176 WTMI: system early-init SVC REV: 5, CPU VDD voltage: 1.155V On 5/28/2019 at 10:17 AM, ebin-dev said: The reason for the different CPU clock speeds with kernel 4.19.20 and 5.1.0 is that the above patches already made it into the kernel (since 4.19.4x). No. The patches from https://github.com/MarvellEmbeddedProcessors/linux-marvell/pull/19 are neither in mainline nor in linux-next. Update: I tried applying the patches, and now it always fails (Same as the one reported here: http://lists.infradead.org/pipermail/linux-arm-kernel/2019-March/638623.html): Loading, please wait... starting version 232 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done. Begin: Waiting for root file system ... [ 5.771744] Internal error: synchronous parity or ECC error: 86000018 [#1] PREEMPT SMP [ 5.777053] Modules linked in: [ 5.780191] CPU: 1 PID: 23 Comm: kworker/1:1 Not tainted 5.2.0-rc2+ #4 [ 5.786906] Hardware name: Globalscale Marvell ESPRESSOBin Board (DT) [ 5.793550] Workqueue: 0x0 (events) [ 5.797211] pstate: 40000085 (nZcv daIf -PAN -UAO) [ 5.802147] pc : dequeue_entity+0x48/0x578 [ 5.806351] lr : dequeue_entity+0x24/0x578 [ 5.810559] sp : ffffff801139bc80 [ 5.813965] x29: ffffff801139bc80 x28: 0000000000000000 [ 5.819430] x27: 0000000000000000 x26: ffffffc03fb012c0 [ 5.824895] x25: ffffff8010bdf400 x24: ffffffc03fde1bc0 [ 5.830360] x23: ffffff8011009000 x22: 0000000000000009 [ 5.835827] x21: 0000000157a1bc9e x20: ffffffc03fb00e00 [ 5.841292] x19: ffffffc03fde1c40 x18: 0000000000000000 [ 5.846758] x17: 0000000000000000 x16: 0000000000000000 [ 5.852224] x15: 0000000000000000 x14: 0000000000000000 [ 5.857688] x13: 0000000000000001 x12: 0000000000000000 [ 5.863154] x11: 0000000000000026 x10: 0101010101010101 [ 5.868620] x9 : 0000000000000000 x8 : 7f7f7f7f7f7f7f7f [ 5.874086] x7 : 0000000000000000 x6 : 0000000018f0cb72 [ 5.879551] x5 : 00ffffffffffffff x4 : 0000000000000008 [ 5.885017] x3 : 000000003b4485a0 x2 : 0000000000000000 [ 5.890483] x1 : 0000000000000000 x0 : 00000001579f2800 [ 5.895949] Call trace: [ 5.898460] dequeue_entity+0x48/0x578 [ 5.902313] dequeue_task_fair+0x54/0x590 [ 5.906437] deactivate_task+0x70/0xc8 [ 5.910290] __schedule+0x2a4/0x4c0 [ 5.913870] schedule+0x38/0xc8 [ 5.917099] worker_thread+0xc8/0x460 [ 5.920862] kthread+0x124/0x128 [ 5.924177] ret_from_fork+0x10/0x1c [ 5.927851] Code: f944e800 cb0002b5 cb0102b5 f9406280 (b5001da0) [ 5.934121] ---[ end trace 21831cae888e94ae ]--- Edited May 31, 2019 by Anders Add result 0 Quote
ebin-dev Posted June 1, 2019 Posted June 1, 2019 10 hours ago, Anders said: No. The patches from https://github.com/MarvellEmbeddedProcessors/linux-marvell/pull/19 are neither in mainline nor in linux-next. If you have a look at armada-37xx-cpufreq.c in the source tree of kernel 4.19.46 and 5.1.6 you will see that i.e. lines 449, 450 have changed to unsigned long u_volt = avs_map[dvfs->avs[load_lvl]] * 1000; freq = base_frequency / dvfs->divider[load_lvl]; This corresponds to new lines 442, 443 of the patch file. So both kernels were patched (I just took the corresponding tar balls from kernel.org). Thank you for testing kernel 4.19.46 on your V7 EspressoBin. It is not stable with the 1000_800 bootloader - not even at the u-boot prompt, the CPU is clocked with 800MHz instead of 1000MHz and the 1200_750 bootloader even bricks your EspressoBin. I think that the vendor must be made aware of this. If you now switch to the other bootloader 800_800 - is your V7 Espresso stable now with kernel 4.19.46 ? cat /etc/default/cpufrequtils #800_800 ENABLE=true MIN_SPEED=200000 MAX_SPEED=800000 GOVERNOR=ondemand 0 Quote
Anders Posted June 1, 2019 Author Posted June 1, 2019 @ebin-dev > If you have a look at armada-37xx-cpufreq.c in the source tree of kernel 4.19.46 and 5.1.6 you will see that i.e. lines 449, 450 have changed to [...] You are right, I didn't catch that. But that is only one of the 3 changes (L172, L423) from https://github.com/MarvellEmbeddedProcessors/linux-marvell/pull/19/commits/8fe9a4c4a024a6353e810a1dbb5e4bc78bff60a8 > Thank you for testing kernel 4.19.46 on your V7 EspressoBin. It is not stable with the 1000_800 bootloader - not even at the u-boot prompt, the CPU is clocked with 800MHz instead of 1000MHz With 4.19.46 I haven't seen any Linux crashes yet (been running for about 8 hours), so in that sense it is stable. But yes, 7zip only reports 800 MHz, and u-boot sometimes fails to boot. > 1200_750 bootloader even bricks your EspressoBin. I think that the vendor must be made aware of this. Alright, anything else that you would like me to test? And what exactly should I report to them? > If you now switch to the other bootloader 800_800 - is your V7 Espresso stable now with kernel 4.19.46 ? Using flash-image-ddr4-1g-1cs-800_800-2019-05-21.bin (md5 09a1b1ecf658ed74802bc65806fa92af), u-boot still fails to boot sometimes, with the last line being "SVC REV: 5, CPU VDD voltage: 1.073V". When the OS is running, everything seems fine so far: http://ix.io/1KCy Also, did you see these two fairly new patches that mentions Espressobin? http://lists.infradead.org/pipermail/linux-arm-kernel/2019-May/655136.html http://lists.infradead.org/pipermail/linux-arm-kernel/2019-May/655147.html Thanks for your help by the way 0 Quote
ebin-dev Posted June 1, 2019 Posted June 1, 2019 1 hour ago, Anders said: With 4.19.46 I haven't seen any Linux crashes yet (been running for about 8 hours), so in that sense it is stable. But yes, 7zip only reports 800 MHz, and u-boot sometimes fails to boot. That are good news for V7 owners. But it also means that the boot loader needs some finishing touches for V7 (boot loader crashes, 1200_750 bricks the V7, CPU runs with 800MHz instead of 1000MHz) and V3-5 EspressoBins too (1000_800 causes kernel panics on boot, CPU runs with 800MHz instead of 1000MHz). These issues occur at least with kernel 4.19.4x and above (cpufreq patch applied). I don't think that we need to report anything else directly to GST/Marvell. Regarding the other patches in the pipeline: There were many undocumented/unannounced changes by GlobalScle to the hardware of V5 and V7 EspressoBins. Most of the issues caused by that were already corrected in the boot_loader by Marvell and in Armbian. I hope that @Igor will have a look at those patches some time. As far as I know the required hardware (V7 EspressoBin with soldered emmc) is not available for testing purposes ... Edit: of course everybody is invited to test patches on their hardware and to post observations in the forum. 0 Quote
Anders Posted June 1, 2019 Author Posted June 1, 2019 > the boot loader needs some finishing touches for V7 (boot loader crashes, 1200_750 bricks the V7, CPU runs with 800MHz instead of 1000MHz) @ebin-dev We agree that the first two (boot loader crashes, 1200_750 bricks the V7) are bootloader problems. But I'm not sure "CPU runs with 800MHz instead of 1000MHz" is a bootloader problem. I just tried flashing flash-image-ddr4-1g-1cs-1000_800-2019-05-21.bin, and booting up 4.4.52-armada-17.10.4-g719fc86-dirty (from http://espressobin.net/tech-spec/) and now 7zip also reports 1000 MHz: http://ix.io/1KCN 0 Quote
ebin-dev Posted June 1, 2019 Posted June 1, 2019 2 minutes ago, Anders said: I just tried flashing flash-image-ddr4-1g-1cs-1000_800-2019-05-21.bin, and booting up 4.4.52-armada-17.10.4-g719fc86-dirty (from http://espressobin.net/tech-spec/) and now 7zip also reports 1000 MHz: http://ix.io/1KCN Very interesting ! Performance is up by 25% too. Is it stable ?? 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.