apollon77 Posted November 10, 2016 Posted November 10, 2016 Hey, after making sure that all packages are current (armbianmobitor upload available at http://sprunge.us/Migh)I still have the problem currently that my three cubietruicks freeze after working 1-3 days. Mostly they are still ping-able, but no "heartbeat led blinking" happens anymore and SSH connect also not possible. The logs simply say nothing. Is there anything knows with uboot 5.23 in case ob unstability? The last uboot I remember that was quite stable was 5.20 ... Can I simply downgrade to that using sudo apt-get install linux-u-boot-cubietruck-next=5.20 or such? Thanks for an advice... Ingo Fischer
Igor Posted November 11, 2016 Posted November 11, 2016 No, it's not a known problem since stability issues are hard to detect and they are sometimes board revision related - different memory chips. Yes, it's usually related to u-boot, where low level inits are done. Because of such problems, we introduced a patch to u-boot and set RAM speed to more conservative levels. This should help and new build should have those settings. I am not sure if this is already added to 5.23 but for sure it's present in our daily build. Try upgrading from here: http://beta.armbian.com/ (beware that this is untested automated daily build) Use one of those: Legacy kernel: http://beta.armbian.com/pool/main/l/linux-u-boot-cubietruck-default/ Vanilla kernel: http://beta.armbian.com/pool/main/l/linux-u-boot-cubietruck-next/
apollon77 Posted November 11, 2016 Author Posted November 11, 2016 Thank you for this solution! Download file and install directly with "dpkg -i" ?! Or what is the preferred way to install it?
Igor Posted November 11, 2016 Posted November 11, 2016 Thank you for this solution! Download file and install directly with "dpkg -i" ?! Or what is the preferred way to install it? Yes, better this way. Alternatively you could switch repository to beta.armbian.com and do apt-get update / upgrade and switch back and do apt-get update ... it's not recommended that this repository is used in production environment.
apollon77 Posted November 11, 2016 Author Posted November 11, 2016 Ok, I will try later today and report then in some days :-)
tpm8 Posted November 11, 2016 Posted November 11, 2016 I think I see similar problems with uboot 5.23 on my cubietruck. Going back to uboot 5.17 fixed it for me for the moment. See my post here: https://forum.armbian.com/index.php/topic/2044-some-discovery-while-trying-520-builds/ I'm going to test the beta version, too. Thx Tobias
tpm8 Posted November 11, 2016 Posted November 11, 2016 ok - here is my feedback, unfortunately uboot 5.24 does not fix it for me. Cubietruck, Jessie Legacy kernel (3.4.112-sun7i) Image 5.20 with all updates. Stability tests witch cpufreq-ljt-stress-test and Lima Memtester. uboot 5.23: Kernel crashes after few seconds during load, Error on console: Unable to handlE kernel NULL pointer dereference at virtual address 000000 uboot 5.24 beta (linux-u-boot-cubietruck_5.24.161109_armhf.deb): Kernel crashes after few seconds during load, Error on console: Unable to handle kernel paging request at virtual address xxxxxx Downgrading uboot to 5.17 or 5.20 - everything is rock solid. No problems during complete runs of cpufreq-ljt-stress-test and Lima Memtester. Tobias 2
Igor Posted November 14, 2016 Posted November 14, 2016 Hopefully fixed with this commit. Lima tester running stable for 30 minutes now.
Igor Posted November 14, 2016 Posted November 14, 2016 Testing now with latest stable u-boot ... v2016.11 ... it seems stable, but will leave until morning. 1
Igor Posted November 15, 2016 Posted November 15, 2016 Testing now with latest stable u-boot ... v2016.11 ... it seems stable, but will leave until morning. Still running, 2
apollon77 Posted November 15, 2016 Author Posted November 15, 2016 So using nightly/beta should work?
Igor Posted November 15, 2016 Posted November 15, 2016 So using nightly/beta should work? Yes, but tomorrow's build.
apollon77 Posted November 15, 2016 Author Posted November 15, 2016 Ok :-) I hopefully find time before the weekend
tpm8 Posted November 17, 2016 Posted November 17, 2016 Can't find any cubietruck uboot packages on beta any more?? E.g. http://beta.armbian.com/pool/main/l/linux-u-boot-cubietruck-default/doesn't exist...
Igor Posted November 17, 2016 Posted November 17, 2016 Can't find any cubietruck uboot packages on beta any more?? E.g. http://beta.armbian.com/pool/main/l/linux-u-boot-cubietruck-default/doesn't exist... True, today's build is broken ... fixing / will be fixed ASAP. 1
tpm8 Posted November 18, 2016 Posted November 18, 2016 Unfortunately - still no success with u-boot 5.24.161116 Unpacking linux-u-boot-cubietruck-default (5.24.161116) over (5.23) ... Setting up linux-u-boot-cubietruck-default (5.24.161116) ... Updating u-boot on device /dev/mmcblk0 I've seen this time two different errors as soon as some load was put on the cubietruck. Sometimes it won't even boot to the prompt. Unable to handle kernel paging request at virtual address xxxxxx or Console: Internal Error: Oops - undefined instruction: 0 [#1] PREEMPT SMP ARM Again, starting LIMA Memtester crashed the cubie immediately. If it is of any use - I've uploaded detailed boot log: root@cubietruck:~# armbianmonitor -u /var/log/armhwinfo.log has been uploaded to http://sprunge.us/NDDH Hope this helps. TIA Tobias
Igor Posted November 19, 2016 Posted November 19, 2016 Reboot and make sure you got a proper u-boot. I made few tests and I could reproduce an error easily while on latest u-boot I could run limatester stable over the night.
zador.blood.stained Posted November 19, 2016 Posted November 19, 2016 If lima-memtester simply crashes, then it sounds more like power supply issue than DRAM instability or some other hardware issue.
apollon77 Posted November 19, 2016 Author Posted November 19, 2016 I just installed the "default" version on a Wheezy Legacy and the next on a Xenial ... will report if I have problems in normal operation
apollon77 Posted November 19, 2016 Author Posted November 19, 2016 Crash on the Xenial server (so vanilla = next) with 5.24 after 9h in normal operation (there is mainly an influxdb+redis slave running). Went back to 5.20 there. On Debian Wheezy (legacy=default) still ok ... will see there. Powering can not be the reason.
tpm8 Posted November 21, 2016 Posted November 21, 2016 Reboot and make sure you got a proper u-boot. I made few tests and I could reproduce an error easily while on latest u-boot I could run limatester stable over the night. I've prepared a new SD card from scratch with 5.20 image + all updates + 5.24 beta u-boot for the test. Several reboots with this clean image but still unstable on my cubietruck. Probably a different hardware revision? On my "production" SD card with u-boot 5.20 (otherwise same packages and same jessie legacy kernel) everything is rock solid. There i can Lima Tester for hours without problems. In order to completely rule out hardware problems I've ordered another (used) cubietruck. As soon as it is delivered I'm able to do more testing. 1
zador.blood.stained Posted November 21, 2016 Posted November 21, 2016 In order to completely rule out hardware problems I've ordered another (used) cubietruck. As soon as it is delivered I'm able to do more testing. Please check what DRAM chips do you have on your board, and ideally you should run lima-memtester with different DRAM clock speeds to find if lower value works for you.
tpm8 Posted November 21, 2016 Posted November 21, 2016 If lima-memtester simply crashes, then it sounds more like power supply issue than DRAM instability or some other hardware issue. I don't think it's power or hardware related, because same configuration works fine with 5.20 u-boot. lima-memtester runs without problems for hours... Maybe I used the wrong words. Calling lima-memtester does not crash itself but immediately freezes / crashes the kernel. The cube is shown on the screen. Anyway I've ordered another cubietruck to completely rule out individual hardware problems.
ssvb Posted November 21, 2016 Posted November 21, 2016 I don't think it's power or hardware related, because same configuration works fine with 5.20 u-boot. lima-memtester runs without problems for hours... Maybe I used the wrong words. Calling lima-memtester does not crash itself but immediately freezes / crashes the kernel. The cube is shown on the screen. Anyway I've ordered another cubietruck to completely rule out individual hardware problems. It would be best if you could compile U-Boot yourself and bisect this problem. You can check http://www.metaltoad.com/blog/beginners-guide-git-bisect-process-elimination or any other tutorial about using git bisect. Just using an older U-Boot release is only a workaround. This is unhealthy in the long run.
tpm8 Posted November 21, 2016 Posted November 21, 2016 Please check what DRAM chips do you have on your board, and ideally you should run lima-memtester with different DRAM clock speeds to find if lower value works for you. Could you please give me some advise - how to identify DRAM chips (dmesg? armbianmonitor?) - how to modify DRAM clock speeds? (do I have to compile an individual kernel for that? boot.cmd parameter?
tpm8 Posted November 21, 2016 Posted November 21, 2016 Crash on the Xenial server (so vanilla = next) with 5.24 after 9h in normal operation (there is mainly an influxdb+redis slave running). Went back to 5.20 there. On Debian Wheezy (legacy=default) still ok ... will see there. Powering can not be the reason. Almost glad that I'm not the only one...
ssvb Posted November 21, 2016 Posted November 21, 2016 Could you please give me some advise - how to identify DRAM chips (dmesg? armbianmonitor?) You can't get this information in a software way. Just check the markings on the four DRAM chips on the board. For example, the original https://linux-sunxi.org/Cubietruck used GT8UB512M8EN-BG chips and you can find them on the pictures. Could you please give me some advise - how to modify DRAM clock speeds? (do I have to compile an individual kernel for that? boot.cmd parameter? It is necessary to recompile U-Boot for this after changing the DRAM clock speed settings in the 'configs/Cubietruck_defconfig' file. You can find some instructions here: https://linux-sunxi.org/Mainline_U-Boot#Compile_U-Boot
apollon77 Posted November 21, 2016 Author Posted November 21, 2016 How it can be HW related when it works rock solid with one version of u-boot and instable with an other ?!
ssvb Posted November 21, 2016 Posted November 21, 2016 How it can be HW related when it works rock solid with one version of u-boot and instable with an other ?! Just try to find the exact commit which introduced the problem, and we will know more.
Recommended Posts