arhi Posted May 20, 2018 Posted May 20, 2018 Something happened to my armbian, I executed rather simple find --execute grep ... troughout the whole sd card, it froze and will not boot any more .. attached a hdmi display (orangepi one is the board in question) and I see it boots the kernel and then just "rebooting..." and reboots and does that in a loop... on a normal linux running grub bootloader I'd stop the boot and add "single" to the kernel parameters, it will ignore all the boot scripts and after booting the kernel would drop me directly into shell as root so I can investigate wth is going on... anything similar I can do to armbian?
chwe Posted May 20, 2018 Posted May 20, 2018 https://docs.armbian.com/User-Guide_Fine-Tuning/#how-to-toggle-boot-output you might mount your sd-card on a linux machine and do it by your own.
arhi Posted May 20, 2018 Author Posted May 20, 2018 16 hours ago, chwe said: https://docs.armbian.com/User-Guide_Fine-Tuning/#how-to-toggle-boot-output you might mount your sd-card on a linux machine and do it by your own. thanks, increased verbosity, and created /boot/.force-verbose so let's see if that will show where the problem is
arhi Posted May 20, 2018 Author Posted May 20, 2018 didn't help .. starting kernel... pause ... reboot
arhi Posted May 20, 2018 Author Posted May 20, 2018 I understood I only need to add verbosity not that I need to recompile it ... how to recompile it on i386/amd64 linux?
chwe Posted May 22, 2018 Posted May 22, 2018 install the u-boot tools (forgot the package name) and then mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr.. if you make changes to boot.cmd
arhi Posted May 22, 2018 Author Posted May 22, 2018 15 minutes ago, chwe said: install the u-boot tools (forgot the package name) and then mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr.. if you make changes to boot.cmd problem is I can't get the darn thing to boot ... can't install anything on it... can edit sd card outside on the regular x86 linux that's all
chwe Posted May 22, 2018 Posted May 22, 2018 u-boot tools should also work on x86 machines (in fact, during the build of images those boot.cmd 's are also compiled on a x86 machine). To my knowledge as long as you set the right flags this should work. mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr I don't play long with bootscripts, so make a backup of the boot.scr before you recompile it... You can easy mess things up there (and if you're not that experienced with it, it drives you nuts until you figure out what's wrong with it)...
arhi Posted May 22, 2018 Author Posted May 22, 2018 clear, so I should make new files on the build env I have, that makes sense ... anyhow I Was not changing boot.cmd, the url https://docs.armbian.com/User-Guide_Fine-Tuning/#how-to-toogle-verbose-boot said it's enough to change verbosity= line in /boot/armbianEnv.txt from 1 to higher number (7 is max) and touch /boot/.force-verbose so I did not touch /boot/boot.cmd (and boot.cmd is already set to "both" so both serial and hdmi are showing output) .. problem is nothing in output makes sense lemme try to record it with hero as phone camera fails to properly focus
arhi Posted May 22, 2018 Author Posted May 22, 2018 not sure what *** Warning - bad CRC, using default environment actually means .. is it reporting crc error in bootloader? sd card? what's "default env" in this context ? looks like safest thing to do is reinstall the darn thing and restore backup from image the weirdest thing, it was working great for a while, then I did "find / -type f -exec grep 101 {} \; " and after 5-6 minutes it crashed linux and wouldn't boot after that
chwe Posted May 22, 2018 Posted May 22, 2018 hmm I don't like videos, but yeah you don't have much other possibilities to show what's going on... First a USB-UART would help yo in case something is visible on uart but not on hdmi, second starting Kernel is normally the last message you get from u-boot before the kernel controls output.. So it seems that he is only shortly able to post something on your HDMI before it f*cks up (maybe he talks a bit more to you on console, might be interesting... ).. But a good thing.. boot.scr (that's where your bootscript lives, which gives u-boot the possibility to set bootargs for the kernel) seems to be loaded and executed properly.. so you can set 'debug commands' for the kernel.. So there's still hope.. but I think you should record when possible on serial console rather than HDMI and hope that the kernel isn't that shy to talk to you.. cp boot.scr boot.scr.orig edit boot.cmd with the arguments you want to send to the kernel from u-boot (sitting in bootargs=blablabla) mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr Cause you consider starting from plain again.. there's not much you can lose... Edit: your u-boot talks much more to you that the one I'm currently working with.. Makes things easier.. 1
arhi Posted May 22, 2018 Author Posted May 22, 2018 noone likes videos but at least I didn't enclose video in word document I have bunch of usb/uart adapters around will attach one now but I doubt I'll see anything new since boot is already set to send data to both devices so I doubt I'll get more on uart but to be on the safe side - testing in few minutes now I don't mind editing and compiling boot.cmd but any arts you think would be useful?
arhi Posted May 22, 2018 Author Posted May 22, 2018 well, hooking up serial was def a good idea... more info (useful info) there .. Autoboot in 1 seconds, press <Space> to stop switch to partitions #0, OK mmc0 is current device Scanning mmc 0:1... Found U-Boot script /boot/boot.scr 3550 bytes read in 223 ms (14.6 KiB/s) ## Executing script at 43100000 U-boot loaded from SD Boot script loaded from mmc 166 bytes read in 192 ms (0 Bytes/s) 4086840 bytes read in 526 ms (7.4 MiB/s) 5818864 bytes read in 595 ms (9.3 MiB/s) Found mainline kernel configuration 26231 bytes read in 452 ms (56.6 KiB/s) 374 bytes read in 549 ms (0 Bytes/s) Applying kernel provided DT overlay sun8i-h3-i2c0.dtbo 4179 bytes read in 407 ms (9.8 KiB/s) Applying kernel provided DT fixup script (sun8i-h3-fixup.scr) ## Executing script at 44000000 ## Loading init Ramdisk from Legacy Image at 43300000 ... Image Name: uInitrd Image Type: ARM Linux RAMDisk Image (gzip compressed) Data Size: 4086776 Bytes = 3.9 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Flattened Device Tree blob at 43000000 Booting using the fdt blob at 0x43000000 Loading Ramdisk to 49c1a000, end 49fffbf8 ... OK reserving fdt memory region: addr=43000000 size=7000 Loading Device Tree to 49c10000, end 49c19fff ... OK Starting kernel ... Uncompressing Linux... done, booting the kernel. Loading, please wait... Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems done. Begin: Will now check root file system ... fsck from util-linux 2.25.2 [/sbin/fsck.ext4 (1) -- /dev/mmcblk0p1] fsck.ext4 -a -C0 /dev/mmcblk0p1 /dev/mmcblk0p1: recovering journal /dev/mmcblk0p1: Superblock needs_recovery flag is clear, but journal has data. /dev/mmcblk0p1: Run journal anyway /dev/mmcblk0p1: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. (i.e., without -a or -p options) fsck exited with status code 4 done. Failure: File system check of the root filesystem failed The root filesystem on /dev/mmcblk0p1 requires a manual fsck Rebooting automatically due to panic= boot argument weird fsck failure but I did a fsck of an image already and wrote the image on top of the SD already ... hm .. weird .. will have to put the card directly on to linux and do fsck of the card that might work..
arhi Posted May 22, 2018 Author Posted May 22, 2018 looks like a dead bloody card ... you write to it, it ignores writes ?!?!?!?! have not had this before .. wow .. well you learn something new every day thanks for help I'd probbly never put the serial on my own 1
chwe Posted May 22, 2018 Posted May 22, 2018 So I'll move this one two to the famous crappy SD-Card/PSU section as well (it was maybe not that crappy in the beginning but for sure.. now it is).... Just out of curiosity.. before you throw away the SD card... Can you format it with whatever you use.. and then let F3 or h2testw running and post the result you get here... It would enhance our collection of 'It's not a software problem - your SD card is crappy' (not to blame you for something, just to show people that those ideas of 'crappy/dead sd-cards' isn't only a myth.. otherwise people start to believe that SD-Cards life forever... ). 21 minutes ago, arhi said: well you learn something new every day thanks for help I'd probbly never put the serial on my own and to my shame.. I came up with UART a way to late.. It's normally the first recommendation... 43 minutes ago, arhi said: boot is already set to send data to both devices so I doubt I'll get more on uart but to be on the safe side well... I quoted this just for fun...
arhi Posted May 22, 2018 Author Posted May 22, 2018 29 minutes ago, chwe said: So I'll move this one two to the famous crappy SD-Card/PSU section as well PSU was the first thing I tested. Then I made IMG of the card (worked flawlesly), then loaded that img on linux, fsckd it and it had errors that were easily fixed and stored back into IMG. Then I dd img to the card and it went "ok" ... but nothing was solved .. running in circles, on and on and ... when you suggested serial it's when I seen the darn thing was not able to fsck the card... This is no-name, made in corea, 8G class 6 SD card that came with one of the raspbery pi boards I think or with some other hardware... but it's bin working 24/7 for few years so I don't care if it died it was worth it and the way it died is actually interesting - it switched to read-only mode. What I do dislike is that it does not report ANY error for writing. Any write you execute is "ok", and of course you read it ok from cache so you think it's there. You remove the card and dang no changes made. Quote Can you format it with whatever you use.. and then let F3 or h2testw running and post the result you get here nope, I dd /dev/zero to card and it shows that card is zeroed, I remove card, return it and the old content is there... no way to write anything to card, it's ignoring writes (silently) ... Quote well... I quoted this just for fun... the problem is my mini 7" hdmi screen was already there as I'm conteplating some project so I hooked the screen instead of uart and was expecting "same output" .. but looks like that "both" thing don't really work Thanks again! now copying, resizing, copying, repairing... to move this to a higher quality kingstop 4G card
chwe Posted May 23, 2018 Posted May 23, 2018 27 minutes ago, arhi said: Quote Can you format it with whatever you use.. and then let F3 or h2testw running and post the result you get here nope, I dd /dev/zero to card and it shows that card is zeroed, I remove card, return it and the old content is there... no way to write anything to card, it's ignoring writes (silently) ... that confirms also that the crappy card is dead.. and that this can fool you cause 'everything looks fine'... Read only is mostly a clear sign that things gone wrong.. And your cheap card costs you in fact more if you count the time you wasted to debug it.. (but you learned something, this might be worth for the future)... 27 minutes ago, arhi said: the problem is my mini 7" hdmi screen was already there as I'm conteplating some project so I hooked the screen instead of uart and was expecting "same output" .. but looks like that "both" thing don't really work It works to the point were everything works as expected... My personal opinion, everyone who deals with SBCs should spend 1-2$ for a USB-UART (IMO every board >20$ should be sent with a el cheapo USB-UART, but different story). Reporting kernel-output to UART works even with a 'near to death' kernel, whereas HDMI.... you figured it out by your own.. It struggles earlier.. All the HDMI port's on my SBCs are 'virgin' cause they are either running headless over Ethernet or connected via UART during troubleshooting.
arhi Posted May 23, 2018 Author Posted May 23, 2018 I'v seen many dead SD cards, from camera's opi's, rpi's .. but never like this, that it lies to you that writes go trough but it's just silently ignoring them .. one more thing to check for in future as for spending 1-2$ for usb-uart, I think I have more then a 100 of those around the house .. electronics is my passion and you need those non stop and everywhere I just trusted in hdmi (for the first time to be honest, the darn thing arrived few days ago and I was thinking if I might want to setup octoprint or some custom made system with some touch screen since I'm making a new enclosure for multiple printers with heated chambers, special part for all the electronics etc etc.. so the screen was "handy" .. well, one learn while one lives
Recommended Posts