0
gailu

Armbian sd card image gets corrupted when power off without proper shutdown

Recommended Posts

Hi Experts,

 

We are using Armbian with Orange Pi zero. Image works fine. We do not have battery in our system and power goes off frequently in India. Sometime when Orange Pi zero is switched off and switched on again, its image gets corrupted and only way to recover is to reflash the card which will not be possible if we send product for field testing. When image is corrupted we see following logs

 

35152 bytes read in 501 ms (68.4 KiB/s)
## Loading init Ramdisk from Legacy Image at 43300000 ...
   Image Name:   uInitrd
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    3113546 Bytes = 3 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
 
Starting kernel ...
 
 
Error: unrecognized/unsupported machine ID (r1 = 0x00000000).
 
Available machine support:
 
ID (hex)        NAME
00001029        sun8i
 
Please check your kernel config and/or bootloader.
 
What could cause change of machine ID and how to fix it?

Share this post


Link to post
Share on other sites

I was curious about this so looked around and there is a lot of raspberry pi ups solutions, most run from a lipo cell and switch power source when the 5v supply fails.

 

Most are $30+ so you may as well just buy a very cheap regular UPS

 

This is the cheapest one I've seen: https://www.aliexpress.com/item/UGEEK-UPS-HAT-with-Battery-for-Raspberry-Pi-3-Model-B-2B-B-Raspberry-Pi-Battery/32786176030.html

 

http://www.raspberrypiwiki.com/index.php/Raspi_UPS_HAT_Board

 

more:

https://www.modmypi.com/raspberry-pi/breakout-boards/pi-modules/ups-pico

 

http://juice4halt.com/

 

a more ghetto way is find a powerbank that is able to do passthrough charging

Share this post


Link to post
Share on other sites

I was curious about this so looked around and there is a lot of raspberry pi ups solutions, most run from a lipo cell and switch power source when the 5v supply fails.

 

There are some that don't allow checking battery voltage/capacity so while they're advertised as UPS they don't allow for a safe shutdown when battery is drained too much. The better idea is to rely on boards with an own PMIC that is made for exactly this purpose. All Allwinner tablet SoCs (the A series) for example have PMIC support (but not all boards provide the necessary circuitry). Getting a CHIP + 3.7V battery is maybe the cheapest solution, Olimex A10 Lime + one of their batteries might still be cheaper compared to OPi + UPS and maybe this device will be priced competitively and batteries are available for the connector used: https://forum.armbian.com/index.php/topic/3251-bpi-r16/

 

I'm still hoping that we see in a few months cheap IoT devices based on R16 with full battery support from more than one (this) vendor...

Share this post


Link to post
Share on other sites

@Igor,

 

Your solution really seems to be bulletproof for my need. However I am not able ti install overlayroot package.

 

Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package overlayroot
 
Do I need to update something to install this package on Pi Zero. I am using image Armbian_5.24_Orangepizero_Debian_jessie_3.4.113.img and run apt-get update before trying to install overlayroot. Any direction to install it will be very helpful.

Share this post


Link to post
Share on other sites

@Igor

 

I tested with Ubuntu Xenial and yes indeed its working out of the box. Thank you very much.

 

I need one more input to make by board really bullet proof against sudden power-off. Hope experts can help.

 

Based on your suggested method I understand that my rootfs is frozen to a known state but I am not clear if my u-boot is also frozen or not. If not how can I do that?

 

I googled for error that I faced "Error: unrecognized/unsupported machine ID (r1 = 0x00000000)" and came to a conclusion that this is due to machine ID mismatch in u-boot and Linux. My Linux is secure with overlayfs but what about u-boot? 

 

Thank You.

Share this post


Link to post
Share on other sites

I checked the back of my Orange Pi Zero and SPI flash is already soldered. So does it means that u-boot is already in SPI nor flash so we don't need to take care of that?

Share this post


Link to post
Share on other sites

@martinayotte

 

Thank your very much for your help.

 

This means that u-boot is also on sd-card and subjected to corruption unless protected properly. Am I right?

 

1) I do not see /dev/mtd0 (using armbian Ubuntu Xenial) entry. Do I need to enable something?

 

2) If I flash u-boot in spi flash how do I ensure that u-boot in spi flash is used?

 

3) How do I tell to u-boot in SPI flash to boot linux from sd-card?

 

4) Will protecting rootfs with overlayfs also protects u-boot on sd-card? In that case I don't need to flash u-boot in SPI flash probably.

 

Is there any guide/wiki/howto available to boot armbian image on sd-card with u-boot in SPI flash?

Share this post


Link to post
Share on other sites

I checked following at http://linux-sunxi.org/Xunlong_Orange_Pi_Zero

Mainline U-Boot
Just use the orangepi_one_defconfig until specific Zero support has materialized.
You can define CONFIG_SPL_SPI_SUNXI=y and CONFIG_SPL_SPI_FLASH_SUPPORT=y to get SPI flash support (if you have a chip soldered). If the U-Boot build boots (test by either writing u-boot-sunxi-with-spl.bin to a uSD card or booting via FEL), you can write this very image to the SPI flash to boot from there:

Looks like SPI flash support is not yet added for orange pi zero images that might be the reason why I do not see /dev/mtd0.

Share this post


Link to post
Share on other sites

@zador.blood.stained

 

Thank you very much for your help and confirmation. That really helps me to avoid all the hassles of putting u-boot in SPI flash.

 

Thanks Again.

Share this post


Link to post
Share on other sites

Just got some more inputs so would like to make this thread live again if that helps improving Armbian reliability.

 

Now when my system is protected with overlayrootfs. I see an important log as below regarding machine id

[FAILED] Failed to start Commit a transient machine-id on disk.
See 'systemctl status systemd-machine-id-commit.service' for details.

To my gut feeling this service seems to be the root cause of machine id mismatch as reported in first post of this thread. Probably it does is job as expected in clean bootup and shutdown but fails to handle race condition for sudden power off may be during boot phase itself that causes to corrupt machine-id on the sd-card.

 

Two short questions:

1)  Is my suspicion justified ?

2) How could I remove this service altogether from Ubuntu Xenial Image?

Share this post


Link to post
Share on other sites

1)  Is my suspicion justified ?

 

I you're still looking after your random corruption issues in u-boot then of course not at all. I'm talking about this here:

Error: unrecognized/unsupported machine ID (r1 = 0x00000000).

This is an issue between mainline u-boot and legacy kernel (no systemd services involved at all!) and 'machine ID' has to be the one set from boot script. You should try to nail your problems (FS corruption for whatever reasons) down and don't play wording games (looking for 'machine id' here and there)

Share this post


Link to post
Share on other sites

"How could I remove this service ..."

 

By removing systemd ;):rolleyes::ph34r: . No, I am kidding (or not ?). But I never paid attention to this machine-id stuff. It seems to be one more "modern linux" invention from systemd. It only exists on machines with systemd and although it is supposed to identify uniquely identify a machine, it seems to be permanently fixed in a specific release and identical on each host ?!?

 

Is that important that a "transient superfluous gadget" cannot be updated because of a RO fs, whereas much more important software fills the log files because it remains in beta version ? :mellow:  :wacko::rolleyes:

Share this post


Link to post
Share on other sites

removing systemd

 

Please don't add to his confusion. He's doing something wrong (allowing power to be cut off without a sane shutdown) and reported about one occurence of FS corruption that led to 'setenv machid 1029' as usually set by boot script to not work anymore. I don't see any traces for this specific 'machine id mismatch' issue happened more than one time so we're simply speaking about FS corruption and paying too much attention on one individual occurence of the broader problem.

 

The solution is to prevent FS corruption and the whole thing has nothing to do with this systemd service or 'machine id' in general. Prevent FS corruption or use suitable hardware (with UPS battery style) and you're done.

Share this post


Link to post
Share on other sites

On the latest images u-boot machine ID is stored in the armbianEnv.txt, and also armhwinfo script changes verbosity on each startup and shutdown by editing this file, so the changes may not be flushed properly on unclean shutdown. So IMO this "crash detection by increasing verbosity" should be disabled in the future since I don't remember the last time we got any useful logs with it.

 

Edit: I remember noticing some garbage in armbianEnv.txt after some tests (where I don't bother with clean shutdowns if I don't expect to boot the image more than one time)

Share this post


Link to post
Share on other sites

So IMO this "crash detection by increasing verbosity" should be disabled in the future since I don't remember the last time we got any useful logs with it.

 

Good spot. Agree with that. So should we just disable the 'verbose logging' at all or revert back to touching an empty file as we did before? This will at least prevent armbianEnv.txt being corrupted on an unclean shutdown.

Share this post


Link to post
Share on other sites

So should we just disable the 'verbose logging' at all or revert back to touching an empty file as we did before? This will at least prevent armbianEnv.txt being corrupted on an unclean shutdown.

Just disable. Editing a plain test file to change the verbosity is easy enough so we don't need any kind of additional automation here IMO.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
0