How to use with write protected filesystems todays?

[ElMariachi2021]

Hello!

 

I wish to make my OrangePi 'failsafe' in regards of spontaneous power losses and such, so I need to have it running write protected, with writes going to a special partition or just RAM.

 

My searches yielded an 'ancient' way to freeze the root filesystem using overlayroot, which isn't supoprted anymore (https://docs.armbian.com/User-Guide_Advanced-Features/)

 

As I think that this kind of 'headless embedded usage' is an usual scenario for SBC running Armbian and having seen there was a method for doing so before, I assume there probably is a popular modern method for doing so, but I couldn' find it.

How would I do this with the current Bullseye release on my Prange Pi Plus 2E?

Or am I suppsed to use the classic kernel?

Thank you!

[Igor]

8 hours ago, ElMariachi2021 said:

probably is a popular

Very few people need industrial grade boot process and those understsand they have to invest into their own custom way of booting and parititoning ... overlayroot is not the best but is the only cheap variant you will find. Does it work well so you can trust your business on? No idea. Look into the code and development history to see if you can trust it.

 

Last time I have tried it, it was working, but never on Debian variant. There something is wrong / missing ...

[ElMariachi2021]

Hello and thank you!

By "it was working" you mean on the new releases with mainline Kernel?

Or did you presuppose installing an legacy version of Armbian?

I guess I'd be able to make it running if there's no essential incompatibility, I just would like to know if it's even worth trying or a waste of time.

[Igor]

1 minute ago, ElMariachi2021 said:

By "it was working" you mean on the new releases with mainline Kernel?

It must work with modern kernel. I think the only kernel dependency is overlay_fs:
 

Spoiler

grep OVERLAY_FS=m *                                                                                                                                                                                                <master>
linux-bcm2711-current.config:CONFIG_OVERLAY_FS=m
linux-bcm2711-edge.config:CONFIG_OVERLAY_FS=m
linux-imx6-current.config:CONFIG_OVERLAY_FS=m
linux-imx6-edge.config:CONFIG_OVERLAY_FS=m
linux-imx7d-legacy.config:CONFIG_OVERLAY_FS=m
linux-media-current.config:CONFIG_OVERLAY_FS=m
linux-media-edge.config:CONFIG_OVERLAY_FS=m
linux-meson64-legacy.config:CONFIG_OVERLAY_FS=m
linux-mvebu64-current.config:CONFIG_OVERLAY_FS=m
linux-mvebu64-edge.config:CONFIG_OVERLAY_FS=m
linux-mvebu64-legacy.config:CONFIG_OVERLAY_FS=m
linux-mvebu-current.config:CONFIG_OVERLAY_FS=m
linux-mvebu-edge.config:CONFIG_OVERLAY_FS=m
linux-mvebu-legacy.config:CONFIG_OVERLAY_FS=m
linux-odroidxu4-edge.config:CONFIG_OVERLAY_FS=m
linux-odroidxu4-legacy.config:CONFIG_OVERLAY_FS=m
linux-rk322x-current.config:CONFIG_OVERLAY_FS=m
linux-rk322x-edge.config:CONFIG_OVERLAY_FS=m
linux-rk322x-legacy.config:CONFIG_OVERLAY_FS=m
linux-rk35xx-legacy.config:CONFIG_OVERLAY_FS=m
linux-rockchip64-legacy.config:CONFIG_OVERLAY_FS=m
linux-rockchip-current.config:CONFIG_OVERLAY_FS=m
linux-rockchip-edge.config:CONFIG_OVERLAY_FS=m
linux-rockchip-legacy.config:CONFIG_OVERLAY_FS=m
linux-rockpis-legacy.config:CONFIG_OVERLAY_FS=m
linux-s5p6818-legacy.config:CONFIG_OVERLAY_FS=m
linux-sun50iw9-legacy.config:CONFIG_OVERLAY_FS=m
linux-sunxi64-current.config:CONFIG_OVERLAY_FS=m
linux-sunxi64-edge.config:CONFIG_OVERLAY_FS=m
linux-sunxi64-legacy.config:CONFIG_OVERLAY_FS=m
linux-sunxi-current.config:CONFIG_OVERLAY_FS=m
linux-sunxi-edge.config:CONFIG_OVERLAY_FS=m
linux-sunxi-legacy.config:CONFIG_OVERLAY_FS=m
linux-uefi-arm64-current.config:CONFIG_OVERLAY_FS=m
linux-uefi-arm64-edge.config:CONFIG_OVERLAY_FS=m
linux-uefi-x86-current.config:CONFIG_OVERLAY_FS=m
linux-uefi-x86-edge.config:CONFIG_OVERLAY_FS=m

 

 

[ElMariachi2021]

Thank you very much!

I will make this running then!

 

[rio]

I have the same issue running Armbian 23.02 Bullseye for an octoprint setup which regularly sees hard power switchoff - making rootfs readonly seems reasonable; is there any update/progress or failure, please?

Edited April 6, 2023 by rio

[Igor]

23 hours ago, rio said:

making rootfs readonly seems reasonable

By default not really  But you can try into this direction https://spin.atomicobject.com/2015/03/10/protecting-ubuntu-root-filesystem/ (try to find more recent guide, perhaps even on this forum). 

[rio]

Quote

By default not really [in response to 'making rootfs readonly seems reasonable']

interesting. To make rootfs readonly was my futile idea to have armbian handle the regular hard power switchoffs with no clean unmounts&shutdowns gracefully. Do you have another suggestion for this recurring situation or do you think that any other means was neccessary at all? overlayroot was just something I found here from the past (but seemed to have stopped working out of the box for newer releases). I am very open to all suggestions.

Thank you

[Igor]

1 hour ago, rio said:

I am very open to all suggestions.

 

In cheap & plug and play range, this is the only option I am aware of. Last time I played with it, it was working on Ubuntu (IIRC Jammy) out of the box, while on Debian it didn't want to start. I didn't proceed in debugging.

[rio]

ok, thanks. Then I may eventually spend some time experimenting. If it works, I will report back. For now, other pressing issues have come up so I have to take chances (and move some of the more frequent changing application specific (log)files to one of the existing tmpfs portions and hope to reduce the risk of fs corruption beyond fsck automatic repairs)

 

I must admit I am surprised this feature was not requested  by most users of armbian - when choosing the sbc+armbian combination I had expected this was a typical scenario and would be covered; error on my side, live and learn, I guess

[Igor]

20 hours ago, rio said:

and move some of the more frequent changing application specific

This was adopted 8-9 years ago and was perfectioned since then. No need to worry about this. Perhaps this is the reason why read-only variants were not requested that much. Also SD card quality improved within years alongside with emergin eMMC / SSD as primary media also in this segment.

[rio]

ok, I will leave it at that and prepare a backup plus another sd-card to boot from.

Thanks

[ValdikSS]

16.02.2022 в 08:12, Igor сказал:

overlayroot… Last time I have tried it, it was working, but never on Debian variant. There something is wrong / missing ...

 

There's a small bug in overlayroot package: it requires grep to be in initramfs, but by default it is not included.

To fix this, you need:

1. Install busybox-static:

apt install busybox-static

 

2. Enable busybox in initramfs:

set

BUSYBOX=y

in /etc/initramfs-tools/initramfs.conf
 

3. Regenerate initramfs:

update-initramfs -c -k all

 

Now it should work.

 

P.S. the logs of overlayfs is in /run/initramfs/overlayroot.log

[VishnuDas]

@ValdikSS

Can you post the full instructions for creating write protected filesystems ?

[ValdikSS]

@VishnuDas

Step 0:

apt install overlayroot
echo 'overlayroot="tmpfs"' > /etc/overlayroot.local.conf

Then follow the steps above and reboot.

[Awesome-O]

I too was initially trying to use overlay-root in Debian and couldn't get it to work. Until I did a serial debug of the boot process.

Here's what I found.

1. Ubuntu has a built in executable called wait-for-root, which the overlay-root calls during boot.
2. I believe the Debian port just used the Ubuntu code, but excluded the wait-for-root.
3. There is no wait-for-root exe in Debian, hence why it was not configuring the overlay.
4. Like @ValdikSS said you also require busybox-static package. (I left my /etc/initramfs-tools/initramfs.conf with BUSYBOX=auto)

 

So I created my own wait-for-root script an placed it on the $PATH. (/usr/local/sbin)

 

#!/bin/sh
device=${1}
timeout=${2}
#
[ "$timeout" = "0" ] && return 1
# wait-for-root writes fstype to stdout, redirect to null
until [ -b $device ]; do
        echo "waiting for root device $dev"
        sleep $timeout
done

 

I worked out that the two parameters come from the overlay-fs, device and timeout.

With that worked out I then created the overlay.local.conf, as per the instructions in /etc to overlay my NVME disk from the eMMC. (No need to edit overlay.conf)

 

#
# Overlay Root Config
# NVME drive 0 partition 1
#
overlayroot="device:dev=/dev/nvme0n1p1,recurse=0,timeout=5"

 

You need to do an update-initramfs after to place the wait-for-root in the initrd.

 

After a reboot I now have this:

 

 

Caveats:

1. You cannot do swap files/partitions in the /etc/fstab when overlayfs is activated, it will comment them out.
2. If you're running docker it will not work with it's built in default overlay-fs. You will need to install the fuse-overlayfs package and create a custom docker config in /etc/docker/daemon.json

Solutions: Create a swap systemd unit that starts after  the overlay is active

1. Create the unit file in /etc/systemd/system
2. Name the file the same as the swap location. (In this case I named it dev-nvme0n1p2.swap, with dashes instead of /'s)

 

[Unit]
Description="Swap device (/dev/nvme0n1p2)"

[Swap]
What=/dev/nvme0n1p2
Options=defaults
Priority=10

[Install]
WantedBy=basic.target

 

 

Docker Daemon needs this added to it:

 

"storage-driver": "fuse-overlayfs",

 

 

 

Edited September 5 by Awesome-O

[ElMariachi2021]

Hello!

 

I have solved this issue back then.

It was due to device specific/bootloader issues on the GoFlexNET I am (still!) using as a printserver.

You can see how I solved it there: https://github.com/nils-werner/raspi-overlayroot/issues/7

The solution of course would work very similar, no matter if one is using Armbian, debian or Arch.

 

Edited September 6 by ElMariachi2021