shrink-backup - a tool for backing up sbc:s

[bedna]

shrink-backup is a bash script for backing up your SBC:s into an img file

 

I made this script because I wanted a universal method of backing up my SBC:s into img files as fast as possible (with rsync), no matter what os is in use.

 

shrink-backup

 

Tested on Raspberry Pi os, Armbian, Manjaro-arm and ArchLinuxARM for rpi with ext4 root partition.
Autoexpansion will not work on ArchLinuxARM (will not fail, only warn) at the moment but works on the other three tested.

Default device that will be backed up unless changed with -d is SD-cards, ie /dev/mmcblk0

 

Do not forget to make the script executable after downloading it.

 

Usage:

sudo shrink-backup -h
Script for creating an .img file and subsequently keeing it updated (-B), autoexpansion is enabled by default
Directory where .img file is created is automatically excluded in backup
########################################################################
Usage: sudo shrink-backup [-Uatyeldh] imagefile.img [extra space (MB)]
  -U         Update the img file (rsync to existing backup .img), no resizing, -a and -d is disregarded
  -a         Let resize2fs decide minimum space (extra space is ignored), disabled if using -U
  -t         Use exclude.txt in same folder as script to set excluded directories
             One directory per line: "/dir" or "/dir/*" to only exclude contents
  -y         Disable prompts in script
  -e         DO NOT expand filesystem when image is booted
  -l         Write debug messages in log file shrink-backup.log in same directory as script
  -d [PATH]  EXPERIMENTAL! Use custom device path. default = /dev/mmcblk0
             MAXIMUM 2 partitions, more and the script will not function correctly!
             Feedback on functionality is apreciated (https://github.com/UnconnectedBedna/shrink-backup/discussions)
  -h --help  Show this help snippet
########################################################################
Example: sudo shrink-backup -at /path/to/backup.img
Example: sudo shrink-backup -e -y /path/to/backup.img 1000
Example: sudo shrink-backup -Ut /path/to/backup.img
Example: sudo shrink-backup -ad /dev/sda /path/to/backup.img
Example: sudo shrink-backup -atd /dev/nvme0n1 /path/to/backup.img

 

The folder where the img file is created will ALWAYS be excluded in the backup.
If -t option is selected, exclude.txt MUST exist (but can be empty) within the directory where the script is located or the script will exit with an error.

 

Use one directory per line in exclude.txt.

/directory/* = create directory but exclude content.
/directory = exclude the directory completely.

 

If -t is NOT selected the following folders will be excluded:

/lost+found
/proc/*
/sys/*
/dev/*
/tmp/*
/run/*
/mnt/*
/media/*
/var/swap

 

Rsync WILL cross filesystem boundries, so make sure you exclude external drives unless you want them included in the backup.

 

Use -l to write debug info into shrink-backup.log file in the same directory as the script.

 

Applications used in the script:

• fdisk (sfdisk)

• dd

• parted

• e2fsck

• truncate

• mkfs.ext4

• rsync

 

Info:

Theoretically the script should work on any device with maximum 2 partitions (boot and root).
The script can handle maximum 2 partitions, if there are more than that on root device the script will fail with an error.
Even if you forget to disable autoexpansion on a non supported system, the backup will not fail.

 

Custom device part can be set with '-d /dev/xxx'. This function is untested simply because I lack good hardware for proper testing.

Feedback on functionality is highly appreciated! If -d is not selected, default device path is used: /dev/mmcblk

 

Order of operations - image creation

1. Reads the block sizes of the partitions
2. Uses dd to create the boot part of the system + a few megabytes to include the filesystem on root (this can be a partition)
3. Removes and recreates the root partition, size depends on options used when starting the script
4. Creates a new ext4 filesystem with the same UUID and LABEL as the system you are backing up from
5. Uses rsync to sync both partitions (if more than one)

 

This means it does not matter if boot is on a partition or f.ex uboot that Armbian uses.

 

Added space is added on top of df reported “used space”, not the size of the partition. Added space is in MB, so if you want to add 1GB, add 1024.

 

The script can be instructed to set the img size by requesting recomended minimum size from e2fsck by using the -a option.
This is not the absolute smallest size you can achieve bit is the “safest” way to create a “smallest possible” img file.
If you do not increase the size of the filesystem you are backing up too much, you can most likely keep it updated with the update function (-U) of the script.

 

To get the absolute smallest img file possible, do NOT set -a option and set “extra space” to 0

Example:

sudo shrink-backup /path/to/backup.img 0

 

This will instruct the script to get the used space from df and adding 192MB “wiggle room”.

If you are like me, doing a lot of testing, rewriting the sd-card multiple times. The extra time it takes each time will add up pretty fast.

Example:

-rw-r--r-- 1 root root 3.7G Jul 22 21:27 test.img # file created with -a
-rw-r--r-- 1 root root 3.3G Jul 22 22:37 test0.img # file created with 0

 

Disclaimer:
Because of how filesystems work, df is never a true representation of what will actually fit on a created img file.
Each file, no matter the size, will take up one block of the filesystem, so if you have a LOT of very small files (running docker f.ex) the “0 added space method” might fail during rsync. Increase the 0 a little bit and retry.

This also means you have VERY little free space on the img file after creation.
If the filesystem you back up from increases in size, an update (-U) of the img file might fail.

 

Order or operations - image update

1. Probes the img file for information about partitions
2. Mounts root partition with an offset for the loop
3. Checks if multiple partitions exists, if true, loops the boot with an offset and mounts it within the root mount
4. Uses rsync to sync both partitions (if more than one)

 

To update an existing img file simply use the -U option and the path to the img file.
Changing size in an update is not possible at the moment but is in the todo list for the future.

 

Disclaimer:

EEPROM updates might not be backed up in an image update, keep that in mind.
To be absolutely sure after doing changes to EEPROM it’s better to make a brand new img rather than updating.
As of this moment, there are no plans to include that functionality in the script.

 

To restore a backup, simply “burn” the img file to an sd-card using your favorite method.

 

A backup is not really a backup until you have restored from it.

Edited July 28, 2023 by bedna

[bedna]

A few changes introduced a bug that made armbian detect 2 partitions instead of one so If you have cloned it since op was made, you have to re clone it, the bug is now fixed.

Sorry about that.

Edited July 28, 2023 by bedna

[bedna]

Good news everyone!

 

Yesterday a new kernel update (5.15.93-sunxi64 > Linux 6.1.42-sunxi64) was pushed to my armbian on an opiPC2 and u-boot was triggered.

Made tests by updating my old image instead of creating a new, then trying to boot from that image with GREAT success!

 

I did not look deeper into what the u-boot actually did, but this time it worked at least.

[bedna]

Attention, attention, version 0.9.1 released!! come get your free copy! xD

 

A lot has happened:

• Device being backed up is what root resides on, ie works on any device now, not just mmcblk. No need to specify, script is smart enough to figure it out.
• ArchLinuxArm autoexpansion now supported without the need for any AUR downloads, only parted is needed.
• Tested to update with same image through kernel and boot changes on Armbian on orangePi-PC2 and no issues detected.
• Tested to update with same image through kernel and boot changes on ManjaroArm and no issues detected.
• Generally a more user friendly interface and the debug function is beyond explanatory. hint hint

Hope you find it useful!

 

Edit:

Sadly I can no longer edit the original post to update with current information, but the link to github is still valid and all new info is available there. 😄

Edited August 14, 2023 by bedna

[bedna]

Version 0.9.5 released.

 

Support added for OrangePi 5. (gpt partition table)

Experimental support for btrfs.

 

For more information, please see https://github.com/UnconnectedBedna/shrink-backup

[bedna]

==> shrink-backup v1.0 <==

 

I have made the decision to not deal with other partitions than boot and root for the 1.0 release.
Instead I introduced the --loop function to let the user expand the img file using the [extra space] option and then manually create partitions by running for example: sudo gparted /dev/loop0 in a terminal to edit partitions in a graphical interface using gparted.
I want to give the user freedom, but I also have to stay true to my initial plan with this script: a very fast utility to create a bootable img file from the system and subsequently keep it updated.

 

I haven't dropped the idea of at least handling /home completely, but the script goes from "kinda basic functionality" to "advanced script" pretty fast when I start working on the feature.
If I do this, I still want the script to be as easy as possible to use, but at the same time give power users the ability to fine tune, ie a lot of work.

 

Features in the release:

• Introduction of --loop, --fix & -z (zoom speed)
• Now crosschecks fstab with lsblk for certain operations.
• Changed MB to MiB etc. Old habits die hard.
• Will now, if needed, check and/or ask for installing gdisk on debian and arch based systems.
• GPT partition table now supported
• Various bug fixes.

I hope you find it useful!