Backup script for block devices

piknew · October 7, 2016

If you have it on SD then put card in SD Card reader and make an backup (by for example Win32DiskImager). If you have it on emmc then you may use script from this thread. Of course you must boot your device from SD card (it can be fresh Armbian, with pv tool installed and enabled authorization by public key).

BTW. I think I will adjust my script to support also "local" image storage (not only by SSH). "local" == locally mounted file system.

tkaiser · October 7, 2016

If you have it on emmc then you may use script from this thread. Of course you must boot your device from SD card

BTW: http://forum.armbian.com/index.php/topic/2033-question-h3-unified-imaging/ (Start reading from post #8 -- eMMC can be mounted as USB thumb drive. On a related note: WinDiskImager does no verify so it's a bad tool. Fortunately the Etcher folks want to add backup/clone functionality to their tool in the future)

earth08 · October 16, 2016

How to backup my whole os installed on emmc

So when ever I need a fresh install I use the backup

piknew · October 16, 2016

How to backup my whole os installed on emmc

So when ever I need a fresh install I use the backup

I answered you already, but you have asked the same question again... If something is not clear you have to point "what" exactly should be elaborated.

earth08 · October 19, 2016

I answered you already, but you have asked the same question again... If something is not clear you have to point "what" exactly should be elaborated.

I am not clear, there

I am on emmc and I would like to back it on my sdcard/pendrive and than copy it to my pc.

Please guide.

Thanks in adv.

tkaiser · December 19, 2016

Addendum: New tools that look promising:

Use case: A eMMC equipped SBC that boots from SD card to make an eMMC backup (clone) to a remote location. The 8GB eMMC is a rather fresh Armbian installation only using 15% of the eMMC's capacity:

/dev/mmcblk1p1  7.1G 1012M  6.0G  15% /

On the remote host we start 'mbuffer -I 9090 >sbc.img.zst'. On the H3 device we now do:

root@bananapim2plus:~# dd if=/dev/mmcblk1 | mbuffer | zstd - -c | mbuffer -O 192.168.83.146:9090 
in @  0.0 KiB/s, out @  0.0 KiB/s,  871 MiB total, buffer   0% full15269888+0 records in
15269888+0 records out
7818182656 bytes (7.8 GB, 7.3 GiB) copied, 221.039 s, 35.4 MB/s

summary: 7456 MiByte in  3min 41.0sec - average of 33.7 MiB/s

summary:  871 MiByte in  3min 41.0sec - average of 4037 KiB/s

The whole stuff was both bottlenecked by CPU and I/O (tested on BPi M2+ that uses a pretty slow eMMC). On boards with faster eMMC (every Orange Pi for example) things would be faster. On average we read from eMMC with 35.4 MB/s. After piping through zstd on the other host just 871MB arrive as a result of the compression (which means we had a network throughput of 4037 KiB/s) and the whole process took 3:41 minutes.

To optimize this we could simply split the whole /dev/mmcblk1 in 500 MB chunks and let two of them being processed in parallel. One process starts from beginning of the device, the other from the end. Please remember: As long as the zstd command is fed with real data CPU is the bottleneck and as soon as the 'empty' parts are processed reading from eMMC becomes the bottleneck. With the aforementioned approach it should be ensured that as long as possible one zstd process is fed with real data and the other running in parallel processing 'empty space'. So on a fast eMMC (read as: not on BPi M2+ or the other newer SinoVoip devices that chose the cheapest Samsung eMMC available) this parallelism should speed up things further.

For more details (eg. how to 'restore' such a bunch of .zst file to an SD card for example) please read through the first linked comments thread above.

tkaiser · December 19, 2016

And now the fun part! My first try was to make a device backup from an eMMC with a fresh Armbian installation on it. But the eMMC itself was quite often in use before (I use the BPi M2+ only for tests in the meantime). So it doesn't really matter how 'full' the eMMC seems to be when looking at it from the filesystem perspective since due to the prior usage the unused space isn't just zeros (that compress pretty good) but on the eMMC is data from the former installations that compresses not that great since 'real data'.

How to improve that? Zero out all the unused space first. So I mounted the ext4 fs on the eMMC and let simply all 'free space' overwrite with zeroes:

root@bananapim2plus:~# dd if=/dev/zero of=zeroes bs=10M || rm zeroes
dd: error writing 'zeroes': No space left on device
617+0 records in
616+0 records out
6468685824 bytes (6.5 GB, 6.0 GiB) copied, 1077.24 s, 6.0 MB/s

OMFG, really just 6.0 MB/s write speed. Did I already mention that SinoVoip uses really cheap/slow eMMC on their devices? (to be honest: as long as you write just a few MB this doesn't matter but such an excercise as above where you want to write +6GB takes some time on such an ultra slow eMMC -- every good SD card is almost four times faster here!)

Now let's repeat this:

root@bananapim2plus:~# dd if=/dev/mmcblk1 | mbuffer | zstd - -c | mbuffer -O 192.168.83.146:9090 
in @  8.0 KiB/s, out @  8.0 KiB/s,  317 MiB total, buffer   0% full15269888+0 records in
15269888+0 records out
7818182656 bytes (7.8 GB, 7.3 GiB) copied, 207.588 s, 37.7 MB/s

summary: 7456 MiByte in  3min 27.5sec - average of 35.9 MiB/s

summary:  317 MiByte in  3min 27.5sec - average of 1563 KiB/s

Processing time got only slightly better but we have huge savings in disk space and transmittion: Before we had to transfer 871 MB compressed data but now it's just 317 MB any more. Since read speeds with this cheap eMMC are limited to ~40 MB/s we got only a slight increase of average read speed (on Orange Pi's it would be twice as much for example).

Lessons to learn:

In case you want to do device backups/clones at least once zero out every unneeded space. This will greatly reduce disk space and network bandwidth needed since zeroes compress pretty good compared to -- existing but unused -- real data patterns (TODO: Have a look whether eMMC implementation support fstrim)
Know your bottlenecks and act accordingly. In our case that means setting up some parallelisms to process different data patterns in parallel

After I zeroed out the empty space the following assumption is both easy and true when using a fresh Armbian installation: If the FS in question is filled with 1012 MB according to df then the first 1012 1MB blocks contain data and the all blocks behind only zeroes. Let's take a look. I started to receiving mbuffer instances on different ports and then process both parts individually:

The data part (1012 MB before compression become 289 MB after) finishes pretty fast:

root@bananapim2plus:~# dd if=/dev/mmcblk1 bs=1M count=1012 | mbuffer | zstd - -c | mbuffer -O 192.168.83.146:9089
in @ 4417 KiB/s, out @ 4417 KiB/s,  289 MiB total, buffer   0% full1012+0 records in
1012+0 records out
1061158912 bytes (1.1 GB, 1012 MiB) copied, 71.4225 s, 14.9 MB/s

summary: 1012 MiByte in  1min 11.4sec - average of 14.2 MiB/s

summary:  289 MiByte in  1min 11.4sec - average of 4146 KiB/s

Now the 'empty space' (6444 MB become just 27.6 MB after compression and the whole process is mainly bottlenecked by slow eMMC):

root@bananapim2plus:~# dd if=/dev/mmcblk1 bs=1M skip=1012 | mbuffer | zstd - -c | mbuffer -O 192.168.83.146:9090
in @ 40.3 MiB/s, out @ 40.2 MiB/s, 6433 MiB total, buffer   0% full6444+0 records in
6444+0 records out
6757023744 bytes (6.8 GB, 6.3 GiB) copied, 179.819 s, 37.6 MB/s

summary: 6444 MiByte in  2min 59.8sec - average of 35.8 MiB/s

summary: 27.6 MiByte in  2min 59.8sec - average of  157 KiB/s

tkaiser · December 19, 2016

Last update: By increasing compression level for 'normal data' (zstd defaults to 3 and now I used '-5') the time to process increased from 1:11 minutes to 3:03 (therefore as long as processing the zeroes needs) but we end up with a smaller backup image now:

root@bananapim2plus:~# dd if=/dev/mmcblk0 bs=1M count=1012 | mbuffer | zstd -5 - -c | mbuffer -O 192.168.83.146:9089
in @ 3826 KiB/s, out @ 3826 KiB/s,  254 MiB total, buffer   0% full1012+0 records in
1012+0 records out
1061158912 bytes (1.1 GB, 1012 MiB) copied, 183.417 s, 5.8 MB/s

summary: 1012 MiByte in  3min 03.4sec - average of 5651 KiB/s

summary:  255 MiByte in  3min 03.4sec - average of 1422 KiB/s

So compared to an archive that needs 871 MB and took 3:41 min to finish we are now at 283 MB archive size and ~3 minutes processing time. If this board would feature faster eMMC and by using more parallelization processing time could further improve.