backup & restore

[hatahata]

for  yonger newbees , i may contribute for ther progress .

 

i backup from 32G sd card , and restore it to 16G  sd card .

 

-----------------------------------------------------------------------

 

i usually use debiandog linux (a kind of debian ) as backup and restore tool .

 

1) insert 32G sd card to sd card reader .

 

2)insert  the sd card reader into usb interface of debiandog  .

 

3) mount /dev/sdb1 /mb1

 

4) mkdir /ArmBack

 

5) ( cd /mb1 ; tar cvpf - .)|(cd /ArmBack ; tar xpf -)

 

6) umoun /mb1

 

7) remove sd card reader and then 32G sd card .

 

 

 

then i prepare 16G sd card .

 

A) insert 16G sd card to sd card reader and insert the  sd card reader into the usb interface of debiandog

 

B)dd if=/Armbian_5.05_Orangepih3_Debian_jessie_3.4.110_desktop.raw  of=/dev/sdb bs=16M

 

but when time laspe 15 second or so , i pus ' ctrl + z '  several  times .

 

C)fdisk -l /dev/sdb  ( only for certainnty)

 

D)mkfs.ext4 /dev/sdb1

 

E)mount /dev/sdb1 /mb1

 

F) ( cd /ArmBack   ; tar cvpf - .)|(cd /mb1 ; tar xpf -)

 

G) umount /mb1

 

H) insert 16 G sd card into orange pi PC , then orange pi PC boots armbian .

 

this is very simple and stable method , i think .

but there are better methods .

please show them .

 

dd is bulky ,so i do not like dd and cannot do downsizing .

upsizing is easy only to use gparted on debiandog .

 

---------------

regards . by tuyosi

[stamasd]

That assumes, of course, that you don't already have more than 16GB of data on the 32GB card. Because if you do it won't work.

 

It also has a chance of losing data even if you have less that 16GB on the card, because of the way ext2/ext4 work. There's no guarantee that some files haven't already been written beyond the first 16GB point.

[martinayotte]

 

It also has a chance of losing data even if you have less that 16GB on the card, because of the way ext2/ext4 work. There's no guarantee that some files haven't already been written beyond the first 16GB point.

What do you means ?

The first "tar" is reading filesystem ext2/ext4 without any issues, wherever the file blocks are located, and the second "tar" restore them on the PC, so there is no 16GB limit.

[tkaiser]

The first "tar" is reading filesystem ext2/ext4 without any issues, wherever the file blocks are located, and the second "tar" restore them on the PC, so there is no 16GB limit.

 

This is true, the 'location' of files before doesn't matter. But in case the 32GB card has been filled with 50% or more then the whole approach is useless.

 

BTW: This whole thing is not about backup but just about creating a clone with maximum efforts. To create such a clone I wouldn't use tar and an intermediate PC/storage but better rsync. And to get u-boot+spl written I wouldn't write a whole Armbian image to the card but instead just this: u-boot+spl. And then this still has nothing to do with backup.

[stamasd]

What do you means ?

The first "tar" is reading filesystem ext2/ext4 without any issues, wherever the file blocks are located, and the second "tar" restore them on the PC, so there is no 16GB limit.

I stand corrected; I hadn't actually looked close at how he makes the backup in the first place. But it still wouldn't work if there is more than 16GB of data, simply because there isn't enough space on the 16GB card to hold it all.

[martinayotte]

Of course, I would try to put 2 litters of milk in a 750ml bottle ...

 

But the beauty of "tar piped in another tar" is that you can even do it across SSH.

(For full backup, you still need to be aware of not including backup of special folders, such as "/lost+found" or "/proc" )

 

Or course, the rsync method is good too !

[tkaiser]

But the beauty of "tar piped in another tar" is that you can even do it across SSH.

(For full backup, you still need to be aware of not including backup of special folders, such as "/lost+found" or "/proc" )

 

This is still not backup since backup needs versioning (to protect from accidentally damaging/deleting files). And 100% identical clones can be created way more efficient.

 

One approach is to use rsync combined with hardlinks (web search for "rsync hardlinks backup" or something like that) and anyone using mainline kernel should have a look into btrfs snapshots and 'btrfs send/receive'.

[rodolfo]

SDcard Cloning 

 

SDcard cloning is simply done with dd if=<source-device> of=<target-device>

 

If your target device is smaller ( 8GB ) than your source device ( 16GB ) and the net content (<=8GB) fits onto the target device 

you first have to move/resize your partitions and the data contained within to the beginning of the source drive with gparted. 

After cloning, you may expand the partitions on the target SDcard again with gparted.

 

The smart solution is to start with master SDcards of SMALL capacities ( 4 GB ) before your shiny new system is reasonably stable

and then maybe copy to larger production SDcard. Keep your basic system small and use readily available other storage for data

( usb flash, hdd, network drives ).

 

 

Backup/Restore

 

Backup/restore of partition data is simply done with rsync. By using hardlinks you can achieve a fully versioned backup-solution allowing

complete disaster restores ( somebody - certainly not you  - did "nothing" and everything blew up ) from generations of FULL backups. You are

also able to restore a file from three days ago ( before you were struck by a bright solution ) into your de-repaired running system.

[RagnerBG]

If we talking about move and/or resize images from card to card, i am using the above method. Here is explained in details - Shrinking images on Linux. But @hatahata method look more faster.

[hatahata]

i only do cloning of { 32GB micro sd card )  to { 16GB  micro sd card } .

 

and  { 16GB  micro sd card } works on orange pi PC .

 

but this method is bad for raspberry pi 2 by unknown reasen (good for ordinal Linux & OpenBSD).

at raspberry pi 2 , 

killall gvfsd-fusecd {hard disk mount point }cp --archive /bin /etc /home /lib /opt /root /run /sbin /srv /tmp /usr /var ./mkdir -p boot dev sys mnt proc 

but this forum is not for raspberry pi .

 

there must be more effective backup methods  (backup to hard disk drive)  using rsync ?

becasu sd card will  be distroyed some day (sd card has shorter life time  than harddisk , i think . ) .

 

for example

rsync -avHx --delete  {sd card mount point}/  { hard disk folder }/

is this OK or not for link  especially hard link ?

 

-----

my page is http://orange-pi-2.blogspot.jp/2016/01/orange-pi-2g.htmlbut written in Japanese .

 

if  tool is permitted , see

http://forum.armbian.com/index.php/topic/897-backup-restore/?hl=backup .

 

i use ' partclone.ext4 -c -s /dev/sdc1 -o /mb1/OrangepiPC-armbian.pcl , then iget  {  5822461748  3月 27 12:30 OrangepiPC-armbian.pcl } from 16GB micro sd ard .
 

[RagnerBG]

but this method is bad for raspberry pi 2 by unknown reasen (good for ordinal Linux & OpenBSD).

at raspberry pi 2

 

On Raspbian, Bananian and some others, you have small bootable fat32 partition. You have to clone it too.

[tkaiser]

there must be more effective backup methods  (backup to hard disk drive)  using rsync ?

 

Cloning is not backup. Never.

 

To create a clone from time to time that can be transferred to a new SD card on a Linux PC two steps are necessary: After connecting the card, you simply transfer the 1st MB to a file by doing:

dd if=/dev/$card of=u-boot_spl_partition-table.img bs=1M count=1

You use the whole device here and not a partition since you want u-boot+spl. Then you mount the first partition to eg. /mnt/armbian/ and then create one huge compressed tar archive excluding uneeded stuff relying on this list (paths need to be relative therefore remove the leading slashes):

cd /mnt/armbian/ && tar -czf clone.tar.xz --options=compression-level=9 --exclude-from=$exclude-list *

Your PC is fast and SD cards are slow so this is the best variant to create an archive with minimal size in minimal time. If you want to use this stuff with a new SD card later simply write u-boot_spl_partition-table.img with dd to the new card, use parted or gparted to adjust/fix the partition table, mount the first partition and then do a

cd /mnt/armbian/ && tar -xf clone.tar.xz

But this is not backup but just a clone. Backups need versions to be able to recover from accidents. In case a x86 Linux box is involved a more reasonable approach would be to use a btrfs FS with maximum ZLIB compression on the PC and then either create snapshots on the SBC (when used with mainline kernel and using btrfs for the rootfs -- Armbian is capable to benefit from btrfs thanks to Zador!) and send them using btrfs send/receive or in case you have to use ext4 on the SBC use rsync to send the changes to the PC (and use btrfs snapshots there or rsync with hardlinks to store multiple copies of data without wasting space).

 

For desaster recovery you still only need the 1MB file u-boot_spl_partition-table.img created sometimes before (or use the appropriate u-boot package from apt.armbian.com) and can then transfer the contents of your backup storage to a new SD card.

 

In case you have a bunch of SBCs that need to be backed up then better use deduplication instead of compression on the Linux PC (also a btrfs feature worth a look)

[hatahata]

i read   the post repetedly .

 

 

10 days ago , i tried with armbian of orange pi PC

 

 dd if=/dev/mmcblk0 of=/ma1/mbr.img bs=512 count=1

 dd if=/mnt/mbr.img of=/dev/mmcblk0 bs=446 count=1

 

but  boot loader is not copied to 16G smaller sd card  and  i cannot boot orange pi with sd card .

 

so i did

---------

B)dd if=/Armbian_5.05_Orangepih3_Debian_jessie_3.4.110_desktop.raw  of=/dev/sdb bs=16M

but when time laspe 15 second or so , i pus ' ctrl + z '  several  times .

-----------

this can copy boot loader and only structure (sd card is composed with one ext4 partition ) .

 

 

i understand 

dd if=/dev/$card of=u-boot_spl_partition-table.img bs=1M count=1

 

by the way 

does {  dd if=u-boot_spl_partition-table.img    o f=/dev/$card   bs=1M count=1 } copy bootloader ?

and

is 1M ok or not ?

 

i heatfuly thanks for vast knowledge and talent .

-----

regards

[tkaiser]

dd if=/dev/mmcblk0 of=/ma1/mbr.img bs=512 count=1

vs.

 

dd if=/dev/$card of=u-boot_spl_partition-table.img bs=1M count=1

 

In your example you copied the first 512 bytes (obviously not that much), in my example I copied the first 1MB (1048576 bytes). This 'little' difference is all that's needed to get the bootloader stuff.

[xwalter]

I think there are too much confusion .....I try but it doens't work 

Could someone reorganize this article and write the procedure as well ?

Image of our orange pi is very important ....I have also GParted , What Can I use it ?....to format ?

[martinayotte]

The general idea is to first get the bootloader written in the first sectors. So, TKaiser suggestion to "dd" the first 1MB is accomplishing this task.

The second step is to have the partition sized properly with the SDCard size, so either fdisk or gparted can do that, deleting the partition and re-create new one that fit on the card.

Third step is to format this partition, either mkfs.ext4 or gparted can be used.

Fourth step is actually copying the whole rootfs from the orinal SD to the new one, to do that, both needs to be mounted, then a "tar piped in a tar" or "rsync" will copy the whole thing.

Last step is to make sure to securely umount sdcards. Then, enjoy your backup ... :-)

[tkaiser]

BTW: The suggested task to shutdown the system, then eject the SD card and do a safe offline clone (no, I still wouldn't call this backup ) won't work that nice with boards where the rootfs is stored on an internal eMMC or NAND.

The other requirement when trying to use dd (same or more amount of available storage at the destination necessary since otherwise cloning will fail) can also be somewhat hard. Given you use an OPi Plus 2E with 16GB eMMC you want to clone to a 16GB SD card you might notice that your eMMC has a few more sectors than the SD card you want to use. So even if your installation on eMMC is only 2 GB in size you would've to buy a 32 GB SD card to be able to clone the whole device.

 

As an alternative our nand-sata-install.sh script could be the basis for an armbian-backup script since it knows how to deal with this bootloader stuff already and can transfer installations to another media using a different size. And since it's already using rsync for the copy process incremental further clones would run way faster than the initial transfer.

 

I created an Request for Enhancement an hour ago for a related topic: https://github.com/igorpecovnik/lib/issues/341

 

Please feel free to think a little bit about this and take your chance to improve our software and get something nice in return. It wouldn't be that hard to create an armbian-backup script based on our nand-sata-install.sh script that simply does it right and saves users from the hassles of dealing with all this stuff.

 

Such a script has to take care of initially transferring all the necessary stuff to the destination (bootloader and the like), then using rsync incrementally updating the clone and by checking the installation state also updating bootloader stuff (u-boot+spl in case of H3/sunxi boards) in case something has changed here.

[Jimmy Belanger]

I resolve this problem

 

clonezilla

 

copy the entire sdcard 32GB but only 8GB of data...

 

I going to create 7.8GB of file 

 

and restore with clonezilla and dont USE the partition resize option ...

 

copy working 32GB to 8GB... if you have really only 7.8 gb of file !