Backup question NanoPiM4V2

[pkfox]

Hi all, first off, I love my NanoPiM4V2 boards, the big drawback with them for me is backing them up, if you are running them from an eMMC and you have the board in a case, you have to remove the cover of the case, if you have a hat ( which I do ) you also have to remove that , then you can remove the eMMC module and clone or do whatever with it. What would be good for me, is, if  I could temporarily boot from an sdcard , unmount the eMMC, and clone it for backup purposes. Before I ruin anything I thought I'd ask the experts. Is this doable ?

[Igor]

31 minutes ago, pkfox said:

backing them up

Backup your data and changes to the system (additional packages and configs).

[pkfox]

1 minute ago, Igor said:

Backup your data and changes to the system (additional packages and configs).

Hi Igor, I would like to create a system image, on my windows boxes I use AOMEI which you can use on a running system and it creates a snapshot of your current box ( applications installed etc... the works ) I would pay for such a system for these boards - when you say Backup your data and changes to the system (additional packages and configs). How can you do that with a live system ?

[Igor]

Sorry, not an backuping expert. I would assume like on any other Linux system.

[pkfox]

29 minutes ago, Igor said:

Sorry, not an backuping expert. I would assume like on any other Linux system.

Ok neither am I :-)

[le51]

@pkfox, once you have installed armbian on emmc, you can't boot on the sd card anymore (unless you remove the emmc) ?

[pkfox]

16 minutes ago, le51 said:

@pkfox, once you have installed armbian on emmc, you can't boot on the sd card anymore (unless you remove the emmc) ?

I thought that might be the case thanks

[le51]

I mean, if you can boot on a sd card with armbian on it, you should be able to copy the content (using dd command) of the emmc to an external hard drive connected via usb.

[Werner]

It depends. Some boards need the sd card with the bootloader at least in place which then redirects the boot process to emmc or whatever is connected.

[le51]

Well, I own a nanopi m4v2 + nVme hat and ssd, but for an unknown reason, the board doesn't start anymore. I suspect that some component has been toasted elsewhere ...

So sad ...

Quote

Some boards need the sd card with the bootloader

First, I've got to say that I'm far from being a guru in all these uboot, bootloader stuff.  But that is the case for the m4v2 + nvme ssd => you need sd card for booting the board and then access root filesystem on the ssd. By swapping the sd card I had the ability to install and choose to use legacy or mainline on two separate partitions.

=> so while booting the board with a fresh armbian sd card, you can backup a ssd partition or even the whole drive with dd. But I don't know if it is feasible on the emmc drive.

 

Btw, there is a "recovery" button on the board, what is that use for, it's a mystery

[mar0ni]

14 hours ago, le51 said:

@pkfox, once you have installed armbian on emmc, you can't boot on the sd card anymore (unless you remove the emmc) ?

I can boot the FriendlyArm image from sd card even if Armbian is installed on emmc. But I also cannot boot an Armbian image from SD. So may be you can try to create an FriendlyArm boot SD to do your backup.

[_r9]

You can simply backup your device with rsync -> https://wiki.archlinux.org/index.php/Rsync#Full_system_backup
 

Because Armbian is Linux and Linux installs all it software as packages it doesn't depend on a registry like Windows.
One big benefit that comes from this is, that you can simply make a "copy" from the whole system. Rsync, in that particular case is able to deal with symlinks and is therefore able to manage a complete image of your system.

 

If the worst case happens and you need to setup your data on a different board you can simply recover it with these steps

1. Write the Armbian image to the emmc disk
2. Overwrite all the data with your rsync data
3. You need to be aware of the /etc/fstab after a fresh installation the UUID might change. If you overwrite it with your old UUID the system might not boot again.
4.  On a full recovery you should also not overwrite following partitions
   1. "/dev/"
   2. "/proc/"
   3. "/sys/"
   4. "/tmp/"
   5. "/run/"
   6. "/mnt/"
   7. "/media/"
   8. "/boot/"

To be safe you also need to save the related Armbian image to each backup. If you change, for example, from Armbian-Stretch to Armbian-Buster and you want to recover it you need to dd the image onto the SD card and overwrite it with your backup like described above.

 

If you simply want to test configurations or new software you should in theory add a USB stick to your device and rsync the whole system to a directory on it. Please don't simply use these lines in your environment. Test such things always in a test environment ; )

 

Backup

mount /dev/YOURUSBSTICK/ /media/usb-backups/
rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / /media/usb-backups/VERSION/

 

Recovery

rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/boot"} --delete /media/backups/VERSION/ /
reboot

 

 

rsync and SSH

Okay now comes the best part. You can combine SSH with rsync to backup and recover your device over the network from another device. Adding this rsync to a cronjob you can create automated backups for your own without buying a backup software. The FLOSS way so to say ; )

 

If you're interested, I've tried to take some time off to develop a backup recovery environment in the past. It's written in bash and is still somewhat in alpha state but maybe you can find some code pieces that'll help you with your backups. https://github.com/rothirschtec/RT-Blog-SBE

In the end, you get the ability to create a backup device with this script and configure it to backup multiple devices automatically.

 

[pkfox]

On 5/12/2020 at 8:46 AM, _r9 said:

You can simply backup your device with rsync -> https://wiki.archlinux.org/index.php/Rsync#Full_system_backup
 

Because Armbian is Linux and Linux installs all it software as packages it doesn't depend on a registry like Windows.
One big benefit that comes from this is, that you can simply make a "copy" from the whole system. Rsync, in that particular case is able to deal with symlinks and is therefore able to manage a complete image of your system.

 

If the worst case happens and you need to setup your data on a different board you can simply recover it with these steps

1. Write the Armbian image to the emmc disk
2. Overwrite all the data with your rsync data
3. You need to be aware of the /etc/fstab after a fresh installation the UUID might change. If you overwrite it with your old UUID the system might not boot again.
4.  On a full recovery you should also not overwrite following partitions
   1. "/dev/"
   2. "/proc/"
   3. "/sys/"
   4. "/tmp/"
   5. "/run/"
   6. "/mnt/"
   7. "/media/"
   8. "/boot/"

To be safe you also need to save the related Armbian image to each backup. If you change, for example, from Armbian-Stretch to Armbian-Buster and you want to recover it you need to dd the image onto the SD card and overwrite it with your backup like described above.

 

If you simply want to test configurations or new software you should in theory add a USB stick to your device and rsync the whole system to a directory on it. Please don't simply use these lines in your environment. Test such things always in a test environment ; )

 

Backup

mount /dev/YOURUSBSTICK/ /media/usb-backups/
rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / /media/usb-backups/VERSION/

 

Recovery

rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/boot"} --delete /media/backups/VERSION/ /
reboot

 

 

rsync and SSH

Okay now comes the best part. You can combine SSH with rsync to backup and recover your device over the network from another device. Adding this rsync to a cronjob you can create automated backups for your own without buying a backup software. The FLOSS way so to say ; )

 

If you're interested, I've tried to take some time off to develop a backup recovery environment in the past. It's written in bash and is still somewhat in alpha state but maybe you can find some code pieces that'll help you with your backups. https://github.com/rothirschtec/RT-Blog-SBE

In the end, you get the ability to create a backup device with this script and configure it to backup multiple devices automatically.

 

Hi there, thanks for replying - what I'm trying to do is clone the eMMC on a running system. If I understand what you are suggesting correctly you are saying rsync everything you need from eMMC card to an external card then install a fresh Armbian image on the eMMC and finally copy the data back

[_r9]

Yes. But I'm not sure about all dependencies. I already described a few of them above. But if anything goes wrong you still have all configuration files and your data inside the backup. So in a worst case scenario you would need to reinstall the software you're using and simply recover the configuration files from the backup. You can recover your home directory at anytime and if you gather a few insights about the software you're using you'd be able to transfer a single application from one system to another on runtime. A research with:
 

whereis <your-application>

 

an the knowledge about the distribution your using (What is systemd?) gives you that ability. But for this scenario it would be more simple to install the software on the new system and transfer the configuration files from the old one.

Okay long story short. If you know what dependencies the installation of Armbian creates (e.g. The UUID= inside the /etc/fstab) you can simply recover all your data with rsync using the --exclude option. It's been a long time I did this so I cannot give you a complete example but if you did the rsync backup first you'll always (in worst case manually) be able to recover your system.

 

Another way I just remembered, if you backup the MBR of your eMMC after installing your system you'd be able to recovery the system like following. Recover the MBR first, create the system partition and recover and overwrite the partition with rsync using your backup data. So you don't need to reinstall the armbian image first. But this one is not for "on runtime" recovery approaches.

 

I have a research project about this.
I did this approach on a few banana pis in the past. Maybe you can find a few code snippets in here.

 

Backup

https://github.com/rothirschtec/ARM-backup-recovery/blob/master/RTbackup.sh

 

Recovery

https://github.com/rothirschtec/ARM-backup-recovery/blob/master/RTrecovery.sh

 

I got lost on this backup projects because the banana pi images made me crazy. But maybe I'll restart the development. So following the projects on github might be interesting. ; )

[JrRockeTer]

On 5/8/2020 at 4:08 PM, pkfox said:

if  I could temporarily boot from an sdcard , unmount the eMMC, and clone it for backup purposes. Before I ruin anything I thought I'd ask the experts. Is this doable ?

You cannot DD the eMMC whilst it i running ?