bedna Posted November 3, 2022 Posted November 3, 2022 Hi! Have limited knowledge in linux, all self learned so bear with me if I sometimes sound like an i**ot in my wording. I have used lots of different Raspberry pi's but since they are near to impossible to get your hands on I figured I'd try out some of the alternate boards. Got myself a nanoPiNeo2, orangePiZero2 and an orangePiPC2 just for the fun of it. I have tried this on the nanoPi and the opiZero2 and run into the same problems, havent even opened the opiPC2 yet but I suspect it will be the same problem on that board. Downloaded armbian for each separate board (instead of debian) because I want to use the same system on all my devices so its easier for me to learn. Got everything working as I wanted (just making a simple pihole to give away as xmas presents) but when it came to backing up, I ran into a wall. With the raspberries I use a script called image-backup located on the raspberrypi forums (awesome script made by RonR) but that obv wont work here since the rpi uses a fat32 as a boot partition. There is a flag you can run when using the script called "ubuntu", but it didn't work. Ok then, lets use win32 imager instead to just rip the card, its only 8gb, not a huge deal, I can probably even shrink the size with e2fsck and parted (as I understand it armbian autoexpands the card on mount anyway?) but I get an i/o error in win32 after a few seconds, this might actually be my computer, going to try it on another computer in a day or two, (I only have one). I tried other programs too, but none seems to work so I decided to try to dd from the mounted card. The backup directory is on a mounted cif on a windows computer if that has to do with anything? sudo dd if=/dev/mmcblk0 of=/path/to/backup/ddbackup.img bs=4M conv=sync status=progress That didn't work, I get (the numbers are bigger I just randomly took a number, the important thing is the +1 that gets added to the "normal" number in "records out", I suspect that is the boot sector?) 15874+1 records in 15875+0 records out I figured it had to do with the card being mounted so I made ANOTHER identical install on another sdcard and inserted the original sd in the usb with a card reader and doing the dd from /dev/sda instead (without it being mounted), but I get the exact same thing and when trying to flash the image and use that card, the board just never comes to life. Network port keeps flashing but nothing else. My plan was to make a backup.img file, mount that with a loop and shrink the "partition" with shrink2fs and parted. But even trying to mount the created image files gives me errors about bad superblocks etc, probably because of what I described above. Running e2fsck also gives me a TON of errors. I have tried for DAYS, searching all over the internet (figuring things out for myself is the best way to learn) but I get nowhere. Is there a simple solution to this? The only time I actually got a working .img file is if I dd a blank img file first from /dev/zero and set the file to become a size (with bs and count) where all data can fit and then rsync / into it. I can mount it with a loop (losetup) and access the files, but naturally it wont boot on the board (if I flash it to an sd card) since I read that the boot sector is "out of bonds" or whatever its called. Same error, just flashing network port. I'm at a complete loss here, I REALLY want to have an img file that the xmas gift holders get that they can just flash onto an sdcard in case of a breakdown of the software somehow. Please have pity on my soul! 0 Quote
denni_isl Posted November 4, 2022 Posted November 4, 2022 (edited) Use at your own risk and practice with less important drive (of course). ----------------- copy hard drive to .img(.gz) and .img(.gz) hard drive ---- * boot your computer up with a live usb or sd wit gparted > unmount the disk you want to copy > gparted > make the last partition smaller but leave some small additional space > check filesystem * to copy used partitions over to your_hard_drive_copy .img.gz ; - calculate > count= (gparted last sector of the alst partition (/) right mouse button, information, last sector * 512 / bs=4096 > * calculate in console; expr 22171647 \* 512 / 4096 = 2771455 -(2771456)) (+1 sector). - last sector +1 * 512 / 4096 = last sector in bs=4096 * copy used partitions /dev/sdc á .img og .img.gz; sudo dd if=/dev/sdc bs=4096 count=2771456 conv=sync,noerror status=progress of=manjaro_vim3_38okt20.img && gzip manjaro_khad_aml_mini.img * .img over to /dev/sda; dd if=hard_drive_backup.img of=/dev/sda bs=1m status=progress ; sync, * when restored one need to boot again with a live usb or cd with gparted and umount the disk > resize > check filesystem ------------------------------------------------------ Edited November 4, 2022 by denni_isl 0 Quote
bedna Posted November 4, 2022 Author Posted November 4, 2022 (edited) I think the reason it didn't work with my first tries I suspect is because I: 1. tried to mount the whole disk, not the partition. 2. Did not probe the "disk". This is the solution I found to best working for me. Backing up and mounting image: Use gnu version of ddrescue (you can back up mounted system drive with it) $ sudo apt install gddrescue -y $ sudo ddrescue -f -n /dev/mmcblk0 /path/to/backup.img GNU ddrescue 1.23 Press Ctrl-C to interrupt ipos: 15634 MB, non-trimmed: 0 B, current rate: 14352 kB/s opos: 15634 MB, non-scraped: 0 B, average rate: 19066 kB/s non-tried: 0 B, bad-sector: 0 B, error rate: 0 B/s rescued: 15634 MB, bad areas: 0, run time: 13m 39s pct rescued: 100.00%, read errors: 0, remaining time: n/a time since last successful read: n/a Finished Find out where the boot sector/partition ends/starts, ie: start(32768) x blocksize(512) = offset(16777216), and mount it (make sure you have directories to mount to, in this case sudo mkdir /mnt/loop) You can also divide the offset by 8 to make sure it's in the right position, should end up with a whole number, in this case 16777216/8=2 097 152. This tells me, even though the bootsector seems to be way bigger than needed, it probably will work. $ sudo fdisk -l /mnt/backup/nanoPiNeo3/piHole/backup.img Disk /path/to/backup.img: 14.56 GiB, 15634268160 bytes, 30535680 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xb90578af Device Boot Start End Sectors Size Id Type /path/to/backup.img1 32768 30212096 30179329 14.4G 83 Linux Mount the img with a loop $ sudo mount -o loop,offset=16777216 /path/to/backup.img /mnt/loop $ ls /mnt/loop -l total 84 -rw-r--r-- 1 root root 3128 Oct 22 18:11 armbian.key lrwxrwxrwx 1 root root 7 Oct 22 08:41 bin -> usr/bin drwxr-xr-x 3 root root 4096 Nov 1 23:50 boot drwxr-xr-x 2 root root 4096 Oct 22 18:11 dev drwxr-xr-x 103 root root 4096 Nov 4 19:19 etc drwxr-xr-x 3 root root 4096 Nov 1 16:34 home lrwxrwxrwx 1 root root 7 Oct 22 08:41 lib -> usr/lib drwx------ 2 root root 16384 Oct 22 18:19 lost+found drwxr-xr-x 2 root root 4096 Oct 22 08:41 media drwxr-xr-x 5 root root 4096 Nov 4 19:19 mnt drwxr-xr-x 3 root root 4096 Nov 1 17:26 opt drwxr-xr-x 2 root root 4096 Oct 22 08:41 proc drwx------ 5 root root 4096 Nov 4 19:18 root drwxr-xr-x 3 root root 4096 Oct 22 18:21 run lrwxrwxrwx 1 root root 8 Oct 22 08:41 sbin -> usr/sbin drwxrwxr-x 2 root root 4096 Oct 22 18:11 selinux drwxr-xr-x 2 root root 4096 Oct 22 08:41 srv drwxr-xr-x 2 root root 4096 Sep 3 14:10 sys drwxrwxrwt 2 root root 4096 Oct 22 18:11 tmp drwxr-xr-x 11 root root 4096 Oct 22 08:41 usr drwxr-xr-x 13 root root 4096 Nov 1 17:23 var Now you can rsync your whole disk to keep files up to date. First create a file with paths to exclude, ie so you don't create a loop and backs the directory you are backing up to, I simply created exclude.txt in my home directory. nano ~/exclude.txt Content of my exclude.txt #directories to exclude when backing up with rsync /proc/* /sys/* /dev/* /tmp/* /run/* /mnt/* /media/* rsync to the mounted drive to update backup sudo rsync -ahv --info=progress2 --stats --delete --force --partial exclude-from=/home/$USER/exclude.txt --delete-excluded / /mnt/loop/ umount /mnt/loop so we can shrink the img file sudo umount /mnt/loop Shrinking the img file Change to superuser (sudo su) so you don't have to type sudo before each line and run following commands. I will use parted. $ sudo su $ modprobe loop $ losetup -f $ losetup /dev/loop0 /path/to/backup.img $ partprobe /dev/loop0 Check how much space is actually used, since it's a copy of my system I can just df that $ df -h /dev/mmcblk0p1 Filesystem Size Used Avail Use% Mounted on /dev/mmcblk0p1 15G 1.7G 13G 12% / Lets make the drive 2500MB, just to be a bit safer and leave space for future rscync backups in case they size increases of what you want to back up $ e2fsck -f /dev/loop0p1 e2fsck 1.46.2 (28-Feb-2021) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information armbi_root: 49784/916864 files (0.1% non-contiguous), 502310/3772416 blocks $ resize2fs /dev/loop0p1 2500M resize2fs 1.46.2 (28-Feb-2021) resize2fs: New size smaller than minimum (659029) Nope, it didn't like that, lets try with 2600M $ resize2fs /dev/loop0p1 2600M resize2fs 1.46.2 (28-Feb-2021) Resizing the filesystem on /dev/loop0p1 to 665600 (4k) blocks. The filesystem on /dev/loop0p1 is now 665600 (4k) blocks long. Nice, it worked. Then recheck it e2fsck -f /dev/loop0p1 e2fsck 1.46.2 (28-Feb-2021) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information armbi_root: 49784/165984 files (0.2% non-contiguous), 454731/665600 blocks Looking good. Now lets shrink the partition using parted. Since I'm not sure how to get all units in the same way I'd rather make the partition a little bigger than the filesystem and then expand the filesystem it up to the partition size, ie lets make it 2700MB. If someone could clarify this part I would be very happy $parted /dev/loop0 GNU Parted 3.4 Using /dev/loop0 Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) print Model: Loopback device (loopback) Disk /dev/loop0: 15.6GB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 16.8MB 15.5GB 15.5GB primary ext4 (parted) resizepart 1 End? [15.5GB]? 2700MB Warning: Shrinking a partition can cause data loss, are you sure you want to continue? Yes/No? y (parted) quit Information: You may need to update /etc/fstab. Then lets resize the filesystem with resize2fs $ resize2fs /dev/loop0p1 resize2fs 1.46.2 (28-Feb-2021) Resizing the filesystem on /dev/loop0p1 to 655083 (4k) blocks. The filesystem on /dev/loop0p1 is now 655083 (4k) blocks long. And one last check before shrinking the img file $ e2fsck -f /dev/loop0p1 e2fsck 1.46.2 (28-Feb-2021) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information armbi_root: 49784/158080 files (0.2% non-contiguous), 454235/655083 blocks Looking good. Now let's see how much we can shave off the file, we are looking for the end sector (5273437) and sector size (512), we use fdisk $ fdisk -l /dev/loop0 Disk /dev/loop0: 14.56 GiB, 15634268160 bytes, 30535680 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x83f3ee4c Device Boot Start End Sectors Size Id Type /dev/loop0p1 32768 5273437 5240670 2.5G 83 Linux The calculation goes: end sector+1 and then multiply that number with the sector size (512), that gives us 5273438 x 512 = 2 700 000 256bytes, about 2,7GB We use truncate to trim the file $ truncate --size=$[(5273437+1)*512] /path/to/backup.img Lets mount the img file again and see if we can access the files $ mount /dev/loop0p1 /mnt/loop $ ls /mnt/loop -l total 84 -rw-r--r-- 1 root root 3128 Oct 22 18:11 armbian.key lrwxrwxrwx 1 root root 7 Oct 22 08:41 bin -> usr/bin drwxr-xr-x 3 root root 4096 Nov 1 23:50 boot drwxr-xr-x 2 root root 4096 Nov 1 23:50 dev drwxr-xr-x 103 root root 4096 Nov 4 19:19 etc drwxr-xr-x 3 root root 4096 Nov 1 16:34 home lrwxrwxrwx 1 root root 7 Oct 22 08:41 lib -> usr/lib drwx------ 2 root root 16384 Oct 22 18:19 lost+found drwxr-xr-x 2 root root 4096 Oct 22 08:41 media drwxr-xr-x 2 root root 4096 Nov 4 19:19 mnt drwxr-xr-x 3 root root 4096 Nov 1 17:26 opt dr-xr-xr-x 2 root root 4096 Jan 21 2016 proc drwx------ 5 root root 4096 Nov 4 19:18 root drwxr-xr-x 2 root root 4096 Nov 4 19:28 run lrwxrwxrwx 1 root root 8 Oct 22 08:41 sbin -> usr/sbin drwxrwxr-x 2 root root 4096 Oct 22 18:11 selinux drwxr-xr-x 2 root root 4096 Oct 22 08:41 srv dr-xr-xr-x 2 root root 4096 Jan 21 2016 sys drwxrwxrwt 2 root root 4096 Nov 4 20:09 tmp drwxr-xr-x 11 root root 4096 Oct 22 08:41 usr drwxr-xr-x 13 root root 4096 Nov 1 17:23 var We can also see that the file is WAY smaller than the 15GB file we started out with. $ df -h /mnt/loop/ Filesystem Size Used Avail Use% Mounted on /dev/loop0p1 2.5G 1.7G 740M 70% /mnt/loop If you want to, you can once again rsync into the img file to back up your system. Just make sure that you don't install massive amounts so the file gets filled, the file will not get bigger by itself. You have to redo these steps here but START with increasing in parted THEN resize it with e2fsck. Finally let's umount and remove the loop $ umount /mnt/loop $ losetup -d /dev/loop0 Don't forget to exit SU by simply typing $ exit You can now flash this img file to an sdcard using win32DiskImager, etcher or whatever your fav prog is. If you want it to expand the partition after you have flashed it to a new card, simply boot up with new card in board, enable armbian-resize-filesystem.service then reboot and disable it again. Your sd card will be max size. $ sudo systemctrl enable armbian-resize-filesystem.service $ sudo reboot now $ sudo systemctrl disable armbian-resize-filesystem.service If someone knows if it's possible to put a file on the sdcard that tells it to autoexpand on boot I would love to know about it. That would remove the whole reboot process when expanding the card. Hope this wall of text can help someone else. Edited November 5, 2022 by bedna 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.