Make SDcard image with sole purpose of eMMC receiving a fresh bootable Armbian image

[wwortel]

Certainly not a new item but havn't found an Armbian how-to.

Hardware: Rockchip rk3588 based hardware with both SDcard slot and eMMC memory.
Wish: make a SD card image that, when inserted in the SDcard slot, causes a bootable Armbian OS image to be written to eMMC from that SDcard, followed by boot from eMMC.
The bootable image should remain in its most compact form, to limit copy time, and only upon first boot from eMMC be made to use the entire eMMC.

Has this been explored already? Ideas on how-to?

[going]

5 часов назад, wwortel сказал:

Certainly not a new item but havn't found an Armbian how-to.

Hardware: Rockchip rk3588 based hardware with both SDcard slot and eMMC memory.
Wish: make a SD card image that, when inserted in the SDcard slot, causes a bootable Armbian OS image to be written to eMMC from that SDcard, followed by boot from eMMC.
The bootable image should remain in its most compact form, to limit copy time, and only upon first boot from eMMC be made to use the entire eMMC.

Has this been explored already? Ideas on how-to?

Do you mean that we need to make a boot image by analogy and similarity, as well-known distributions such as openSUSE, ubuntu do?

[wwortel]

Well, apparently my question was not clear. No, the purpose is to have the option of creating a .img Armbian OS for a rk3588s based system that when written onto a SDcard and inserted in the rk3588s hardware, that also has eMMC on board, writes a working Armbian image to eMMC, such that the SD card slot is only used once to get the image into the eMMC.

Reason being that not so experienced users find it a lot easier to write and insert a SD card, than to write into eMMC. Latter requires either access to serial console, or the use of Rockchip helper programs.

Thinking about it it would be like the original .img as binary file inside a .img that can be written to the SD card, with the effect that just enough u-boot and perhaps kernel start is done from the SD card to get the .img file, also on that card, into eMMC.  

[going]

19.11.2024 в 20:01, wwortel сказал:

Well, apparently my question was not clear. No, the purpose is to have the option of creating a .img Armbian OS for a rk3588s based system that when written onto a SDcard and inserted in the rk3588s hardware, that also has eMMC on board, writes a working Armbian image to eMMC, such that the SD card slot is only used once to get the image into the eMMC.

This is not a viable option.

 

[wwortel]

@going

What makes you think this is not viable? After all u-boot is able to start Armbian from either SD or eMMC, so why could it not be made to copy an image from one to the other?

[going]

4 часа назад, wwortel сказал:

What makes you think this is not viable?

The functionality of the armbian-install script is quite enough to meet your needs.

 

But the community requests are a bit more than this script can handle.

In addition, you will probably want to check how the system boots and works before deciding whether to install it on an internal drive.

 

 

 

[Igor]

4 hours ago, wwortel said:

why could it not be made to copy an image from one to the other?

@going Probably @wwortel means to develop a minimal boot image which would download and dd image to eMMC. I agree that would be nice to have, but develop that requires time / resources we don't have. If you look into the script that installs OS to eMMC (and other media) its pretty complex and hard to maintain https://github.com/armbian/build/blob/main/packages/bsp/common/usr/bin/armbian-install Not to mention that people doesn't pile up to help us around that. There is / always was just a few people that sacrifices their precious family time to provide you current experience. Where we can always find something to fix and improve. 

[going]

10 минут назад, Igor сказал:

Probably @wwortel means to develop a minimal boot image which would download and dd image to eMMC.

I understood perfectly well, but this functionality is necessary for mass deployment.
Previously, this was used by lazy and smart system administrators when the task was to expand the number of new jobs.

But there seems to be something different here.

 

On which hardware devices does the author want to massively deploy operating systems?

[robertoj]

1 hour ago, Igor said:

@going Probably @wwortel means to develop a minimal boot image which would download and dd image to eMMC. I agree that would be nice to have, but develop that requires time / resources we don't have. If you look into the script that installs OS to eMMC (and other media) its pretty complex and hard to maintain https://github.com/armbian/build/blob/main/packages/bsp/common/usr/bin/armbian-install Not to mention that people doesn't pile up to help us around that. There is / always was just a few people that sacrifices their precious family time to provide you current experience. Where we can always find something to fix and improve. 

 

I had no idea there is a script for THAT  Thanks Armbian team  

 

To the original poster: all beginners should stick to microSD. Please gain experience, make the tool you want, and share it here.

 

Also look at this wonderful tool that bedna made: (it might be useful for you)

 

[wwortel]

Thanks for all the answers, that certainly go beyond 'not a viable option'.
The hardware is a very dedicated motherboard with a Radxa CM5 compute module (RK3588s2 SoC) and some non-standard peripherals for software defined radio reception and transmission purpose, and an Armbian OS for the CM5 in which extensive use has been made of user patches and customize-image. A large part of the end users though are not into software, and an even larger part not into Linux based software. And, as with any product, a large part of the users will not read a manual.
Rather than explaining how to use 'rkdeveloptool' or equivalents to put the Armbian into eMMC, it would be much simpler to have a one time SD card insertion, wait, power off, remove SD card, and power-on with boot from eMMC. Certainly normal use of the system has to be from eMMC as it boots faster than SD. Hope this explains the motives for my question that also aimed at not re-inventing the wheel. But a ready to go solution seems not yet available.
Was looking into the one-time script 'armbian-resize-filesystem'. Perhaps that would be a good place to add the functionality I am looking for and, in case booted from SDcard, do a OS copy into eMMC before the OS on the SDcard is resized to memory size. I've read that when the copy is made one must make sure that no writing goes on into the source image that is being copied, to ensure integrity of the root file system that is landing on the eMMC. Ideas welcome how and where to do a copy into eMMC from SDcard in Armbian of the yet unexpanded OS!
Igor's point taken that preferably it should be as simple as possible and minimally disruptive to current standard Armbian. Basically a switch in configuration that, when set, makes the first boot process detect whether it starts from SD, whether eMMC is present, whether the switch has been set, and only then does the dd copy and ends with the invitation to remove the SD card and reboot.

[SteeMan]

I would suggest you consider a hybrid approach.  Boot partition on SD and root partition on emmc.  Leave the uboot on the SD card and only transition the rest to emmc.  You get the benefits of performance you are looking for (as everything other than uboot is running from emmc), but you don't have the complexities of switching your boot environment from SD to emmc all in a script.  Basically you on first boot need to format your emmc, copy the root filesystem over and adjust your fstab and armbianenv.txt entries to point to the new location of your root partion.  Now that itself doesn't come with protential issues, but that is a valid way to have Armbian configured (I think it is an option in armbian-install to have boot and root partitions on different media).

 

There is a script that the amlogic tv box builds use: https://github.com/armbian/build/blob/main/config/optional/boards/aml-s9xx-box/_packages/bsp-cli/root/install-aml.sh

This script is just a fairly simple example of the steps involved in moving from SD to emmc, with some very specific amlogic tvbox stuff mixed in.  It may be a good starting point to think from (as opposed to armbian-install which is a lot more complex as it is generically handling a bunch of different requirements)

[eselarm]

I know Ext4 and 1st boot resize is used, but I have manually recreated rootfs to be Btrfs. On NanoPi-R6C it is u-boot code in first 16M of SD or eMMC, then a 256M FAT32 partition with Image uInitrd DTB overlays. I made an extra script for update-iniitramfs to copy those for a certain kernel version to the FAT partition.

 

In a qemu VM, I use a custom enhanced u-boot that has Btrfs support, so no bootFAT needed anymore.  This should also work on real HW,

 

The advantage of Btrfs for root is that you can move the rootfs from SD to eMMC (or NVME) I a live running system (btrfs replace start ...) At finish, only dd with u-boot loader to eMMC is needed. Then take out SD-card and done, no reboot needed, same rootfs UUID. Creating partition on eMMC is still needed of course, but can be just max of eMMC. Btrfs can always be shrunk later on in live system. So that is what I did several times also to install other OS variants in other partitions of the eMMC (and mostly NVME as that is much faster).

 

I haven't looked in armbiam-installer. I know when selecting Btrfs as rootfs in builder, you get Ext4 for boot and Btrfs for root. This is what I use on NanoPi-NEO and BananaPi-M1 (created manually long time ago). Those don't have eMMC or SPI-flash and it works fine. But tempting to write a Btrfs enabled u-boot to the proper place at the beginning of the SD-card and merge Ext4 /boot into rootfs.

I am a bit unsure about building 32-bit u-boot but i think i can temporary used systemd-nspawn of a 32-biit rootfs on RK3588 so fast building.

 

[eselarm]

I managed to build u-boot-2024.10 with Btrfs for 32-bit (using systemd-nspawn). See:

https://docs.u-boot.org/en/latest/build/index.html

It would be more easy to do for native aarch64, but that I already knew when building for 'machine' 'qemu-arm64'.

 

As example for my NanoPi-NEO, all in extra separate src/build folder:

dpkg -L linux-u-boot-nanopineo-current

- add/change

CONFIG_CMD_BTRFS=y
CONFIG_FS_BTRFS=y
in nanopi_neo_defconfig

make menuconfig and save the .config

make -j8

- see platform_install.sh how and where to write the newly build u-boot-sunxi-with-spl.bin (about 570kByte, so fits in my MBR partition starting at sector 2048)

 

So after merging boot files from the mmcblk0p1 Ext4 partition to a folder /boot on mmcblk0p2 Btrfs rootfs, I started fdisk and made Btrfs rootfs p1 (using the same start-end sectors), so only 1 partition. Same as you actually have on standard Armbian images. And comment out the line in fstab for mount of separate boot partition (or add nofail option maybe).

 

Then reboot and restarted without problems. I did actually all via remote ssh, although i also had serial USB console cable connected as I wanted to see the boot log, but was as expected. A remark I saw was that it could not write environment to FAT. That is something I need to think about. Do I gain something if that is possible. Currently not on single OS SBCs for me I guess. A bootFAT might be more convenient for people who rely more or only on a Windows/MacOS computer as their main desktop. I use Linux so no issue with fixing SD-cards in case of non-boot because I messed up something testing or so.

 

What went wrong afterwards is my backup script via ssh. For SoCs that use binary blob bootloader, I copy/compare/backup the first sectors till p1 (usually 2048 or so, so a 1 MiB file). That size determination went wrong, it created a 257MiB file in /tmp, so fails with a 512MiB SBC. But learning more now after 10+ years SBCs, I think I should do that smarter, just re-use platform_install.sh or so.

 

 

I think I will have to look at 1st boot resize script(s). It is max SD-card, so all space gone for Ext4. I really hate that. Even more when done on fixed/soldered eMMC or NVME.
Ext4 cannot be shrunk in a running system later on. Btrfs can, so last hack I did on an Ext4 downloaded image (RPiOS64 Lite) was double the image file, max Ext4 on that image and then convert to Btrfs with btrfs-convert. It also need kernel cmdline and fstab change still, but that is more or less it.

[wwortel]

Thanks for all suggestions so far. Have been experimenting and actually managed to get the behavior wanted by editing the init file inside the uInitrd ramdisk.

A test is done now whether the root_partition equals /dev/mmcblk1p1 , where the blk1 indicates boot from sdcard, and if indeed it booted from the SDcard dd is used to copy the /dev/mmcblk1 to /dev/mmcbkl0 , latter being the eMMC memory, followed by reboot. Upon reboot this copy is skipped because u-boot prefers to boot now from eMMC and one ends up with the normal first run interaction about passwords and locale.

Remaining question: I did the edit by decompressing and unpacking the uInitrd from the SDcard, and after edit repacking and recompressing it and writing it back.

How can I make the Armbian build system, when it does update-initramfs, make it include a modified init file in the /boot/uInitrd<uname -r> ?

Edited December 19 by wwortel
typo

[wwortel]

Let me answer my last question. The init script at the top of the ramdisk tree stems from the initramfs-tools package and can be changed from customize-image.sh by writing an edited version from /tmp/overlay/ (build/userpatches/overlay/) into /usr/share/initramfs-tools/