The boot process and various devices

[ManoftheSea]

1 hour ago, usual user said:

UUID always requires user space support because there is no native support for this and thus always an initramfs

This is untrue.  The Helios64 can find the uuid of ext4 root without initrd.  However, yes, that's probably because ext support is compiled in.

 

1 hour ago, usual user said:

Keyword "bootflow", and also UEFI systems are to be supported.

Hey, great!  I look forward to it.

 

1 hour ago, usual user said:

I prefer the way mainline uboot goes, each distribution uses its own configuration and uboot uses distro-boot, uefi-boot, lecacy boot.scr

I can understand it.  Maybe with additional experience, I'll learn it too.

 

1 hour ago, usual user said:

When can I use your planned solution out-of-the-box for each distribution

Come on, you know the answer is "probably never".  I'm in the wishes and dreams stage, and the learning what the problem even looks like.  I do appreciate your guidance on other available solutions.  I definitely like the extlinux menu option situation, where the system can know its own list of kernels and cmdlines and present them to u-boot (and through u-boot to the user), I just want to know how to deconflict it with other distros.  And yes, that's very unusual for embedded boards... but doesn't something like the Station P2 want to be more like a PC?  Everyone's looking at the RK3588 boards as "desktop replacements" (in many years).

 

1 hour ago, usual user said:

only the install routine decides the details where to put the boot artifacts

Yeah, that seems like a good idea.  Armbian uses Debian's kernel packages.  Deviating from their standards should be done in a controlled way - And that's the install routine, as you said.

 

Thanks a lot for the education.

[usual user]

22 hours ago, ManoftheSea said:

This is untrue.  The Helios64 can find the uuid of ext4 root without initrd.

The time when I had the experience on which I base my statement is already many years ago, so you might be right and it is now available. Since I actively maintain the partition ID of my drives with talking values, I also use it for physical identification.

Spoiler

NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT      PARTUUID
sda            8:0    1  29,8G  0 disk.................
├─sda1         8:1    1   512M  0 part                 00000013-01
└─sda2         8:2    1  28,9G  0 part                 00000013-02
sdb            8:16   0 931,5G  0 disk.................
├─sdb1         8:17   0     2G  0 part                 50000002-01
├─sdb2         8:18   0     6G  0 part                 50000002-02
└─sdb3         8:19   0   900G  0 part /mnt/ssd-002-03 50000002-03
mmcblk2      179:0    0  14,6G  0 disk.................
mmcblk2boot0 179:8    0     4M  1 disk.................
mmcblk2boot1 179:16   0     4M  1 disk.................
mmcblk1      179:24   0  29,7G  0 disk.................
└─mmcblk1p1  179:25   0    20G  0 part /               696d6701-01
zram0        252:0    0   2,8G  0 disk [SWAP]..........
nvme0n1      259:0    0 931,5G  0 disk.................
├─nvme0n1p1  259:1    0     2G  0 part                 50000003-01
├─nvme0n1p2  259:2    0     6G  0 part                 50000003-02
└─nvme0n1p3  259:3    0   900G  0 part                 50000003-03

 

And since the PARTUUID use has worked equally since the beginning, I have never questioned it again.

 

22 hours ago, ManoftheSea said:

I just want to know how to deconflict it with other distros.

As long as you do not want to control two systems from one menu, there are no conflicts. Only the selection of the system to be used is not quite as comfortable but it is still possible. And removable media always work.

 

22 hours ago, ManoftheSea said:

Armbian uses Debian's kernel packages.

I still build my kernels from the original fedora source packages, but no longer use fedora binary packages. My kernels are installed by simply copying in place and can be used via a tar archive in any system. No complicated install routine. KISS
Try this portability with a deb or rpm package.

[usual user]

22 hours ago, ManoftheSea said:

I'm in the wishes and dreams stage

Now I would also like to dream:
When commissioning a new system, I imagine that I will be offered a menu at the first boot in which all configurations available for selection are displayed. Since the SBC can't read my mind, at least I hope so, I choose the one I want. The system then starts and performs its first boot routine. At the next start of the system I wish for a dedicated menu of my previously selected boot option.

 

I'll leave it up to you to explore what changes you think are necessary.

 

Nevertheless, let's see how I would approach the matter in my dream. First, I would create a numbered extlinux.conf for each configuration selection with the following naming scheme: extlinux.conf-xx  with xx=01, 02, ...
The initial extlinux.conf contains only the fail save boot options of the available configurations. In addition, I append a boot parameter of the following form to each append line: bootoption=xx with xx=01, 02, ...
Then I write a shell script that searches /proc/cmdline for bootoption= and then copies the corresponding extlinux.conf-xx to extlinux.conf depending on the value found. Now I make sure that the script is executed as part of the first boot routine, and my dream should come true.
Wait, do I have just developed a method how distro-boot can communicate to the running system which boot option for the successful boot has been used. Am I still dreaming now or is all this already really feasible?

[ManoftheSea]

u-boot has a configuration option "CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_DATA_PART_OFFSET".  If I understand it correctly, it says the number of sectors to skip after jumping over MBR.  If set to 33, I believe that means uboot will skip over the region used by the gpt, and begin at sector 34.  Not needed on rockchips which load from 64 to begin, but useful for amlogic which start with sector 0.

 

Even better, it appears u-boot can load from a specific partition type from mmc/sd.  This ought to be set as the BIOS type.  I'm going to have to try that.  (21686148-6449-6E6F-744E-656564454649)

[usual user]

Uboot is only the last component in the firmware running chain. Before passing control to uboot, several other firmware artifacts are executed in different security levels. ARM Trusted firmware  is a open source reference implementation of secure world software for ARMv8-A including Exception Level 3 (EL3) software. For amlogic no TF-A exists, hence you have to use encrypted binay amlogic blob artifacts. Check from which components the Armbian uboot build composes (aml_encrypt_g12b --bootmk) the fip for e.g. Odroid N2+. The uboot code is located at the end of the fip. Read this how to try to bypass the firmware location restrictions.

[usual user]

And the story goes on. I decided to put my ODROID-M1 into operation. Initial research shows that the SPI flash firmware supports distro-boot to a certain extent, but it is so poorly configured that it is not really usable. I then used the image of @rpardini to test that the hardware works. He uses extlinux.conf, so the minimum requirements for distro-boot must be met. In the first step, of course, I changed it to my structure. Then I built my kernel package with @tobetter his commit diff and added it to the rootfs. To my surprise, my very first attempt worked immediately. The image only starts as a command line version, so I started the SD card I used for my Odroid-N2+ bringup. Unfortunately, it only runs up to the greeter screen, as I have long forgotten the "secure password" that Armbian enforces at the first start. My fedora also works in command line mode, but as soon as I start a graphical user, the kernel explodes. I did not expect this to be any different, because mali support is still in its infancy.
Guess now I have to stay closer to the rk3568 development.
I used the @rpardini image as an ESP partition without FAT32 and UEFI. And all this was only possible with the attached extlinux.conf and my kernel package.

[surenz]

I'll try to hijack this topic with my problems migrating a working Armbian from one SBC to another - Orange Pi One to Orange Pi PC and vice a versa. 

The SBCs have very small differences but at least they have so it's a good and (almost) safe experiment.

So far I succeeded to use the appropriate dtb file for every SBC as easy as adding the parameter fdtfile= to the file /boot/armbianEnv.txt

It works on the level to detecting the right hardware in the linux but u-boot is still detecting the old model board.

In the linux the /proc/device-tree/model is showing the right model and the hardware is working as it should.

But some cosmetic issues exist - armbian-config is still showing the old board and the MOTD on ssh login has the wrong model banner (I suspect that this info is taken from the file armbian-release).

 

So my questions:

1. Do you know what I have to do to change the u-boot to show the right board? (I tried menu config but saveenv failed)

2. What is exactly the process of detecting the right board (perhaps it's done in the linux image which is downloaded in ready state and compiled for that specific board) either in first run or in the compile image stage?

3. Can we make changes to already installed linux to change those small discrepancies for more comfortable feeling (as more issues seem cosmetic than real)?

 

 

The changes between the SBCs I mentioned are small (at least the SoC is same) but some of them are important:

- the ports are different (OPi One has two USBs instead of 4 in OPi PC, audio and IR connectors are missing)

- voltage regulator is missing on OPi One

[ManoftheSea]

4 hours ago, surenz said:

Do you know what I have to do to change the u-boot to show the right board? (I tried menu config but saveenv failed)

I would not be surprised to find these are compiled in to the u-boot image.  For Rockchip boards, my understanding is that u-boot is on the SD card at sectors 64-32767.  You should generate a u-boot image for your board to write over that part of your SD card.  My guess is something like dd skip=64 seek=64 count=32703 if=new-board-armbian.img of=/dev/sdX

 

5 hours ago, surenz said:

What is exactly the process of detecting the right board (perhaps it's done in the linux image which is downloaded in ready state and compiled for that specific board) either in first run or in the compile image stage?

u-boot is told upon compile what board it's being built for, and there are some dtb modification routines that change the file read from disk before handing the memory location to the linux kernel.  I don't know how much modification for any given board.  Linux just takes the devices from the dtb.  As for the login text splash, yes, that's in armbian specific files on the image, which I think are in the armbian-bsp-board packages.

[surenz]

2 minutes ago, ManoftheSea said:

I would not be surprised to find these are compiled in to the u-boot image.  For Rockchip boards, my understanding is that u-boot is on the SD card at sectors 64-32767.  You should generate a u-boot image for your board to write over that part of your SD card.  My guess is something like dd skip=64 seek=64 count=32703 if=new-board-armbian.img of=/dev/sdX

 

The u-boot can be reflashed very easily and secure with the armbian-config but it takes the image from some package which is downloaded from apt repo. Of course those packages are still not updated so they don't change even if flashed.

[surenz]

Just as update I can say that the cosmetic issues in my previous question 3 are resolved with the change of BOARD_NAME and BOARD in  /etc/armbian-release to comply with the real board.

The only problem I see is the message at the top of the file PLEASE DO NOT EDIT THIS FILE and if the file is updated during some of the regular updates.

[balbes150]

13.05.2022 в 23:20, usual user сказал:

First step: prepare kernel package (copy files around)

placing boot files (kernel\initrd\dtb\config, etc.) in the root system itself is very bad advice. all startup-critical files\config must be placed outside the root file system.

what will you do when using an unsupported u-boot file system (not ext4\fat) , encrypted file sharing, or network hosting of a root system?

[balbes150]

19.05.2022 в 22:24, ManoftheSea сказал:

I definitely like the extlinux menu option situation, where the system can know its own list of kernels and cmdlines and present them to u-boot (and through u-boot to the user), I just want to know how to deconflict it with other distros.  And yes, that's very unusual for embedded boards... but doesn't something like the Station P2 want to be more like a PC?  Everyone's looking at the RK3588 boards as "desktop replacements" (in many years).

Kernel (system) selection management has been working for rk3399 for a long time.   including automatic installation (updating) of the kernel by standard and simple means (apt\synaptic), as on ordinary PCs

Which version of u-boot is used on Helios 64 ?

[ManoftheSea]

@balbes150In the case of an UEFI booting system, I agree with you, the files necessary for boot go in the ESP (EFI System Partition) which is required to be a FAT filesystem.  This is not a u-boot requirement, but a bootloader specification.  https://systemd.io/BOOT_LOADER_SPECIFICATION/ .  Type 1 entries have a .conf file which looks much like SYSLINUX (also EXTLINUX) format in ESP/loader/entries/.  Type 2 entries are EFI applications in ESP/EFI/Linux/*.efi .  Systemd-boot in Debian 12 (currently sid) can create these entries, I don't know the status in Ubuntu.

 

For a system which is able to have firmware that goes with the board, rather than the SD card (e.g. espressobin with NAND flash), the bootloader (e.g. u-boot) can use a completely standard GPT-partitioned block device or attempt a UEFI network boot (which I know little about).  For a system with eMMC, it *might* be possible to put a u-boot in place in the hardware eMMC partitions, such that it acts like the previous case, and finds the normal system on the GPT partitioned eMMC block device; but the bootloader can save its environment somewhere with the SBC.  Finally, the last case is a system with no long-term storage, where even the u-boot is written to the SD card.  These systems are inherently non-standard, and need workarounds like 0xEA partition on MBR or moving the GPT table entries. 

 

 

 

 

[balbes150]

18 часов назад, ManoftheSea сказал:

I don't know the status in Ubuntu.

Everything works fine in EFI mode.

Do I understand correctly, you have Helios64 (rk3399) and Espressobin (marvel 370)?

 

[balbes150]

@ManoftheSeaCan you test running this image on your Helios 64 and Espressobin ? To run on Helios 64 - write the image to the SD card and try to run. For Espresso bin, you will need to make a small edit and use a USB flash drive for recording. This is an image with EFI\GRUB support. On rk3399 there should be a picture with a menu at startup with the option to select the system\kernel. If the output to the screen in u-boot works on the Espresso bar, there should also be a corresponding picture there.

 

https://disk.yandex.ru/d/-8s9ffTWG6i1nw

[iav]

there boot log on Helios64, image from yandex disk

screen.txt

[balbes150]

14 часов назад, iav сказал:

here boot log on Helios64, image from yandex disk

What is connected to the HDMI output, is there a picture there? Judging by the log, the system is up and running.

[iav]

Sorry, there nothing to connect to HDMI.

And there no login prompt on serial console.

But yes, network scan found an IP obtained with DHCP, then I successful login into armbian first login wizard.

Quote

Linux helios 6.0.0-media #trunk SMP PREEMPT Fri Oct 7 15:23:21 MSK 2022 aarch64 aarch64 aarch64 GNU/Linux

System cooler works on full speed all the time.

[iav]

shutdown and reboot commands stays system shutted down but powered on.

[balbes150]

1 час назад, iav сказал:

But yes, network scan found an IP obtained with DHCP, then I successful login into armbian first login wizard.

Tnx.

Now I was only interested in the general launch of the system using EFI, you confirmed its operation. I am not checking the operation of the system (kernel) yet, perhaps additional patches \ settings for the kernel are needed, but this requires access to the hardware. By the way, you can disable UEFI mode and switch to using extlinux.conf, with which full output and control via UART should work. Remove the "boot" flags from the first FAT partition and the bootloader will stop using EFI. Or you can remove all files from the FAT partition.

[Igor]

New fixes for armbian-install https://github.com/armbian/build/pull/4271 Common UEFI ARM64 image can boot from u-boot 2022.07 installed on NanoPC T4 eMMC. GRUB works even in serial console ... UEFI installation to eMMC is covered with this MR, perhaps some hardening / code cleaning is needed. It also works to NVME, but u-boot does not recognise it, so we can't boot from it. Alternatively we could hack this - to ask user where he wants to have EFI partition (eMMC). Perhaps adding that option too?

[balbes150]

10 часов назад, Igor сказал:

UEFI installation to eMMC is covered with this MR, perhaps some hardening / code cleaning is needed. It also works to NVME, but u-boot does not recognise it, so we can't boot from it. Alternatively we could hack this - to ask user where he wants to have EFI partition (eMMC). Perhaps adding that option too?

In the current state, the installation does not work correctly (there is no proper partition creation, there is no u-boot installation, etc.). I am currently working on a fix and will create a PR as testing progresses.

[ManoftheSea]

On 10/8/2022 at 9:50 AM, balbes150 said:

Remove the "boot" flags from the first FAT partition and the bootloader will stop using EFI.

I really wish you'd distinguish the ESP (a FAT-formatted partition on a GPT labeled disk with the partition type of C12A7328-F81F-11D2-BA4B-00A0C93EC93B (gparted identifies this as 0xEF00) ) from "the first fat partition" which may NOT be the ESP.

 

But if I understand what you're trying to do, you're using GRUB2 as the EFI app to then launch the linux kernel.  I don't know anything about GRUB2 anymore.  Certainly, it is a method of booting.  But better exists.

[balbes150]

9 часов назад, ManoftheSea сказал:

I really wish you'd distinguish the ESP (a FAT-formatted partition on a GPT labeled disk with the partition type of C12A7328-F81F-11D2-BA4B-00A0C93EC93B (gparted identifies this as 0xEF00) ) from "the first fat partition" which may NOT be the ESP.

1. I wrote about the "first fat section" - because I know for SURE (as the author of this image) that it is him (and why it is needed).

2. When creating an image, this section has the "boot" and "esp" flags, when removing the "boot" flag, the "esp" flag is automatically removed and "msftdata" is installed.

 

9 часов назад, ManoftheSea сказал:

But better exists.

At the moment, a bundle of u-boot + efi + grub is the best option for a universal image. As far as I know, only u-boot can provide global support for all devices (which we use), all other variants of startup systems, at best, have fragmentary support (on some type of hardware\platforms).

[balbes150]

08.10.2022 в 16:43, iav сказал:

shutdown and reboot commands stays system shutted down but powered on.

Added  version 20221012 with the current 5.19 kernel. I wonder how it will work on Helios 64.

[iav]

I run headless, with serial console. jammy.0.txt

`reboot` does reboot, `poweroff` not power off after shutdown.

boot takes a lot of time, you can see by timestamps.

 

`armbian-config` allows me to set scheduler to schedutil and set min and max frequences. System cooler was off until I setup 7zip and run benchmark `7zz b`. then system cooler turns on until test was completed, then turns off.

system diagnostics http://ix.io/4cXm

[iav]

debian version — console jammy.0.txt

diagnostic http://ix.io/4cXz

all the same.

[ManoftheSea]

11 hours ago, balbes150 said:

as the author of this image

But let's talk more generally about what is actually required, not just what is done in this instance.  Otherwise, we end up with a cargo cult - we do it this way because we've always done it this way.  The more "normal" we make the boot process (do it like debian or red hat, etc), the easier to support these systems with an installer rather than just flashing images to SD.

11 hours ago, balbes150 said:

the "esp" flag is automatically removed

What is an "ESP flag"?  Bit 2 is "BIOS bootable" flag in a GPT entry, and u-boot looks for this flag as part of its autodiscovery.  But there is no ESP flag - the GUID identifies the ESP on a given block device.  It looks like Parted may have its own non-standard terminology here.  Some of the SBCs in Armbian cannot use GPT formatted disks because vendor bootloader code is in the way.  In which case, u-boot can still find EFI apps on MBR formatted disks.

 

11 hours ago, balbes150 said:

a bundle of u-boot + efi + grub is the best option for a universal image.

At the moment with stable versions of Debian, I agree with you.  With systemd 251, systemd-boot improves on GRUB in a distro-agnostic way.  This would enable running multiple distributions on the same block device without conflict.

- If the SBC can keep u-boot with the board in flash, this is the best case, as an EFI app for the architecture ought to be automatically found and launched by u-boot.  U-boot fades as a component about which to worry.

- If u-boot can be loaded from eMMC hardware partition, this can work like the previous case, if the board knows how to keep its boot variables

- if u-boot must be loaded from SD card, the image can never be universal, so there are only bad choices.  But most of these devices need specially patched kernels anyway, so it doesn't matter how universal the bootloader is.

 

IF u-boot provides the UEFI environment (which I think it does in most cases), then the choice of systemd-boot or GRUB or just loading the kernel directly can be made.  Systemd-boot makes the most sense to me.

[balbes150]

13 часов назад, iav сказал:

boot takes a lot of time, you can see by timestamps.

judging by the log, there may not be enough files \ settings for network cards to work. without access to real hardware, it is not possible to search for and fix these problems. if desired, you can use my GIT and create new versions of images yourself (add the necessary components\files to the reset system settings) and solve this problem. you can use Helios64 itself to build the system from an SD card or USB drive, so as not to interfere with the system on eMMC. and according to the results (when you get the solution), send the PR to the official GIT Armbian.

 

 

[balbes150]

11 часов назад, ManoftheSea сказал:

But let's talk more generally about what is actually required, not just what is done in this instance.  Otherwise, we end up with a cargo cult - we do it this way because we've always done it this way.  The more "normal" we make the boot process (do it like debian or red hat, etc), the easier to support these systems with an installer rather than just flashing images to SD.

at this stage, I am interested in a conceptual check of the general possibility of using the resulting images (including on marvel equipment) in order to work out the basic parameters \ principles that need to be used for a universal image. in the future, you can use any installation/ startup algorithms and optimize them for any tasks (installation and easy use of different cores or systems, with an easy choice of what you need at startup, as on a regular PC, etc.).

 

11 часов назад, ManoftheSea сказал:

At the moment with stable versions of Debian, I agree with you.  With systemd 251, systemd-boot improves on GRUB in a distro-agnostic way.  This would enable running multiple distributions on the same block device without conflict.

I'll tell you the "secret" for a long time (several years) I have been using several systems on different devices at the same time (Ubuntu\Debian\Altlinux\Libreelec), which still have several kernel variants installed, and I choose the right system banally, through the keyboard, as on a regular PC or from the UART console. By the way, many years ago I released multi-system (eMMC hosted several Android\Linux\Libreelec systems at once) versions for khadassevitch with AML chips and users could easily choose which system to run. And for Rk and AW, this is a simple task.

 

12 часов назад, ManoftheSea сказал:

if u-boot must be loaded from SD card, the image can never be universal,

You're wrong Even in this case, you can have one universal image, but you need to change the algorithm of its use by users somewhat (another step is added when writing the image to the SD card), after recording the image, the bootloader image file for a specific model is also recorded on the same medium. I've been using this option for a long time and it works great.