ManoftheSea 6 Posted Friday at 03:46 AM Share Posted Friday at 03:46 AM I'm writing from a limited perspective, so constructive criticism appreciated. In this thread, I'm recording some thoughts about how the OS gets from files into the memory of various boards, and what might make this more unified and easier and/or more flexible. Current U-boot My knowledge comes from the mvebu64 and rockchip64 families. At this time, the board boot process starts from code in the processor that attempts to find u-boot, and u-boot runs a script from its environment that searches for a UEFI application or a boot.scr file. They call this "Distro-boot" Armbian uses the boot.scr to load an uncompressed kernel image and a compressed initramdisk, along with a device tree. These use u-boot legacy image format. U-boot then passes control to the kernel. On mvebu64 (espressobin), u-boot is contained within the SPI on the board, along with its environment. U-boot 2022.04 on this board can do network booting as well as searching emmc, sd, or sata devices for a boot device based on a GPT type or simply the existence of files within the first partition on each device. On rk3399, some earlier stage attempts to launch idbloader from sector 64, which loads u-boot.itb from sector 16384, from whatever device can be detected: SPI, USB, eMMC, SD. The boot.scr loads the legacy uboot images from the first partition on the device, then applies overlays - I believe this enables HATs on boards that can use them. U-boot capabilities Boot process: UEFI. U-boot can load the linux kernel through the UEFI stub, or the GRUB bootloader. As far as I can tell, it's not a drop-in replacement for a legacy kernel boot. Boot file format: Flattened uImage Tree. This is a monolithic file that has one or more of kernels, initramdisks, and device trees, along with "configurations" that can be selected to tell u-boot which files to actually load. Additionally, the components of this file can be compressed, as u-boot can uncompress the kernel before passing control. Boot media: dhcp(v6) to tftp. U-boot can attempt to boot an EFI application from tftp, before it attempts to run a script from dhcp, before it attempts to PXE boot. Extlinux script: u-boot can read /extlinux/extlinux.conf where a menu of kernels can be provided, with paths to kernel, initrd, and dtb, and an entry for the cmdline. Each kernel on the system would need to contribute a fragment of this config, which isn't currently done. Image Layout Currently, many armbian images use an MBR partition table. It's the default, and a device needs to request GPT during the build to get it. But with GPT come a few advantages. The systemd/freedesktop people seem to have a vision of a single layout that would allow a linux system to boot on several architectures, and then auto-configure based on "discoverable partitions". There are partition type uuids for such things as ARM64 root partition, or dm-verity superblock and signature partitions. These features would give a read-only, integrity checked, signed image. There are partition flags for things like "read-only" (60), "no auto" (63), or "grow-file-system" (59). Beyond this, the ESP (EFI System Partition) has a structure to allow multiple kernels/OSes to deconflict the storage of their files. Boot Sequence Type #1 Boot Loader Specification: $BOOT is the (typically) vfat partition with type "bc13c2ff-59e6-4262-a352-b275fd6f7172", or the ESP (VFAT, "c12a7328-f81f-11d2-ba4b-00a0c93ec93b"). Under $BOOT/loader/entries/, there would be files (e.g. ${machine-id}-${uname -r}-${os-release}.conf) that give configuration to find kernel and initrd at $BOOT/${machine-id}/${uname}/{linux,initrd,dtb,dtbo} Type #2: Unified kernel image - EFI PE w/ stub loader, initrd, and commandline, and can be signed cryptographically. These are at $BOOT/EFI/Linux/*.efi I assume these are handled by GRUB, as they do not seem to match u-boot's distro-boot. Armbian currently uses the boot.scr from u-boot's distroboot sequence to boot off the filesystem where boot.scr was found. First-boot and nand-sata-install When armbian builds the image, it does so through chroot and debootstrap. This results in artifacts like /etc/machine-id or /etc/ssh/ssh_host_*_key, which shouldn't be on multiple systems. They're removed as part of the image creation. Partition UUIDs ought to be changed too, but these are less likely to be a problem. The nand-sata-install attempts to create partitions to make a filesystem on some other block device, and then rsync's the running armbian system over to the new block device. It also tries to manage the u-boot bootloader and all the possible ways that can exist. (Goal) SD card partitioning sgdisk -n 14:${biosstart}:${biosend} -c 14:bios -t 14:ef02 -n 15:${uefistart}:${uefiend} -c 15:efi -t 15:ef00 -n 1:${rootstart}:0 -c 1:root -t 1:8305 -A 1:set:59 ${SDCARD}.raw This produces a GPT-formatted disk with the first partition (numbered 14) with a BIOS type (recommended 1 mb, or covering u-boot and files), an ESP (recommended 550 MB, to mount on /efi/), and an auto-detected ARM64 root partition that is labeled to grow to fill the disk when booted. Linux should be stored in a UEFI or FIT image, preferably as a UEFI boot method. U-boot environment variables for a given board should be able to select the appropriate armbian kernel, allowing the root filesystem to have multiple kernels for different boards installed and an SD card that boots on multiple boards. 0 Quote Link to post Share on other sites More sharing options...
jethome 1 Posted Friday at 05:17 AM Share Posted Friday at 05:17 AM on amlogic gpt will conflict with u-boot written on device. this can only be avoided via emmc-boot partitions or spi-flash 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted Friday at 04:07 PM Author Share Posted Friday at 04:07 PM I see on meson64, write_uboot_platform writes 442 bytes to the beginning of the disk (out of 446 reserved for bootloader code, why the missing 4 bytes?) and then continues to write the rest of uboot. MBR fits in the 66 bytes that get skipped, while GPT would need to use the next 16 kB (LBA 1-33 with 512b sectors) and that's conflicting with the u-boot code. I wonder if it would be possible to get the uboot stage 1 code in the first sector to jump to 1 MB rather than just over the MBR, perhaps with some setting in the u-boot code. Better still would be searching for a BIOS type GPT partition, that's what that type is for. This is a problem, but one that looks like it can be handled at image creation without having to run commands on the amlogic board. Thanks for the note. 0 Quote Link to post Share on other sites More sharing options...
jethome 1 Posted Friday at 04:11 PM Share Posted Friday at 04:11 PM 2 минуты назад, ManoftheSea сказал: I wonder if it would be possible to get the uboot stage 1 code in the first sector to jump to 1 MB rather than just over the MBR, perhaps with some setting in the u-boot code. Better still would be searching for a BIOS type GPT partition, that's what that type is for. amlogic use closed bins for bl2/bl30/bl31 stages. https://github.com/LibreELEC/amlogic-boot-fip 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted Friday at 04:27 PM Author Share Posted Friday at 04:27 PM So, "u-boot.bin" isn't just BL31 u-boot, but is lower levels of bootloader stages too. And it looks like they may be encrypted to the processor too, so there's no chance of a human-readable "jump" instruction that could be redirected. Disappointing. 0 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted Friday at 05:08 PM Share Posted Friday at 05:08 PM 13 hours ago, ManoftheSea said: allowing the root filesystem to have multiple kernels for different boards installed and an SD card that boots on multiple boards. I already have this with native mainline uboot without special boot partition or even a boot directory. My SD card boots on several SBCs (NanoPC-T4, ODROID-N2+, Honeycomb, ...) and the several kernels I have installed concurrently are only a reboot and a keypress, to select the desired one, away. Booting different systems (on separate partitions) is also working, but I use it seldom. And the good thing, to configure everything, only a simple text file is needed. I have already used this boot schema with various Armbian images for some reverse engineering tasks. Armbian had everything necessary available when using mainline uboot, at least with all images I used. It was only a question of creating a suitable configuration file and copy some files around. If you are eager to experiment, I can try to talk you thru the configuration. For our experiment an SBC with rk3399 and an SD card with an Armbian image which uses mainline uboot would be best suited because I am most familiar with this SOC. 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted Friday at 05:53 PM Author Share Posted Friday at 05:53 PM Yeah, tell me about "suitable configuration file and copy files around". How does uboot match the board with the proper kernel to launch? 0 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted Friday at 06:39 PM Share Posted Friday at 06:39 PM 43 minutes ago, ManoftheSea said: Yeah, tell me about "suitable configuration file and copy files around". As a first step, I need some information about the running system on which we want to do the experiment. Please post the output of the following commands: cat /proc/cmdline ls -hal /boot lsblk -o +PARTUUID so I can prepare the experiment. 45 minutes ago, ManoftheSea said: How does uboot match the board with the proper kernel to launch? It is the right DTB what really matters. 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted Friday at 07:25 PM Author Share Posted Friday at 07:25 PM I've got access to both the Helios64 (rk3399) and the espressobin (mvebu64). Both are on 5.15 in "current". Since the rk3399 loads firmware from the sd card, I expect that would be an easier base from which to work - the ebin loads u-boot from SPI. Is it part of the boot.scr that looks up the board on which its running, so as to get u-boot to load the proper DTB? Surely it can't use the symlink to simply "dtb/" but needs to be customized to 5.15.31-mvebu64 or 5.15.31-rockchip64. Spoiler mmcblk0 179:96 0 14.8G 0 disk └─mmcblk0p1 179:97 0 14.7G 0 part / 8b029f90-01 root=UUID=939c3717-edfc-4ff6-b0b5-4b0058bda414 rootwait rootfstype=ext4 console=ttyS2,1500000 console=tty1 consoleblank=0 loglevel=1 ubootpart=8b029f90-01 usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1 drwxr-xr-x 3 root root 4.0K May 12 16:38 . drwxr-xr-x 19 root root 4.0K Mar 28 11:23 .. -rw-r--r-- 1 root root 166 Mar 28 11:23 armbianEnv.txt -rw-r--r-- 1 root root 1.5K Sep 8 2021 armbian_first_run.txt.template -rw-r--r-- 1 root root 38K Sep 8 2021 boot.bmp -rw-r--r-- 1 root root 3.1K Sep 8 2021 boot.cmd -rw-rw-r-- 1 root root 3.2K Sep 8 2021 boot.scr -rw-r--r-- 1 root root 226K Mar 28 01:30 config-5.15.31-rockchip64 lrwxrwxrwx 1 root root 22 Mar 28 09:58 dtb -> dtb-5.15.31-rockchip64 drwxr-xr-x 6 root root 4.0K Mar 28 09:58 dtb-5.15.31-rockchip64 lrwxrwxrwx 1 root root 26 Mar 28 09:59 Image -> vmlinuz-5.15.31-rockchip64 -rw-r--r-- 1 root root 15M May 7 21:53 initrd.img-5.15.31-rockchip64 -rw-r--r-- 1 root root 0 Mar 28 09:59 .next -rw-r--r-- 1 root root 6.1M Mar 28 01:30 System.map-5.15.31-rockchip64 lrwxrwxrwx 1 root root 26 May 7 21:53 uInitrd -> uInitrd-5.15.31-rockchip64 -rw-r--r-- 1 root root 15M May 7 21:53 uInitrd-5.15.31-rockchip64 -rw-r--r-- 1 root root 29M Mar 28 01:30 vmlinuz-5.15.31-rockchip64 0 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted Friday at 08:20 PM Share Posted Friday at 08:20 PM OK, here is the preperation for our experiment: First step: prepare kernel package (copy files around) cd /usr/lib/modules/5.15.31-rockchip64 cp --preserve --recursive /boot/dtb . cp --preserve /boot/System* . cp --preserve /boot/config* . cp --preserve /boot/Image . cp --preserve /boot/vmlinuz* . cp --preserve /boot/initrd* . Second step: prepare for dristro-boot mkdir /extlinux - Copy the attached extlinux.conf into /extlinux/extlinux.conf over. - Reboot and check if the device is now booting with distro-boot method The auto boot timeout is set to 10 seconds, so be patient. Please post the uboot output because I wrote the extlinux.conf without any testing possibility and hopefully did not make any typos. extlinux.conf 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted Friday at 08:33 PM Author Share Posted Friday at 08:33 PM I forgot about extlinux. To support that, our kernel packages would need to be able to contribute fragments to the generated extlinux.conf, which probably belongs somewhere in /etc/initramdisk/post-install/ scripts. I've updated my first post to mention it. This also seems like a good way to have different kernels even for the same board, such as what I hear about a "media kernel". 0 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted Friday at 09:01 PM Share Posted Friday at 09:01 PM 27 minutes ago, ManoftheSea said: kernel packages would need to be able to contribute fragments to the generated extlinux.conf For standard kernel updates, this is not even strictly necessary. We will see this as soon as my offered extlinux.conf works. 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted Friday at 09:35 PM Author Share Posted Friday at 09:35 PM I was able to test the extlinux.conf (though I moved some things around). I used the existing /boot/Image and /boot/uInitrd symlinks. The fdtfile name, that comes from the u-boot environment on the device, right? So while a kernel that targets aarch64 might be able to boot on multiple systems, the u-boot on the system has to know the right name to find the dtb. And if that changes between kernel versions, then u-boot's environment must be kept current to seek out the fdt and load it. 0 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted Friday at 09:58 PM Share Posted Friday at 09:58 PM 21 minutes ago, ManoftheSea said: though I moved some things around Please do not do this, it will prevent the usability of kernels installed at the same time. In the end, we won't use anything in /boot. 22 minutes ago, ManoftheSea said: the u-boot on the system has to know the right name to find the dtb. Firmware (uboot) is the only device specific code and is dedicated to each SBC. Having it on the SD card prevents it usability for different devices. The best is if you can offload it in a SPI flash. There is not one uboot that suits everyone. 1 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted Saturday at 05:59 AM Share Posted Saturday at 05:59 AM 8 hours ago, ManoftheSea said: I was able to test the extlinux.conf I am still awaiting the uboot messages log. We are still in the stage of redoing the legacy boot features with distro-boot method. If I have verified that it is working as I expect, I can start teach how to use the now possible new features of distro-boot. 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted Saturday at 05:24 PM Author Share Posted Saturday at 05:24 PM I did not move the files from /boot to /usr/lib/modules, and I edited the extlinux.conf to point at the files in /boot. I think I understand what would be accomplished by moving the files, though. I actually did the test on ebin (mvebu64) with a default uboot environment and saw that it was able to boot the kernel and bring up the system. Do you really need the actual console output? It looked like it just says it's booting the kernel, same as it ever does. As far as ebin goes, I moved that to using FIT images indicated by the boot.scr.uimg. The extlinux.conf can't tell uboot to use this itb file, I don't think. Which means no compressed kernels and no uefi boots. Why avoiding anything in /boot, though? I think the desired state (which has been pointed out that it has conflicts with bootloader code) would be an ESP that has files from any number of OS's or distributions and one or more linux partitions for the Armbian system, identified through the root=UUID=xyz (or autodiscovered), which don't conflict with other distributions. But uboot would need to save efivars to point at files and cmdlines. If the Armbian image has a generic aarch64 kernel and dtbs for multiple boards, and uboot can pick the specific dtb to be used, wouldn't this allow for a universal Armbian image that can coexist with other OS's? 0 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted Saturday at 08:29 PM Share Posted Saturday at 08:29 PM 2 hours ago, ManoftheSea said: did not move the files from /boot to /usr/lib/modules II never asked for, the cp command does a copy, i.e. the source file keeps in place. We only add files to the root filesystem and the lagacy support stays in place. You can always cancel our experiment by renaming extlinux conf, e.g. mv extlinux.conf extlinux.conf-disabled You are then back to your old method with some more files in the root file system. Of course, you can also delete all added files to remove them completely. 2 hours ago, ManoftheSea said: I think I understand what would be accomplished by moving the files With your wrong assumptions you come to wrong conclusions. Please let's do the experiment as I suggest, and you'll ask your questions at the end when you know the whole story. 2 hours ago, ManoftheSea said: I actually did the test on ebin (mvebu64) Please use the Helios64 for the experiment. I don't have any of your devices and using a well-known SOC is already complicated when I have to rely on your eyes and hands. We can come back to espressobin if we were successful with Helios64. 3 hours ago, ManoftheSea said: Do you really need the actual console output? When your boot process happens as always it sounds to me as if it has fallen back to the legacy method. It would therefore be important for me to know the exact messages. Oh, I forgot to ask, does your current uboot support the USB keyboard so you can interrupt autoboot and make keyboard inputs? If not, we need the serial console for now and take care of uboot USB keyboard support afterwards. 3 hours ago, ManoftheSea said: Which means no compressed kernels and no uefi boots. The Image link point to vmlinuz and that is a compressed kernel and we are using distro-boot and that has nothing to do with the uefi methode. 3 hours ago, ManoftheSea said: Why avoiding anything in /boot, though? By the end of our experiment, it will be obvious to you. 3 hours ago, ManoftheSea said: But uboot would need to save efivars to point at files and cmdlines. If the Armbian image has a generic aarch64 kernel and dtbs for multiple boards, and uboot can pick the specific dtb to be used, wouldn't this allow for a universal Armbian image that can coexist with other OS's? By the end of our experiment, you will know how it works. 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted Saturday at 09:57 PM Author Share Posted Saturday at 09:57 PM Do you really have to be so mysterious? vmlinuz is NOT compressed on aarch64. The FIT image allows telling u-boot that the kernel image is compressed, and uboot does the decompression, not the kernel. Here's uboot log: Spoiler U-Boot 2022.04-armbian (May 13 2022 - 05:30:12 +0000) DRAM: 512 MiB Core: 38 devices, 20 uclasses, devicetree: fit WDT: Not starting watchdog@8300 Comphy chip #0: Comphy-0: USB3_HOST0 5 Gbps Comphy-1: PEX0 5 Gbps Comphy-2: SATA0 6 Gbps SATA link 0 timeout. AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode flags: ncq led only pmp fbss pio slum part sxs PCIe: Link down MMC: sdhci@d0000: 0, sdhci@d8000: 1 Loading Environment from SPIFlash... SF: Detected is25wp032 with page size 256 B OK Model: Globalscale Marvell ESPRESSOBin Board Net: eth0: ethernet@30000 Hit any key to stop autoboot: 0 MMC Device 1 not found no mmc device at slot 1 MMC: no card present starting USB... Bus usb@58000: Register 2000104 NbrPorts 2 Starting the controller USB XHCI 1.00 Bus usb@5e000: USB EHCI 1.00 scanning bus usb@58000 for devices... 2 USB Device(s) found scanning bus usb@5e000 for devices... 1 USB Device(s) found scanning usb for storage devices... 1 Storage Device(s) found Device 0: Vendor: PNY Rev: 8192 Prod: USB 2.0 FD Type: Removable Hard Disk Capacity: 7728.0 MB = 7.5 GB (15826944 x 512) ... is now current device Scanning usb 0:1... Found /extlinux/extlinux.conf Retrieving file: /extlinux/extlinux.conf Boot Options 1: default 2: fit Enter choice: 2 2: fit Retrieving file: /boot/espressobin.itb append: root=UUID=705ef14a-c1c1-4f59-9fc9-b4a489e68873 rootfstype=ext4 rootwait ## Loading kernel from FIT Image at 07000000 ... Using 'v5' configuration Verifying Hash Integrity ... OK Trying 'kernel' kernel subimage Description: Vanilla Linux kernel Type: Kernel Image Compression: lzma compressed Data Start: 0x070000d0 Data Size: 4801338 Bytes = 4.6 MiB Architecture: AArch64 OS: Linux Load Address: 0x07000000 Entry Point: 0x07000000 Hash algo: sha1 Hash value: 837ab9c436ee8b1a71d3504c6bb8fb50f3f2bd84 Verifying Hash Integrity ... sha1+ OK ERROR: Did not find a cmdline Flattened Device Tree Could not find a valid device tree 1: default Retrieving file: /boot/uInitrd Retrieving file: /boot/Image append: root=UUID=705ef14a-c1c1-4f59-9fc9-b4a489e68873 rootfstype=ext4 rootwait Retrieving file: /boot/dtb/marvell/armada-3720-espressobin.dtb ## Loading init Ramdisk from Legacy Image at 0a000000 ... Image Name: uInitrd Image Type: AArch64 Linux RAMDisk Image (gzip compressed) Data Size: 6635288 Bytes = 6.3 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Flattened Device Tree blob at 06f00000 Booting using the fdt blob at 0x6f00000 Loading Ramdisk to 1f4a3000, end 1faf6f18 ... OK Loading Device Tree to 000000001f49d000, end 000000001f4a2d85 ... OK Starting kernel ... Loading, please wait... Starting version 247.3-7 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Here's /extlinux/extlinux.conf Spoiler menu title Boot Options timeout 100 label default fdtdir /boot/dtb/ kernel /boot/Image initrd /boot/uInitrd append root=UUID=705ef14a-c1c1-4f59-9fc9-b4a489e68873 rootfstype=ext4 ro otwait loglevel=1 usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u label fit kernel /boot/espressobin.itb append root=UUID=705ef14a-c1c1-4f59-9fc9-b4a489e68873 rootfstype=ext4 rootwa it loglevel=1 usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u Here's /boot/ Spoiler root@espressobin:/boot# ls -hal total 42M drwxr-xr-x 3 root root 4.0K May 14 09:37 . drwxr-xr-x 20 root root 4.0K May 13 09:04 .. -rw-r--r-- 1 root root 144 May 14 09:37 armbianEnv.txt -rw-r--r-- 1 root root 1.5K May 13 02:28 armbian_first_run.txt.template -rw-r--r-- 1 root root 38K May 13 02:28 boot.bmp -rw-r--r-- 1 root root 567 May 13 02:22 boot.cmd -rw-rw-r-- 1 root root 639 May 13 02:29 boot.scr.uimg -rw-r--r-- 1 root root 123K May 13 02:18 config-5.17.5-mvebu64 lrwxrwxrwx 1 root root 18 May 13 02:27 dtb -> dtb-5.17.5-mvebu64 drwxr-xr-x 3 root root 4.0K May 13 02:27 dtb-5.17.5-mvebu64 -rw-r--r-- 1 root root 11M May 13 02:31 espressobin.itb lrwxrwxrwx 1 root root 22 May 13 02:27 Image -> vmlinuz-5.17.5-mvebu64 -rw-r--r-- 1 root root 6.4M May 13 02:31 initrd.img-5.17.5-mvebu64 -rw-r--r-- 1 root root 0 May 13 02:27 .next -rw-r--r-- 1 root root 2.4M May 13 02:18 System.map-5.17.5-mvebu64 lrwxrwxrwx 1 root root 22 May 13 02:31 uInitrd -> uInitrd-5.17.5-mvebu64 -rw-r--r-- 1 root root 6.4M May 13 02:31 uInitrd-5.17.5-mvebu64 -rw-r--r-- 1 root root 16M May 13 02:18 vmlinuz-5.17.5-mvebu64 I'm using serial console in the case of the ebin, and I will use serial console for helios64. That'll be a little slower to run, I'll get to it. 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted Sunday at 03:32 AM Author Share Posted Sunday at 03:32 AM Here's the Helios64: I had to add `cp --preserve --recursive /boot/dtb-5.15.31-rockchip64` to the instructions given, as the first command only copied a symlink Directories: Spoiler root@iron:/usr/lib/modules/5.15.31-rockchip64/boot# ls -lah total 79M drwxr-xr-x 3 root root 4.0K May 14 23:03 . drwxr-xr-x 4 root root 4.0K May 14 23:00 .. -rw-r--r-- 1 root root 226K Mar 28 01:30 config-5.15.31-rockchip64 lrwxrwxrwx 2 root root 22 Mar 28 09:58 dtb -> dtb-5.15.31-rockchip64 drwxr-xr-x 6 root root 4.0K Mar 28 09:58 dtb-5.15.31-rockchip64 -rw-r--r-- 1 root root 29M Mar 28 01:30 Image -rw-r--r-- 1 root root 15M May 7 21:53 initrd.img-5.15.31-rockchip64 -rw-r--r-- 1 root root 6.1M Mar 28 01:30 System.map-5.15.31-rockchip64 -rw-r--r-- 1 root root 29M Mar 28 01:30 vmlinuz-5.15.31-rockchip64 root@iron:/usr/lib/modules# ls -hal total 12K drwxr-xr-x 3 root root 4.0K May 14 23:05 . drwxr-xr-x 82 root root 4.0K May 11 17:19 .. drwxr-xr-x 4 root root 4.0K May 14 23:00 5.15.31-rockchip64 lrwxrwxrwx 1 root root 19 May 14 23:05 linux -> 5.15.31-rockchip64/ /extlinux/extlinux.conf - slight differences Spoiler menu title Boot Options timeout 100 label default fdtdir /usr/lib/modules/linux/boot/dtb/ kernel /usr/lib/modules/linux/boot/vmlinuz initrd /usr/lib/modules/linux/boot/uInitrd append loglevel=4 root=PARTUUID=8b029f90-01 cma=896M coherent_pool=2M selinux=0 audit=0 console=uart8250,mmio32,0xff1a0000 console=tty0 fbcon=nodefer rootwait rootfstype=ext4 raid=noautodetect label 5.15.31-rockchip64 fdtdir /usr/lib/modules/5.15.31-rockchip64/boot/dtb/ kernel /usr/lib/modules/5.15.31-rockchip64/boot/Image initrd /usr/lib/modules/5.15.31-rockchip64/boot/initrd.img-5.15.31-rockchip64 append root=PARTUUID=8b029f90-01 rootwait rootfstype=ext4 console=ttyS2,1500000 console=tty1 consoleblank=0 loglevel=1 ubootpart=8b029f90-01 usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1 uboot log: Spoiler => boot switch to partitions #0, OK mmc1 is current device Scanning mmc 1:1... Found /extlinux/extlinux.conf Retrieving file: /extlinux/extlinux.conf 909 bytes read in 5 ms (176.8 KiB/s) Boot Options 1: default 2: 5.15.31-rockchip64 Enter choice: 2 2: 5.15.31-rockchip64 Retrieving file: /usr/lib/modules/5.15.31-rockchip64/boot/initrd.img-5.15.31-rockchip64 15271936 bytes read in 658 ms (22.1 MiB/s) Retrieving file: /usr/lib/modules/5.15.31-rockchip64/boot/Image 30149120 bytes read in 1287 ms (22.3 MiB/s) append: root=PARTUUID=8b029f90-01 rootwait rootfstype=ext4 console=ttyS2,1500000 console=tty1 consoleblank=01 Retrieving file: /usr/lib/modules/5.15.31-rockchip64/boot/dtb/rockchip/rk3399-kobol-helios64.dtb 83425 bytes read in 19 ms (4.2 MiB/s) Moving Image from 0x2080000 to 0x2200000, end=3f60000 ## Flattened Device Tree blob at 01f00000 Booting using the fdt blob at 0x1f00000 Loading Ramdisk to f505e000, end f5eee800 ... OK Loading Device Tree to 00000000f5046000, end 00000000f505d5e0 ... OK Starting kernel ... Armbian 21.08.6 Bullseye ttyS2 iron login: Addl info: The espressobin is using uboot 2022.04, while the helios64 self-identifies as "ver=U-Boot 2020.10-armbian (Sep 08 2021 - 10:55:50 +0000)". It is interesting to me that the helios u-boot has more uboot variables than the espressobin. The serial# and ethaddr are bogus, coming from the Armbian image because of uboot variables on the sd card, right? Compression: Spoiler root@iron:/boot# ls -l vmlinuz-5.15.31-rockchip64* -rw-r--r-- 1 root root 30149120 Mar 28 01:30 vmlinuz-5.15.31-rockchip64 -rw-r--r-- 1 root root 8430130 Mar 28 01:30 vmlinuz-5.15.31-rockchip64.lzma 0 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted Sunday at 06:21 AM Share Posted Sunday at 06:21 AM 8 hours ago, ManoftheSea said: Do you really have to be so mysterious? I am still in the phase to learn what your system exactly provides. I know what to look for, since I don't know the results before, I can't deliver a turnkey system. I therefore have to check individual points step by step and, if necessary, establish appropriate workarounds. Therefore, it is necessary to follow my instructions as unmodified as possible. 8 hours ago, ManoftheSea said: vmlinuz is NOT compressed on aarch64 For me it is: file vmlinuz vmlinuz: gzip compressed data, max compression, from Unix, original size modulo 2^32 46146048 A point that kann be tackled for Armbina afterwards. 2 hours ago, ManoftheSea said: I had to add `cp --preserve --recursive /boot/dtb-5.15.31-rockchip64` Well caught, you have applied the right solution, link and directory are needed. 2 hours ago, ManoftheSea said: Here's the Helios64: The logs are already very promising, but do not yet confirm all the required points. I need the outputs with my unmodified extlinux.conf as I attached it. Please repeat the process by removing the link /usr/lib/modules/linux, restore my extlinux.conf, restart the device, wait for the autoboot timeing out and post the log after the boot is complete. If everything is as I expect it to be, there is only one imponderable to check and we can finally start our distro-boot exercise. 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted 19 hours ago Author Share Posted 19 hours ago On 5/15/2022 at 2:21 AM, usual user said: For me it is How? This isn't a kernel configuration option, so what is doing so afterward? And then how are you telling uboot to decompress it, that's not in your extlinux config? https://www.kernel.org/doc/Documentation/arm64/booting.txt "The AArch64 kernel does not currently provide a decompressor and therefore requires decompression (gzip etc.) to be performed by the boot loader if a compressed Image target (e.g. Image.gz) is used. For bootloaders that do not implement this requirement, the uncompressed Image target is available instead." I will remove the /usr/lib/modules/linux symlink. But I'm still going to leave the extra /usr/lib/modules/5.15.31-rockchip64/boot/ directory. It looks like you expect the default option to fail and you want to see if it will fallthrough. 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted 18 hours ago Author Share Posted 18 hours ago Bottom of log, default with timeout: Spoiler Hit any key to stop autoboot: 0 switch to partitions #0, OK mmc1 is current device Scanning mmc 1:1... Found /extlinux/extlinux.conf Retrieving file: /extlinux/extlinux.conf 909 bytes read in 5 ms (176.8 KiB/s) Boot Options 1: default 2: 5.15.31-rockchip64 Enter choice: 1: default Retrieving file: /usr/lib/modules/linux/boot/vmlinuz Failed to load '/usr/lib/modules/linux/boot/vmlinuz' Skipping default for failure retrieving kernel 2: 5.15.31-rockchip64 Retrieving file: /usr/lib/modules/5.15.31-rockchip64/boot/initrd.img-5.15.31-rockchip64 15271936 bytes read in 655 ms (22.2 MiB/s) Retrieving file: /usr/lib/modules/5.15.31-rockchip64/boot/Image 30149120 bytes read in 1286 ms (22.4 MiB/s) append: root=PARTUUID=8b029f90-01 rootwait rootfstype=ext4 console=ttyS2,1500000 console=tty1 consoleblank=0 loglevel=1 ubootpart=8b029f90-01 usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1 Retrieving file: /usr/lib/modules/5.15.31-rockchip64/boot/dtb/rockchip/rk3399-kobol-helios64.dtb 83425 bytes read in 18 ms (4.4 MiB/s) Moving Image from 0x2080000 to 0x2200000, end=3f60000 ## Flattened Device Tree blob at 01f00000 Booting using the fdt blob at 0x1f00000 Loading Ramdisk to f505e000, end f5eee800 ... OK Loading Device Tree to 00000000f5046000, end 00000000f505d5e0 ... OK Starting kernel ... Armbian 21.08.6 Bullseye ttyS2 Extlinux.conf Spoiler menu title Boot Options timeout 100 label default fdtdir /usr/lib/modules/linux/boot/dtb/ kernel /usr/lib/modules/linux/boot/vmlinuz append loglevel=4 root=PARTUUID=8b029f90-01 cma=896M coherent_pool=2M selinux=0 audit=0 console=uart8250,mmio32,0xff1a0000 console=tty0 fbcon=nodefer rootwait rootfstype=ext4 raid=noautodetect label 5.15.31-rockchip64 fdtdir /usr/lib/modules/5.15.31-rockchip64/boot/dtb/ kernel /usr/lib/modules/5.15.31-rockchip64/boot/Image initrd /usr/lib/modules/5.15.31-rockchip64/boot/initrd.img-5.15.31-rockchip64 append root=PARTUUID=8b029f90-01 rootwait rootfstype=ext4 console=ttyS2,1500000 console=tty1 consoleblank=0 loglevel=1 ubootpart=8b029f90-01 usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1 0 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted 13 hours ago Share Posted 13 hours ago 5 hours ago, ManoftheSea said: It looks like you expect the default option to fail and you want to see if it will fallthrough. Exactly. 5 hours ago, ManoftheSea said: But I'm still going to leave the extra /usr/lib/modules/5.15.31-rockchip64/boot/ directory. Again you have modified my suggestions, so slowly I lose the desire to continue this training. A boot directory is a legacy boot artefact that isn't needed for distro-boot. Either you want to learn from me my distro-boot deployment and follow my suggestions, or we leave it and I save myself a lot of frustration. Since I am convinced from the previous observations that your system has the necessary prerequisites, here is a last attempt to continue the training. Use my unmodified extlinux.conf and use the locations that evoke these commands: cd /usr/lib/modules/5.15.31-rockchip64 cp --preserve --recursive /boot/dtb* . cp --preserve /boot/System* . cp --preserve /boot/config* . cp --preserve /boot/Image . cp --preserve /boot/vmlinuz* . cp --preserve /boot/initrd* . After the training, you can then make all the changes you want when you implement it for Armbian. As a next step, we will now install another kernel. Since I can assume that you do not have a suitable kernel package available, I have uploaded my kernel, as I currently use it on all my devices, here. It is a pure mainline kernel built as a generic kernel, i.e. it should be suitable for all aarch64 devices that have corresponding mainline support. Helios64 support is built, but I'm not sure if everything necessary is included, but the backup 5.15.31-rockchip64 is in place to keep your System usable. Unpack it with this command in your /usr/lib/modules/ branch: tar -C /usr/lib/modules/ -xzf 5.18.0-0.rc3.27.fc34.aarch64.tgz This will populte a new kernel package branch in /usr/lib/modules/ and aktivate it for the default boot option. Reboot your Helios64 and report if the new kernel fully starts. 0 Quote Link to post Share on other sites More sharing options...
ManoftheSea 6 Posted 11 hours ago Author Share Posted 11 hours ago So, how's fedora compressing the vmlinuz? How is uboot recognizing it and unpacking it? Can I just point uboot at any old compressed kernel? Spoiler Boot Options 1: default 2: 5.15.31-rockchip64 Enter choice: 1: default Retrieving file: /usr/lib/modules/linux/vmlinuz 15310030 bytes read in 657 ms (22.2 MiB/s) append: loglevel=4 root=PARTUUID=8b029f90-01 cma=896M coherent_pool=2M selinux=0 audit=0 console=uart8250,mmio32,0xff1a0000 console=tty0 fbcon=nodefer rootwait rootfstype=ext4 raid=noautodetect Retrieving file: /usr/lib/modules/linux/dtb/rockchip/rk3399-kobol-helios64.dtb 56725 bytes read in 16 ms (3.4 MiB/s) Uncompressing Kernel Image Moving Image from 0x2080000 to 0x2200000, end=58a0000 ## Flattened Device Tree blob at 01f00000 Booting using the fdt blob at 0x1f00000 Loading Device Tree to 00000000f5ede000, end 00000000f5eeed94 ... OK Starting kernel ... [ 4.674723] rockchip-usb2phy ff770000.syscon:usb2phy@e460: failed to create phy [ 4.732922] rockchip-drm display-subsystem: [drm:rockchip_drm_platform_probe] *ERROR* No available vop found for display-subsystem. [ 15.365803] rk_gmac-dwmac fe300000.ethernet: cannot get clock clk_mac_speed [ 16.045562] r8152 2-1.4:1.0 (unnamed net_device) (uninitialized): netif_napi_add() called with weight 256 Armbian 21.08.6 Bullseye ttyS2 iron login: root Password: _ _ _ _ __ _ _ | | | | ___| (_) ___ ___ / /_ | || | | |_| |/ _ \ | |/ _ \/ __| '_ \| || |_ | _ | __/ | | (_) \__ \ (_) |__ _| |_| |_|\___|_|_|\___/|___/\___/ |_| Welcome to Armbian 22.05.0-trunk with Linux 5.18.0-0.rc3.27.fc34.aarch64 No end-user support: built from trunk System load: 59% Up time: 0 min Memory usage: 12% of 3.75G IP: CPU temp: 51°C Usage of /: 43% of 15G Last login: Mon May 16 16:50:01 EDT 2022 from XXXX:XXX:XXXX:XXXX:2365:131b:8635:ea86 on pts/0 root@iron:~# uname -a Linux iron 5.18.0-0.rc3.27.fc34.aarch64 #1 SMP PREEMPT_DYNAMIC Mon Apr 18 17:15:42 CEST 2022 aarch64 GNU/Linux root@iron:~# So a successful boot of the fedora kernel. Full log attached. I also attempted the 5.16 kernel from debian, but it did not find the console or network, so I can't claim it ever successfully booted. minicom-iron.cap 0 Quote Link to post Share on other sites More sharing options...
usual user 23 Posted 10 hours ago Share Posted 10 hours ago Congratulations, now we are in business. 52 minutes ago, ManoftheSea said: So, how's fedora compressing the vmlinuz? How is uboot recognizing it and unpacking it? It uses exactly what is described in your quote (https://www.kernel.org/doc/Documentation/arm64/booting.txt). The kernel is build with the Image .gz target and the firmware (uboot) decompresses it. And, surprise, uboot can automatically detect at what kind of kernel the extelinux.conf kernel key points. I have so much more to elaborate about what has been achieved so far, but no time right now. So I have to put you off on a follow-up post, please stay tuned. 1 Quote Link to post Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.