denni_isl

I'm writing from a limited perspective, so constructive criticism appreciated. In this thread, I'm recording some thoughts about how the OS gets from files into the memory of various boards, and what might make this more unified and easier and/or more flexible.
Current U-boot
My knowledge comes from the mvebu64 and rockchip64 families.  At this time, the board boot process starts from code in the processor that attempts to find u-boot, and u-boot runs a script from its environment that searches for a UEFI application or a boot.scr file.  They call this "Distro-boot"  Armbian uses the boot.scr to load an uncompressed kernel image and a compressed initramdisk, along with a device tree.  These use u-boot legacy image format.  U-boot then passes control to the kernel.
On mvebu64 (espressobin), u-boot is contained within the SPI on the board, along with its environment.  U-boot 2022.04 on this board can do network booting as well as searching emmc, sd, or sata devices for a boot device based on a GPT type or simply the existence of files within the first partition on each device.
On rk3399, some earlier stage attempts to launch idbloader from sector 64, which loads u-boot.itb from sector 16384, from whatever device can be detected: SPI, USB, eMMC, SD.  The boot.scr loads the legacy uboot images from the first partition on the device, then applies overlays - I believe this enables HATs on boards that can use them. 
 
U-boot capabilities
Boot process: UEFI.  U-boot can load the linux kernel through the UEFI stub, or the GRUB bootloader.  As far as I can tell, it's not a drop-in replacement for a legacy kernel boot.
Boot file format: Flattened uImage Tree.  This is a monolithic file that has one or more of kernels, initramdisks, and device trees, along with "configurations" that can be selected to tell u-boot which files to actually load.  Additionally, the components of this file can be compressed, as u-boot can uncompress the kernel before passing control.
Boot media: dhcp(v6) to tftp.  U-boot can attempt to boot an EFI application from tftp, before it attempts to run a script from dhcp, before it attempts to PXE boot.
Extlinux script: u-boot can read /extlinux/extlinux.conf where a menu of kernels can be provided, with paths to kernel, initrd, and dtb, and an entry for the cmdline.  Each kernel on the system would need to contribute a fragment of this config, which isn't currently done.
 
Image Layout
Currently, many armbian images use an MBR partition table.  It's the default, and a device needs to request GPT during the build to get it.  But with GPT come a few advantages.  The systemd/freedesktop people seem to have a vision of a single layout that would allow a linux system to boot on several architectures, and then auto-configure based on "discoverable partitions".  There are partition type uuids for such things as ARM64 root partition, or dm-verity superblock and signature partitions.  These features would give a read-only, integrity checked, signed image.  There are partition flags for things like "read-only" (60), "no auto" (63), or "grow-file-system" (59).  Beyond this, the ESP (EFI System Partition) has a structure to allow multiple kernels/OSes to deconflict the storage of their files. 
 
Boot Sequence
Type #1 Boot Loader Specification:
$BOOT is the (typically) vfat partition with type "bc13c2ff-59e6-4262-a352-b275fd6f7172", or the ESP (VFAT, "c12a7328-f81f-11d2-ba4b-00a0c93ec93b").  Under $BOOT/loader/entries/, there would be files (e.g. ${machine-id}-${uname -r}-${os-release}.conf) that give configuration to find kernel and initrd at $BOOT/${machine-id}/${uname}/{linux,initrd,dtb,dtbo}
Type #2:
Unified kernel image - EFI PE w/ stub loader, initrd, and commandline, and can be signed cryptographically.  These are at $BOOT/EFI/Linux/*.efi
 
I assume these are handled by GRUB, as they do not seem to match u-boot's distro-boot.  Armbian currently uses the boot.scr from u-boot's distroboot sequence to boot off the filesystem where boot.scr was found. 
 
First-boot and nand-sata-install
When armbian builds the image, it does so through chroot and debootstrap.  This results in artifacts like /etc/machine-id or /etc/ssh/ssh_host_*_key, which shouldn't be on multiple systems.  They're removed as part of the image creation.  Partition UUIDs ought to be changed too, but these are less likely to be a problem. 
The nand-sata-install attempts to create partitions to make a filesystem on some other block device, and then rsync's the running armbian system over to the new block device.  It also tries to manage the u-boot bootloader and all the possible ways that can exist.
 
(Goal) SD card partitioning
sgdisk -n 14:${biosstart}:${biosend} -c 14:bios -t 14:ef02    -n 15:${uefistart}:${uefiend} -c 15:efi  -t 15:ef00    -n  1:${rootstart}:0 -c  1:root -t 1:8305 -A 1:set:59 ${SDCARD}.raw
This produces a GPT-formatted disk with the first partition (numbered 14) with a BIOS type (recommended 1 mb, or covering u-boot and files), an ESP (recommended 550 MB, to mount on /efi/), and an auto-detected ARM64 root partition that is labeled to grow to fill the disk when booted.  Linux should be stored in a UEFI or FIT image, preferably as a UEFI boot method.  U-boot environment variables for a given board should be able to select the appropriate armbian kernel, allowing the root filesystem to have multiple kernels for different boards installed and an SD card that boots on multiple boards.

Thank you,it helps me a lot.Change to rk3399-nanopi-m4.dtb really works well.I have booted Armbian_21.02.3_Firefly-rk3399_buster_current_5.10.21.img from SD card,and it seems nothing worng but HDMI.Did anyone tested the M.2 PCIE slot?

 

So, this is how it works (and advance apologies if I misread the conversation and you're talking about something else):
 
First off, if you want to talk semantics, sha1 is technically a cryptographic hash function, not a checksum algorithm. One could even argue that it's overkill for this use-case and that an algortithm made for checksums, such as CRC, is more appropriate (less secure but by far enough to detect random write or read errors, but more performant).
 
The data on the card is read, bit-by-bit (well, on a lover level the reads are buffered for performance reasons, but it makes no practical difference otherwise). The data is then put through the sha1 hash function, which produces a 160-bit hash. A single bit being changed produces a completely different hash.
 
The odds of the hashes matching up in case every single bit doesn't is 1 in ~10^18. For practical purposes, low enough odds to be practically impossible. When we're at those odds, you might as well start thinking about the probability that the read during your verification steps gives you the expected output due to a read error exactly matching the inverse of the write error during the write, RAM errors and cosmic radiation. If you're concerned about well-funded adversaries with a huge dedicated compute cluster deliberately giving you a fake image with the same matching hash, then you might want to look at more cryptographically secure hash functions - in which case replace the `sha1sum` command above with the slower but more secure `sha512sum` (512 bits, or 1 in 10^154). The chances of a collision are small enough that on average, with all the worlds current computing power, we'd be talking millions of years. The number of atoms in the entire universe is believed to be around 10^80. Pick two random atoms in the known universe and the odds of those being the same atom is still vastly higher than SHA-2 producing the same hash after a write error.
 
TLDR; For your purposes, if the checksum validates, so does the written data (given that the checksum verification here is made between data read back from the written card vs the source file and not just between the source file and a website). Etcher did CRC32 until 2018, and is now on SHA512. Looking at the source, usbimager actually reads back the source and the target into memory and does a memcmp (straight-up byte-by-byte comparison): https://gitlab.com/bztsrc/usbimager/-/blob/master/src/main_libui.c#L147
These are all valid approaches, only caveat with the last one being that you need to be able to fit 2x the image size into RAM during the validation

I just use vanilla linux tools.
 
To write image:
$ dd if=image.img of=/dev/sda bs=4M status=progress && sync  
To verify sha1sum of the data just written to the card:
$ head -c $(stat -c '%s' image.img) /dev/sda | sha1sum $ sha1sum image.img  

I have some work-in-progress patches at https://github.com/Kwiboo/linux-rockchip/compare/next-20200501...next-20200501-drm-rockchip that should enable up to 4k30hz hdmi modes on rk3399.
 
My focus switched to rkvdec so these patches have been on hold for some time, they need to be synced with some of my older rk3328 patches for 420-mode support before they can be sent upstream. With 10-bit decoding now working I am expecting readying 4k/10-bit hdmi patches to be more fun :-)

I expected this, the reason is not correct (not correct) data for DTB. To solve the problem, I need to have the test equipment. I sent a request to Fyrefly, but for now it's the New Year in China and the response will be after it.

This has to be set
 
# max cpu frequency in MHz unit if test "${variant}" = "n2_plus"; then setenv fdtfile meson-g12b-odroid-n2-plus.dtb setenv max_freq_a73 "2400" setenv max_freq_a53 "2016" else setenv fdtfile meson-g12b-odroid-n2.dtb setenv max_freq_a73 "1800" setenv max_freq_a53 "1896" fi
... but I it looks like it doesn't work anymore. I found the problem. Will be fixed at next release.
 
Edit: Tested
 

I checked it again now. NVMe works with all ArmbianTV and Armbian images with the main core. I didn't check the legacy kernel. Here is a screenshot. Please note, the hat is very sensitive to the contact fit, if you put it on too tightly, NVMe does not work. You need to slightly raise the hat over the main board, so that there is no contact with other elements.
 
https://yadi.sk/i/cWmayDV-hH0D5Q
 
Disk /dev/mtdblock0: 16 MiB, 16777216 bytes, 32768 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mmcblk2: 29.12 GiB, 31268536320 bytes, 61071360 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x72dd71e9
Device         Boot   Start      End  Sectors  Size Id Type
/dev/mmcblk2p1        30720  1038335  1007616  492M  c W95 FAT32 (LBA)
/dev/mmcblk2p2      1040384 61071359 60030976 28.6G 83 Linux

Disk /dev/nvme0n1: 232.91 GiB, 250059350016 bytes, 488397168 sectors
Disk model: Samsung SSD 970 EVO 250GB               
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x95cda040
Device         Boot Start       End   Sectors   Size Id Type
/dev/nvme0n1p1       2048 488396799 488394752 232.9G 83 Linux

Disk /dev/sda: 14.73 GiB, 15795748864 bytes, 30851072 sectors
Disk model: Transcend 16GB  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd59e383c
Device     Boot   Start      End  Sectors  Size Id Type
/dev/sda1         32768  1081343  1048576  512M  e W95 FAT16 (LBA)
/dev/sda2       1081344 30542559 29461216   14G 83 Linux
 
 
armbianmonitor
https://yadi.sk/d/iw5-pIbZWEucmQ
 
 

But note that the PoE contacts must be isolated so that they do not touch the power connector housing. And no need to disconnect the WiFi antenna connector from the board, it does not interfere with the installation of the hat
 
I pay attention, when I checked, I did not insert the connectors to the full depth, I controlled that the board did not touch and did not bend. I was left with about 1-2 mm of contacts not inserted into the connector. The Samsung EVO 970 256Gb NVMe module is perfectly detected and works on ArmbianTV. I haven't tested NVMe on other systems.

I tested the latest versions from this link, booting from USB media works and all items work with DTB by default (except NVMe). By default, DTB does not support NVMe, if you need NVMe, you need to replace DTB with a second link.
 
https://yadi.sk/d/TnDjwCsvG9-nuA?w=1
 
DTB for NVMe
https://yadi.sk/d/UgUTCAw8mxijYw?w=1

What model do you want to run on ?
 
You can take these files (need two files\libraries) from your x86 host system, or from any of my images (ArmbianTV\Armbian for M1\P1). Or from my GIT (these libraries are included in all my builds, so that users can easily run the build themselves on any ARM device). by the way, the site has versions of images with these libraries for n2 and t4. 
 
https://users.armbian.com/balbes150/

I forgot to mention how to build ATF (Arm Trusted Firmware)
 
git clone https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git
 
make PLAT=rk3399 bl31
cp build/rk3399/release/bl31/bl31.elf ../u-boot
 
You need bl31.elf to build your u-boot!

On roc-rk3399-pc you can short pin 6 of the 8pin SPI-Flash-Chip to GND during
power on. Then rk3399 boots from SD/eMMC.

The tag was always there but at a different configuration level so no need to change it manually ;-)
 
Nonetheless the ideas to explore for this issue (which I will unfortunately have no time for in foreseeable future) would be:
Disable HDMI support for rk3399 boards in u-boot again - something like this PR (may need adjustments)
It was already disabled once because of similar issues (which were supposed to be fixed with v2020.07) but I am leaning towards disabling it for all rk3399 boards again; Blacklist panfrost module, reboot and see if it helps - there was quite some development in panfrost between 5.4 and 5.8
This however does not apply to issues found in legacy; Try both of the above at the same time if none helps on its own; If you can spare some time - try testing the above scenarios.

Boot from SPI directly to NVME works:
 
Latest U-Boot v2021.01-rc4 + this patch:
rockchip: roc-pc-rk3399: fix boot from SPI flash on spi1
https://patchwork.kernel.org/project/linux-rockchip/list/?series=403611
 
To build uboot on roc-pc:
make roc-pc-mezzanine-rk3399_defconfig
make -j6
./tools/mkimage -n rk3399 -T rkspi -d tpl/u-boot-tpl.bin:spl/u-boot-spl.bin idbloader-spi.img
 
To deploy uboot on spi:
dd if=idbloader-spi.img of=spi.img
                  (#0x60000 = 393216 = 4096 * 96)
dd if=u-boot.itb of=spi.img bs=4096 seek=96
flashcp -v spi.img /dev/mtd0
 
in U-Boot:
edit boot_targets to have nvme0 as first entry
 
On NVME drive (e.g. Samsung 970EVOPlus):
 
/boot/extlinux/extlinux.conf:
 
label mainline-kernel
    kernel /boot/Image
    initrd /boot/Initrd
#    fdt /rk3399-roc-pc.dtb
    fdt /boot/rk3399-roc-pc-mezzanine.dtb
    append earlycon=uart8250,mmio32,0xff1a0000 console=tty1 console=ttyS2,1500000 swiotlb=1 root=/dev/nvme0n1p1 rootwait rw rootfstype=ext4 init=/sbin/init
 
ls /boot:
Image (Image-5.10.0-next-20201223)
Initrd (Initrd-5.10.0-next-20201223)
rk3399-roc-pc-mezzanine.dtb (rk3399-roc-pc-mezzanine.dtb-5.10.0-next-20201223)
 
Kernel built on roc-pc:
Latest linux-next
 
make -j6
make modules_install
kernelversion=`cat ./include/config/kernel.release`
cp .config /boot/config-$kernelversion
mkinitramfs -c gzip -o ./initramfs-$kernelversion $kernelversion
mkimage -A arm64 -O linux -T ramdisk -C none -a 0 -e 0 -n initramfs -d ./initramfs-$kernelversion ./Initrd
cp arch/arm64/boot/Image /boot/Image
cp Initrd /boot
cp arch/arm64/boot/dts/rockchip/rk3399-roc-pc.dtb /boot
cp arch/arm64/boot/dts/rockchip/rk3399-roc-pc-mezzanine.dtb /boot
cp arch/arm64/boot/Image /boot/Image-$kernelversion
cp Initrd /boot/Initrd-$kernelversion
cp arch/arm64/boot/dts/rockchip/rk3399-roc-pc.dtb /boot/rk3399-roc-pc.dtb-$kernelversion
cp arch/arm64/boot/dts/rockchip/rk3399-roc-pc-mezzanine.dtb /boot/rk3399-roc-pc-mezzanine.dtb-$kernelversion
 
example .config:
config

Finally found the problem - HW "issue". My SPI flash is not soldered (no place on motherboard) - it is connected with wires which are (presumably) too long. Workaround is to reduce SPI speed in u-boot and Linux.
 
U-boot:

diff --git a/arch/arm/dts/rk3399-rockpro64.dtsi b/arch/arm/dts/rk3399-rockpro64.dtsi index 9bca258012..797dd80d38 100644 --- a/arch/arm/dts/rk3399-rockpro64.dtsi +++ b/arch/arm/dts/rk3399-rockpro64.dtsi @@ -677,7 +677,7 @@         flash@0 {                 compatible = "jedec,spi-nor";                 reg = <0>; -               spi-max-frequency = <10000000>; +               spi-max-frequency = <1000000>;         };  };
 
Linux:
Recompile your device tree with reduced SPI speed:

        flash@0 {                 compatible = "jedec,spi-nor";                 reg = <0>; -               spi-max-frequency = <10000000>; +               spi-max-frequency = <1000000>;         };

EUREKA!
Thank you balbes150 and Panzerknaker - did make my third donatian to armbian through paypal. 
It did boot successfully from the sd card after -
=> sf erase 0 0x200000                                                          
 
 
What is the best way for one to get himself familiar with memory address management.   To know offsett and size of memory addresses.   It is a bit a Hebrew to me and probably most others. 
 
This is a good source of information about practical use of u-boot commands, with examples and explanations. 
https://www.linuxsecrets.com/xilinx/U-Boot.html
 
 
 

To get rid of all the tails that are now on your SD card, I recommend doing a clean check. Download the image from the link, unzip it and burn it to the SD card. Do not change anything on the SD card, connect to the device and check the launch.
 
https://users.armbian.com/balbes150/firefly_station_p1/
 
Model Fyrefly Station P1 = ROC-RK3399-pc-plus + eMMC module.

thank you denni

I installed this u-boot ontop of my already running armbian eMMC install and boot time improved dramatically... and it will boot from sdcard if one is inserted

i just downloaded files and followed instructions
 
https://github.com/pcm720/u-boot-build-scripts/releases/tag/v2020.07-1

Thank everybody for MASSIVE support...
 
Either 5.x kernels are not working with MIPI/DSI, or the DTB structure has changed so hard so I cannot get even a dmesg on whether or not the panel is detected at all.
Managed to boot ubuntu with 4.4.179 (armbian REFUSED to boot whatever I do), decompiled the old successful DTB, modded the distro one (still no source files; there's only nonworking crap in source tree) and combined.
After 2 more days I do have the panel alive.

There is no such overview - parameters can be different from kernel to kernel. Too much work for something almost nobody will notice. One parameter that is common for all is extraargs= which is directly translated to https://www.kernel.org/doc/html/v4.14/admin-guide/kernel-parameters.html Peek into boot.cmd script to understand how those parameters are passed on.
 
Then others - like verbosity - is just extracted from there. Then you have overlays and its parameters. That is different from kernel to kernel. Its directly tied to the hardware.

As an assumption, it is possible that u-boot specifies the name of the DTB file, which is not suitable for recent kernels. You can try to force the use of a different DTB (via armbianuEnv.txt) and try to check other DTB options. Sometimes there are "miracles" and someone else's DTB can work.

Hi all.
For my next camping trip I've bought a big USB3 2.4G/5G wifi antenna with the rtl88x2bu chip to have better camping wifi reception.
It didn't work out of the box in Armbian. I've installed it on all my images manually.
Question is if this driver can be baked into Armbian? It would save me a lot of work. And maybe many others too?
 
 
For others who need this driver:
Go to : sudo armbian-config -> software -> install headers
Download the zip file : https://github.com/cilynx/rtl88x2BU_WiFi_linux_v5.3.1_27678.20180430_COEX20180427-5959
Unzip
You'll also need to install dkms :  sudo apt install dkms
Then follow the instructions in the readme file in the rtl88... folder
cd rtl88x2BU_WiFi_linux_v5.3.1_27678.20180430_COEX20180427-5959 VER=$(sed -n 's/\PACKAGE_VERSION="\(.*\)"/\1/p' dkms.conf) sudo rsync -rvhP ./ /usr/src/rtl88x2bu-${VER} sudo dkms add -m rtl88x2bu -v ${VER} sudo dkms build -m rtl88x2bu -v ${VER} sudo dkms install -m rtl88x2bu -v ${VER} sudo modprobe 88x2bu  

That's supported too. You need to change the files extlinux.conf and uenv.ini to make them point to the correct .dtb file(look into dtb folder) for the vim3 with a311d. Works great. I'm running it on an ssd. Greetings.