Pali

The whole thing is broken. For 1.2 GHz variant is officially documented CPU errata related to CPU voltage together with workaround. But source of the issue is not documented and is Marvell secret. That workaround is implemented in kernel for a long time (including 5.15). But for some unknown reasons since some kernel version 1.2 GHz variant of CPU crashes even with workaround. And because only Marvell knows source of one documented crahes, we can just guess... if this that documented issue, or some other issue... Only HW engineer of that A3720 CPU can answer to those questions. Anyway, if somebody is interested in, wants to debug this issue and is skilled in compiling, patching and debugging kernel, I can forward some new test info which I have (write me an email).

Here is the issue. 1.2 GHz variant of A3720 CPU is not supported by Linux. Trying to do frequency scaling on that variant cause CPU crashes. Hence it was disabled. Yes, on many places and there is just a silence: https://lore.kernel.org/linux-pm/20210809040224.j2rvopmmqda3utc5@vireshk-i7/ https://github.com/MarvellEmbeddedProcessors/linux-marvell/issues/20 Contacting your support departement or directly Marvell FAE. Opening RMA or returning broken product back to seller.

You should add that chunk of dts code to the original source device tree file and not into the decompiled device tree binary (which you attached into the first post).

Use upstream U-Boot and not some OpenWRT fork which was broken. And after upgrading reset env variables to default and beware to not erase mac address of network controller (stored in env too). Armbian boot scripts explicitly erased mac address which lead to your U-Boot errors. Recent upstream U-Boot from this year used to work without any issues on Espressobin v5.

There are Espressobin variants with just 512MB of memory. So if your image needs to be compatible with more variants, ensure that you place all parts in first 512 MB of RAM. I'm not sure where is running U-Boot after relocation and where it is malloc area. So choose address which does not conflict with it. You can look at RAM layout used on A3720 CPU: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/marvell/armada/a3k/common/include/platform_def.h?h=v2.6

IIRC those were chosen as the first available space for some common image sizes. I do not have full knowledge of FIT images, but above addresses seems to be wrong. They are above 1GB of space and therefore it is required to have at least 2GB of RAM. If you have less then this is expected - CRASH. Could you chose lower address for loading of those FIT sub-images?

USB 3.0 is 5Gbps, PCIe 2.1 is 5Gbps and SATA is 6Gbps. For loading you should use some unused space in RAM. Default value of of those variables should contain appropriate values. And they may change between releases, that is why addresses are in variables.

I have tested it on two different A3720 devices and on both it worked. And yes, it is clean U-Boot 2022.04 without any additional patches. So really do not know what can trigger this issue. There can be memory corruption or any other issue which could depend on linker layout, memory usage, etc... But as I cannot reproduce it, it is really hard for me to debug.

I tried to debug this issue, but I'm not able to reproduce it. I always have correct scriptaddr env name. See: TIM-1.0 mv_ddr-devel-g7753d4b3e420 DDR3 16b 1GB 2CS WTMI-devel-18.12.1-1d977157e371 WTMI: system early-init CPU VDD voltage default value: 1.155V Setting clocks: CPU 1000 MHz, DDR 800 MHz CZ.NIC's Armada 3720 Secure Firmware v2021.09.07-48-geb1ac8f98b81 (Apr 14 2022 10:11:16) Running on ESPRESSObin NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.6(release):v2.6-573-g1000e7dbd297 NOTICE: BL1: Built : 12:47:53, Mar 23 2022 NOTICE: BL1: Booting BL2 NOTICE: BL2: v2.6(release):v2.6-573-g1000e7dbd297 NOTICE: BL2: Built : 12:47:53, Mar 23 2022 NOTICE: BL1: Booting BL31 NOTICE: BL31: v2.6(release):v2.6-573-g1000e7dbd297 NOTICE: BL31: Built : 14:04:46, Mar 23 2022 U-Boot 2022.04 (Apr 14 2022 - 10:33:14 +0200) DRAM: 1 GiB Core: 38 devices, 20 uclasses, devicetree: separate WDT: Not starting watchdog@8300 Comphy chip #0: Comphy-0: USB3_HOST0 5 Gbps Comphy-1: PEX0 5 Gbps Comphy-2: SATA0 6 Gbps SATA link 0 timeout. AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode flags: ncq led only pmp fbss pio slum part sxs PCIe: Link up MMC: sdhci@d0000: 0, sdhci@d8000: 1 Loading Environment from SPIFlash... SF: Detected w25q32dw with page size 256 Bytes, erase size 4 KiB, total 4 MiB OK Model: Globalscale Marvell ESPRESSOBin Board Net: eth0: ethernet@30000 [PRIME] => env default -a ## Resetting to default environment => printenv boot_a_script=load ${devtype} ${devnum}:${distro_bootpart} ${scriptaddr} ${prefix}${script}; source ${scriptaddr} boot_efi_binary=load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootaa64.efi; if fdt addr ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r};else bootefi ${kernel_addr_r} ${fdtcontroladdr};fi boot_efi_bootmgr=if fdt addr ${fdt_addr_r}; then bootefi bootmgr ${fdt_addr_r};else bootefi bootmgr;fi boot_extlinux=sysboot ${devtype} ${devnum}:${distro_bootpart} any ${scriptaddr} ${prefix}${boot_syslinux_conf} boot_net_usb_start=usb start boot_pci_enum=pci enum boot_prefixes=/ /boot/ boot_script_dhcp=boot.scr.uimg boot_scripts=boot.scr.uimg boot.scr boot_syslinux_conf=extlinux/extlinux.conf boot_targets=mmc1 mmc0 usb0 scsi0 pxe dhcp bootcmd=run distro_bootcmd bootcmd_dhcp=devtype=dhcp; run boot_net_usb_start; run boot_pci_enum; if dhcp ${scriptaddr} ${boot_script_dhcp}; then source ${scriptaddr}; fi;setenv efi_fdtfile ${fdtfile}; setenv efi_old_vci ${bootp_vci};setenv efi_old_arch ${bootp_arch};setenv bootp_vci PXEClient:Arch:00011:UNDI:003000;setenv bootp_arch 0xb;if dhcp ${kernel_addr_r}; then tftpboot ${fdt_addr_r} dtb/${efi_fdtfile};if fdt addr ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r}; else bootefi ${kernel_addr_r} ${fdtcontroladdr};fi;fi;setenv bootp_vci ${efi_old_vci};setenv bootp_arch ${efi_old_arch};setenv efi_fdtfile;setenv efi_old_arch;setenv efi_old_vci; bootcmd_mmc0=devnum=0; run mmc_boot bootcmd_mmc1=devnum=1; run mmc_boot bootcmd_pxe=run boot_net_usb_start; run boot_pci_enum; dhcp; if pxe get; then pxe boot; fi bootcmd_scsi0=devnum=0; run scsi_boot bootcmd_usb0=devnum=0; run usb_boot distro_bootcmd=scsi_need_init=; setenv nvme_need_init; for target in ${boot_targets}; do run bootcmd_${target}; done efi_dtb_prefixes=/ /dtb/ /dtb/current/ eth1addr=XX:XX:XX:XX:XX:XX eth2addr=XX:XX:XX:XX:XX:XX eth3addr=XX:XX:XX:XX:XX:XX ethaddr=XX:XX:XX:XX:XX:XX fdt_addr=0x6f00000 fdt_addr_r=0x6f00000 fdtfile=marvell/armada-3720-espressobin.dtb kernel_addr=0x7000000 kernel_addr_r=0x7000000 load_efi_dtb=load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} ${prefix}${efi_fdtfile} mmc_boot=if mmc dev ${devnum}; then devtype=mmc; run scan_dev_for_boot_part; fi nvme_boot=run boot_pci_enum; run nvme_init; if nvme dev ${devnum}; then devtype=nvme; run scan_dev_for_boot_part; fi nvme_init=if ${nvme_need_init}; then setenv nvme_need_init false; nvme scan; fi pxefile_addr_r=0x6e00000 ramdisk_addr_r=0xa000000 sata_boot=if sata dev ${devnum}; then devtype=sata; run scan_dev_for_boot_part; fi scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${distro_bootpart}...; for prefix in ${boot_prefixes}; do run scan_dev_for_extlinux; run scan_dev_for_scripts; done;run scan_dev_for_efi; scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists devplist || setenv devplist 1; for distro_bootpart in ${devplist}; do if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then run scan_dev_for_boot; fi; done; setenv devplist scan_dev_for_efi=setenv efi_fdtfile ${fdtfile}; for prefix in ${efi_dtb_prefixes}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${efi_fdtfile}; then run load_efi_dtb; fi;done;run boot_efi_bootmgr;if test -e ${devtype} ${devnum}:${distro_bootpart} efi/boot/bootaa64.efi; then echo Found EFI removable media binary efi/boot/bootaa64.efi; run boot_efi_binary; echo EFI LOAD FAILED: continuing...; fi; setenv efi_fdtfile scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${boot_syslinux_conf}; then echo Found ${prefix}${boot_syslinux_conf}; run boot_extlinux; echo SCRIPT FAILED: continuing...; fi scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run boot_a_script; echo SCRIPT FAILED: continuing...; fi; done scriptaddr=0x6d00000 scsi_boot=run boot_pci_enum; run scsi_init; if scsi dev ${devnum}; then devtype=scsi; run scan_dev_for_boot_part; fi scsi_init=if ${scsi_need_init}; then scsi_need_init=false; scsi scan; fi usb_boot=usb start; if usb dev ${devnum}; then devtype=usb; run scan_dev_for_boot_part; fi Environment size: 4227/65532 bytes => I just masked MAC addresses from the output.

About criptaddr - this looks like a bug in U-Boot and needs to be fixed. It really should be scriptaddr. I will try to look at it where is the issue. Last time I tested it I did not see such issue.

U-Boot is part of the firmware package and cannot be flashed separately on Espressobin. So you always have to flash full firmware, not just U-Boot. There are 3 variants of Armada 3720 CPU used in Espressobin: 800 MHz, 1 GHz and 1.2 GHz. And more variants of soldered DDR chips on Espressobin. You have to choose firmware which matches your CPU and DDR (3 or 4 and correct size). If you do not know what you have, you can look at chips markings to recognize it (so in this case you need to look what is under headsink). You can also look at frequences, DDR sizes and other debug information on the serial console to deduce it (but this works only in case you already have running correct firmware; factory firmware should be correct).

Before upgrading U-Boot, in case your MAC addresses were not wiped out, you should print them in U-Boot console and remember/backup them. After upgrading U-Boot to the new version, you should reset enviroment to default. Latest U-Boot preserve MAC addresses variables. But if your U-Boot version uses different storage offset for env variables then all env variables (including MAC addresses) would be lost. So that is why backup is required. Some Espressobin boards have MAC addresses printed on the sticker/label, which is suitable backup. To access env variables from Linux system, you need to set /etc/fw_env.config correctly. With new U-Boot version, partition number is available in /proc/mtd at u-boot-env line and offset is always zero. Old U-Boot versions have it different.

In new U-Boot you should not change addr and boot env variables.

Ad eMMC detection in U-Boot: It is enabled by default and disabled+removed at runtime if no eMMC module is detected. See code: https://source.denx.de/u-boot/u-boot/-/blob/v2022.04/board/Marvell/mvebu_armada-37xx/board.c#L125-134 https://source.denx.de/u-boot/u-boot/-/blob/v2022.04/arch/arm/dts/armada-3720-espressobin-u-boot.dtsi#L32-38

How could the mainline sources be patched with this hotfix? It is already in the u-boot master branch.

This is a bug in U-Boot for which I sent a hotfix recently: http://patchwork.ozlabs.org/project/uboot/patch/20220323161942.7308-1-pali@kernel.org/ Hotfix should appear in master branch ASAP and will be part of U-Boot 2022.04 release. Ideally please test this hotfix, to ensure that it will be fixed in 2022.04. TF-A and U-Boot is 64-bit ARM code and therefore must be built with aarch64 cross compiler. WTMI firmware (including the new one in mox-boot-builder repo) is 32-bit ARM code and must be built with armv7 cross compiler for Cortex M3. It does not matter if you use softfp or hardfp or linux or gnueabi cross toolchain as firmware is standalone freestanding code and Makefiles override all CPU / arch gcc flags to use correct one. together with ignoring / skipping system pre-compiled CRT code (which is gnueabi/softfp/hardfp specific). So it always should generate correct code or throw compile errors (when compiler does not support required code generation). See TF-A documentation. You can use either precompiled crypto++ libraries (e.g. provided by OS by libcrypto++-dev) or you can compile crypto++ libraries from sources as part of build process. crypto++ libraries are just dependency for host tool which generate flashable A3720 firmware image and it is your choice which version of crypto++ libraries do you want to use. Somebody prefer to use system host pre-compiled libraries, somebody prefer to compile everything from sources.

U-Boot automatically fills variable ${fdtfile} to marvell/armada-3720-espressobin-v7.dtb or marvell/armada-3720-espressobin.dtb (or with -emmc.dtb suffix) at the runtime based on detected espressobin variant. Variable ${fdtfile} is standard distroboot variable so you can use it in your boot script to make booting universal.

Espressobin v5 has WAN port on different position as Espressobin v7. This is the fact (you can look into the official Espressobin manuals) and espressobin v7 DTS file has it reflected compared with espressobin v5 DTS file.

Just quick search in my emails found this link: https://github.com/armbian/build/issues/2861 And as @y52 pointed, file /boot/armbianEnv.txt has problems too.

Interesting... I do not know. Just in case did not lan0 and lan1 got configured some ip address? Or Is not dhcpd also listening (for some reason) also on lan0 and lan1? Or is not there some dangling lease (stored in disk lease file) which could conflict? I do not know how exaclty is dhcpd detecting these duplicates, but it smells like a loop in switching/networking.

If you have configured bridge interface br0 based on lan0 and lan1 then you should not touch lan0 and lan1 and should not run on these interfaces any SW. Last time I did it, it was closed. Why should I do it again? Sorry, thanks, I will not spend my time with it again with something which would be closed/dropped.

Yea... it is known that Armbian wants users to not use correct MAC addresses. I have reported it many times and the last thing which was done, was removal of permanent MAC addresses from the board, which is totally insane...

I hope that after calling setenv you can also called saveenv so it was deleted also from env storage... ... because this is not correct. You should have that your one MAC address on all lan0, lan1 and wan interfaces. Could you check if boot script does not overwrite these mac addresses? Also could you check if you system does not have in some (probably autogenerated?) file stored those different MAC addresses?

Now I spotted that you forgot to remove rest of mtdparts ,-(reserved). Via journalctl. Log should be in some file in /var/log. Some of them are text, but soem (journald) are binary. Espressobin does not have power switch like ATX PC boards, so software cannot power off Espressobin HW. For this reason that function a3700_system_off (in TF-A) is not implemented. Kernel after stopping all servides, sent instruction to firmware (TF-A) to power down board and kernel itself stopped exection. Firmware (TF-A) received command and because it cannot do anything it just printed that error message and went to infinite loop (hang). This state is like in old compuers... when they printed message you can now halt... It is possible to implement a3700_system_off function in TF-A. But question is, what it should do?

So if they are in boot script then remove them from boot script. Do not set loglevel= at all. IIRC Setting loglevel to zero disables it completely. Your removal of mtdparts= is OK, it should be removed completely.