Unmodified Armbian build script reproducably failing for nanopifire3

Thinky101 · September 26, 2022

I am trying to compile the full OS image (incl. kernel and u-boot) for nanopifire3 target but fail every time. I am following the provided build instructions.
I have tried compiling on multiple machines, but the one I am using has 8 cores and 12gb RAM. I have tried in an Ubuntu 22.04 LXC container, Ubuntu 22.04 in a VM, and the the docker container provided by Armbian. My kernel has the binfmt_misc module loaded (verified with lsmod), no spaces are present in build path, and I have tried both root user and sudo as a regular user. I have also checked the similar issues on the armbian forum

Please forgive me if anything is incorrectly formatted or so, I have tried to provide as much detail as I can. If I didn't provide something, please tell me and I will do my best to resolve the issue. Additionally, I know this board is CSC status and not supported. I am not asking for support with anything in the OS, I am simply trying to build a new kernel for the device.

Here are the exact steps required to produce the issue.

1. Install Ubuntu 22.04 on bare metal/VM and install git from apt

2. Run

git clone --depth 1 https://github.com/armbian/build; cd build; ./compile.sh

3. Select "Full OS build for flashing"

4. Select "Show a kernel configuration menu before compiling" (but I never got this far so it probably doesn't matter)

5. Show CSC and pick nanopifire3 from list

6. Pick bullseye (I believe all of them error the same but my target is bullseye)

7. Pick image with server interface and standard/minimal image (both error same)

8. The error shown below will be produced when compiling u-boot.

Error log from console

[ o.k. ] Looking for user patches in [ userpatches/u-boot/u-boot-s5p6818 ]
[ o.k. ] * [l][c] script-loading.patch
[ o.k. ] * [l][c] update-nanopi-fire3-support.patch
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  SHIPPED scripts/kconfig/zconf.tab.c
  SHIPPED scripts/kconfig/zconf.lex.c
  SHIPPED scripts/kconfig/zconf.hash.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf
#
# configuration written to .config
#
scripts/kconfig/conf  --silentoldconfig Kconfig
#
# configuration written to .config
#
  CHK     include/config.h
  UPD     include/config.h
  GEN     include/autoconf.mk
  GEN     include/autoconf.mk.dep
  CHK     include/config/uboot.release
  CHK     include/generated/timestamp_autogenerated.h
  UPD     include/generated/timestamp_autogenerated.h
  UPD     include/config/uboot.release
  CFG     u-boot.cfg
  CHK     include/generated/version_autogenerated.h
  UPD     include/generated/version_autogenerated.h
  CC      lib/asm-offsets.s
  CC      arch/arm/lib/asm-offsets.s
  CHK     include/generated/generic-asm-offsets.h
  CHK     include/generated/asm-offsets.h
  UPD     include/generated/generic-asm-offsets.h
  UPD     include/generated/asm-offsets.h
  HOSTCC  tools/bmp_logo
  HOSTCC  tools/img2srec
  HOSTCC  tools/mkenvimage.o
  HOSTCC  tools/os_support.o
  WRAP    tools/lib/crc32.c
  HOSTCC  tools/aisimage.o
  HOSTCC  tools/atmelimage.o
  HOSTCC  tools/fip_create/fip_create.o
  HOSTCC  tools/nexell/SECURE_BINGEN.o
  HOSTCC  tools/default_image.o
  WRAP    tools/common/bootm.c
  WRAP    tools/lib/fdtdec_common.c
  WRAP    tools/lib/fdtdec.c
  HOSTCC  tools/nexell/BOOT_BINGEN.o
  HOSTCC  tools/fit_common.o
  HOSTCC  tools/fit_image.o
  HOSTLD  tools/nexell/SECURE_BINGEN
  HOSTLD  tools/nexell/BOOT_BINGEN
  HOSTLD  tools/fip_create/fip_create
  HOSTCC  tools/gpimage.o
  HOSTCC  tools/gpimage-common.o
  WRAP    tools/common/image-fit.c
  HOSTCC  tools/image-host.o
  WRAP    tools/common/image.c
  LDS     u-boot.lds
  HOSTCC  tools/imagetool.o
[ error ] ERROR in function compile_uboot [ main.sh:595 -> main.sh:481 -> compilation.sh:241 -> general.sh:0 ]
[ error ] U-boot compilation failed
[ o.k. ] Process terminated

Full output log (/build/output/debug/output.log) here https://files.electrohaxz.host/file/d240f5c497e8db0f53dbf940329fb4f6/output.log

Edited September 26, 2022 by Thinky101

Werner · September 26, 2022

2 hours ago, Thinky101 said:

I have tried compiling on multiple machines, but the one I am using has 8 cores and 12gb RAM. I have tried in an Ubuntu 22.04 LXC container, Ubuntu 22.04 in a VM, and the the docker container provided by Armbian. My kernel has the binfmt_misc module loaded (verified with lsmod), no spaces are present in build path, and I have tried both root user and sudo as a regular user. I have also checked the similar issues on the armbian forum

First of all thank you for taking the time to check for common pitfalls beforehand. There aren't many people doing that unfortunately before complaining.

LXC or similar containerized/PVM solution besides docker will not work. Proper HVM like KVM, VirtualBox and so on are perfectly fine.

Virtualization type: microsoft

That was hypervisor I assume?

s5p6818

Oh boy, that thing did not get attention for quite a while I think. Well it's csc and if nobody from the community steps up to take care it will compost over time 😄

Anyway there are further logs in debug since raw output log does not go into detail. Check compilation log and other for deeper insights.

Thinky101 · September 27, 2022

Thanks for your feedback and advice. Appologies for late response, I can only post once a day as a new user. I am not using Microsoft virtualization, despite what Linux says. This issue has nothing to do with virtualization though, it's easily reproducable on bare metal with Ubuntu 22.04/Debian 11, or in docker, hyper-v, Xen, KVM, even LXC as this u-boot compilation error is before anything that LXC might not support (though this doesn't mean LXC is supported, it might fail later on for some other reason!).

Upon further investigation, I did find that the compilation log file had some useful information that led to a solution. The error listed there was a ton of syntax errors and such. I've included the full log file below for anyone with the same issue who might want to actually fix the issue rather than work around it. The error I found led me to this page https://github.com/hardkernel/u-boot/issues/73. While there was no solution there, it did give me the idea to try an older OS since perhaps Ubuntu 22.04 and Debian 11 packages are too new for this old code. Turns out I was right.

Using Peppermint 10 (Based on Ubuntu 18.04, I'm sure it'd work fine on Ubuntu 18.04 too) this error is gone. In order to get it to run on Ubuntu 18.04, you have to edit line 1447 of ./build/lib/general.sh to include bionic as a supported version. You also need to install libncurses-5.dev if you don't have it already or else it errors later on due to missing ncurses. I know this is obviously unsupported but if you need the u-boot compile to actually work and this isn't fixed by the time anyone else is reading this in the future, that's how I got it to work. I didn't try Ubuntu 20.04 but it could have a chance of working too since it has older packages than 22.04 but as of now, is listed as a supported version in the general.sh file.

---------------------------- Start u-boot error compilation.log -----------------------------------
== u-boot make ==

In file included from tools/../include/libfdt.h:54,
                 from tools/fdt_host.h:11,
                 from tools/imagetool.h:24,
                 from tools/aisimage.c:8:
/usr/include/libfdt_env.h:27:30: error: conflicting types for ‘fdt64_t’; have ‘uint64_t’ {aka ‘long unsigned int’}
   27 | typedef uint64_t FDT_BITWISE fdt64_t;
      |                              ^~~~~~~
In file included from <command-line>:
././include/libfdt_env.h:19:16: note: previous declaration of ‘fdt64_t’ with type ‘fdt64_t’ {aka ‘long long unsigned int’}
   19 | typedef __be64 fdt64_t;
      |                ^~~~~~~
In file included from ././include/libfdt_env.h:12,
                 from <command-line>:
/usr/include/libfdt_env.h:47:24: error: expected ‘)’ before ‘x’
   47 | static inline uint32_t fdt32_to_cpu(fdt32_t x)
      |                        ^~~~~~~~~~~~
././include/compiler.h:66:16: error: expected ‘)’ before ‘&’ token
   66 |         ((((x) & 0xff000000) >> 24) | \
      |                ^
././include/compiler.h:66:30: error: expected ‘)’ before ‘>>’ token
   66 |         ((((x) & 0xff000000) >> 24) | \
      |                              ^~
././include/compiler.h:66:37: error: expected ‘)’ before ‘|’ token
   66 |         ((((x) & 0xff000000) >> 24) | \
----------------------------- End u-boot error compilation.log ------------------------------------
Full log (LOTS more errors I left out for brevity): https://files.electrohaxz.host/file/e3872aa444605cab978c45b683c00b44/compilation.log

Incase something is missing, here's an archive of the entire debug folder contents for the u-boot compilation error (all logs) https://files.electrohaxz.host/file/f1df43da7dbaffb65bd634e7ba1e2c07/uboot-err-logs.7z

Now my issue is unrelated to u-boot. The whole reason for recompiling was to enable kvm support (and preferably support lxc too but didn't attempt that yet), however after enabling KVM in the compile options, the compile fails with the below error. Is this something you or someone else would be able to help with? I'm not too sure since this is a CSC board and I don't want to ask too much. I figured there's a reason KVM isn't enabled in the kernel by default on this platform, despite it being enabled on others like rock64. Perhaps this is something unsolvable for one reason or another, I don't know. Any help regarding this would be greatly apprecaited as I've got about $1000 worth of these boards in a cluster and without lxc or kvm, I'm not sure what I can use them for unfortunately. If this isn't something I can get help on then I understand too. I would also be willing to donate a bit to whomever could help get kvm and lxc working in this kernel if I can donate with crypto. If anyone willing to help needs a development system to build on if you don't have your own, I can also provide remote access to a server with any OS of your choice and plenty of compute and fast networking for that for as long as needed (for free, of course!). I can also provide remote access to as many nano pi fire 3's as you need if that's needed for any reason.

---------------------------- Start KVM error Output.log ---------------------------------------
arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.c:574:9: error: implicit declaration of function ‘arch_timer_get_kvm_info’; did you mean ‘arch_timer_get_cntkctl’? [-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
make[1]: *** [arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.o] Error 1
make: *** [arch/arm64/kvm] Error 2
[ error ] ERROR in function compile_kernel [ main.sh:595 -> main.sh:489 -> compilation.sh:509 -> general.sh:0 ]
[ error ] Kernel was not built [ @host ]
[ o.k. ] Process terminated
----------------------------- End KVM error Output.log ----------------------------------------
Full log: https://files.electrohaxz.host/file/338487ce6b7e665d521325acadc0791a/output.log

---------------------------- Start KVM error compilation.log -----------------------------------
== kernel ==

arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.c: In function ‘kvm_timer_hyp_init’:
arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.c:574:9: error: implicit declaration of function ‘arch_timer_get$
info = arch_timer_get_kvm_info();
         ^~~~~~~~~~~~~~~~~~~~~~~
         arch_timer_get_cntkctl
arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.c:574:7: warning: assignment to ‘struct arch_timer_kvm_info *’ f$
info = arch_timer_get_kvm_info();
       ^
cc1: some warnings being treated as errors
make[1]: *** [arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.o] Error 1
make: *** [arch/arm64/kvm] Error 2
make: *** Waiting for unfinished jobs....
fs/proc/task_mmu.c: In function ‘show_smap.isra.13’:
fs/proc/task_mmu.c:764:7: warning: ‘last_vma’ may be used uninitialized in this function [-Wmaybe-uninitialized]
bool last_vma;
       ^~~~~~~~
----------------------------- End KVM error compilation.log ------------------------------------
Full log: https://files.electrohaxz.host/file/e555ae72e83a95e2909bd5bc448909e3/compilation.log