Menion Posted August 17, 2017 Posted August 17, 2017 Hi all Is it possible to compile a full 32bit Armbian (32bit Kernel and 32bit userland) for targets that run aarch64 SoC (H5, A64, etc....)? Bye
chrisf Posted August 22, 2017 Posted August 22, 2017 Why would you want to? So it's slower? I don't see why you couldn't just change the target, but you'd need to have the bootloader to set the right mode first wouldn't you? That's going to be SoC specific
zador.blood.stained Posted August 22, 2017 Posted August 22, 2017 1 hour ago, chrisf said: Why would you want to? So it's slower? And lower memory footprint (which may make sense for devices with only 512MiB RAM). Currently it's not possible to compile 32-bit images using the unmodified build script. Using 32-bit userland is easy - build an image for any 32-bit board and overwrite the kernel, u-boot and board support package. Compiling a 32-bit kernel is harder and I'm not sure how it should be done exactly (what ARCH, CROSS_COMPILE and defconfig combination should be used). AFAIK you also need to patch the ATF to start a 32-bit kernel instead of 64-bit, and u-boot will still be 64-bit in any case.
tkaiser Posted August 22, 2017 Posted August 22, 2017 1 hour ago, chrisf said: Why would you want to? So it's slower? Well, https://forum.armbian.com/index.php?/topic/4583-rock64/&do=findComment&comment=37015 (though using ARM crypto extensions is the better option of course). And the smaller memory footprint might have a huge performance impact once the system is running out of physical memory and has to use zram/swap instead.
Menion Posted August 22, 2017 Author Posted August 22, 2017 Hi Yes the problem is the memory footprint I have an Orange Pi PC2 running some services, in particular amule, deluge, apache, freeradius, tvheadend, minidlna (plus the default Armbian userland services) This board has 1Gb of RAM. The problem is that I often run out of physical memory especially when the apt update package list run., or when I compile onboard some cpp stuff I do have swap set on the external USB HD, but I don't know if this is a bug or what, when the swap happen, all the user space application hang, since kswapd0 (kernel thread) run 100% of all the 4 cores. The result is that I can ping the board (ping is terminated in the kernel IP stack) but all the application services, including ssh are unresponsive. The behaviour is strange, it could also be a bug, or some irq affinity problem, don't know, but this is not the point The point is that for this kind of application (which are 99.99% of the application of this board, since it is hard that someone does some heavy computing with this SoC), there is no good reason to run 64bit. Consider that the Raspberry PI consortium basically does not eve provide an official 64bit distribution for the same reason. I think that with the current uboot and kernel makefile it is enough to select ARCH=arm and point to an armhf toolchain for CROSS_COMPILE in order to compile everything in 32bit The defconf (and the config) for kernel, should be a no issue, since it should get all the proper settings from the ARCH variable, you just need to select the proper SoC Quote Using 32-bit userland is easy - build an image for any 32-bit board and overwrite the kernel, u-boot and board support package Are you sure? I think that the 32bit libc must be compiled against the 64 bit kernel (syscall parameters are different). I remember years ago, with the first ubuntu64 bit experiment, if you wanted to upgrade an existing 32 bit userland to 64bit kernel also the libc got updated
zador.blood.stained Posted August 22, 2017 Posted August 22, 2017 42 minutes ago, Menion said: I think that with the current uboot and kernel makefile it is enough to select ARCH=arm and point to an armhf toolchain for CROSS_COMPILE in order to compile everything in 32bit ARCH=arm has no effect for the u-boot AFAIK, and I highly doubt that you will be able to compile the current mainline u-boot (2017.07) using the orangepi_pc2_defconfig without changes. Compiling 32-bit kernel should be possible, but you will probably need a different config for it and hope that dependencies on ARM64 don't break anything (i.e. compiling a 32bit kernel for A64 will probably be complicated because of dependencies like this one config SUN50I_A64_CCU bool "Support for the Allwinner A64 CCU" default ARM64 && ARCH_SUNXI depends on (ARM64 && ARCH_SUNXI) || COMPILE_TEST I would strongly recommend testing a 32-bit userland with a 64-bit kernel first. Edit: Quote Are you sure? I think that the 32bit libc must be compiled against the 64 bit kernel (syscall parameters are different). I remember years ago, with the first ubuntu64 bit experiment, if you wanted to upgrade an existing 32 bit userland to 64bit kernel also the libc got updated AFAIk it just works. You can even run armhf apps on arm64 system, like by default we install installed armhf Chromium on arm64 desktop images.
Menion Posted August 22, 2017 Author Posted August 22, 2017 Quote I would strongly recommend testing a 32-bit userland with a 64-bit kernel first. Can you point me where the cross compiler for userland is selected in the build script for the specific target? So I can try to swith to the 32bit version
zador.blood.stained Posted August 22, 2017 Posted August 22, 2017 2 minutes ago, Menion said: Can you point me where the cross compiler for userland is selected in the build script for the specific target? So I can try to swith to the 32bit version As I said, grab an image from an armhf board like Orange Pi PC, replace the u-boot, kernel and other contents of the /boot directory and try to boot it. Currently there are no separate architecture/cross-compiler switches for the kernel and rootfs, everything depends on the ARCH variable. And userland is not cross-compiled since we are using upstream Debian/Ubuntu package base for like 99%, architecture affects debootstrap arguments, package list, repository list and some scripts like uInitrd generation helper.
Menion Posted August 23, 2017 Author Posted August 23, 2017 Hi I have built and image, attaching the first 2Mb of OrangePi PC2 image to the Orange Pi PC image after having removed the first 1Mb (yes the partition start address is different) and I have replaced /boot, /lib/modules, /lib/firmware and /usr/include/linux with the content of OrangePi PC 2 image. I will try it this evening. One question: the libc binaries are coming from upstream or cross compiled? Bye
zador.blood.stained Posted August 23, 2017 Posted August 23, 2017 7 minutes ago, Menion said: One question: the libc binaries are coming from upstream or cross compiled? Upstream
Menion Posted August 23, 2017 Author Posted August 23, 2017 Hi I tried it but it cannot work, I suspect the problem is the uinitrd image that should be 32bit also But I don't know if I can just take it from Orange Pi PC image U-Boot SPL 2017.01-rc1-g5df570f-dirty (Jun 13 2017 - 06:32:47) DRAM: 1024 MiB Trying to boot from MMC1NOTICE: BL3-1: Running on H5 (1718) in SRAM A2 (@0x44000) NOTICE: Configuring SPC Controller NOTICE: BL3-1: v1.0(debug):aa75c8d NOTICE: BL3-1: Built : 06:32:43, Jun 13 2017 INFO: BL3-1: Initializing runtime services INFO: BL3-1: Preparing for EL3 exit to normal world INFO: BL3-1: Next image address: 0x4a000000, SPSR: 0x3c9 U-Boot 2017.01-rc1-g5df570f-dirty (Jun 13 2017 - 06:32:54 +0200) Allwinner Technology CPU: Allwinner H5 (SUN50I) Model: OrangePi PC 2 DRAM: 1 GiB MMC: SUNXI SD/MMC: 0 *** Warning - bad CRC, using default environment Setting up a 720x576i composite-pal console (overscan 32x20) Error: no valid bmp image at 66000000 In: serial Out: vga Err: vga Net: phy interface7 eth0: ethernet@1c30000 Hit any key to stop autoboot: 0 38518 bytes read in 116 ms (324.2 KiB/s) switch to partitions #0, OK mmc0 is current device Scanning mmc 0:1... Found U-Boot script /boot/boot.scr 3362 bytes read in 132 ms (24.4 KiB/s) ## Executing script at 4fc00000 U-boot loaded from SD Boot script loaded from mmc 116 bytes read in 93 ms (1000 Bytes/s) 19253 bytes read in 533 ms (35.2 KiB/s) 4179 bytes read in 647 ms (5.9 KiB/s) Applying kernel provided DT fixup script (sun50i-h5-fixup.scr) ## Executing script at 44000000 4729916 bytes read in 587 ms (7.7 MiB/s) 11407368 bytes read in 843 ms (12.9 MiB/s) ## Loading init Ramdisk from Legacy Image at 4fe00000 ... Image Name: uInitrd Image Type: AArch64 Linux RAMDisk Image (gzip compressed) Data Size: 4729852 Bytes = 4.5 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Flattened Device Tree blob at 4fa00000 Booting using the fdt blob at 0x4fa00000 reserving fdt memory region: addr=4fa00000 size=6b000 Loading Ramdisk to 49b7d000, end 49fffbfc ... OK Loading Device Tree to 0000000049b0f000, end 0000000049b7cfff ... OK Cannot setup simplefb: node not found Starting kernel ... Loading, please wait... starting version 229 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems done. Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done.
zador.blood.stained Posted August 23, 2017 Posted August 23, 2017 4 minutes ago, Menion said: I tried it but it cannot work, I suspect the problem is the uinitrd image that should be 32bit also Initrd image must match the kernel (since it contains kernel modules) and architecture (since it contains userspace tools). The easiest way to fix this would be chrooting from another machine (with qemu static emulation) and regenerating it using update-initramfs
Menion Posted August 23, 2017 Author Posted August 23, 2017 mmmmm I am thinking about it What is this: Quote Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Begin: Running /scripts/local-block ... done. Is it some script in initrd? Because this initrd image is taken straight direct from the Orange Pi PC2 image (I have replaced all the /boot content), so actually it does match the kernel.... Unless initrd call some executable in the root file system, in this case it is definitely required to update it, due to the rootfs being 32bit
zador.blood.stained Posted August 23, 2017 Posted August 23, 2017 7 minutes ago, Menion said: Is it some script in initrd? Yes, but usually if you see messages like this it means that it can't find the rootfs device (it will print that after a while). 10 minutes ago, Menion said: (I have replaced all the /boot content), so actually it does match the kernel.... Did you also replace the /boot/armbianEnv.txt from another image? It contains the rootfs UUID and it is unique on each image. Also this means that you initrd userspace tools are 64-bit, so you will probably still need to regenerate it.
Menion Posted August 23, 2017 Author Posted August 23, 2017 Yes I have replaced also armbianEnv.txt!!! So there are two things to do, update the UUID and update initrd One extra thing: on my running aarch64 image, this is the content of armbianEnv.txt verbosity=1 console=both overlay_prefix=sun50i-h5 rootdev=UUID=00f42023-d4d4-499b-9a57-e3c8d029e8e4 rootfstype=ext4 apart from the UUID, how should the rest look for this hybrid image? In particular I wonder about "overlay_prefix" Do you know if blkid can read UUID from a img file or from a loop mounted img?
zador.blood.stained Posted August 23, 2017 Posted August 23, 2017 3 minutes ago, Menion said: Do you know if blkid can read UUID from a img file or from a loop mounted img? If you loop-mount the image and partprobe it to display partitions, you should be able to get a correct UUID from /dev/loopXp1. 5 minutes ago, Menion said: I wonder about "overlay_prefix" It should match the SoC type, so for 64-bit targets it's either sun50i-a64 or sun50i-h5. Obviously if you don't use DT overlays it doesn't matter.
Menion Posted August 24, 2017 Author Posted August 24, 2017 So, I have launched in chroot+qemu: root@menion-VirtualBox:/# update-initramfs -u -v but for some reason it pretends that the kernel is still the OrangePi PC one: Available versions: 4.11.7-sun8i update-initramfs: Generating /boot/initrd.img-4.11.7-sun8i WARNING: missing /lib/modules/4.11.7-sun8i but I have the OrangePi PC2 stuff, such: root@menion-VirtualBox:/lib/modules# ls 4.11.1-sun50iw2 root@menion-VirtualBox:/lib/modules# So, something else should be changed, but I cannot understand what/where. --edit-- Found, I have to generate a md5sum file with the kernel version as filename under /var/lib/initramfs-tools Or actually I could launch: update-initramfs -v -c -k all One thing: in the update-initramfs I get this two errors: /usr/share/initramfs-tools/hooks/klibc^i-t ignored: not alphanumeric or '_' file end Calling hook resume Unsupported ioctl: cmd=0x5331 Nothing to be worried of?
zador.blood.stained Posted August 24, 2017 Posted August 24, 2017 1 hour ago, Menion said: /usr/share/initramfs-tools/hooks/klibc^i-t ignored: not alphanumeric or '_' file According to apt-file it should be either one of those (or both) ➜ ~ % apt-file search '/usr/share/initramfs-tools/hooks/klibc' initramfs-tools-core: /usr/share/initramfs-tools/hooks/klibc klibc-utils: /usr/share/initramfs-tools/hooks/klibc-utils ➜ ~ % 1 hour ago, Menion said: Calling hook resume Unsupported ioctl: cmd=0x5331 Can be ignored in chroot
Menion Posted August 24, 2017 Author Posted August 24, 2017 Quote According to apt-file it should be either one of those (or both) ➜ ~ % apt-file search '/usr/share/initramfs-tools/hooks/klibc' initramfs-tools-core: /usr/share/initramfs-tools/hooks/klibc klibc-utils: /usr/share/initramfs-tools/hooks/klibc-utils ➜ ~ % Then it is possible it will cause problem, right?
zador.blood.stained Posted August 24, 2017 Posted August 24, 2017 9 minutes ago, Menion said: Then it is possible it will cause problem, right? Don't know. As you see, this file was ignored because of its name, but it's not clear whether the original file is there and why does this file (with the strange name) even exist. If it causes any problems you can try verifying initramfs-tools-core package integrity with dpkg --verify
Menion Posted August 24, 2017 Author Posted August 24, 2017 Hi Actually both exist: -rwxr-xr-x 1 root root 743 dic 9 2016 klibc^i-t -rwxr-xr-x 1 root root 678 gen 26 2016 klibc-utils -rwxr-xr-x 1 root root 743 dic 9 2016 klibc^i-t -rwxr-xr-x 1 root root 678 gen 26 2016 klibc-utils and both does the same, actually some non sense to me root@menion-VirtualBox:/usr/share/initramfs-tools/hooks# ./klibc^i-t /bin/cp: '/lib/klibc-wBxlC8vEZykm9g94Kf1Dr2tj4ts.so' and '/lib/klibc-wBxlC8vEZykm9g94Kf1Dr2tj4ts.so' are the same file root@menion-VirtualBox:/usr/share/initramfs-tools/hooks# ./klibc-utils /bin/cp: '/lib/klibc-wBxlC8vEZykm9g94Kf1Dr2tj4ts.so' and '/lib/klibc-wBxlC8vEZykm9g94Kf1Dr2tj4ts.so' are the same file
Menion Posted August 24, 2017 Author Posted August 24, 2017 No, it does not work U-Boot 2017.01-rc1-g5df570f-dirty (Jun 13 2017 - 06:32:54 +0200) Allwinner Technology CPU: Allwinner H5 (SUN50I) Model: OrangePi PC 2 DRAM: 1 GiB MMC: SUNXI SD/MMC: 0 *** Warning - bad CRC, using default environment Setting up a 720x576i composite-pal console (overscan 32x20) Error: no valid bmp image at 66000000 In: serial Out: vga Err: vga Net: phy interface7 eth0: ethernet@1c30000 Hit any key to stop autoboot: 0 38518 bytes read in 115 ms (326.2 KiB/s) switch to partitions #0, OK mmc0 is current device Scanning mmc 0:1... Found U-Boot script /boot/boot.scr 3362 bytes read in 129 ms (25.4 KiB/s) ## Executing script at 4fc00000 U-boot loaded from SD Boot script loaded from mmc 116 bytes read in 90 ms (1000 Bytes/s) 19253 bytes read in 527 ms (35.2 KiB/s) 4179 bytes read in 633 ms (5.9 KiB/s) Applying kernel provided DT fixup script (sun50i-h5-fixup.scr) ## Executing script at 44000000 4779271 bytes read in 620 ms (7.4 MiB/s) 11407368 bytes read in 919 ms (11.8 MiB/s) ## Loading init Ramdisk from Legacy Image at 4fe00000 ... Image Name: uInitrd Image Type: ARM Linux RAMDisk Image (gzip compressed) Data Size: 4779207 Bytes = 4.6 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK No Linux AArch64 Ramdisk Image Ramdisk image is corrupt or invalid SCRIPT FAILED: continuing... starting USB... USB0: USB EHCI 1.00 USB1: USB OHCI 1.0 So, U-Boot looks for an aarch64 initramfs while the generated one in arm(hf)... I hadn't the time to check the uboot environment, but I am not sure how to move on, in the end now initramfs is an hybrid
zador.blood.stained Posted August 24, 2017 Posted August 24, 2017 18 minutes ago, Menion said: I hadn't the time to check the uboot environment, but I am not sure how to move on, in the end now initramfs is an hybrid You need to change INITRD_ARCH in /etc/armbian-release (and use update-initramfs to regenerate the image) It should not matter what is inside the initrd if it is packaged with the right arguments.
Menion Posted August 24, 2017 Author Posted August 24, 2017 Ok, then I will change only that row so it looks like: INITRD_ARCH=arm64
Menion Posted August 25, 2017 Author Posted August 25, 2017 Tried, still the initramfs is detected as no aarch64. One thing worth to mention is that this hybrid image never ran on any target, it is just a downloaded image with uboot and kernel replaced, so I wonder if update-initramfs needs these info somewhere else, generated by the first run on target
zador.blood.stained Posted August 25, 2017 Posted August 25, 2017 8 minutes ago, Menion said: One thing worth to mention is that this hybrid image never ran on any target, it is just a downloaded image with uboot and kernel replaced, so I wonder if update-initramfs needs these info somewhere else, generated by the first run on target U-boot only checks the architecture of uInitrd - standard initrd image packaged by mkimage. It is done by this script and INITRD_ARCH is passed from /etc/armbian-release You can use "mkimage -l /boot/uInitrd" to check the image info
Menion Posted August 25, 2017 Author Posted August 25, 2017 Then there is something strange: root@menion-VirtualBox:/etc# cat armbian-release # PLEASE DO NOT EDIT THIS FILE BOARD=orangepipc BOARD_NAME="Orange Pi PC" VERSION=5.32.170629 LINUXFAMILY=sun8i BRANCH=dev ARCH=arm IMAGE_TYPE=nightly BOARD_TYPE=conf INITRD_ARCH=arm64 notice that the original armbian-release didn't have any INITRD_ARCH row root@menion-VirtualBox:/etc# update-initramfs -c -k all -v Available versions: 4.11.1-sun50iw2 Execute: /usr/sbin/update-initramfs -c -k "4.11.1-sun50iw2" -b /boot -v -t update-initramfs: Generating /boot/initrd.img-4.11.1-sun50iw2 Copying module directory kernel/drivers/hid ............................................. ............................................. Calling hook fixrtc Adding binary /bin/date Adding binary /sbin/hwclock Adding binary /sbin/dumpe2fs Building cpio /boot/initrd.img-4.11.1-sun50iw2.new initramfs update-initramfs: Converting to u-boot format but root@menion-VirtualBox:/etc# mkimage -l /boot/uInitrd Image Name: uInitrd Created: Fri Aug 25 17:56:33 2017 Image Type: ARM Linux RAMDisk Image (gzip compressed) Data Size: 4778579 Bytes = 4666.58 kB = 4.56 MB Load Address: 00000000 Entry Point: 00000000 So it is still 32 bit arm....
Menion Posted August 25, 2017 Author Posted August 25, 2017 Found the problem! This is the 99-uboot script in the Orange Pi PC image #!/bin/sh echo "update-initramfs: Converting to u-boot format" >&2 tempname="/boot/uInitrd-$1" mkimage -A arm -O linux -T ramdisk -C gzip -n uInitrd -d $2 $tempname > /dev/null ln -sf $(basename $tempname) /boot/uInitrd > /dev/null 2>&1 || mv $tempname /boot/uInitrd exit 0 modified to arm64 and going to test
zador.blood.stained Posted August 25, 2017 Posted August 25, 2017 12 minutes ago, Menion said: notice that the original armbian-release didn't have any INITRD_ARCH row Please check the code of /etc/initramfs/post-update.d/99-uboot On older images INITRD_ARCH was not sourced from /etc/armbian-release but was hardcoded directly in the script Edit: too late
Recommended Posts