Jump to content

Compile 32bit Armbian for aarch64 capable SoC

Menion

By Menion
in Advanced users - Development

 Share

Recommended Posts

chrisf

chrisf

Posted
Posted

Why would you want to? So it's slower?

 

I don't see why you couldn't just change the target, but you'd need to have the bootloader to set the right mode first wouldn't you? That's going to be SoC specific

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
1 hour ago, chrisf said:

Why would you want to? So it's slower?

And lower memory footprint (which may make sense for devices with only 512MiB RAM).

 

Currently it's not possible to compile 32-bit images using the unmodified build script.

Using 32-bit userland is easy - build an image for any 32-bit board and overwrite the kernel, u-boot and board support package. Compiling a 32-bit kernel is harder and I'm not sure how it should be done exactly (what ARCH, CROSS_COMPILE and defconfig combination should be used).

AFAIK you also need to patch the ATF to start a 32-bit kernel instead of 64-bit, and u-boot will still be 64-bit in any case.

Link to comment
Share on other sites
tkaiser

tkaiser

Posted
Posted
1 hour ago, chrisf said:

Why would you want to? So it's slower?

 

Well, https://forum.armbian.com/index.php?/topic/4583-rock64/&do=findComment&comment=37015 (though using ARM crypto extensions is the better option of course).

 

And the smaller memory footprint might have a huge performance impact once the system is running out of physical memory and has to use zram/swap instead.

 

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

Hi

Yes the problem is the memory footprint

I have an Orange Pi PC2 running some services, in particular amule, deluge, apache, freeradius, tvheadend, minidlna (plus the default Armbian userland services)

This board has 1Gb of RAM. The problem is that I often run out of physical memory especially when the apt update package list run., or when I compile onboard some cpp stuff

I do have swap set on the external USB HD, but I don't know if this is a bug or what, when the swap happen,  all the user space application hang, since kswapd0 (kernel thread) run 100% of all the 4 cores. The result is that I can ping the board (ping is terminated in the kernel IP stack) but all the application services, including ssh are unresponsive.  The behaviour is strange, it could also be a bug, or some irq affinity problem, don't know, but this is not the point

The point is that for this kind of application (which are 99.99% of the application of this board, since it is hard that someone does some heavy computing with this SoC), there is no good reason to run 64bit. 

Consider that the Raspberry PI consortium basically does not eve provide an official 64bit distribution for the same reason.

I think that with the current uboot and kernel makefile it is enough to select ARCH=arm and point to an armhf toolchain for CROSS_COMPILE in order to compile everything in 32bit

The defconf (and the config) for kernel, should be a no issue, since it should get all the proper settings from the ARCH variable, you just need to select the proper SoC

 

Quote

Using 32-bit userland is easy - build an image for any 32-bit board and overwrite the kernel, u-boot and board support package

 

Are you sure? I think that the 32bit libc must be compiled against the 64 bit kernel (syscall parameters are different). I remember years ago, with the first ubuntu64 bit experiment, if you wanted to upgrade an existing 32 bit userland to 64bit kernel also the libc got updated

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
42 minutes ago, Menion said:

I think that with the current uboot and kernel makefile it is enough to select ARCH=arm and point to an armhf toolchain for CROSS_COMPILE in order to compile everything in 32bit

ARCH=arm has no effect for the u-boot AFAIK, and I highly doubt that you will be able to compile the current mainline u-boot (2017.07) using the orangepi_pc2_defconfig without changes.

Compiling 32-bit kernel should be possible, but you will probably need a different config for it and hope that dependencies on ARM64 don't break anything (i.e. compiling a 32bit kernel for A64 will probably be complicated because of dependencies like this one 

config SUN50I_A64_CCU
	bool "Support for the Allwinner A64 CCU"
	default ARM64 && ARCH_SUNXI
	depends on (ARM64 && ARCH_SUNXI) || COMPILE_TEST

 

I would strongly recommend testing a 32-bit userland with a 64-bit kernel first.

 

Edit:

Quote

Are you sure? I think that the 32bit libc must be compiled against the 64 bit kernel (syscall parameters are different). I remember years ago, with the first ubuntu64 bit experiment, if you wanted to upgrade an existing 32 bit userland to 64bit kernel also the libc got updated

AFAIk it just works. You can even run armhf apps on arm64 system, like by default we install installed armhf Chromium on arm64 desktop images.

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted
Quote

I would strongly recommend testing a 32-bit userland with a 64-bit kernel first.

 

Can you point me where the cross compiler for userland is selected in the build script for the specific target?
So I can try to swith to the 32bit version

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
2 minutes ago, Menion said:

Can you point me where the cross compiler for userland is selected in the build script for the specific target?
So I can try to swith to the 32bit version

As I said, grab an image from an armhf board like Orange Pi PC, replace the u-boot, kernel and other contents of the /boot directory and try to boot it. Currently there are no separate architecture/cross-compiler switches for the kernel and rootfs, everything depends on the ARCH variable. And userland is not cross-compiled since we are using upstream Debian/Ubuntu package base for like 99%, architecture affects debootstrap arguments, package list, repository list and some scripts like uInitrd generation helper.

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

Hi

I have built and image, attaching the first 2Mb of OrangePi PC2 image to the Orange Pi PC image after having removed the first 1Mb (yes the partition start address is different) and I have replaced /boot, /lib/modules, /lib/firmware and /usr/include/linux with the content of OrangePi PC 2 image.

I will try it this evening. One question: the libc binaries are coming from upstream or cross compiled?

Bye

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

Hi

I tried it but it cannot work, I suspect the problem is the uinitrd image that should be 32bit also

But I don't know if I can just take it from Orange Pi PC image

  


U-Boot SPL 2017.01-rc1-g5df570f-dirty (Jun 13 2017 - 06:32:47)
DRAM: 1024 MiB
Trying to boot from MMC1NOTICE:  BL3-1: Running on H5 (1718) in SRAM A2 (@0x44000)
NOTICE:  Configuring SPC Controller
NOTICE:  BL3-1: v1.0(debug):aa75c8d
NOTICE:  BL3-1: Built : 06:32:43, Jun 13 2017
INFO:    BL3-1: Initializing runtime services
INFO:    BL3-1: Preparing for EL3 exit to normal world
INFO:    BL3-1: Next image address: 0x4a000000, SPSR: 0x3c9


U-Boot 2017.01-rc1-g5df570f-dirty (Jun 13 2017 - 06:32:54 +0200) Allwinner Technology

CPU:   Allwinner H5 (SUN50I)
Model: OrangePi PC 2
DRAM:  1 GiB
MMC:   SUNXI SD/MMC: 0
*** Warning - bad CRC, using default environment

Setting up a 720x576i composite-pal console (overscan 32x20)
Error: no valid bmp image at 66000000
In:    serial
Out:   vga
Err:   vga
Net:   phy interface7
eth0: ethernet@1c30000
Hit any key to stop autoboot:  0
38518 bytes read in 116 ms (324.2 KiB/s)
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
3362 bytes read in 132 ms (24.4 KiB/s)
## Executing script at 4fc00000
U-boot loaded from SD
Boot script loaded from mmc
116 bytes read in 93 ms (1000 Bytes/s)
19253 bytes read in 533 ms (35.2 KiB/s)
4179 bytes read in 647 ms (5.9 KiB/s)
Applying kernel provided DT fixup script (sun50i-h5-fixup.scr)
## Executing script at 44000000
4729916 bytes read in 587 ms (7.7 MiB/s)
11407368 bytes read in 843 ms (12.9 MiB/s)
## Loading init Ramdisk from Legacy Image at 4fe00000 ...
   Image Name:   uInitrd
   Image Type:   AArch64 Linux RAMDisk Image (gzip compressed)
   Data Size:    4729852 Bytes = 4.5 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 4fa00000
   Booting using the fdt blob at 0x4fa00000
   reserving fdt memory region: addr=4fa00000 size=6b000
   Loading Ramdisk to 49b7d000, end 49fffbfc ... OK
   Loading Device Tree to 0000000049b0f000, end 0000000049b7cfff ... OK
Cannot setup simplefb: node not found

Starting kernel ...

Loading, please wait...
starting version 229
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems
done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.

 

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
4 minutes ago, Menion said:

I tried it but it cannot work, I suspect the problem is the uinitrd image that should be 32bit also

Initrd image must match the kernel (since it contains kernel modules) and architecture (since it contains userspace tools). The easiest way to fix this would be chrooting from another machine (with qemu static emulation) and regenerating it using update-initramfs

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

mmmmm I am thinking about it

What is this:

Quote 

Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.

Is it some script in initrd? Because this initrd image is taken straight direct from the Orange Pi PC2 image (I have replaced all the /boot content), so actually it does match the kernel.... Unless initrd call some executable in the root file system, in this case it is definitely  required to update it, due to the rootfs being 32bit

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
7 minutes ago, Menion said:

Is it some script in initrd?

Yes, but usually if you see messages like this it means that it can't find the rootfs device (it will print that after a while).

10 minutes ago, Menion said:

(I have replaced all the /boot content), so actually it does match the kernel....

Did you also replace the /boot/armbianEnv.txt from another image? It contains the rootfs UUID and it is unique on each image.

 

Also this means that you initrd userspace tools are 64-bit, so you will probably still need to regenerate it.

 

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

Yes I have replaced also armbianEnv.txt!!!

So there are two things to do, update the UUID and update initrd

One extra thing: on my running aarch64 image, this is the content of armbianEnv.txt 

verbosity=1
console=both
overlay_prefix=sun50i-h5
rootdev=UUID=00f42023-d4d4-499b-9a57-e3c8d029e8e4
rootfstype=ext4

apart from the UUID, how should the rest look for this hybrid image? In particular I wonder about "overlay_prefix"

Do you know if blkid can read UUID from a img file or from a loop mounted img?

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
3 minutes ago, Menion said:

Do you know if blkid can read UUID from a img file or from a loop mounted img?

If you loop-mount the image and partprobe it to display partitions, you should be able to get a correct UUID from /dev/loopXp1.

 

5 minutes ago, Menion said:

I wonder about "overlay_prefix"

It should match the SoC type, so for 64-bit targets it's either sun50i-a64 or sun50i-h5.

Obviously if you don't use DT overlays it doesn't matter.

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

So, I have launched in chroot+qemu:

  

root@menion-VirtualBox:/# update-initramfs -u -v

but for some reason it pretends that the kernel is still the OrangePi PC one:

  

Available versions: 4.11.7-sun8i
update-initramfs: Generating /boot/initrd.img-4.11.7-sun8i
WARNING: missing /lib/modules/4.11.7-sun8i

but I have the OrangePi PC2 stuff, such:

  

root@menion-VirtualBox:/lib/modules# ls
4.11.1-sun50iw2
root@menion-VirtualBox:/lib/modules#

So, something else should be changed, but I cannot understand what/where.

 

--edit--

Found, I have to generate a md5sum file with the kernel version as filename under  

/var/lib/initramfs-tools

Or actually I could launch:

  

update-initramfs -v -c -k all

One thing: in the update-initramfs I get this two errors:

  

/usr/share/initramfs-tools/hooks/klibc^i-t ignored: not alphanumeric or '_' file

end 

Calling hook resume
Unsupported ioctl: cmd=0x5331

Nothing to be worried of?

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
1 hour ago, Menion said: 

/usr/share/initramfs-tools/hooks/klibc^i-t ignored: not alphanumeric or '_' file

 

 

According to apt-file it should be either one of those (or both) 

➜  ~  % apt-file search '/usr/share/initramfs-tools/hooks/klibc'
initramfs-tools-core: /usr/share/initramfs-tools/hooks/klibc
klibc-utils: /usr/share/initramfs-tools/hooks/klibc-utils
➜  ~  %

 

1 hour ago, Menion said:

Calling hook resume Unsupported ioctl: cmd=0x5331

Can be ignored in chroot

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted
Quote

According to apt-file it should be either one of those (or both) 


➜  ~  % apt-file search '/usr/share/initramfs-tools/hooks/klibc'
initramfs-tools-core: /usr/share/initramfs-tools/hooks/klibc
klibc-utils: /usr/share/initramfs-tools/hooks/klibc-utils
➜  ~  %

 

 

Then it is possible it will cause problem, right?

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
9 minutes ago, Menion said:

Then it is possible it will cause problem, right?

Don't know. As you see, this file was ignored because of its name, but it's not clear whether the original file is there and why does this file (with the strange name) even exist.

If it causes any problems you can try verifying initramfs-tools-core package integrity with dpkg --verify

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

Hi

Actually both exist:

 

-rwxr-xr-x 1 root root  743 dic  9  2016 klibc^i-t
-rwxr-xr-x 1 root root  678 gen 26  2016 klibc-utils
  

-rwxr-xr-x 1 root root  743 dic  9  2016 klibc^i-t
-rwxr-xr-x 1 root root  678 gen 26  2016 klibc-utils

and both does the same, actually some non sense to me :)

  

root@menion-VirtualBox:/usr/share/initramfs-tools/hooks# ./klibc^i-t 
/bin/cp: '/lib/klibc-wBxlC8vEZykm9g94Kf1Dr2tj4ts.so' and '/lib/klibc-wBxlC8vEZykm9g94Kf1Dr2tj4ts.so' are the same file
root@menion-VirtualBox:/usr/share/initramfs-tools/hooks# ./klibc-utils 
/bin/cp: '/lib/klibc-wBxlC8vEZykm9g94Kf1Dr2tj4ts.so' and '/lib/klibc-wBxlC8vEZykm9g94Kf1Dr2tj4ts.so' are the same file

 

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

No, it does not work

  

U-Boot 2017.01-rc1-g5df570f-dirty (Jun 13 2017 - 06:32:54 +0200) Allwinner Technology

CPU:   Allwinner H5 (SUN50I)
Model: OrangePi PC 2
DRAM:  1 GiB
MMC:   SUNXI SD/MMC: 0
*** Warning - bad CRC, using default environment

Setting up a 720x576i composite-pal console (overscan 32x20)
Error: no valid bmp image at 66000000
In:    serial
Out:   vga
Err:   vga
Net:   phy interface7
eth0: ethernet@1c30000
Hit any key to stop autoboot:  0
38518 bytes read in 115 ms (326.2 KiB/s)
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
3362 bytes read in 129 ms (25.4 KiB/s)
## Executing script at 4fc00000
U-boot loaded from SD
Boot script loaded from mmc
116 bytes read in 90 ms (1000 Bytes/s)
19253 bytes read in 527 ms (35.2 KiB/s)
4179 bytes read in 633 ms (5.9 KiB/s)
Applying kernel provided DT fixup script (sun50i-h5-fixup.scr)
## Executing script at 44000000
4779271 bytes read in 620 ms (7.4 MiB/s)
11407368 bytes read in 919 ms (11.8 MiB/s)
## Loading init Ramdisk from Legacy Image at 4fe00000 ...
   Image Name:   uInitrd
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    4779207 Bytes = 4.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
No Linux AArch64 Ramdisk Image
Ramdisk image is corrupt or invalid
SCRIPT FAILED: continuing...
starting USB...
USB0:   USB EHCI 1.00
USB1:   USB OHCI 1.0

So, U-Boot looks for an aarch64 initramfs while the generated one in arm(hf)...

I hadn't the time to check the uboot environment, but I am not sure how to move on, in the end now initramfs is an hybrid

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
18 minutes ago, Menion said:

I hadn't the time to check the uboot environment, but I am not sure how to move on, in the end now initramfs is an hybrid

You need to change INITRD_ARCH in /etc/armbian-release (and use update-initramfs to regenerate the image)

It should not matter what is inside the initrd if it is packaged with the right arguments.

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

Tried, still the initramfs is detected as no aarch64. 

One thing worth to mention is that this hybrid image never ran on any target, it is just a downloaded image with uboot and kernel replaced, so I wonder if update-initramfs needs these info somewhere else, generated by the first run on target 

Link to comment
Share on other sites
zador.blood.stained

zador.blood.stained

Posted
Posted
8 minutes ago, Menion said:

One thing worth to mention is that this hybrid image never ran on any target, it is just a downloaded image with uboot and kernel replaced, so I wonder if update-initramfs needs these info somewhere else, generated by the first run on target 

U-boot only checks the architecture of uInitrd - standard initrd image packaged by mkimage.

It is done by this script and INITRD_ARCH is passed from /etc/armbian-release

You can use "mkimage -l /boot/uInitrd" to check the image info

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

Then there is something strange:

  

root@menion-VirtualBox:/etc# cat armbian-release 
# PLEASE DO NOT EDIT THIS FILE
BOARD=orangepipc
BOARD_NAME="Orange Pi PC"
VERSION=5.32.170629
LINUXFAMILY=sun8i
BRANCH=dev
ARCH=arm
IMAGE_TYPE=nightly
BOARD_TYPE=conf
INITRD_ARCH=arm64

notice that the original armbian-release didn't have any INITRD_ARCH row

  

root@menion-VirtualBox:/etc# update-initramfs -c -k all -v
Available versions: 4.11.1-sun50iw2
Execute: /usr/sbin/update-initramfs -c -k "4.11.1-sun50iw2" -b /boot -v -t
update-initramfs: Generating /boot/initrd.img-4.11.1-sun50iw2
Copying module directory kernel/drivers/hid
.............................................
.............................................
Calling hook fixrtc
Adding binary /bin/date
Adding binary /sbin/hwclock
Adding binary /sbin/dumpe2fs
Building cpio /boot/initrd.img-4.11.1-sun50iw2.new initramfs
update-initramfs: Converting to u-boot format

but 

root@menion-VirtualBox:/etc# mkimage -l /boot/uInitrd
Image Name:   uInitrd
Created:      Fri Aug 25 17:56:33 2017
Image Type:   ARM Linux RAMDisk Image (gzip compressed)
Data Size:    4778579 Bytes = 4666.58 kB = 4.56 MB
Load Address: 00000000
Entry Point:  00000000

So it is still 32 bit arm....

 

Link to comment
Share on other sites
Menion

Menion

Posted
  • Author
Posted

Found the problem!

This is the 99-uboot script in the Orange Pi PC image

  

#!/bin/sh
echo "update-initramfs: Converting to u-boot format" >&2
tempname="/boot/uInitrd-$1"
mkimage -A arm -O linux -T ramdisk -C gzip -n uInitrd -d $2 $tempname > /dev/null
ln -sf $(basename $tempname) /boot/uInitrd > /dev/null 2>&1 || mv $tempname /boot/uInitrd
exit 0

modified to arm64 and going to test

Link to comment
Share on other sites
Guest
This topic is now closed to further replies.
 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept