Unable to boot 'focal' or 'buster' images on SOPine Clusterboard

[robocone]

I am trying to boot armbian on the Pine64 clusterboard but I cannot get past a 'Starting kernel...' message on the serial console.

 

I am not able to boot either of the images from https://www.armbian.com/sopine-a64/#kernels-archive-all

 

I'm not sure if the serial is just turning off or the system is freezing. It does not connect to my router.

 

Here is the serial output for for 'focal' image.

 

Spoiler

U-Boot SPL 2020.10-armbian (Mar 08 2021 - 19:07:13 +0100)
DRAM: 2048 MiB
Trying to boot from MMC1
NOTICE:  BL31: v2.4(debug):2c62b00-dirty
NOTICE:  BL31: Built : 19:07:11, Mar  8 2021
NOTICE:  BL31: Detected Allwinner A64/H64/R18 SoC (1689)
NOTICE:  BL31: Found U-Boot DTB at 0x4094bd8, model: SoPine with baseboard
INFO:    ARM GICv2 driver initialized
INFO:    Configuring SPC Controller
INFO:    PMIC: Probing AXP803 on RSB
INFO:    PMIC: dcdc1 voltage: 3.300V
INFO:    PMIC: dcdc5 voltage: 1.200V
INFO:    PMIC: dcdc6 voltage: 1.100V
INFO:    PMIC: dldo1 voltage: 3.300V
INFO:    PMIC: dldo2 voltage: 3.300V
INFO:    PMIC: dldo4 voltage: 3.300V
INFO:    PMIC: fldo1 voltage: 1.200V
INFO:    PMIC: Enabling DC SW
INFO:    BL31: Platform setup done
INFO:    BL31: Initializing runtime services
INFO:    BL31: cortex_a53: CPU workaround for 843419 was applied
INFO:    BL31: cortex_a53: CPU workaround for 855873 was applied
INFO:    BL31: cortex_a53: CPU workaround for 1530924 was applied
INFO:    PSCI: Suspend is unavailable
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x4a000000
INFO:    SPSR = 0x3c9


U-Boot 2020.10-armbian (Mar 08 2021 - 19:07:13 +0100) Allwinner Technology

CPU:   Allwinner A64 (SUN50I)
Model: SoPine with baseboard
DRAM:  2 GiB
MMC:   mmc@1c0f000: 0, mmc@1c11000: 1
Loading Environment from FAT... Card did not respond to voltage select!
In:    serial
Out:   serial
Err:   serial
Net:   phy interface7
eth0: ethernet@1c30000
230454 bytes read in 13 ms (16.9 MiB/s)
starting USB...
Bus usb@1c1a000: USB EHCI 1.00
Bus usb@1c1a400: USB OHCI 1.0
Bus usb@1c1b000: USB EHCI 1.00
Bus usb@1c1b400: USB OHCI 1.0
scanning bus usb@1c1a000 for devices... 1 USB Device(s) found
scanning bus usb@1c1a400 for devices... 1 USB Device(s) found
scanning bus usb@1c1b000 for devices... 1 USB Device(s) found
scanning bus usb@1c1b400 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Autoboot in 1 seconds, press <Space> to stop
=>                      
=> 
=> 
=> 
=> 
=> 
=> 
=> 
=> 
U-Boot SPL 2020.10-armbian (Mar 08 2021 - 19:07:13 +0100)
DRAM: 2048 MiB
Trying to boot from MMC1
NOTICE:  BL31: v2.4(debug):2c62b00-dirty
NOTICE:  BL31: Built : 19:07:11, Mar  8 2021
NOTICE:  BL31: Detected Allwinner A64/H64/R18 SoC (1689)
NOTICE:  BL31: Found U-Boot DTB at 0x4094bd8, model: SoPine with baseboard
INFO:    ARM GICv2 driver initialized
INFO:    Configuring SPC Controller
INFO:    PMIC: Probing AXP803 on RSB
INFO:    PMIC: dcdc1 voltage: 3.300V
INFO:    PMIC: dcdc5 voltage: 1.200V
INFO:    PMIC: dcdc6 voltage: 1.100V
INFO:    PMIC: dldo1 voltage: 3.300V
INFO:    PMIC: dldo2 voltage: 3.300V
INFO:    PMIC: dldo4 voltage: 3.300V
INFO:    PMIC: fldo1 voltage: 1.200V
INFO:    PMIC: Enabling DC SW
INFO:    BL31: Platform setup done
INFO:    BL31: Initializing runtime services
INFO:    BL31: cortex_a53: CPU workaround for 843419 was applied
INFO:    BL31: cortex_a53: CPU workaround for 855873 was applied
INFO:    BL31: cortex_a53: CPU workaround for 1530924 was applied
INFO:    PSCI: Suspend is unavailable
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x4a000000
INFO:    SPSR = 0x3c9


U-Boot 2020.10-armbian (Mar 08 2021 - 19:07:13 +0100) Allwinner Technology

CPU:   Allwinner A64 (SUN50I)
Model: SoPine with baseboard
DRAM:  2 GiB
MMC:   mmc@1c0f000: 0, mmc@1c11000: 1
Loading Environment from FAT... Card did not respond to voltage select!
In:    serial
Out:   serial
Err:   serial
Net:   phy interface7
eth0: ethernet@1c30000
230454 bytes read in 13 ms (16.9 MiB/s)
starting USB...
Bus usb@1c1a000: USB EHCI 1.00
Bus usb@1c1a400: USB OHCI 1.0
Bus usb@1c1b000: USB EHCI 1.00
Bus usb@1c1b400: USB OHCI 1.0
scanning bus usb@1c1a000 for devices... 1 USB Device(s) found
scanning bus usb@1c1a400 for devices... 1 USB Device(s) found
scanning bus usb@1c1b000 for devices... 1 USB Device(s) found
scanning bus usb@1c1b400 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Autoboot in 1 seconds, press <Space> to stop
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
3202 bytes read in 3 ms (1 MiB/s)
## Executing script at 4fc00000
U-boot loaded from SD
Boot script loaded from mmc
155 bytes read in 2 ms (75.2 KiB/s)
41034 bytes read in 5 ms (7.8 MiB/s)
3821 bytes read in 4 ms (932.6 KiB/s)
Applying kernel provided DT fixup script (sun50i-a64-fixup.scr)
## Executing script at 45000000
11754630 bytes read in 564 ms (19.9 MiB/s)
21792776 bytes read in 1041 ms (20 MiB/s)
Moving Image from 0x40080000 to 0x40200000, end=41730000
## Loading init Ramdisk from Legacy Image at 4fe00000 ...
   Image Name:   uInitrd
   Image Type:   AArch64 Linux RAMDisk Image (gzip compressed)
   Data Size:    11754566 Bytes = 11.2 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 4fa00000
   Booting using the fdt blob at 0x4fa00000
EHCI failed to shut down host controller.
   Loading Ramdisk to 494ca000, end 49fffc46 ... OK
   Loading Device Tree to 0000000049457000, end 00000000494c9fff ... OK

Starting kernel ...

 

 

Can anyone provide any advice for diagnosing/fixing this?

Board: Not on the list

Edited March 22, 2021 by TRS-80
put long output inside spoiler

[Igor]

1 hour ago, robocone said:

I am trying to boot armbian on the Pine64 clusterboard but I cannot get past a 'Starting kernel...' message on the serial console.

Default log goes to HDMI ... how long do you wait?

[robocone]

The clusterboard has no HDMI connection, so I am using the serial console so I can get networking going and then use ssh.

 

Thanks, I will try and find out how to change the output from HDMI to serial

[robocone]

Thank you very much, mounting the image and changing armbianEnv.txt to

console=serial
#disp_mode=1920x1080p60

had it go further:

 

Starting kernel ...

Loading, please wait...
Starting version 241
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems
done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  UUID=b9866ace-f260-437c-9794-8e1a363a3c71 does not exist.  Dropping to a shell!
(initramfs) 

 

Now it is unable to find the root filesystem (but when I check the UUID on my PC it does match

[robocone]

I was able to boot successfully by downloading 'Armbian_20.11.7_Pine64so_buster_current_5.10.4.img.xz' from the archives (I did not try all previous versions but this one worked so I will upgrade from there).

[dippywood]

I have the same problem -so, to help things along a little - 21.02.1 WILL boot - 21.02.3 does not.

 

Upgrading via apt will prevent booting unless frozen.

 

@TRS-80(it seems that you might be a good person to ask) I note that, in the first post, this has been flagged as having a board that is not on the list - this board is the Sopine A64 compute module, which is intended for use in the 'baseboard' and the 'clusterboard'  The Sopine A64 is explicitly listed as supported. Is it that this compute module is only supported when used with the baseboard? It seems as though the clusterboard has been supported until now, with changes having been made to support networking just recently

[TRS-80]

17 hours ago, dippywood said:

@TRS-80(it seems that you might be a good person to ask) I note that, in the first post, this has been flagged as having a board that is not on the list - this board is the Sopine A64 compute module, which is intended for use in the 'baseboard' and the 'clusterboard'  The Sopine A64 is explicitly listed as supported. Is it that this compute module is only supported when used with the baseboard? It seems as though the clusterboard has been supported until now, with changes having been made to support networking just recently

 

To clarify,  OP wrote "Board: Not on the list" which was the last line in his post.  No idea what they meant by that.

 

If you look closely at the edit note below that, you will see a reason.  It is the same as your own post immediately above this one.  Anyway, in my case, my edit to OP was simply to "put long output inside spoiler" and that's all I did.

[dippywood]

Thank you for the clarification - and apologies for my misunderstanding.

[Igor]

On 3/23/2021 at 11:14 AM, dippywood said:

The Sopine A64 is explicitly listed as supported.

 

That is correct. Support: https://docs.armbian.com/#what-is-supported creates costs which are 99% covered by us, which means it will wait for a free slot. This can be stretched from days to years. You have to understand that this vendor never covered us a single minute for supporting troubles their customers have by using (one if not the best available) OS for their hardware. Nor "you" see and feel the necessity to cover support costs you make. Since none of you see value in getting help, there is no warranty we will fix this and you might need to wait months ... 

 

Workaround = use old builds.

[dippywood]

@IgorPlease do not think that your efforts are unappreciated - they are, and greatly so. I was simply concerned that the board status was not what I expected. 

 

I also agree with the work-around - and thus me taking the builds and working out which would work for @robocone

 

I am also not adverse to providing workarounds when I can, and, as time allows, I will attempt to identify exactly where the problem is, and share same.

 

Once more, the efforts here are greatly appreciated (at least by myself) and I remain grateful for all of your efforts. If I can find the cause then I shall do so, but. for me. the 'free' is very valuable

 

[dippywood]

Just in case this provokes an 'ah-ha!' - this problem is somewhere within the DTB.

 

I purchased a baseboard so that I can reboot on command, and by selectively releasing the hold on package upgrades, this shows that the problem is only apparent when linux-dtb-current-sunbix64 is upgraded 

 

If there is no light-bulb moment then I'll start digging through the dtbs and see what I can find.

[pinecorn]

Thanks, holding back 'linux-dtb-current-sunxi64' works for me too.

 

Performing a diff on the dts with

dtc -I dtb -O dts -o /tmp/sopine.dts /boot/dtb/allwinner/sun50i-a64-sopine-baseboard.dtb

 

Finds, apart from some odd white space changes (< x > becomes <x> and <x> becomes < x >) the following differences in the dts:
 

Spoiler

 

 

I have not been able to confirm whether the removing the dts changes is enough to make it boot but looking at the contents of the package 'dpkg -L linux-dtb-current-sunxi64' it looks like it could be

[karlitos]

Hello,

 

I just stumbled upon this topic, after I spend few hours trying to run the current Armbian (buster)  I downloaded from the Sopine64 Product page. I got the same error:

 

Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  UUID=b9866ace-f260-437c-9794-8e1a363a3c71 does not exist.  Dropping to a shell!
(initramfs) 

 

The UUID in the armbianEnv.txt did match, so my conclusion is, that there is something wrong with the current linked image. I have a Pine64 LTS which is identical with the Sopine board. Downloading the Armbian_20.11.7 image did solve my problems.

[pinecorn]

Spoiler

 

 

Dude, see the suggestions by @dippywood the latest version is 21.02.1

The broken package can be held back with

echo "linux-dtb-current-sunbix64 hold" | sudo dpkg --set-selections

 

[karlitos]

15 hours ago, pinecorn said:

 

Dude, see the suggestions by @dippywood the latest version is 21.02.1

The broken package can be held back with

echo "linux-dtb-current-sunbix64 hold" | sudo dpkg --set-selections

 

Thanks for the suggestion, I will hold the brocken package back. Does the 21.02.1. version boot ?

[dippywood]

So, here's the problem, and its resolution:

 

Comparing the decompiled DTBs for the SOPine for 21.02.1 and 21.02.3, diff gives me 

565d564
<                       non-removable;
600c599
<                       max-frequency = <0xbebc200>;
---
>                       max-frequency = <0x8f0d180>;
681a681,682
>                       phys = <0x2d 0x00>;
>                       phy-names = "usb";
691a693,694
>                       phys = <0x2d 0x00>;
>                       phy-names = "usb";

 

Working through these show that only the first of these seems to make a difference. The definition for mmc@1c0f000 no longer contained the 'non-removable' entry. 

 

Therefore, inserting "non-removable;" at line 565 of sun50i-a64-sopine-baseboard.dts and compiling to /boot/dtb/allwinner/sun50i-a64-sopine-baseboard.dtb resolves this issue.

 

I can supply a patched focal image if requested.

 

 

[lanefu]

Any chance you can share the patched version of the dts.  Will make it easier to add it back into armbian build

[dippywood]

In my stupidity when renaming my temporary files I used the wrong extension - the file previously attached is the DTS.

 

So, here it is, with the proper extension.

 

I have asked for a new brain for my birthday! 

sun50i-a64-sopine-baseboard.dts

[SecT0uch]

Having the exact same problem with my sopine and the baseboard.

@lanefu I can't see the related PR upstream, has it been done yet ? I would submit it, but I'm not sure on which file (on Github) I should apply it.

[dippywood]

@SecT0uchIf you are able to mount the image on another system then you could make the update in place

 

Something along the lines of (adjust paths to suit your setup)

mkdir /tmp/image
mount -v -o offset=4194304 -t ext4 /Path/To/Armbian_21.02.3_Pine64so_focal_current_5.10.21.img /tmp/image
dtc -O dtb -o /tmp/image/boot/allwinner/dtb/sun50i-a64-sopine-baseboard.dtb -b 0 sun50i-a64-sopine-baseboard.dts
umount /tmp/image

 

Then write the image as usual

 

You can mount this from another Linux system. from WSL2 or whatever, as long as you have the dtc command avaialble. 

[SecT0uch]

9 hours ago, dippywood said:

You can mount this from another Linux system.

 

I wasn't sure of that, but thanks, it worked like a charm!

[dippywood]

13 hours ago, SecT0uch said:

 

I wasn't sure of that, but thanks, it worked like a charm!

You are most welcome.

[pfeerick]

Thanks for finding out the cause of the problem! I just updated all seven nodes of my clusterboard, only to have them not boot... It was probably time to clean them up, but it's still annoying, and affects more than just the clusterboard, but also the Pine64 A64 LTS. I thought I was having trouble with getting this to take, but I was probably being too imatient, and not waiting long enough for the microSD resize to complete...

 

If I understand right, the patch would go in here - https://github.com/armbian/build/tree/master/patch/kernel - but I'm not sure which of the sunxi branches it goes in... current? Making it so the patch belongs in `archive/sunxi-5.10` ... Anyway... all seven nodes will be happy once I apply this to the remaining six

[kravietz]

Any idea where did this non-removable change come from? I checked the DTS file https://github.com/armbian/build/blob/master/packages/blobs/sunxi/a64/pine64so.dts#L2117 but the non-removable flag is still there in sdmmc@01c0f000 section. But then some changes in DTS files are introduced by kernel patches apparently. The only patch that removes the flag is this one https://github.com/armbian/build/blob/0cdffb29b07305209efb12cf3b5ac6032d3a1153/patch/kernel/archive/sunxi-5.4/patch-5.4.46-47.patch#L44 from 24 March but the Armbian runs Linux 5.10.21-sunxi64 so I can't see how it's even relevant... I would like to create a PR to fix this but I'm a bit confused here.

[Buckzilla311]

On 4/8/2021 at 5:23 PM, dippywood said:

dtc -O dtb -o /tmp/image/boot/allwinner/dtb/sun50i-a64-sopine-baseboard.dtb -b 0 sun50i-a64-sopine-baseboard.dts

 

Should be:

 

dtc -O dtb -o /tmp/image/boot/dtb/allwinner/sun50i-a64-sopine-baseboard.dtb -b 0 sun50i-a64-sopine-baseboard.dts

 

Not that I'm critiquing or anything, I hope you got that new brain for your birthday 

 

Anyway, I can also confirm that this is indeed working. My clusterboard thanks you!

[dippywood]

@Buckzilla311- I did caveat with 'something along the lines of' to ensure that another brain replacement was not warranted due to lack of forethought :-)

 

In any case, I am glad it is working for you!

[goathunter]

Has anyone tried the latest upgrade to 5.10.34 to see if this problem has been resolved?

[poVoq]

Seems to be resolved. I was able to use both Focal with Kernel 5.10.60 as well as the Bullseye image with the same kernel.

[goathunter]

Thanks. I gave up and retired my Pine A64 LTS. It had some apparent hardware issues that were a pain. I replaced it with a non-LTS Pine, and I've had no problems since.