Odroid C4

[mboehmer]

I have my C4 now, with eMMC card, and tried first the standard download image. Works nicely!

 

I started compiling my own system, so I can assist in debugging - anything which I should start with?

 

 

[Igor]

43 minutes ago, mboehmer said:

I started compiling my own system, so I can assist in debugging - anything which I should start with?

With the nastiest problem we have  Board stuck at u-boot (modern 2020.04 & kernel 5.6.y only) on reboot ... we don't know why or where to look at this point. Perhaps observing commit log from the stock u-boot might reveal something? Perhaps this is already solved in a u-boot trunk? Any information surrounding this bug would be helpful.

[mboehmer]

OK, I asked for it, so I got it. Is reboot affected by uSD or eMMC?

[Igor]

1 hour ago, mboehmer said:

Is reboot affected by uSD or eMMC?

Both.

[mboehmer]

Which image should I test? downloadable one or self compiled?

 

[mboehmer]

Hm. I just did a loop of reboots on the download image (Armbian Focal with Linux 5.6.15-meson64), and had no stuck. I use an Hardkernel 16GB eMMC and a 2A 12V power supply.

EDIT: I can't get a reboot problem reproduced with eMMC card. With uSD card, it happens very frequently (same image used). The reboot command either is the last message I get in the console, or we end up like that:

[  262.787662] reboot: Restarting system
bl31 reboot reason: 0xd
bl31 reboot reason: 0x0
system cmd  1.
SM1:BL:511f6b:81ca2f;FEAT:A0F83180:20282000;POC:F;RCY:0;EMMC:800;NAND:81;SD?:0;SD:0;READ:0;0.0;CHK:0;
bl2_stage_init 0x01
bl2_stage_init 0x81
hw id:øSM1:BL:511f6b:81ca2f;FEAT:A0F83180:20282000;POC:F;RCY:0;EMMC:800;NAND:81;SD?:0;SD:400;USB:8;LOOP:1;EMMC:800;NAND:81;SD?:0;SD:400;USB:8;

It seems to me that the reboot problem only affects uSD operation.

Edited June 4, 2020 by mboehmer
New information added

[Igor]

2 hours ago, mboehmer said:

and had no stuck

Can you perhaps repeat this like 10 times? I hope to get some input from others.

[mboehmer]

Hi Igor, eMMC reboot was 20x so far with no lockup.

It seems a uSD card problem - I have not managed to get a "reboot" with uSD card done properly. I use a SanDisk Extreme 16GB Class 3 HC I card.

The power supply should be sufficient (12V 2A).

With uSD card, the following errors occur when I log in after booting and do a "reboot" as root:

root@odroidc4:~# reboot
[   59.522866] reboot: Restarting system
bl31 reboot reason: 0xd
bl31 reboot reason: 0x0
system cmd  1.
SM1:BL:511f6b:81ca2f;FEAT:A0F83180:20282000;POC:F;RCY:0;EMMC:800;NAND:81;SD?:0;SD:0;READ:0;0.0;CHK:0;
bl2_stage_init 0x01
bl2_stage_init 0x81
hw id:øSM1:BL:511f6b:81ca2f;FEAT:A0F83180:20282000;POC:F;RCY:0;EMMC:800;NAND:81;SD?:0;SD:400;USB:8;LOOP:1;EMMC:800;NAND:81;SD?:0;SD:400;USB:8;

After that, the system is stuck and needs a power cycle (didn't find a reset yet).

 

The same image on an eMMC card does the following on a "reboot":

root@odroidc4:~# reboot
[   50.423833] reboot: Restarting system
bl31 reboot reason: 0xd
bl31 reboot reason: 0x0
system cmd  1.
SM1:BL:511f6b:81ca2f;FEAT:A0F83180:20282000;POC:F;RCY:0;EMMC:0;READ:0;0.0;CHK:0;
bl2_stage_init 0x01
bl2_stage_init 0x81
hw id: 0x0000 - pwm id 0x01
bl2_stage_init 0xc1
bl2_stage_init 0x02

L0:00000000
L1:00000703
L2:00008067
L3:15000020
S1:00000000
B2:20282000
B1:a0f83180

TE: 67922

BL2 Built : 20:29:41, Jun 18 2019. g12a ga659aac - luan.yuan@droid15-sz

Board ID = 1
Set cpu clk to 24M
Set clk81 to 24M
Use GP1_pll as DSU clk.
DSU clk: 1200 Mhz
CPU clk: 1200 MHz
Set clk81 to 166.6M
eMMC boot @ 0
sw8 s

So there are differences in the output, I have no clue yet where this output comes from.

Hope this helps.

 

 

[mboehmer]

I compiled a Buster system with new kernel (5.6.16)  now with the build system, and tested on eMMC card. Same results, reboot works fine on eMMC card.

 

[Arpel]

Hi,

My 2 cents here : I'm using Armbian on C4 since 14 days now and I only had reboot issue (on eMMC) with the early versions (5.4 kernel) : 5.6 never got stuck.

[mboehmer]

1 hour ago, Arpel said:

Hi,

My 2 cents here : I'm using Armbian on C4 since 14 days now and I only had reboot issue (on eMMC) with the early versions (5.4 kernel) : 5.6 never got stuck.

Could you try with uSD cardf (while we both know that uSD is always second choice

My system stucks very often with uSD while eMMC seems to work fine.

[mboehmer]

Made some tests now with Armbian 20.05.2 and 4.9.224-odroidc4 - reboot works with uSD cards.

For me it looks like a timing problem in Uboot... I will have to make some measurements on uSC pins to see if the problem can be seen there.

 

Removing the uSD card after the first boot attempt and reinserting it doesn't help, power cycle is needed.

[vkozlov]

7 minutes ago, mboehmer said:

Made some tests now with Armbian 20.05.2 and 4.9.224-odroidc4 - reboot works with uSD cards.

 

Same for me, Focal-based 4.9 works just fine, but 5.6.15 almost always stucks.

 

Could it be an issue with module unloading? Maybe it sounds stupid; but I had the very similar issue with Atomic PI and Ubuntu 20.04 - it wasn't able to reboot properly and dw_dmac module was the root cause, I have blacklisted it and problem disappeared. I wonder if we have something like it on Odroid-C4. 

[Igor]

3 hours ago, vkozlov said:

Could it be an issue with module unloading? Maybe it sounds stupid; but I had the very similar issue with Atomic PI and Ubuntu 20.04 - it wasn't able to reboot properly and dw_dmac module was the root cause, I have blacklisted it and problem disappeared. I wonder if we have something like it on Odroid-C4. 

No, AFAIK it always hangs at boot loader, below Linux and Ubuntu user land.

[mboehmer]

If you take a look on my previous post, it looks like the bootloader starting, and then being broken by a reset, immediately following a new boot attempt.

 

The "normal" boot messages are like that (cold starting a 5.x kernel from uSD card):

SM1:BL:511f6b:81ca2f;FEAT:A0F83180:20282000;POC:F;RCY:0;EMMC:800;NAND:81;SD?:0;SD:0;READ:0;0.0;CHK:0;
bl2_stage_init 0x01
bl2_stage_init 0x81
hw id: 0x0000 - pwm id 0x01
bl2_stage_init 0xc1
bl2_stage_init 0x02

Doing a "reboot" or "shutdown -r now" as root leads to the following messages, compare the "hw id" line:

[   95.523748] reboot: Restarting system
bl31 reboot reason: 0xd
bl31 reboot reason: 0x0
system cmd  1.
SM1:BL:511f6b:81ca2f;FEAT:A0F83180:20282000;POC:F;RCY:0;EMMC:800;NAND:81;SD?:0;SD:0;READ:0;0.0;CHK:0;
bl2_stage_init 0x01
bl2_stage_init 0x81
hw id:øSM1:BL:511f6b:81ca2f;FEAT:A0F83180:20282000;POC:F;RCY:0;EMMC:800;NAND:81;SD?:0;SD:400;USB:8;

The first "SM1" line is the same, no clue about the meanings of the status message, but "SD:0" seems to say "SDcard is fine", and booting starts. The next "bl2_stage_init" lines are identical, but while the "hw id" line is fine in the cold start, it is interrupted in the warm start (reset) by a SM1 line again, this time mit "SD:400" - which obviously is bad.

I could figure out something so far:

• SD:20000 -> no SDcard inserted
• SD:0 -> SD card fine (?)
• SD:400 -> our problem

As this message occurs without any boot media, it must originate directly from the SoC.

It doesn't originate from the power on reset (VDDIO_AO), this line is accessible on the UART header, no dip visible.

 

I will dig deeper...

 

Edit: my best guess so far is a forgotten watchdog timer which resets the system while it tries to reboot. For reasons unknown to me this seems not to hit when eMMC is booting. According to the data sheet the eMMC card is probed earlier then uSD, and this may already make a difference?

 

[mboehmer]

Some news: it is not the TEST_N pin. There is no pulse or noise visible which could reset the CPUs.

I will check the VDDIO_AO next, which could cause a reset also.

Only glitch so far I have seen: Amlogic recommends a 1nF cap on RESET_N line, which is missing in the C4.

 

Can anyone explain at what point of the boot sequence the "SM1:" message is being generated? Any clues for a reset source hitting here?

[Igor]

https://github.com/armbian/build/pull/2024#issuecomment-644629168

[Sean Mathews]

On 6/8/2020 at 7:04 AM, Igor said:

Currently we have only one(1) board and no official maintainer(s). I already blow several days solo for this board and I think / hope the biggest problems were fixed but I have no intention to add more long term work on my shoulders. We need two persons to be around to deal with this board or support stays on the at least same level as on other Linux distros.

 

I am currently working a project and want to use this board.

@igor Would it be possible to see the diff on github of what was done to bring up the ODROID-C4 board.

 

I looked over the commits on github but was not able to find much on this board. I think I saw one commit that was bringing over settings from another similar board?

 

Best

 SM

[Igor]

2 hours ago, Sean Mathews said:

and want to use this board.

 

And you are willing to cover yearly support costs?

 

3 hours ago, Sean Mathews said:

Would it be possible to see the diff

All code and all transactions are public.

[mboehmer]

As we know now where the uSD card problem resides, is there anyone working on that already?

[Igor]

21 hours ago, mboehmer said:

As we know now where the uSD card problem resides, is there anyone working on that already?

I have no info.

[Sean Mathews]

I can help contribute a little to the development. Send me info on how and how much

[Igor]

1 hour ago, Sean Mathews said:

I can help

 

You are helping yourself while we don't know yet if we can afford to support this hw / your attempt - for dealing with a board that makes some noise and have some users we easily lost 20.000 - 30.000 EUR per year. Some initial work was already done, but it's not too late to unplug and go slow as everyone else.

 

1 hour ago, Sean Mathews said:

Send me info on how and how much

Here is a test you have to pass:

https://github.com/armbian/build#support

[Technicavolous]

@Sean Mathews if you make any progress please give a share!

 

As for "how and how much," consider a monthly subscription. Some of us do that ;]

Or Buy Igor a Beer or tank of gas.

Or watch the wish list for something you'd like to contribute.  ( @Igor is there still an Amazon wish list page somewhere??)

 

@mboehmer I've rebooted my C4 over and over with several downloaded images,  I've not had a single stick. Finally compiled my own image. Sticks every time on shutdown -r now.  

I have the Hardkernel eMMC 16GB, but also tried Sandisk 32GB U3.  I am using lab power supplies ...

[mboehmer]

On 6/23/2020 at 3:02 AM, Technicavolous said:

@mboehmer I've rebooted my C4 over and over with several downloaded images,  I've not had a single stick. Finally compiled my own image. Sticks every time on shutdown -r now.  

I have the Hardkernel eMMC 16GB, but also tried Sandisk 32GB U3.  I am using lab power supplies ...

Which kernel / Uboot did you use for compiling?  I assume you compiled a complete image?

WIth eMMC reboot works like a charm (tried Hardkernel eMMC 16GB, as well as Radxa ones), with uSD cards it fails every time.

 

I assume we need to make sure that Uboot switches the card to 3.3V mode during startup. Putting this into the kernel is dangerous, as the reboot may or may not happen by software.

[Technicavolous]

Last night I noticed a slight difference in shutdowns. 

 

sudo shutdown -h now

appears to shut down the board properly, the red light goes out.

 

sudo halt

leaves the red light on indefinitely.

 

I have only recently got my build system working, and frankly this is the first time I've ever done that ... I just left everything default, selected the board, current kernel, no kernel config, os with desktop. I've not even opened any config files yet. 

 

My coding days were back in the MS C++ 7 era, predating VC ;] so it will take me a bit to get acclimated to the build system.  

 

 

 

 

[jjjbenderjjj]

Can somebody write the output of the commands for 5.4 kernel. It would be very helpful for me (I try to figure out is there SVE or not in this CPU)

• uname -a
• cat /proc/cpuinfo
• lshw

Thx you all in advance.

[mboehmer]

17 hours ago, jjjbenderjjj said:

Can somebody write the output of the commands for 5.4 kernel. It would be very helpful for me (I try to figure out is there SVE or not in this CPU)

• uname -a
• cat /proc/cpuinfo
• lshw

Thx you all in advance.

For 5.6 kernel attached. I have no 5.4 system running,. Hope it helps nevertheless.

5.6.txt

[jjjbenderjjj]

1 hour ago, mboehmer said:

For 5.6 kernel attached. I have no 5.4 system running,. Hope it helps nevertheless.

5.6.txt 4.29 kB · 4 downloads

God bless YOU!!!

According to this portion of a code [arch/arm64/kernel/cpuinfo.c: 384] (and other one): 

	if (IS_ENABLED(CONFIG_ARM64_SVE) &&
	    id_aa64pfr0_sve(info->reg_id_aa64pfr0))
		info->reg_zcr = read_zcr_features();

Of course, I double-check this [if] statement and {IS_ENABLED(CONFIG_ARM64_SVE)} part gives [1] but other one not (this is obvious to us because the already present in our defconfig) ;(
And if WE all try to read this feature register [res_from_aa64pfr0_el1 (64-bit width)] WE will get (if I am right) some unpleasantness results ;( :

register uint64_t res_from_aa64pfr0_el1;
__asm__ __volatile__("mrs %[res_from_aa64pfr0_el1], id_aa64pfr0_el1 \n\t"
					 ::[res_from_aa64pfr0_el1]"r"(res_from_aa64pfr0_el1)
					 :"memory", "cc");

printf("ID_AA64PFR0_EL1[0] 0x%" PRIX64 "\n", res_from_aa64pfr0_el1);
printf("ID_AA64PFR0_EL1[1] 0x%" PRIX64 "\n", res_from_aa64pfr0_el1 >> 32);
printf("ID_AA64PFR0_EL1.SVE 0x%" PRIX64 "\n", (res_from_aa64pfr0_el1 >> 32) & 0xF);

Our [CPU] doesn't have [SVE]. Please somebody proves me wrong!

[mboehmer]

12 hours ago, jjjbenderjjj said:

Our [CPU] doesn't have [SVE]. Please somebody proves me wrong!

Hm. How about starting with the existing problems (like the 1.8V/3.3V switching for the uSD card), before you try to find out something about the CPU instruction set?

A system not being able to properly reboot with uSD card will not be of use forf anyone.

Seems like you know what you are doing, so you help us here.