Odroid HC4 - No network connection after boot

[klippe]

Armbianmonitor:

http://ix.io/2Rh8

Hello,

 

   I'm facing a weird issue where a network connection doesn't work after my HC4 is started (after a power off), but then after calling a `reboot` network starts to work. 

 

   I'm running a fresh install of the latest "Armbian Buster mainline kernel 5.10.y". More details in armbianmonitor's output: http://ix.io/2Rh8 http://ix.io/2Req

 

   I've tried playing with /etc/network/interfaces by putting stuff like the following there:

allow-hotplug eth0
no-auto-down eth0
auto eth0
iface eth0 inet dhcp

   

also tried static ip but nothing works.

 

What exactly is happening?

1. When I power on my HC4 it awaits for network for a longer while (A start job is running for Raise network interfaces) and then finally boots with no network working.

2. ifconfig shows eth0 and lo interfaces. eth0 has no IP set. I cannot connect to the device from outside not reach network from it.

3. then I run `reboot` (need to connect a screen and a keyboard to the device :/) and the device boots quickly with network up and working

 

Any ideas how to resolve that? 

[Igor]

51 minutes ago, klippe said:

   I'm running a fresh install of the latest "Armbian Buster mainline kernel 5.10.y". More details in armbianmonitor's output: http://ix.io/2Req

This log is not from HC4. Not sure if this a problem of pastebin service or you made some mistake. Check it and do it again.

[klippe]

Oops. Sorry, here's the correct link to the armbianmonitor's output: http://ix.io/2Rh8

[tparys]

According to the below, your board thinks there is nothing plugged in. No link was established, so there's nothing to talk to.

 

2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP>

 

When it does establish, it only negotiates 100 Mb, not gigabit.

 

[   17.233329] meson8b-dwmac ff3f0000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
[   17.233371] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

 

Perhaps your Ethernet cable is bad, or the port on your router/switch?

[klippe]

No, it's not the case

9 hours ago, tparys said:

Perhaps your Ethernet cable is bad, or the port on your router/switch?

Thanks but it's not the case. I've tried different cables and ports but the main thing is that all's good with network when I go back to armbian based on 4.9 kernel. Just in that case it doesn't see my HDDs. So kind of stuck here.

More in the thread on the Odroid forum: https://forum.odroid.com/viewtopic.php?f=203&t=41456&p=323150

[Igor]

10 hours ago, klippe said:

Thanks but it's not the case. I've tried different cables and ports but the main thing is that all's good with network when I go back to armbian based on 4.9 kernel. Just in that case it doesn't see my HDDs. So kind of stuck here.

We added one related improvement / fix which will be available in nightly repository beta.armbian.com when CI finishes. But I don't have hardware (nor time tbh) to test.
https://github.com/armbian/build/commit/b355a3110907be526028531a402955482b2af926

[legogris]

This sounds similar to the issue where eth0 comes up during initramfs but fails to come up after startup.

 

See if this solves it for you:

ip link set eth0 down; sleep 1; rmmod realtek; sleep 1; modprobe realtek; sleep 1; ip link set eth0 up

(And then do dhcp/add ip and routes)

Most of that delay is probably unnecessary - I noticed it would sometimes fail without them but didn't bother to pinpoint exactly where and how long is necessary.

 

If it does work, until the issue is solved upstream, you can add and enable this systemd unit to have it done on startup:

 

[Unit]
Description=Load Realtek drivers.
Before=network-online.target

[Service]
Type=simple
ExecStartPre=/usr/bin/sh -c '/sbin/rmmod realtek && sleep 1'
ExecStart=/usr/sbin/modprobe realtek

[Install]
WantedBy=multi-user.target

 

Separate but similar issue to https://superuser.com/questions/1520212/realtek-ethernet-not-working-after-kernel-update

[klippe]

28 minutes ago, legogris said:

This sounds similar to the issue where eth0 comes up during initramfs but fails to come up after startup.

 

See if this solves it for you:

ip link set eth0 down; sleep 1; rmmod realtek; sleep 1; modprobe realtek; sleep 1; ip link set eth0 up

(And then do dhcp/add ip and routes)

 

If it does work, until the issue is solved upstream, you can add and enable this systemd unit to have it done on startup:

Thanks. I've tried that but with no effect. Still eth0 not active after that (tried dhclient eth0 to re-fetch ip etc.).

[klippe]

14 hours ago, Igor said:

We added one related improvement / fix which will be available in nightly repository beta.armbian.com when CI finishes. But I don't have hardware (nor time tbh) to test.
https://github.com/armbian/build/commit/b355a3110907be526028531a402955482b2af926

Thanks. However the problem is that after switching to nightly my device no longer boots so I've no way to check network o_O

[Igor]

8 minutes ago, klippe said:

Thanks. However the problem is that after switching to nightly my device no longer boots

Can you try self build image?... today's nightly rebuild didn't finished and will be available later on after things get sorted out.

[legogris]

Just to make sure, you're still on 5.10 trying this, correct? Just checking since I saw you mentioning downgrading to 4.x in the hardkernel forum thread.

 

I self-built buster on 5.10 (default current) and got it working with the above

[klippe]

49 minutes ago, Igor said:

Can you try self build image?... today's nightly rebuild didn't finished and will be available later on after things get sorted out.

Have never done it before but I can try. Anything specific that should be included in the config? Guessing amlogic support.

[klippe]

2 minutes ago, legogris said:

Just to make sure, you're still on 5.10 trying this, correct? Just checking since I saw you mentioning downgrading to 4.x in the hardkernel forum thread.

 

I self-built buster on 5.10 (default current) and got it working with the above

Yes. I've installed 5.10 again. What I did was:

 

$ ip link set eth0 down; sleep 1; rmmod realtek; sleep 1; modprobe realtek; sleep 1; ip link set eth0 up
$ dhclient -r
$ dhclient eth0

 

[legogris]

and can you show the output of `ip a` after the first but before dhclient?

[legogris]

a tail of dmesg could be helpful too

[klippe]

Here we go:

`ip a`

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:1e:06:49:08:b4 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e90d:fe83:a2ab:20a6/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

dmesg

[   49.219978] EXT4-fs (mmcblk0p1): resizing filesystem from 450560 to 7712800 blocks
[   55.903177] random: crng init done
[   55.903184] random: 7 urandom warning(s) missed due to ratelimiting
[   60.237243] EXT4-fs (mmcblk0p1): resized to 3145728 blocks
[   70.748887] EXT4-fs (mmcblk0p1): resized to 5767168 blocks
[   78.799163] EXT4-fs (mmcblk0p1): resized filesystem to 7712800
[   80.669353] meson8b-dwmac ff3f0000.ethernet eth0: PHY [0.0:00] driver [RTL8211F Gigabit Ethernet] (irq=41)
[   80.683035] meson8b-dwmac ff3f0000.ethernet eth0: No Safety Features support found
[   80.683050] meson8b-dwmac ff3f0000.ethernet eth0: PTP not supported by HW
[   80.683403] meson8b-dwmac ff3f0000.ethernet eth0: configuring for phy/rgmii link mode
[   94.359985] meson8b-dwmac ff3f0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[   94.360023] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[  103.828834] logitech-hidpp-device 0003:046D:2007.0005: HID++ 1.0 device connected.
[  178.512669] meson8b-dwmac ff3f0000.ethernet eth0: Link is Down
[  181.713569] meson8b-dwmac ff3f0000.ethernet eth0: PHY [0.0:00] driver [RTL8211F Gigabit Ethernet] (irq=41)
[  181.728888] meson8b-dwmac ff3f0000.ethernet eth0: No Safety Features support found
[  181.728911] meson8b-dwmac ff3f0000.ethernet eth0: PTP not supported by HW
[  181.728928] meson8b-dwmac ff3f0000.ethernet eth0: configuring for phy/rgmii link mode
[  195.627538] meson8b-dwmac ff3f0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[  195.627582] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

 

[Igor]

Here you can find latest builds:

https://imola.armbian.com/dl/odroidhc4/nightly/

[legogris]

The interface looks fully up, which makes me think it's actually working as intended and your problem lies elsewhere (network, DHCP server, cables, etc)

 

Notice how it;s changed now as opposed to before

[klippe]

21 minutes ago, legogris said:

The interface looks fully up, which makes me think it's actually working as intended and your problem lies elsewhere (network, DHCP server, cables, etc)

I know it may look like that but:

1. I've tried different cables and router ports. Also no other device connected to the router has any network issues.

2. networks works correctly on 4.9

3. the issue only happens on 5.10 and only for the first boot (when the device is powered on). Then after a soft `reboot` network is up and working with no issues.

 

[klippe]

Ok, so... mindblow.

 

1. I built image from trunk and booted my hc4 with it.

2. No network.

3. Followed steps from legogris. No network

4. added eth0 to /etc/network/interfaces (auth eth0 and iface eth0 inet dhcp and hotplug)

5. then some reboots dhclient calls, tried disconnecting-connectnig cable (because of hotplug) etc. and... suddenly network started to work.

6. power off, power on and network still works.

 

Cool, right? So then:

1. I tried with stable image

2. booted - network works out of the box

 

No idea what's going on here but hope it'll stay like that. Thanks for the support guys.

[RussianNeuroMancer]

Getting similar issue with meson8b-dwmac on Libre Computer AML-S905X-CC.

 

Cable definitely plugged in (orange and green leds light up), DHCP is totally fine, and even meson8b-dwmac report that link in up in dmesg. Yet systemd-networkd could not receive IP address from DHCP. Detaching and attaching cable back let me workaround this issue, but if I just reboot - it's almost always happening again. Tested both of currnet 5.10.21 and edge 5.11.11 kernels, reproducible in both cases, although with 5.11 success rate (of getting address from DHCP server) is noticeable higher, yet it's not 100% stable yet.

 

Logs: http://ix.io/2Wbc

 

When I attach exactly same cable to any other Rockchip or AllWinner boards - issue no longer reproducible.

 

EDIT Another similar issue with meson8b-dwmac:

 

[Igor]

Yes, it happens on random. Again.

[RussianNeuroMancer]

It's seems like rollback to uboot 2020.04 (specifically, I used this package) fixed it (found this idea here).

 

[Igor]

8 hours ago, RussianNeuroMancer said:

It's seems like rollback to uboot 2020.04 (specifically, I used this package) fixed it (found this idea here).

My C4 behaves better with latest u-boot. Its confusing, but @chewittgave me some tips to look into. Sadly time is the problem.