Lost network after upgrading to Armbian 23.8.1

[aoleynik]

Hello, I have  Orange Pi 3 LTS with Armbian 22.*.1 5.15.80-sunxi64.

It have been working well for about a year, but after upgrading to 23.8.16.1.47-current-sunxi64 network interfaces stop working at all.

I can't downgrade kernel with arbian-config because it need internet access.

 

Here is output from armbianmonitor:

armbianmonitor.txt

 

Could you please guide me how to solve this issue?

[Igor]

3 hours ago, aoleynik said:

Could you please guide me how to solve this issue?

Install new image that will be released this week. 

[Gunjan Gupta]

4 hours ago, aoleynik said:

Could you please guide me how to solve this issue?

If you need instant fix, edit /etc/apt/sources.list.d/armbian.list. Use beta repository by change apt.armbian.com to beta.armbian.com and install the kernel from beta repository

 

Note: issue is already reported and fix is also discussed on another thread. In future, please take some time searching the forum before opening a new issue

[aoleynik]

1 hour ago, Gunjan Gupta said:

issue is already reported and fix is also discussed on another thread

 

Thank you for the suggestion, but it seems that this is another case.

I've updated to Armbian 23.11.0-trunk.110 using WiFi dongle, but the problem is still here.

I see no wifi interface and eth0 is down and won't get up.

There is a lot of errors in logs:
 

[  308.574504] dwmac-sun8i 5020000.ethernet eth0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  308.575858] dwmac-sun8i 5020000.ethernet eth0: __stmmac_open: Cannot attach to PHY (error: -19)

 

[Gunjan Gupta]

you need to update your linux-uboot-<something> package as well. Don't remember the exact name, but name should be visible in the output of "dpkg -l | grep linux". The issue will then get resolved after a cold boot. That is shut it down, unplug the power and plug the power back in. The network should continue to work fine after that. To properly test the fix, make sure to also do a soft reboot as the issue you have reported is only visible on reboot and doesn't happen on cold boot

[aoleynik]

As I can see I have a latest version

 

linux-u-boot-orangepi3-lts-edge/jammy,now 23.11.0-trunk.110 arm64 [installed]
u-boot-tools/jammy-updates,now 2022.01+dfsg-2ubuntu2.4 arm64 [installed]

 

Cold boot didn't helped.

Any thoughts?

[Gunjan Gupta]

hmm... What is the version reported by u-boot on serial console during boot?

[aoleynik]

U-Boot SPL 2023.07.02-armbian (Sep 06 2023 - 11:20:36 +0000)

[Gunjan Gupta]

This should have had yesterday's date. Found in docs. upgrade only installs the package. You then have to install u-boot manually using steps mentioned here https://docs.armbian.com/User-Guide_Getting-Started/#how-to-update-u-boot

[aoleynik]

I've updated boot loader as described but nothing changed: serial console reported previous version of U-Boot.

[Gunjan Gupta]

ok...Thats weird. Could you please upload logs using armbianmonitor -u and share the link?

 

I don't have this issue on my Opi3 LTS. So need to check if yours uses same chip as mine. Mine uses motorcom chip for ethernet. I heard there is also some batches with a realtek chip. Also whats the version of your board. Mine is version 1.4

[aoleynik]

I couldn't upload so just attached it here.

 

orangepi3-lts:~:% armbianmonitor -u
System diagnosis information will now be uploaded to
Network/firewall problem detected.
Trying fallback...

 

mon.txt

 

I've the same version of the board 1.4

Edited September 20, 2023 by aoleynik

[Gunjan Gupta]

And you also have motorcomm chip like the one shown in the attached photo. Motorcomm is the black chip below 859, behind IR sensor. If you zoom in, you can read motorcomm written on it

 

[Igor]

51 minutes ago, aoleynik said:

I couldn't upload so just attached it here.

Use sudo next time.

 

BTW. Fixed images are already uploaded while package update will follow shortly.

[Gunjan Gupta]

 

2 minutes ago, Igor said:

BTW. Fixed images are already uploaded while package update will follow shortly.

Yeah, about that. He doesn't have issue with wifi. instead he seems to be having issues with ethernet. Same one that you found with Orange Pi One Plus. Its strange that I am not having that issue on my Opi3 lts.

 

@aoleynikyou can use armbian-config to switch to legacy 5.15.125 or what ever kernel is available. Once you do and if it brings back ethernet. Could you please again share logs for comparision with the non-working state?

[Igor]

27 minutes ago, Gunjan Gupta said:

. Same one that you found with Orange Pi One Plus. Its strange that I am not having that issue on my Opi3 lts.

Yes, network breaking came with some of recent upstream "improvements". Last 5.15.y could also break ... a version or two back.

[aoleynik]

I switched to Armbian 23.11.0-trunk.110 Jammy with Linux 5.15.80-sunxi64 and network interfaces are working now.

Here is armbianmonitor output: https://paste.armbian.com/sixalidoni

[aoleynik]

With Linux 5.15.127-legacy-sunxi64 ethernet works only not wifi.

armbianmonitor output: https://paste.armbian.com/hinododiyi

[Gunjan Gupta]

3 hours ago, aoleynik said:

With Linux 5.15.127-legacy-sunxi64 ethernet works only not wifi.

5.15.130+ kernels should have wifi working as well. 5.15.131 should be available from beta repository. You can give this a try.

 

This issue is specially a hard one to trace. I will try recreating your environment by installing 22.11.4 and upgrading from the same. Lets hope I face the same issue.

[Gunjan Gupta]

@aoleynikJust to rule power supply out. How are you powering your orangepi3 lts? For example, I am using official Raspberry Pi power supply rated at 15.3 watts, delivers 5.1 volt with maximum 3 Amps

[aoleynik]

I’m using it for 3D printer and power it by power supply module 5v 5 amps. 
While debugging I’ve used Apple 20w charger. 

[Gunjan Gupta]

13 minutes ago, aoleynik said:

I’m using it for 3D printer and power it by power supply module 5v 5 amps. 
While debugging I’ve used Apple 20w charger. 

Is it safe to assume those chargers produce exactly 5v and nothing above the same like 5.1v produced by Raspberry pi charger? I faintly remember reading somewhere that orange pi boards require slightly above 5v to function correctly. I have a Xiaomi PD adapter that can deliver 5V 3A. Will try with the same as well.

[Gunjan Gupta]

Tried upgrading from 22.11.1 image. Wifi was already broken in that image. Ethernet worked. 

 

First tried upgrading using apt (I am new to Armbian and personally I refer docs only after when something doesn't work 😁) . Used --allow-unauthenticated as apt was complaining about key

 

 apt-get --allow-unauthenticated update 
 apt-get --allow-unauthenticated dist-upgrade

 

This upgraded most packages to 23.8.3. However kernel was not upgraded. So I used armbian-config > system > firmware to upgrade further. That upgraded the kernel to 6.1.53. Rebooted OPi 3 LTS to boot from new kernel.

 

During reboot, noticed that u-boot version was still 2022.07 as seen on serial console. After reboot completed, ethernet still works and wifi also worked now. Used armbian-config > system > install > install/update the bootloader on sd/emmc option. Rebooted the system and noticed u-boot is now 2023.07.02. Ethernet still works, wifi still works. 

 

Powered off and restarted using onboard power button. Logged in to find ethernet still works, wifi still works.

 

Powered off. Disconnected power cable. Waited 30 seconds and plugged the power cable back in. Logged in and everything still works

 

All this with Raspberry Pi 15.3 watt power supply. Logs uploaded for reference https://paste.armbian.com/zibuyibeqi 

 

Now going to do all of the above again with Xiaomi 15 watt mobile phone power adapter.

[Gunjan Gupta]

@aoleynik Another small question. My Opi3 lts is running headless. No monitor keyboard, mouse or any other accessories connected. For you I know you have some lcd screen connected, Is there anything else connected as well? The 3d printer you have mentioned before, is it connected via USB?

[aoleynik]

8 hours ago, Gunjan Gupta said:

Is it safe to assume those chargers produce exactly 5v and nothing above the same like 5.1v produced by Raspberry pi charger?

I can't say - didn't measure, but It has been working for a year without errors.

 

7 hours ago, Gunjan Gupta said:

Is there anything else connected as well?

MKS TS35 V2.0 screen and since yesterday usb wifi dongle because on chip wifi won't connect to Unifi 6 AP.

The printer is connected to Orange Pi through UART and there is one more PIN used to power off printer.

[Gunjan Gupta]

Tried with 5v 3A supply. My Orange Pi 3 LTS behaved bit unstable but ethernet still worked.

 

@aoleynikCould you please try the following

 

Install 6.1.53. If the following command doesn't work, then try armbian-config > system > other and choose 6.1.53 kernel to install

 

apt-get install --upgrade linux-image-current-sunxi64 linux-dtb-current-sunxi64

 

Create a file called sun50i-h6-ethernet-fix.dts with following content

/dts-v1/;
/plugin/;

&emac {
	snps,reset-active-low;
	snps,reset-delays-us = <0 10000 1000000>;
	snps,reset-gpio = <&pio 3 14 1>;
};

&ext_rgmii_phy {
	/delete-property/ reset-gpios;
	/delete-property/ reset-assert-us;
	/delete-property/ reset-deassert-us;
};

 

Compile the file to dtbo

dtc -@ -I dts -O dtb -o sun50i-h6-ethernet-fix.dtbo sun50i-h6-ethernet-fix.dts

 

Copy sun50i-h6-ethernet-fix.dtbo file to /boot/dtb/allwinner/overlay directory and then enable the overlay by using armbian-config > system > hardware. Reboot the system for the changes to apply. See if ethernet works after reboot.

 

PS: The above actually breaks the working ethernet for me. But as yours is already broken, this might fix it for you

[张军]

I don't know if you're still talking about Ethernet not working after hot reset. The kernel I use is 6.1.54-current-sunxi64, After a hot reboot, wifi automatically connects but Ethernet does not work. I connected the network cable, HDMI cable, serial cable, and I used a regulated power supply with a limited power of 5V1A. Hope to help troubleshoot this problem.

armbianmonitor -u:
https://paste.armbian.com/iwufeqaxit

@Gunjan Gupta

[Gunjan Gupta]

4 hours ago, 张军 said:

I don't know if you're still talking about Ethernet not working after hot reset. The kernel I use is 6.1.54-current-sunxi64, After a hot reboot, wifi automatically connects but Ethernet does not work. I connected the network cable, HDMI cable, serial cable, and I used a regulated power supply with a limited power of 5V1A. Hope to help troubleshoot this problem.

armbianmonitor -u:
https://paste.armbian.com/iwufeqaxit

@Gunjan Gupta

 

Yes this thread is about broken ethernet on 6.x kernels. Thanks for reporting the problem. Sadly the logs are not much helpful in this case. Combining with the fact that ethernet works fine for me on both 6.1.54 and 6.5.4 on my Opi 3 LTS makes it pretty hard to trouble shoot the problem. This mostly leaves me with suggesting random things based on the differences between 5.15.x kernel and 6.x kernels to the user and wait for them to confirm if it worked or not.

 

If you are willing to help, could you please try the device tree overlay I posted above and see if it fixes ethernet for you?

[Gunjan Gupta]

I worked with another contributor named @ALIGMSTEN, For him ehternet worked on coldboot but was broken after reboot. We tested a couple of device-tree overlays and this worked for him

 

/dts-v1/;
/plugin/;

&reg_gmac_3v3 {
	startup-delay-us = <150000>;
};

&mdio {
	motorcomm,clk-out-frequency-hz = <125000000>;
};

 

@aoleynik@张军 would you guys try this out. Once you guys confirm that it works for you, I will merge it in so that it will be available for everyone.

[张军]

1 hour ago, Gunjan Gupta said:

would you guys try this out. Once you guys confirm that it works for you, I will merge it in so that it will be available for everyone.

I did several tests and the problem was fixed after using the above device tree. But there seem to be some problems with the U-Boot.

U-Boot SPL 2023.07.02-armbian (Sep 19 2023 - 18:28:54 +0000)
DRAM: 4096 MiB
Trying to boot from MMC1

I used an SD card to boot and often the card would not move after the above print after reboot, it seems that the identified DRAM capacity is wrong.