nanopi r4s - LAN doesn't work after kernel upgrade

[elyograg]

I started this journey with DietPi on a nanopi r4s.  I asked about it on the DietPi forum, and they basically said "Our distro is a bunch of scripts on top of Armbian.  We can't really help you with a kernel problem."  So I reimaged with Armbian and I'm here now.

Background with DietPi:

When I first boot the DietPi image, the LAN port works.  This is on kernel 5.10.63.  Then as soon as I log into it via ssh on the IP address acquired by the LAN port, it proceeds to auto-upgrade software, including the kernel, which goes to 5.15.25.  During the upgrade, I got messages saying a bunch of firmware for the r8169 driver was not there.  Installing armbian-firmware-full gets rid of that error, but does not help.  Once it reboots into the new kernel, the LAN port will not get link, and cannot even be seen by Linux.  I cannot make it work no matter what I do.  Some things I tried made the system unbootable.  The WAN port does continue to work.

So I grabbed the latest Armbian image for the r4s.  When I boot that, the LAN port never works, but the WAN port works just fine.  For what I am trying to do with the device, I need both ports working. 

I tried updating all the software on the system, including installing the -edge-rockchip64 linux packages.  I just cannot make it work.

Looking deeper into the dietpi initial boot with 5.10.63, I discovered that it loads the r8169 driver, but the newer versions don't.

[TRS-80]

1 hour ago, elyograg said:

Looking deeper into the dietpi initial boot with 5.10.63, I discovered that it loads the r8169 driver, but the newer versions don't.

 

So I would try uninstalling all those -edge-rockchip64 packages (or just flash new clean Armbian image) and then try to load that driver.

[Werner]

Providing logs with 

armbianmonitor -u

 helps with troubleshooting and significantly raises chances that issue gets addressed.

[elyograg]

The DietPi folks gave me the following commands:
 

apt -y --allow-downgrades install linux-dtb-current-rockchip64=21.08.2 linux-image-current-rockchip64=21.08.2 linux-headers-current-rockchip64=21.08.2
apt-mark hold linux-dtb-current-rockchip64 linux-image-current-rockchip64 linux-headers-current-rockchip64

This has gotten both ports working, but it isn't a good long-term solution.  It means I'm going to miss out on bug fixes and improvements because I won't be able to update my kernel.

Here are some system reports using armbianmonitor -u ... the first one is running a kernel where the LAN port doesn't work, the second one is running the older kernel where it does work:

http://ix.io/3U37
http://ix.io/3U7o

Can this post serve as a bug report, or do I need to make a post in another forum?

 

I found another distro that works on the R4S - FriendlyWRT.  This image has kernel 5.15.11.  The LAN port does not work on this either.

 

One other data point:  Although the LAN port does work with kernel 5.10.63, the link light never illuminates.  Neither the link light on the port nor the one on the top of the box is lit.  If I plug into the WAN port, then that link light does work.

 

Why would the nanopi designers use two different realtek LAN chipsets?  Seems like the board would be a lot simpler if they used a single chipset managing two ports.

[Igor]

13 hours ago, elyograg said:

I found another distro

 

Special purpose driven hardware is not amateur distro hopper friendly. This might help to understand. 

 

13 hours ago, elyograg said:

do I need to make a post in another forum?

 

This is forum and not a technical support service. Moderators will move it to correct place. Double posting is an offence / disrespect on forums. Please try to understand the difference - making friends and your contribution is essential. 

 

Professional support service is very far away. R&D even further. If you need that everything works perfectly, you need to hire someone and share your investment into FOSS with everyone. This is a support contract you have with Armbian, where it doesn't say anything about sponsoring bug fixing users find. Support is done in "best effort" manner and "no effort" if you are using a product based on Armbian.

 

IMO if you want to use this hardware, stick to Armbian with 5.10.y kernel. If you want that device follow kernel development, sponsor weeks of full-time engineer time and contribute to this community and project best way you can.

• https://forum.armbian.com/staffapplications/
• https://www.armbian.com/donate/

Help people which are responsible that you don't need to throw this device into the trash. Cover for - what works - first.

 

13 hours ago, elyograg said:

Seems like the board would be a lot simpler if they used a single chipset managing two ports.

FA knows what they are doing. HW is well designed, within reference design, don't worry about that. SoC has one internal phy and the other is attached either to usb or pci bus. 

 

P.S.
The reason why you were unable to reply at once after registering is a spam prevention and fighting against forum transforming into technical support service. Where "customer" is a king. Once again. This is forum - perhaps contributing first? Asking for services worth tens of thousands of euros can wait. Weeks - years.

New users also tend to ask too many questions in a very short period of time - like "i found out this is not working for me", while most of those problems are known, just there are no resources to address them. Users should also be using knowledge and information search function first to save precious resources which might be able to help them some day. Then they should search forum if topic they are interested in is not already open ...

 

Anyway, welcome!

[elyograg]

My takeaway from this is that there is a problem with the kernel, especially as I have seen it on multiple distros.  I only went distro-hopping because I had a problem, and wanted to understand exactly where the problem is.

 

I guess the remaining question is whether you will accept this as a bug report on Armbian, where your best kernel expert pursues it with kernel.org, or whether I need to be the one to take it to kernel.org.

If this is not affecting all NanoPi R4S users, I will be surprised.  That would most likely mean that I have a defective piece of hardware.  As I only have one unit, I cannot make that determination.

 

I've just created an account on bugzilla.kernel.org ... on the main page for that site, it says "If you did not compile your own kernel from scratch, you are probably in the wrong place."  They prefer bugs filed with the distribution.

 

Is there a guide for compiling your own kernel on Armbian?  A *really* long time ago on ancient linux distributions, I compiled my own kernel.  This was in the days when lilo was king and I had never heard of grub.  I would compile virtually everything I needed for a specific machine into its kernel, not really using modules at all.

Eventually the distribution-provided kernels got to the point where they were sufficiently optimized and would support virtually all hardware with zero problems, and so I stopped compiling my own kernels.  Now, although I remember enough about *compiling* a custom kernel, I have almost no knowledge about how to *install* such a beast on a modern linux distribution.  The best route might be to get the source package, customize that, and build a new package, then just let the package installer handle all the magic.

[Werner]

5 hours ago, elyograg said:

Is there a guide for compiling your own kernel on Armbian?

https://docs.armbian.com/Developer-Guide_Build-Preparation/

[elyograg]

I am in the process of building an "edge" package set with the branch of kernel source pulled down changed from 5.16.y to 5.17.y, hoping that works.  if it doesn't I can reinstall from scratch and downgrade the kernel again.

[elyograg]

10 minutes ago, elyograg said:

I am in the process of building an "edge" package set with the branch of kernel source pulled down changed from 5.16.y to 5.17.y, hoping that works.  if it doesn't I can reinstall from scratch and downgrade the kernel again.

The build failed.  Probably because the patchset for the kernel was intended be applied to a 5.16 kernel. not a 5.17 kernel.  Would there be anyone here who knows how to modify the build to use a 5.17 or 5.18 kernel?

[Werner]

1 hour ago, elyograg said:

Would there be anyone here who knows how to modify the build to use a 5.17 or 5.18 kernel?

The same way as we do when bumping kernel versions: Check the whole patchset and modify accordingly. To say remove stuff that has been upstreamed and check existing patches why they fail and to adjustments to match upstream.

For that check here https://github.com/armbian/build/tree/master/patch/kernel

and go to the matching folder for your family and tree. In your case (Nanopi N4S) that would be rockchio64-edge.

[elyograg]

14 hours ago, Werner said:

check existing patches why they fail and to adjustments to match upstream.

None of the patches failed to apply to the 5.17.y branch, which REALLY surprised me.   I thought for sure at least one of them would have failed.  I do not have enough experience with kernel code to make any adjustments to the patches.  I'm just guessing when I say that because they are designed for 5.16, they break the code for 5.17.

[3735943886]

NanoPi R4s, enp1s0 ethernet device not showed up after reboot - Beginners - armbian forum

[elyograg]

Today I updated packages.  That update included armbian-firmware-full version 22.08.0-trunk.0038 ... on reboot, the LAN port now shows up as enp1s0 and the WAN port is still eth0.  Problem solved.

[David Manouchehri]

The pcie_rockchip_host driver is buggy. Give latest a go, should work after https://github.com/armbian/build/pull/4308.

[elyograg]

  

I upgraded my kernel again today and now the LAN port shows up in the OS.

It works with both a cold boot and a warm reboot.

 

elyograg@legolas:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 54:10:ec:69:2a:04 brd ff:ff:ff:ff:ff:ff
    inet 192.168.217.165/24 brd 192.168.217.255 scope global dynamic noprefixroute eth0
       valid_lft 10776sec preferred_lft 10776sec
    inet6 fe80::e711:cec1:6580:b9c9/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: enp1s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether be:4f:ec:fc:14:a2 brd ff:ff:ff:ff:ff:ff
elyograg@legolas:~$ uname -a
Linux legolas 6.0.5-rockchip64 #22.08.8 SMP PREEMPT Fri Oct 28 15:15:06 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
elyograg@legolas:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.1 LTS
Release:	22.04
Codename:	jammy

 

[Igor]

17 hours ago, elyograg said:

 

I upgraded my kernel again today and now the LAN port shows up in the OS.

Today upgrade will break it as this fix broke many other boards. We need to find a better solution. 

[Werner]

For reference: https://github.com/armbian/build/pull/4369#event-7698677956

[elyograg]

In the last few months, some upgrades would break the port and others would fix it.

One of the recent upgrades I made was to update to kernel 6.0 ... since then it looks like the LAN port has been there consistently.   Curiously, the WAN port got renamed from eth0 to eth1 and that also seems to be consistent after every reboot.   My netplan config does not have any network device names in it, so that name change doesn't break anything.

[elyograg]

Today there was a new firmware package.  After reboot, the LAN interface is gone again.  A power cycle did not help.

[elyograg]

The WAN port is back to eth0 now.