NanoPi NEO3: dtb gone since linux-dtb-current-rockchip64 22.02.1

[lpirl]

Dear community,
dear @Igor (as maintainer of NEO3 CSC),

 

thanks a lot for the NEO3 support. This board ran rock solid for me approx. two years straight – much more stable than my other SBCs running Armbian (due to various reasons, no offense :)).

 

However, the board failed to boot with "ERROR: Did not find a cmdline Flattened Device Tree". It appears that ``rk3328-nanopi-neo3-rev02.dtb`` went missing. This must have happened when upgrading the package ``linux-dtb-current-rockchip64`` from 21.08.2 to 22.02.1.

 

As a workaround, it helps to (download the CSC image for the board again, get the file out of the image,) place the file in ``/boot`` and set ``fdtfile=../rk3328-nanopi-neo3-rev02.dtb`` in ``/boot/armbianEnv.txt``

 

I think it's a pity that support for this board broke, which works so well with Armbian. Any chance we can get the dtb back into the package?

 

Cheers,

 

Lukas

[Igor]

46 minutes ago, lpirl said:

as maintainer of NEO3 CSC

CSC targets does not have maintainers. My name was there by mistake. Fixed. 

 

https://docs.armbian.com/User-Guide_FAQ/

[lpirl]

Thanks for clarification, I took armbian.com/nanopineo3 for granted despite CSC.

 

BTW, I am also willing to contribute a PR (again ) but it is not yet clear to me what to fix. Happy to receive pointers…

[Igor]

18 hours ago, lpirl said:

Happy to receive pointers

 

We upgraded kernel and probably DT name changed. (without looking into the code)

 

18 hours ago, lpirl said:

I am also willing to contribute a PR

Welcome! For start it would be ideal to take this role related to this particular board, which currently doesn't have anyone. Or this. We also have a board in test facility. 

[siljaer]

On 3/23/2022 at 1:01 PM, lpirl said:

As a workaround, it helps to (download the CSC image for the board again, get the file out of the image,) place the file in ``/boot`` and set ``fdtfile=../rk3328-nanopi-neo3-rev02.dtb`` in ``/boot/armbianEnv.txt`

 

Thank you! I was stuck and that solved my problem. I agree with you: wonderful board and wonderful armbian image.

[lpirl]

On 3/24/2022 at 8:38 AM, Igor said:

We upgraded kernel and probably DT name changed. (without looking into the code)

 

That was my intuition too but I could find neither another adequately named file in ``boot/dtb`` nor a commit that changes (the patch file containing) the file name.

 

Could it be that removal of deprecated build targets (9d8a424) causes ``rk3328-nanopi-neo3-rev02.dtb`` not to be included in deb ``linux-dtb-current-rockchip64``? If yes, it this on purpose? If yes, should we maybe introduce ``…-csc`` debs or similar?

 

On 3/24/2022 at 8:38 AM, Igor said:

Welcome! For start it would be ideal to take this role related to this particular board, which currently doesn't have anyone. Or this. We also have a board in test facility. 

 

I actually would not hesitate much to maintain this board. Problem is, I have only one which is in permanent remote use. I'll think about it.

Also, I need to understand the test facility. I just built the CSC image for the board and I'd expect it won't boot (same issue as in OP) while the test is green.

[Igor]

4 hours ago, lpirl said:

If yes, it this on purpose?

 

We just change status since we are unable to take care of it anymore. There are archived images for emergency. They will boot. Upgrade - we can't provide warranty.

 

When we stop investing our time into supporting hardware, it usually starts the journey of falling apart. Upstream changes can break once this board, then another one or some of its vital functions. This is happening all the time. Unless you are o.k. with frozen kernel and no upgrades ... In this case it is just a very direct symptom - not booting at all. Keeping all SoC functions alive and functional is another level. This board is now like Debian, Arch, Manjaro, ... supported, works in theory, but ... you have to fix it in case not. This forum will help on best effort, but we can not provide any dedication.

 

4 hours ago, lpirl said:

I actually would not hesitate much to maintain this board.

Its about taking the load away from people that work 24/7 on this project. We have to drop support since we have enormous of other work, new HW is coming all the time. I waste 4-6 hours every day just for talking and planning without any touch with HW. Playing with hardware is usually more fun, but no time. Time and managing capacity is the problem.

 

4 hours ago, lpirl said:

Also, I need to understand the test facility.

 

Current test facility is basic and simple. Trust me. Also no need to understand it - it is currently used to estimate overall support state. As a maintainer you should know if it works or not - that is among job descriptions. A spare hardware would be nice, yes, so you don't use production HW for upgrade test and similar. I can send you some hardware, not this one, since I don't have spare, but something else. In case you are interested?

 

4 hours ago, lpirl said:

I just built the CSC image for the board and I'd expect it won't boot (same issue as in OP) while the test is green.

I am afraid its false positive  Bug in error detection in first step. I made this script and I fix it when I find time. But there are other more critical problems all the time 
https://github.com/armbian/scripts/runs/5629245369?check_suite_focus=true

[lpirl]

Thanks for all your explanations, @Igor. I think I could have well-anticipated the reasons why the team had to drop support for some boards to save some resources and it makes perfect sense, all good.

My question ("If yes, it this on purpose?") has probably been imprecise. I meant to ask "Is it expected that the dtb files of CSC boards are not in the dep package (anymore)?". Since you did not seem surprised, I assume the answer is "yes".

Thank you also very much for your offer so send over some hardware. I decline with thanks. However, without the specific board, I cannot plausibly take part in the release process and test upgrades etc.
Hence, if someone has a spare Neo3, I am interested. I'll also keep my eyes open if I can buy a used one. If I get one spare Neo3, I'll report back and I'll be happy to be the maintainer for the board (issue, PR, etc. volume for the specific board seems to be quite low) and see how long I can keep up.

[MichaIng]

Just found the same issue. It is fine when general support for boards is dropped, but it is a critical issue when users who flashed the image when it was still officially supported just do an "apt upgrade" and cannot boot their board anymore. There is no info or warning of course somewhere during the apt upgrade process that would inform users that their board becomes unbootable by proceeding.

 

I tried using the R2S device tree, the green LED then lights after some seconds, but doesn't blink and boot doesn't proceed. A problem is also that I cannot get any serial console output on debug console or UART1. Not sure whether one of the other R2 device trees is closer, or probably U-Boot needs to be reflashed as well? I used 21.08.1 image from the archive, with Linux 5.10.60 to start with.

 

I will upgrade U-Boot (EDIT: Ah, not possible as there is no new U-Boot package available anymore) and go through the other R2 device trees. Generally, if there is any chance to re-enable the NEO3 device tree (without implying official support), that would be highly welcome and prevent all current NEO3 users from breaking their system earlier or later. And, CSC hardware does not generally imply that the device tree is removed, there are a lot of CSC board device trees present in the kernel and images can be still manually build in many cases.

[MichaIng]

I got it working with the R2 rev00 device tree:

 

verbosity=4
bootlogo=false
overlay_prefix=rockchip
fdtfile=rockchip/rk3328-nanopi-r2-rev00.dtb
rootdev=UUID=31d2bbea-e3ea-497d-a5cd-69a0d829f5fc
rootfstype=ext4
console=serial
earlycon=on
usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u

 

Ethernet, USB (including USB 3.0 and UAS, from what I see) work well, even GPIO devices show up. Not sure whether the missing UART output is related to the kernel upgrade or not. 1,500,000 baudrate assured, pins should have been attached correct, however at least the board is back up. We can ship the device tree switch for our users, but I can only encourage Armbian to re-enable the NEO3 device tree or find another way to prevent users from breaking their system, for reputation reasons. The NEO3 is a small board, but there will be some important production systems live as well, and there are always users who do not have an up-to-date SD card backup, in which cases they run into their personal GAU, without (working) debug console not even a way to get any related output of the failing boot 😉.

[StoneHead]

I was actualy looking at modding the neo3 dtb file just the other day. Found it had gone missing from the dts makefile . (arch/arm64/boot/dts/rockchip/Makefile)

 

--- Makefile.orig       2022-03-31 23:09:30.840184181 +0200
+++ Makefile    2022-03-31 23:14:43.734675836 +0200
@@ -25,6 +25,7 @@
 dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3326-odroid-go2.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3328-a1.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3328-evb.dtb
+dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3328-nanopi-neo3-rev02.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3328-nanopi-r2s.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3328-rock64.dtb
 dtb-$(CONFIG_ARCH_ROCKCHIP) += rk3328-rock-pi-e.dtb

 

[Alan L.]

@Igor

 

Can I donate either hardware, money or both to resume official Armbian support for NanoPi Neo3?  I use several of these little machines for a variety of purposes.  Thanks!

[Werner]

4 hours ago, Alan L. said:

@Igor

 

Can I donate either hardware, money or both to resume official Armbian support for NanoPi Neo3?  I use several of these little machines for a variety of purposes.  Thanks!

Check for personalized support: https://github.com/armbian/build#support

[Igor]

On 4/2/2022 at 2:10 AM, Alan L. said:

Can I donate either hardware, money or both to resume official Armbian support for NanoPi Neo3?

 

Currently your donations only covers electricity for building images so you are very far away.

I am sure donate button is easy to find and when donation(s) comes on the levels that we will be able to hire people that will support you, then we will proceed. Until then, we are forced to drop support in order to maintain sanity and contact with our families.

[lpirl]

Meanwhile, I got a used device and I'll see how I setup the infrastructure for continuously maintaining the device.

[MichaIng]

Let me know if I can help with testing, reviews or such. However, as being owned by full time job and own OOS projects, I cannot be a reliable official maintainer.