Armbian + NVME = fail to boot

[jrstarke]

I tried the "Armbian_22.11.1_Rock-5b_bullseye_legacy_5.10.72.img.xz" image on my Rock 5B. It worked on an SD card, but when I loaded it on an NVME drive on the build in M.2 slot, it worked with neither the SPI Boot image with armbian-install.sh, nor the SPI image from radxa.

 

When I have SPI loaded, and I have the SD Card installed, it just seems to load from the SD Card. When I remove the SD card, and boot the system. It seems to just stay with the LED with the blue and green elements on. I've waited several minutes without it booting. I was able to get it to successfully boot once, but never again.

[UdoA]

I can confirm that the SPI boot with NVMe is NOT working at the moment with the Jammy + XFCE image.

[fraz0815]

At the moment, SPI boot is only working with builds from here:

Butt will sure be ported to armbian in future.

[Igor]

I have successfully installed to SPI / NVME from official Armbian builds. Without any hacks, regular armbian-install.

[balbes150]

18 часов назад, Igor сказал:

have successfully installed to SPI / NVME from official Armbian builds. Without any hacks, regular armbian-install.

And now try to start the system from the SD card (imitation of problems with updating the kernel and you need a recovery system from an external media).

[Igor]

1 hour ago, balbes150 said:

And now try to start the system from the SD card

Aha, that I haven't try. So we would need to change priorities in the boot script or proceeding into some other method?

[balbes150]

35 минут назад, Igor сказал:

So we would need to change priorities in the boot script or proceeding into some other method?

Using boot.scr is initially a bad idea. This old junk has been "obsolete" for a long time and it's time to get rid of it. Changing boot.scr - you will not solve the problem, it is only a disguise laying a time bomb.

[Igor]

1 hour ago, balbes150 said:

Using boot.scr is initially a bad idea.

 

Not saying its good idea. This is all what was possible back then. Odoroids  are still on boot.ini which is textual interpretation of boot.scr ...  

 

1 hour ago, balbes150 said:

This old junk has been "obsolete" for a long time and it's time to get rid of it.

I agree that some old junk has to be remove. But this has to be done together and systematically or we made one step forward, two back. Upgrades / downgrades has to work. Install tools has to work. Build tools has to work and everything has to be checked twice. And again - what do we sacrifice?

[balbes150]

1 час назад, Igor сказал:

This is all what was possible back then.

No, initially there was an attempt to use the normal version (extlinux.conf) for rock 5b, but under the pressure of retrogrades, there was a rollback to this old stuff.

 

1 час назад, Igor сказал:

Odoroids  are still on boot.ini which is textual interpretation of boot.scr

Odroid is the stupidest choice and obsession with all sorts of old shit (boot.ini, piteboot).

 

1 час назад, Igor сказал:

But this has to be done together and systematically or we made one step forward, two back.

rk35xx (rk356x rk3588) - should initially focus on normal, new versions of startup systems (EFI\Grub). For rk356x - EDK2 has been available for a long time with a choice of system\kernel on the screen (for the headless option, this also works fine). For rk3588 - test versions are already available and in the coming weeks \ month there will be a working version with monitor support.

 

With whom and what to discuss if retrogrades don't even understand how extlinux or efi\grub works and squeal at every corner that these are "very bad decisions"? 

At the same time, they do not understand elementary things - that switching at least to extlinux gives a huge step forward and allows you to atomize the installation of several cores at the same time (it is not necessary to make a choice of the core on the screen, it is important to keep the ability to easily roll back to the primary core if the new one has problems). Try using boot.scr to implement two\three\N cores installed in the system at the same time. When using boot.scr - updating the system\kernel, this is a lottery, with a high probability of getting a serious problem (strictly there is no kernel anymore. and the new one is not working).

[Igor]

4 minutes ago, balbes150 said:

there was a rollback to this old stuff.

Current only working software for this HW is very old stuff anyway 

 

5 minutes ago, balbes150 said:

Odroid is the stupidest choice and obsession with all sorts of old shit (boot.ini, piteboot).

 

Vendors objectives are different than ours. They want to sell hardware and they want to have control over software. By doing on their own, they control it and they don't help their competitors. It this aspect it makes sense. 

But yes, I totally agree. 

 

9 minutes ago, balbes150 said:

With whom and what to discuss

 

Research has been done on the topic, we know many pros and cons and I think we do agree to some point.

https://forum.armbian.com/topic/20847-the-boot-process-and-various-devices/

A good topic to discuss it next week - how to proceed from current stage? @ManoftheSea Too early?

 

10 minutes ago, balbes150 said:

updating the system\kernel, this is a lottery

 

If preparation and transition is not done well, we also have a lottery. It is nice that you experiment with all this, I know others also did some. We found some issues and we need to make some decisions ... 

[jrstarke]

On 12/19/2022 at 5:05 AM, Igor said:

I have successfully installed to SPI / NVME from official Armbian builds. Without any hacks, regular armbian-install.

 

I just tested this again with the latest Debian CLI Bullseye_Legacy build from December 9th. I can boot from the SD card fine. Rock 5 running stock radxa 30W power supply. Ran fdisk on my WD Black SSD, deleted all partitions, the created just one using mkfs.ext4 /dev/nvme0n1p1. Once complete, I ran armbian-install and installed on to /dev/nvme0n1p1, and then flashed the SPI when prompted. Once complete. I selected power off when prompted, then unplugged the power and removed the SD card. I powered the Rock 5 back on, and the activity LED has been solid green-blue ever since.

 

If I boot to the SD card, but power it down, inserting the SD card, then powering it back on. It's solid green-blue for 5 seconds before it drops to just green with a double blink of the blue LED once a second.

 

I've installed the radxa Rock 5 image on the NVME, and loaded up the SPI and that boots fine, but for some reason, I can't get it to boot on Armbian.

 

I also tried flashing the NVME directly with BalenaEtcher with the same image, then initially booting with an SD card to run armbian-install to install the SPI. Same thing, when it boots, the activity LED is just solid green-blue. I think I let it run for 30 minutes at one point with no change.

Edited December 21, 2022 by jrstarke
Add additional context.

[DMC]

On 12/21/2022 at 6:01 AM, jrstarke said:

I just tested this again with the latest Debian CLI Bullseye_Legacy build from December 9th. I can boot from the SD card fine. Rock 5 running stock radxa 30W power supply. Ran fdisk on my WD Black SSD, deleted all partitions, the created just one using mkfs.ext4 /dev/nvme0n1p1. Once complete, I ran armbian-install and installed on to /dev/nvme0n1p1, and then flashed the SPI when prompted. Once complete. I selected power off when prompted, then unplugged the power and removed the SD card. I powered the Rock 5 back on, and the activity LED has been solid green-blue ever since.

 

I tried same thing and found out that something is wrong about spi image included there for armbian-install. For now jammy is working fine, but not debian (tried cli on ext4), it will not find nvme drive and don't try to boot from it. 

replacing spi with this one: https://github.com/huazi-yg/rock5b/releases/download/rock5b/rkspi_loader.img made it working again.  

[jrstarke]

14 hours ago, DMC said:

replacing spi with this one: https://github.com/huazi-yg/rock5b/releases/download/rock5b/rkspi_loader.img made it working again.  

 

I tried this out, and at first it didn't work. Then I reflashed the NVMe from the armbian image and restarted. Boom, works like a charm!

[ManoftheSea]

@IgorI'm sorry I missed this ping. 

Extlinux is nice enough, but keeps all of the configuration within a single file, which makes it difficult to manage a multi-boot setup.  GRUB is a long known bootloader, but it's full of so much legacy - I don't know anything it does better than another option, and plenty it does worse.  It does have an advantage of drop-in include files, but it also likes to repeat itself over and over and over again.

 

systemd-boot is nice and clean, but doesn't have support in Debian Stable. 

[balbes150]

9 часов назад, ManoftheSea сказал:

which makes it difficult to manage a multi-boot setup

How does it make it difficult to manage multiple systems\cores? On the contrary, a single control point, you add any number of systems, cores. The only limitation is that to use the selection, you need support for monitor output and USB for u-boot. For rk3399 - it is implemented.

 

9 часов назад, ManoftheSea сказал:

GRUB is a long known bootloader, but it's full of so much legacy - I don't know anything it does better than another option, and plenty it does worse

Everything written is wrong. grub does an excellent job with the AUTOMATIC (no manual manipulation needed) addition of a new kernel when installing it, supports all standard functions, as on regular PCs, and so on.

 

10 часов назад, ManoftheSea сказал:

systemd-boot is nice and clean, but doesn't have support in Debian Stable. 

excessively complex and poorly supported in Debian\Ubuntu systems, it is not widespread even on PCs.

[ManoftheSea]

What is a "system" or "core" as you use the words?  systemd-boot can select next-boot from a running linux system, no monitor/USB needed.

 

GRUB does terrible at detecting other distributions.

 

systemd-boot is simple, a single app on ESP and some standardized config files.  How can you claim it's complex without identifying GRUB as more so? It is supported fine in any new enough distro.

[balbes150]

12 часов назад, ManoftheSea сказал:

What is a "system" or "core" as you use the words? 

The system is a specific version of a full-fledged Armbian system (for example, Debian-22.11-XFCE, Ubuntu 22.08-server, etc.). The kernel is several versions of the kernel installed simultaneously (6.0.10 and 6.0.15 and 6.1.1). Which the user can choose independently at startup.

 

12 часов назад, ManoftheSea сказал:

systemd-boot can select next-boot from a running linux system, no monitor/USB needed.

I really want to see how system-boot will "guess" the thoughts of users, which version of the kernel (from the installed 2-3) needs to be launched at a given time.

 

12 часов назад, ManoftheSea сказал:

GRUB does terrible at detecting other distributions.

I work with Armbian and it is Armbian support that is being discussed here, so I am absolutely indifferent to "other distributions". Let their developers puzzle over how to work with them. Or are you suggesting that Armbian should pay out of pocket for fun for "other distributions"? "other distributions" too often use ready-made Armbian solutions and get money and bonuses for it.

 

12 часов назад, ManoftheSea сказал:

and some standardized config files. 

Which the user must manually configure. let "other distributions" deal with this masochism.

 

12 часов назад, ManoftheSea сказал:

How can you claim it's complex without identifying GRUB as more so?

show simple instructions for regular users for easy installation and automatic configuration for system-boot.

[ManoftheSea]

How will systemd-boot guess the thoughts of users?  Users will tell systemd-boot.  As I said, you can use bootctl to select the next boot.  Usually, the desired kernel is "the latest".  But if you have an opinion, go ahead and type it in to the computer, over SSH or console or whatever.

"I don't want to play nice with others" - Well, I can't help you there, but being interoperable with the best solution sounds like it's beneficial enough just for our own use, and the interoperability is a side benefit.

"User must manually configure" - you're kidding, right?  These configuration files are automatically generated and drop into the standard place.  It's easier on ourselves, not harder.

"Simple instructions" - apt install linux-image systemd-boot; kernel-install Balbes-ornery /usr/lib/linux/linux-6.1-balbes150 /usr/lib/linux/initrd; bootctl set-default Balbes-ornery

[balbes150]

2 часа назад, ManoftheSea сказал:

apt install systemd-boot

I tried Jammy in Ubuntu - the package is missing. Further discussion does not make sense.

 

 

P.S. Installing the kernel root system (kernel-install Balbes-ornery /usr/lib/linux/linux-6.1-balbes150 /usr/lib/linux/initrd) is very bad advice.

[ManoftheSea]

In jammy, it's part of the systemd package, not broken out.  In 22.10 and debian unstable, it's broken out to the systemd-boot package. 

If you keep your kernel at /boot/vmlinuz-6.0.0-6-amd64, then you instead run "kernel-install myKernel /boot/vmlinuz-6.0.0-6-amd64".  At least one thought is going to be required to take manual control of installing non-standard kernels.

[balbes150]

20 часов назад, ManoftheSea сказал:

In jammy, it's part of the systemd package, not broken out.  In 22.10 and debian unstable, it's broken out to the systemd-boot package. 

Checked the startup using system-boot. Two variants of u-boot with EFI support and EDK2-EFI. I took the original image from u-boot, which includes support for EFI, it is guaranteed to work and the system starts in EFI\grub mode without any problems. With the standard installation of additional kernel with the "apt install" command, the new kernel is automatically added to the grub startup list and works fine without any manual manipulations (I installed 4 kernel to check and checked their launch). Re-recorded launched this image. I uninstalled GRUB and installed system-boot ("bootctl install"), checked that the necessary files were created in the EFI partition and there were no errors during installation. Rebooted the system and ... the system does not start. Conducted  similar check with EDK2-EFI. The result is similar, grub works fine and automatically adds the kernel during installation by standard means (without any manual manipulations), allows  to select any kernel at startup and runs it perfectly. System-boot behavior with EDK2 - no startup works. I admit that if it takes a long and tedious tinkering with manual settings and commands,  can make it work, but the question was about an easy installation by an ordinary user.

 

20 часов назад, ManoftheSea сказал:

If you keep your kernel at /boot/vmlinuz-6.0.0-6-amd64, then you instead run "kernel-install myKernel /boot/vmlinuz-6.0.0-6-amd64".  At least one thought is going to be required to take manual control of installing non-standard kernels.

I pay special attention to the fact that all additional cores are standard DEB packages (from network repositories) with the correct structure and content, which are perfectly installed by standard APT or SYNAPTIC modules without manual manipulation.

 

By the way, I am currently testing the installation on rk3399\rk356x\rk3588 Armbian on eMMC\NVMe in EFI mode (u-boot and EDK2) and it works fine with GRUB.

[UdoA]

with the image from @balbes150 with efi support I got armbian to boot from spi and nvme.