Installing Ambian Bookworm to NVME / M2 SSD

[AaronNGray]

I am trying to install Ambian on to the BananaPi-M7 to eMMC using option 6. there is no option for NVME / M2 SSD.

 

I had a failed install as the uSDCard image was 256GB larger than the eMMC, so it balked. I tried a 128GB uSDCard, followed by a 64GB uSDCard, but got the same error each time even on the 64GB one. I seem to have two partitions on the eMMC, mmcblk0boot0, and mmcblk0boot2 now,.

I am not sure if they were on the boards eMMC originally or what the boot sequence for the board is If the Masked ROM tries to boot the uSDCard if present and if that fails it boots the eMMC ?

 

Do I  clear mmcblk0 and try again with the 64GB uSDcard ? Or is mmcblk0 required to boot mmcblk1 ?

 

I have done an

Quote

armbianmonitor -u

  :-

 

https://paste.armbian.com/ehanixecoh

 

Also

```
aaronngray@bananapim7:~$ lsblk -a
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
ram0           1:0    0     4M  0 disk
loop0          7:0    0     0B  0 loop
loop1          7:1    0     0B  0 loop
loop2          7:2    0     0B  0 loop
loop3          7:3    0     0B  0 loop
loop4          7:4    0     0B  0 loop
loop5          7:5    0     0B  0 loop
loop6          7:6    0     0B  0 loop
loop7          7:7    0     0B  0 loop
mmcblk1      179:0    0  59.7G  0 disk
└─mmcblk1p1  179:1    0  59.1G  0 part /var/log.hdd
                                       /
mmcblk0      179:32   0 115.3G  0 disk
mmcblk0boot0 179:64   0     4M  1 disk
mmcblk0boot1 179:96   0     4M  1 disk
zram0        252:0    0  15.5G  0 disk [SWAP]
zram1        252:1    0    50M  0 disk /var/log
zram2        252:2    0     0B  0 disk
nvme0n1      259:0    0   1.8T  0 disk
```

Edited November 5 by AaronNGray

[AMR]

Apologies if i misunderstood, are you attempting to install armbian on eMMC or on your m.2 SSD?

 

16 hours ago, AaronNGray said:

I am not sure if they were on the boards eMMC originally or what the boot sequence for the board is If the Masked ROM tries to boot the uSDCard if present and if that fails it boots the eMMC ?

Well the first preference is SD card / USB device, and the second preference is eMMC.

 

 

Regarding installing on eMMC you can follow these steps:

 

 - Remove all the devices which might power the device

- Not officially mentioned but just to be safe remove any storage devices

- Connect the USB to TYPEC port on your Banana Pi M7 and the other end to your PC

- Make sure drivers are installed for your device (I am using windows for this so i only know how to do so via windows)

- Use RKDevTool v2.96 (In the official docs they provided an older version, just use 2.96 since i've personally tested it and it works)

 

- Hold the reset button (if flashing for first time) Or Markrom button, and only THEN insert the DCIN (make sure it has atleast 35W of power)

 

- If your drivers were installed, it should successfully show your device in Maskrom mode in your RKDevtool 2.96

- Do this:

 

- Make sure to check Write by Address, it's important

- Another thing, the image you select here should be .img, and not .img.xz (extract it first if its .xz)

 

- It should install armbian on your device. 

 

Here's my sources:

Getting Started BPI-M7 | BananaPi Docs  (Install Image to eMMC 2 section)

 

 

 

Edited November 6 by AMR

[AaronNGray]

Hi, I thought the eMMC is separate from the MaskedROM and the eMMC is write programmable while running the device booted from the uSDCard. Where as the MaskedROM boots either the SDCard or the eMMC in that order.

 

I am actually looking to be able to either :-

 

i ) boot directly from the NVME M2 SSD
ii) boot a bootloader on eMMC that then boots from the NVME M2 SSD

 

 

[AMR]

15 hours ago, AaronNGray said:

I am actually looking to be able to either :-

 

i ) boot directly from the NVME M2 SSD
ii) boot a bootloader on eMMC that then boots from the NVME M2 SSD

 

First of all, Have you tried installing the OS directly on your NVMe via balenaetcher? If not i think you should give it a shot. I am not entirely sure if thats possible though.

 

 

For what you are looking for, bootloader on eMMC which boots from Nvme sounds very reasonable. 

 

It does provide an option in RKDevTool to select a device. It should be PCIE from my guess, but if it doesn't work try SATA. Other than that, I honestly suggest you install the OS directly on your EMMC and keep the /var data or  /home (depending on your requirement) on the NVMe. 

 

Oh by the way, Loader is flashed directly on the device in perhaps some unreadable memory, However I am not very sure of that (Wasn't able to see it anywhere personally in lsblk)

 

 

EDIT: I've found this: https://docs.armbian.com/User-Guide_Getting-Started/#how-to-install-to-emmc-nand-sata-usb

 

 

 

Edited November 7 by AMR
removed: had some incorrect info, my bad

[amazingfate]

Hi, there is a simple way to install armbian to NVME SSD.

1, Boot armbian from sd card

 

2, Burn u-boot to emmc:

You can find u-boot firmware at 

$ ls /usr/lib/linux-u-boot-*/
idbloader.img  rkspi_loader.img  u-boot.itb

Use dd command to flash u-boot:

dd if=./idbloader.img of=/dev/mmcblk0 seek=64 conv=notrunc status=none
dd if=./u-boot.itb of=/dev/mmcblk0 seek=16384 conv=notrunc status=none

 

3, Burn armbian image to NVME SSD, you can use dd in the armbian system on sd card with command:

sudo dd if=./Armbian.img of=/dev/nvme0n1 bs=1M status=progress

or use application like balenaEtcher on windows to flash image to SSD.

 

4, Unplug sd card and plug NVME SSD, now you should be able to boot armbian system on NVME SSD.

[AaronNGray]

@amazingfateHi, can I fdisk the NVME SSD as I want to have a separate SWAP partition. How do I get the eMMC to boot the NVME SSD ir will it booted from the on on board "Masked ROM" ? Sorry not replied for nearly a week  : |

[amazingfate]

Armbian is using zram to configure swap by default, so it has no swap partitrion in its image. I you want separate swap partition you have to install the system on your own, by rsync the whole system to parted NVME root patition, and write correct root partition UUID to /boot/armbianEnv.txt, and config swap in /etc/fstab.

[AaronNGray]

@amazingfateIs it possible to eleborate on the details of installing to a fdisked NVME please ?  on what device and partition does the /boot/armbianEnv.txt live and how do I specify the UUID. Do I create a boot partition on the NVME as well as a swap and main/root partition ? Its would be nice to create a HOWTO document on how to do all this with possible variations, Many thanks for your help.

Edited November 18 by AaronNGray

[Igor]

22 minutes ago, AaronNGray said:

Its would be nice to create a HOWTO document

I think it is better to expand script to support those scenarios. 

https://github.com/armbian/build/blob/main/packages/bsp/common/usr/sbin/armbian-install

 

Worse case, write things here:
https://docs.armbian.com/User-Guide_Getting-Started/#how-to-install

[AaronNGray]

@Igor I need to understand what I am doing first as I dont have a lot of time I and also to verify the method. @amazingfate seems to know what he is doing and it would be good to document it first. Then we can look at the install-script., which might actually work anyway ? @Igor Please read the whole of this thread if you have not already done so. 

Edited November 18 by AaronNGray

[AaronNGray]

@amazingfate I have the board booting from eMMC, but /dev/nvme0n1 or any /dev/nvme* does not exist. I have installed a Samsung 970 EVO Plus 2 TB PCIe NVMe M.2 Internal Solid State Drive (SSD) (MZ-V7S2T0) device on the board.

 

[AaronNGray]

@Igor @amazingfateokay whats happening on installing to eMMC, then to NVME, is on an 'sudo apt-get update' and 'sudo apt-get upgrade' either at the eMMC stage or the NVME stage the kernel is being upgraded and it no longer supports NVME.

I also had a totally unstable state at NVME stage after presumably a cron job and bash was not recognising any external commands.

 

[eselarm]

Check you powersupply I think.

And run

 armbianmonitor -u

and post the URL so we see what versions and so on you are running.

Edited December 6 by eselarm

[Igor]

9 hours ago, AaronNGray said:

unstable state at NVME

I am afraid this is current state of Rockchip vendor kernel. PCI support works, but its unstable. Just FYI - we needed years to stabilise PCI on previous RK3399 SoC. Situation there was just worse - we were contracted by one of the vendors to do this, so we invested more efforts, but they "forgot to pay" for services even they were backed by contract ... damages to our work helping you is done also by purpose.

 

Workarounds is - cold reboot few times, trying to use some older kernels, where PCI is stable, but there are other problems. Waiting, supporting our work is solution, but that both is hard, when everyone expects software support perfection - that everything just work  

[going]

12 часов назад, AaronNGray сказал:

okay whats happening on installing to eMMC, then to NVME, is on an 'sudo apt-get update' and 'sudo apt-get upgrade' either at the eMMC stage or the NVME stage the kernel is being upgraded and it no longer supports NVME.

I also had a totally unstable state at NVME stage after presumably a cron job and bash was not recognising any external commands.

By the way, it may also be the specifics of this particular board and/or this particular NVME disk manufacturer.
If you want to deal with this problem, you need to build the EDGE kernel with a little more debugging options and publish the output here.
Maybe we can see the reason and maybe someone can fix it.

[AaronNGray]

@eselarm - The PSU is a 45Watt PSU and am pretty sure it is fine. I had issues with PSU current rating and voltage drop on BananaPi M3's

 

It seems to be okay on bootup but fails after running `stress --verbose --hdd 4' after a period of time. Looks like according to `/etc/fstab` it becomes a readonly filing system on errors. 

 

I will go through everything methodically now and post `armbianmonitor -u` reports.

Edited December 7 by AaronNGray

[AaronNGray]

@Igor Its a shame there was not a public record of this. I saw the https://www.armbian.com/bananapi-m7/ page, looked at the Armbian forum for issues, saw none and presumed the board was okay. We should really have detailed uptodate reports on all boards.

[AaronNGray]

@going yes hopefully I will get round to doing this stage. I was a low level programmer but never mastered the Linux Kernel other than the basics. Hopefully I can act as a tester and we can get this thing moving forward.

[AaronNGray]

I am trying Armbian 24.11.1 Noble Gnome - MESA / VPU - Kernel: 6.1.75, Size: 1.4 GB, Release date: Nov 17, 2024

 

armbianmonitor -u  -  https://paste.next.armbian.com/bakevuvuya

 

Install onto eMMC using armbian-install

 

Device seems to be overheating, as I had a failed boot, untill I put a 5V (of 12V) fan ontop. I have found this in the past running `stress`.

 

armbianmonitor -u  -  https://paste.next.armbian.com/sitakelipu

 

I am getting on both `armbianmonitor -u` 's a "Failed grabbing info (pipe 3 result 22) and sending to server paste.armbian.com

 

`lsblk -a` is not showing the nvme !!!

 

rebooting its there !!!

 

`sudo armbian-install` is hanging before display comes up, and now 'lsblk -a` is not displaying the NVME !

 

rebooting eMMC allowed me to see the NVME. so continuing...

 

`df` shows Armbian 24.11.1 Noble Gnome, launched on NVME as `/` !

 

armbianmonitor -u  -  https://paste.next.armbian.com/pekoxalapo

 

`stress --verbose --hdd 1` ran for 9 minutes befor failing and creating readonly fs.

 

Edited December 7 by AaronNGray

[Igor]

2 hours ago, AaronNGray said:

there was not a public record of this

 

There is, but one need to follow forums and GitHub issues to catch everything. If we (which includes everyone) would like to keep download pages updated with this, we will need to do something about. I already do more what is possible.

 

It is complex to assemble and forward right bits of information at right time to right people that needs that information. Bugs are constantly changing information, so we are looking at serious work, nothing what we can finance. Also some bugs are fixed fast and if we already have no resources for fixing bugs, how we could possible have for a complex info system? :) 

One alternative is official news channel. Currently we have three people working on preparing weekly newsletter. If they didn't catch this on their own, its not published. And last one mention some other problems for example https://www.armbian.com/newsflash/armbian-weekly-highlights-5/

 

1 hour ago, AaronNGray said:

I am getting on both `armbianmonitor -u` 's a "Failed grabbing info (pipe 3 result 22) and sending to server paste.armbian.com

 

There are many bugs in open source software that are pinned on us and we also produce a lot of code. This one is ours, yes, but its also free software, best effort and what comes with it.

 

WIP

https://github.com/armbian/build/pull/7542

 

More are waiting for good Samaritans:
https://github.com/armbian/build/issues

 

2 hours ago, AaronNGray said:

We should really have detailed uptodate reports on all boards.

 

Anyone can establish this. I tried that in several ways, but except few people that did testing there was no help in the most problematic part - organizing and leading this operation.

[AaronNGray]

Quote

That is for us impossible to organise and / or finance.

@Igor Either a databsae or even just a wiki page for each board/device with tables for OS, release,  kernel and support would go some way to satisfying this requirement.

 

Edited December 7 by AaronNGray

[Igor]

8 minutes ago, AaronNGray said:

just

 

Providing a tool means nothing,

 

What about the most important part, organising and leading?

 

- regularly promoting that we have this and how this operate

- establish and maintaining a team of people that will do that. developers don't have time for this and will mainly not cooperate

- you need to talk with those people on regular basis

- you need to contact them 3 weeks before you need feedback, and a week before to remind them what you are looking for

- you need to thank them for helping you

- this should not be 1 out of 100 projects you are running, but a project you do in your free time.

... and on and on.

 

This is not technical problem, its organisational one.

[AaronNGray]

I have run CPU stress tests starting at 33.3 degrees C, with 5V (12V) fan on top of metal case.

`stress --cpu 4` stabalizing at 55.4 degrees C and `stress --cpu 8` reachiing 57.3 degrees C, with a fan on top of the device and watched the loading and temperature with `armbianmonitor -m` and it seems quite stable.

 

Now have run `stress --verbose --hdd 1` for 20 minutes so far reaching 45.3 degrees C, stabalizing a 44.4 degrees C.

 

run `stress --verbose --hdd 2` for 20 minutes so far rising to 56.4 degrees C, CPU clk 1800/2352 MHz, load about 2.63 max.

 

Waiting for temperature to reduce to 33 degrees C then runnig `stress --verbose --hdd 4` 

 

`stress --verbose --hdd 4` is sky rocketting in 5 minutes temperature hitting 62.8 degrees C, CPU clk 1800/2352 MHz, load about 5.00 max.

 

Okay fallen over after 8 minutes, inconclusive results as a result : (

 

I rerun this test, okay its not temperature it time and load as is susspected, but load and temperature are also a crashing factor for the BananaPi-M7.

 

I would like to monitor the temperature of the NVME.

 

Need to take in to account loading and kernel versions ....., I was getting this happen in less than 5 minutes on Bookworm without a GUI with `stress --hdd 4`

 

I cannot seem to do a select all or highlight more than a pages worth of console text on Gnome, otherwise I would have provided detailed results. This process and instrumentation would be good to automate. I got some DS18B20 TO92 Digital Temperature Sensors which I would like to be able to write a script to enable internal and external case monitoring.

 

I need to run this on debian again but need a GUI to perform the tests without writing a script to combine `stress` and `armbianmonitor -m`

 

Do we have a kernel version that was though to be stable ? Also it might be worth looking at OrangePi 5 and see what results that community is getting.

Edited December 7 by AaronNGray

[eselarm]

Ii your first log with 6.1.75 kernel I see:

[   48.606684] nvme nvme0: controller is down; will reset: CSTS=0xffffffff, PCI_STATUS=0x10
[   48.606692] nvme nvme0: Does your device have a faulty power saving mode enabled?
[   48.606694] nvme nvme0: Try "nvme_core.default_ps_max_latency_us=0 pcie_aspm=off" and report a bug
[   48.643584] nvme0n1: I/O Cmd(0x2) @ LBA 0, 8 blocks, I/O Error (sct 0x3 / sc 0x71) 
[   48.643602] I/O error, dev nvme0n1, sector 0 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2
[   48.666942] nvme 0000:01:00.0: enabling device (0000 -> 0002)
[   48.667012] nvme nvme0: Removing after probe failure status: -19
[   48.680284] Buffer I/O error on dev nvme0n1, logical block 0, async page read
[   48.680286] nvme0n1: detected capacity change from 3907029168 to 0

I recognise 'aspm' from RPi5 related PCIe, but don't know more. But you might look into that.

 

My NanoPi-R6C with alu casing goes to 76 degrees when room 20degres and kernel compile in armbian build. If I put metal can with water on it 47 degrees or so. No fan,  Samsung 970 512G PCie 2 x1 NVME. U-boot, BootFAT is on eMMC. Only GMAC ethernet.

 

I currently use 6.1.84 vendor, but if troubles I will boot with 6.12.2 mainline. I have no trouble with NVME or PCIE, only get reboots if I mess with power supply like adding meters or using cheap USB-C PB DC/DC from 12V car battery.

 

Your NVME is PCIe3x4 I think, it is more demanding I think, I am not sure how PD operates on RK3588 boards.  vendor and mainline might differ, My understanding is vendor can negotiate >5V, mainline not yet. But I am guessing now. You should put USB-C PD analyser on the powerline for actual status.

[AaronNGray]

$ uname -a
Linux bananapim7 6.1.75-vendor-rk35xx #1 SMP Tue Nov 12 08:48:32 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
 

[AaronNGray]

@eselarm Many thanks I will follow these leads up either tomorrow if I get time or next week if not.

 

https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/6/html/power_management_guide/aspm

 

Okay I need to try 6.1.84-vendor then ! 

 

Ordered a USB-C PD analyser, many thanks !

Edited December 7 by AaronNGray

[AaronNGray]

@Igor It would be lovely to have temperature monitoring on the System Monitor Resources page !

[AaronNGray]

I turned off the errors readonly mode off in `/etc/fstab` and its running along the stress --verbose --hdd 4` test as suspected. I am wondering if errors can be detected and acted upon in a different manor by the filing system ? I may try btrfs as amother test when I can do so.

[eselarm]

Btrfs can detect and even correct block errors when dup or raid1 profile is used, but from the dmesg kernel log you can see that the whole NVME storage device disappears.

 

I use 'btop' CLI tool via ssh, shows temperature as well.

 

It is no guarantee that with 6.1.84 it works. If it is {old) SW implementation in the vendor code for PCIe as igor hints, then mainline kernel might be a better test/try. If power issue then I don't know.

 

I used 6.10.10 kernel with ubuntu 22 KDE neon image as test, very well working GPU accel operation. Now there is noble based I see in downloads, but I use headless and bookworm based(don't need GUI/HDMI).

Edited December 8 by eselarm

[going]

15 часов назад, AaronNGray сказал:

I turned off the errors readonly mode off in `/etc/fstab` and its running along the stress --verbose --hdd 4` test as suspected.

Why do you use a stress test for NVME?

Do you want its resource to run out faster?

 

You can find out the status\information using the smartctl program.

Example:

old disk

sudo smartctl --all /dev/sda
...
=== START OF INFORMATION SECTION ===
Device Model:     SSD 512GB
Serial Number:    YS202010005170AA
...
ATA Version is:   ACS-2 T13/2015-D revision 3
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
...
175 Program_Fail_Count_Chip 0x0032   100   100   050    Old_age   Always       -       117440512
176 Erase_Fail_Count_Chip   0x0032   100   100   050    Old_age   Always       -       2175044
177 Wear_Leveling_Count     0x0032   100   100   050    Old_age   Always       -       6113508
178 Used_Rsvd_Blk_Cnt_Chip  0x0032   100   100   050    Old_age   Always       -       1
181 Program_Fail_Cnt_Total  0x0032   100   100   050    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   050    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   050    Old_age   Always       -       22
194 Temperature_Celsius     0x0032   100   100   050    Old_age   Always       -       40
195 Hardware_ECC_Recovered  0x0032   100   100   050    Old_age   Always       -       117242
...

 

a new disk

sudo smartctl --all /dev/sdb
...
=== START OF INFORMATION SECTION ===
Device Model:     SSD 128GB
Serial Number:    YS20211100077037
...
ATA Version is:   ACS-2 T13/2015-D revision 3
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
...
175 Program_Fail_Count_Chip 0x0032   100   100   050    Old_age   Always       -       0
176 Erase_Fail_Count_Chip   0x0032   100   100   050    Old_age   Always       -       0
177 Wear_Leveling_Count     0x0032   100   100   050    Old_age   Always       -       33617462
178 Used_Rsvd_Blk_Cnt_Chip  0x0032   100   100   050    Old_age   Always       -       0
181 Program_Fail_Cnt_Total  0x0032   100   100   050    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   050    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   050    Old_age   Always       -       19
194 Temperature_Celsius     0x0032   100   100   050    Old_age   Always       -       40
195 Hardware_ECC_Recovered  0x0032   100   100   050    Old_age   Always       -       0
...