NVME drive not recognized anymore after recent kernel upgrade

[Armling]

After a recent kernel upgrade on a nanopi-r5s, the NVME device files /dev/nvme0* are no longer present. The relevant part of /var/log/syslog shows:

 

2025-07-05T20:55:35.716376+02:00 nanopi-r5s kernel: [    0.000000] Linux version 6.12.35-current-rockchip64 (build@armbian) (aarch64-linux-gnu-gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #1 SMP PREEMPT Fri Jun 27 10:11:46 UTC 2025
...
2025-07-05T20:55:35.719736+02:00 nanopi-r5s kernel: [    4.491538] nvme nvme0: pci function 0002:01:00.0
2025-07-05T20:55:35.719739+02:00 nanopi-r5s kernel: [    4.491584] nvme 0002:01:00.0: enabling device (0000 -> 0002)

 

There is a single NVME drive installed and the output of a previous kernel where /dev/nvme0* was present is as follows.

 

2025-07-05T02:17:37.940170+02:00 nanopi-r5s kernel: [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x412fd050]
2025-07-05T02:17:37.954269+02:00 nanopi-r5s kernel: [    0.000000] Linux version 6.12.35-current-rockchip64 (build@armbian) (aarch64-linux-gnu-gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #1 SMP PREEMPT Fri Jun 27 10:11:46 UTC 2025
...
2025-07-05T02:17:37.969044+02:00 nanopi-r5s kernel: [    4.406406] nvme nvme0: pci function 0002:01:00.0
2025-07-05T02:17:37.969049+02:00 nanopi-r5s kernel: [    4.406455] nvme 0002:01:00.0: enabling device (0000 -> 0002)
2025-07-05T02:17:37.969053+02:00 nanopi-r5s kernel: [    4.540002] nvme nvme0: D3 entry latency set to 10 seconds
2025-07-05T02:17:37.969074+02:00 nanopi-r5s kernel: [    4.628689] nvme nvme0: allocated 64 MiB host memory buffer.
2025-07-05T02:17:37.969080+02:00 nanopi-r5s kernel: [    4.737972] nvme nvme0: 4/0/0 default/read/poll queues

 

The contents of /etc/armbian-image-release currently are as follows:

 

# PLEASE DO NOT EDIT THIS FILE
BOARD=nanopi-r5s
BOARD_NAME="NanoPi R5S"
BOARDFAMILY=rockchip64
BUILD_REPOSITORY_URL=https://github.com/armbian/build
BUILD_REPOSITORY_COMMIT=f149a11b4
LINUXFAMILY=rockchip64
ARCH=arm64
BOOT_SOC=rk3568
IMAGE_TYPE=nightly
BOARD_TYPE=csc
INITRD_ARCH=arm64
KERNEL_IMAGE_TYPE=Image
KERNEL_TARGET=current,edge
KERNEL_TEST_TARGET=current
FORCE_BOOTSCRIPT_UPDATE=
FORCE_UBOOT_UPDATE=
OVERLAY_DIR="/boot/dtb/rockchip/overlay"
VENDOR="Armbian_community"
VENDORCOLOR="247;16;0"
VENDORDOCS="https://docs.armbian.com"
VENDORURL="https://github.com/armbian/build"
VENDORSUPPORT="https://community.armbian.com/"
VENDORBUGS="https://github.com/armbian/community/issues"
BOOTSCRIPT_FORCE_UPDATE="no"
BOOTSCRIPT_DST="boot.cmd"
VERSION=25.8.0-trunk.245
REVISION=25.8.0-trunk.245
IMAGE_UUID=bef97f49-0290-4f2c-9d7a-0229540b307a
INSTALLED=true

 

The contents of /etc/armbian-release currently are as follows:

 

# PLEASE DO NOT EDIT THIS FILE
BOARD=nanopi-r5s
BOARD_NAME="NanoPi R5S"
BOARDFAMILY=rockchip64
BUILD_REPOSITORY_URL=https://github.com/armbian/build
BUILD_REPOSITORY_COMMIT=034e9253c
LINUXFAMILY=rockchip64
ARCH=arm64
BOOT_SOC=rk3568
IMAGE_TYPE=nightly
BOARD_TYPE=csc
INITRD_ARCH=arm64
KERNEL_IMAGE_TYPE=Image
KERNEL_TARGET=current,edge
KERNEL_TEST_TARGET=current
FORCE_BOOTSCRIPT_UPDATE=
FORCE_UBOOT_UPDATE=
OVERLAY_DIR="/boot/dtb/rockchip/overlay"
VENDOR="Armbian"
VENDORCOLOR="247;16;0"
VENDORDOCS="https://docs.armbian.com"
VENDORURL="https://www.armbian.com/"
VENDORSUPPORT="https://forum.armbian.com"
VENDORBUGS="https://www.armbian.com/bugs"
BOOTSCRIPT_FORCE_UPDATE="no"
BOOTSCRIPT_DST="boot.cmd"
VERSION=25.8.0-trunk.319
REVISION=25.8.0-trunk.319
BRANCH=6.1.0

 

Unfortunately, I don't have a version of the file when the NVME device files were still present. If I remember correctly, the working version was 25.8.0-trunk.313

https://github.com/armbian/os/tags

 

Strangely, the current version 25.8.0-trunk.319 shows "(Ubuntu 13.2.0-23ubuntu4) 13.2.0" in the kernel boot output whereas the previous version (probably 25.8.0-trunk.313) shows "(Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0", i.e., a later version.

 

Is there a way to try version 25.8.0-trunk.313 again or figure out what might be the problem with the current version?

Edited Saturday at 09:58 PM by Armling

[Werner]

Providing logs with 

armbianmonitor -u

 helps with troubleshooting and significantly raises chances that issue gets addressed.

[Armling]

I can do that but it does not provide any useful information on the subject in my opinion.

 

Is there a way to downgrade to 25.8.0-trunk.313?

 

According to this post, it might be useful to set the kernel option nvme_core.default_ps_max_latency_us resp. the DT overlay parameter pciex1_gen=3.

How can the kernel (cmdline) option be set in Armbian? Regarding the DT overlay parameter pciex1_gen, should I follow this part of the documentation?