FriendlyElec CM3588Plus NAS - nvme's overheating (75C)

[serverlesslove]

Hi,

 

I'm trying to setup CM3588Plus NAS

3 x 4tb vp4300 nvme drives

 

OpenMediaVault updated.

 

Drives are setup to RAID5 with ext4.

 

the OS is reporting that ssd temps getting to 63-68C while idling, and likes to creep up to 75C where if I'm correct thermal throttling starts to kick in.
the SBC is in open air, no case/enclosure. room temp 21C.

 

How to find the cause and solve it? Amy recommendations setup (change the fs) etc. Thanks a mill.

Edited April 16 by serverlesslove
updated details.

[Werner]

Hi

 

Providing logs with

 

PASTE_SERVER_HOST=paste.armbian.de armbianmonitor -u

 

 helps with troubleshooting and significantly raises chances that issue gets addressed.

 

 

[Sirmalinton]

@serverlesslove, your mount options for the device might not be ideal. If the device is used as a multimedia server, then you can avoid, for example, updating file access times on each read, reducing disk writes and improving performance, using the `noatime` option. Also increase the file system journal commit interval to 120 seconds (instead of the default 5 seconds) using `commit=120`.

 

Check your current `mount` options for the device and try these:
```

defaults,noatime,commit=120,errors=remount-ro
```

 

[Vijay Gill]

@Sirmalinton Are you sure this could be the reason? I have NVME's with btrfs running with following options in fstab

defaults,noatime,nodiratime,compress=zstd,nodiscard,max_inline=0

And temperature is quite acceptable (see screenshot below).

 

I had initially set up my NAS is the standard metallic case which was quite useless. So I got a 3d-printed one with a fan on the top to keep everything cool. See my post in another thread https://forum.armbian.com/topic/48705-cm3588-overlay-for-fan-speed-change/#findComment-214042.

 

Now I don't have to worry about the temperatures at all. 

 

I must also mention that my NVME's have heatsink attached.

 

[serverlesslove]

Core Problem: Drives Stuck in PS0 at Idle

drives remain in Power State (PS): 0 (4.5W consumption) and get burning hot in idle. While copying 500gb of video files over wifi, under low load, temps settled at 35-37C.

---

All three of your NVMe drives (nvme0, nvme1, nvme2) are detected by the kernel.

They are identified as Viper VP4300L 4TB drives.

They are correctly initialized and made operational within your md0 RAID array.

---

The output parm: default_ps_max_latency_us:max power saving latency for new devices; use PM QOS to change per device (ulong) confirms that the nvme_core module recognizes this parameter.

However, this modinfo command only tells us about the parameter itself (its description and type), not the value that is currently active in your running kernel. We already confirmed that active value is 100000 from your previous cat /sys/module/nvme_core/parameters/default_ps_max_latency_us command.

So, the situation remains: Your NVMe drives are staying in Power State 0 (consuming 4.5W each) and getting burning hot, even though the kernel is telling them "No Workload" and the default_ps_max_latency_us parameter is set generously high (100000 µs), which should allow them to enter deep sleep states.

This strongly suggests that something is actively preventing them from entering sleep states, despite the ideal configuration.
---

In the span of a few seconds:

pi@omv:~$ sudo nvme get-feature /dev/nvme0n1 -f 0x02 -H
get-feature:0x02 (Power Management), Current value:0x00000004
       Workload Hint (WH): 0 - No Workload
       Power State   (PS): 4
pi@omv:~$ sudo nvme get-feature /dev/nvme1n1 -f 0x02 -H
get-feature:0x02 (Power Management), Current value:0x00000003
       Workload Hint (WH): 0 - No Workload
       Power State   (PS): 3
pi@omv:~$ sudo nvme get-feature /dev/nvme2n1 -f 0x02 -H
get-feature:0x02 (Power Management), Current value:0x00000004
       Workload Hint (WH): 0 - No Workload
       Power State   (PS): 4
pi@omv:~$ sudo nvme get-feature /dev/nvme0n1 -f 0x02 -H
get-feature:0x02 (Power Management), Current value:0x00000004
       Workload Hint (WH): 0 - No Workload
       Power State   (PS): 4
pi@omv:~$ sudo nvme get-feature /dev/nvme1n1 -f 0x02 -H
get-feature:0x02 (Power Management), Current value:00000000
       Workload Hint (WH): 0 - No Workload
       Power State   (PS): 0
pi@omv:~$ sudo nvme get-feature /dev/nvme0n1 -f 0x02 -H
get-feature:0x02 (Power Management), Current value:00000000
       Workload Hint (WH): 0 - No Workload
       Power State   (PS): 0
pi@omv:~$ sudo nvme get-feature /dev/nvme2n1 -f 0x02 -H
get-feature:0x02 (Power Management), Current value:0x00000004
       Workload Hint (WH): 0 - No Workload
       Power State   (PS): 4

---

sudo nvme id-ctrl /dev/nvme0n1 -H | less
 

ps      0 : mp:4.5000W operational enlat:0 exlat:0 rrt:0 rrl:0
           rwt:0 rwl:0 idle_power:- active_power:-
           active_power_workload:-
ps      1 : mp:2.4000W operational enlat:0 exlat:0 rrt:1 rrl:1
           rwt:1 rwl:1 idle_power:- active_power:-
           active_power_workload:-
ps      2 : mp:0.6000W operational enlat:0 exlat:0 rrt:2 rrl:2
           rwt:2 rwl:2 idle_power:- active_power:-
           active_power_workload:-
ps      3 : mp:0.0250W non-operational enlat:5000 exlat:5000 rrt:3 rrl:3
           rwt:3 rwl:3 idle_power:- active_power:-
           active_power_workload:-
ps      4 : mp:0.0040W non-operational enlat:5000 exlat:25000 rrt:4 rrl:4
           rwt:4 rwl:4 idle_power:- active_power:-
           active_power_workload:-
---
 

Edited 7 hours ago by serverlesslove

[serverlesslove]

I've also checked for good couple minutes with iotop while system was idle, only the rootfs was accessed which is on the internal eMMC.

Next I will investigate more thoroughly using iotop and atop.