NPU and RKLLM support on Rockchip RK3588 (NanoPC-T6) and RK3576 (NanoPi M5)

[fever_wits]

Hello,

I am currently working on a project involving local LLM inference on Rockchip SoCs using the rknn-llm framework. I have two specific questions regarding Armbian support for the NPU:
 1. NanoPi M5 (RK3576): FriendlyElec recently released the NanoPi M5 based on the RK3576. If I use the Armbian image for this board, will I be able to run LLM models using the NPU? I am considering purchasing this board for LLM
    testing and would like to know if the driver support is already there.
 2. NanoPC-T6 (RK3588): I have a NanoPC-T6 running Armbian, but I cannot find the /dev/rknpu device node. I have tested this with both 6.1.115-vendor-rk35xx and 6.12.58-current-rockchip64 kernels, but the NPU device is not appearing.
    Is there a specific overlay or a manual configuration step needed to enable NPU support on these kernels?
 tried adding tags npu and nanopi m5, but there was no such option.
I am happy to provide any additional information, logs (such as `armbianmonitor -u`), or command outputs if needed to help diagnose the issue.
Thank you in advance for your help and guidance!

Best regards,
A.H.
 

[Werner]

7 hours ago, fever_wits said:

but I cannot find the /dev/rknpu device node

npu is there in both vendor bsp kernel 6.1.y. There is no node exposed afaik. Everthing beyond like libraries to use npu is userspace and out of scope of Armbian.

edge 6.18.y also has reverse-engineered npu support with a driver called Rocket.

[fever_wits]

Hello,
I think the problem with not seeing /dev/rknpu is that it is not defined in the rk3588-nanopc-t6.dtb file.
In earlier versions, there was no fan control and with each update I made corrections.
I have a saved dts file from 14.01.2025
 

206220 Dec 28 21:12 rk3588-nanopc-t6.dts
341802 Jan 14 2025 rk3588-nanopc-t6.dts.14.01.2025

There is a big difference between the two. I know that NanoPC-T6 is Community maintained, who can I contact to see what the problem is?
It is possible that I am also making a mistake in trying to run an LLM model.

Best regards,
A.H.

[Werner]

7 hours ago, fever_wits said:

NanoPC-T6 is Community maintained, who can I contact to see what the problem is?

You are community. Contact yourself

As there is no maintainer for this board everyone and nobody the same time is responsible for it. If nobody cares, functionality  will fall apart over time.

[usual user]

The rk3588 NPU support has been working as expected for me for some time now, but I don't use legacy software.

$ ./classification
Loading external delegate from /usr/lib64/libteflon.so with args: {}
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
0.901961: military uniform
0.023529: Windsor tie
0.011765: mortarboard
0.007843: bulletproof vest
0.003922: cornet
time: 15.706ms

$ ./classification
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
0.901961: military uniform
0.023529: Windsor tie
0.007843: bulletproof vest
0.007843: mortarboard
0.003922: cornet
time: 70.607ms

versus

[Werner]

https://patchwork.kernel.org/project/linux-rockchip/patch/20251230-arm64-dts-rockchip-rk3588-npu-enablements-v1-1-d38b182a59e3@pardini.net/ fyi

[fever_wits]

Good morning

@Werner  Thanks for the patch. I'll probably wait for the new kernel and test it. The project is long-term. For the first answer "to address myself" :). I think I understand what you're saying. Thanks :).
If I find a solution to this, I'll share it with the community, as I've been doing so far  
@usual user Thanks for the answer.
Today is the 31st and I'm thinking of not working on servers, happy new year and may the next one be fruitful for all of us.

Best regards,
A.H.

[usual user]

51 minutes ago, fever_wits said:

I'll probably wait for the new kernel and test it.

Waiting for others to do the work is a good strategy. If everything goes well, it will land at the earliest in 6.20, if improvements are needed, 7.x is probably more likely. When the corresponding kernel will be included in the chosen distribution is the next data point. You are clearly aware of what you have to wait for.
Until then, I will simply continue using the necessary modifications as a self-written DTBO, which I have been doing for some time.
But maybe you'll get lucky and someone else takes the burdon to apply the patches as an early adaptation, so you can just sit back and do nothing and still enjoy the upcoming availability.

[Werner]

I think this patch can also be backported to 6.18.y in Armbian if @rpardini didn't plan to do anyway.

[fever_wits]

@usual user In essence, I'm not waiting for someone to do the work for me.
Realistically, if I knew exactly how to do it, I would have done it and described it here.
If I make something that could be useful for someone or the community, I share it.
I rarely write because in most cases I use standard things and there's not much to write about, or I use quite non-standard solutions that for most people aren't solutions.

 

 

[usual user]

1 hour ago, Werner said:

I think this patch can also be backported to 6.18.y in Armbian

Armbian is carrying it for quite some time. So no excuse to not just use it already.

[Werner]

Okay, didn't bother checking beforehand

[fever_wits]

I have the following (maybe stupid) question, what needs to be done to be able to use all the features of the RK3588 by adding the dts file. Example: there used to be /dev/tpm0 which has been missing for several versions. As I said earlier, the latest dts is much shorter.

My question is not an attempt to nitpick, but a serious question. And what can I help with. As in my case, there is a peculiarity. I have a FriendlyElec NanoPC-T6 and it is used in "production" in my network. And I do not have the opportunity to reinstall it.

I would like to make a dts file that allows using the full functionality of the RK3588.

Again, I want to say, I do not want to blame anyone or nitpick, but to find a solution, and whether I can help with anything and if Yes, how and with what.

P.S.
Since today is the 31st, if you want we can postpone the discussion until January 1st or 2nd

 

Edited 1 hour ago by fever_wits