Does the Orange Pi 6 Plus NPU work natively with Ollama, or do models need conversion? Also, experiences running larger LLMs with its high RAM?

[lucass]

Hey everyone, I'm eyeing the Orange Pi 6 Plus for some edge AI projects, given its 12-core CIX SoC, up to 28.8 TOPS NPU, and massive RAM options (16/32/64GB LPDDR5). Has anyone gotten Ollama running on it with NPU acceleration? Does it support it out of the box, or do you need to convert models (e.g., to INT4/INT8 formats) using custom tools like rkllama or similar?

Additionally, with the higher RAM, can it handle bigger LLMs (like 13B+ models) more smoothly than lower-spec SBCs? Any benchmarks or tips on setups (e.g., Ubuntu/Debian installs, frameworks like MLC-LLM)? I'd love to hear real-world experiences—thanks!