Unsloth Desktop adds AMD Ryzen AI NPU support for 35B models
★ Add us as a preferred source on GoogleTL;DR
Unsloth Desktop now runs local AI models on AMD XDNA 2 NPUs under Windows 11 and Linux, letting agent apps use hardware acceleration without touching the GPU.
AMD and Unsloth announced that Unsloth Desktop now supports local NPU inference on AMD XDNA 2 engines, marking the first time the app drives a neural processing unit. The update connects local language models to more than 100 agent apps, including GitHub Copilot and LangChain, via an OpenAI-compatible API powered by Lemonade and FastFlowLM.
Supported hardware includes Ryzen AI 300, Ryzen AI Max, and Ryzen AI 400 chips on Windows 11 and x64 Linux. Linux users must use kernel 7.0 or newer, the amdxdna driver, an unlimited memlock limit, and current firmware. Models must use the Q4NX 4-bit format, which requires conversion through Unsloth or AMD before loading. Unsloth reports that a 35B mixture-of-experts model runs at 17 tokens per second, while a dense 27B model runs at 2 tokens per second due to memory bandwidth limits. Unsloth claims NPUs consume two to four times less energy per token than GPUs, though performance depends on active power modes.
More on AMDAI in gaming and hardwareLinux gaming
Via Unsloth ↗ · Also from Hardware Busters ↗ · Other coverage: Phoronix ↗, VideoCardz ↗
Spot a mistake? Tell us



