Console

Garlic LLM runs Qwen3.5-9B on original PlayStation 4

By One Percent LowsCovered by 2 outlets
★ Add us as a preferred source on Google

TL;DR

Leonardo Alegría developed Garlic LLM to run a 9-billion-parameter Qwen3.5-9B model on the 2013 PlayStation 4 at speeds up to 39 tokens per second.

Developer Leonardo Alegría created Garlic LLM, an open-source application that runs the 9-billion-parameter Qwen3.5-9B model natively on the original 2013 PlayStation 4. The application runs directly on the console's AMD GCN GPU without a Linux installation or an external computer, Alegría says.

To fit the model within the PlayStation 4's 8 GB of shared GDDR5 memory, Alegría quantized the model to approximately 4.5 GiB. This includes 4-bit precision for most weights, 3-bit quantization for larger MLP matrices, and a 4-bit format for the KV cache. Alegría used custom OpenCL C GPU kernels and manually constructed PM4 command buffers to communicate with the GPU.

The setup supports a 16,000-token context window, which Alegría describes as modest and better suited for general AI chat or basic tool tasks than complex programming. Enabling speculative decoding, which uses the Multi-Token Prediction block in Qwen3.5 to forecast up to two tokens ahead, increases generation speeds from roughly 22 tokens per second to between 31 and 39 tokens per second. Alegría says speeds occasionally reach 43 tokens per second during coding tasks.

More on PlayStation