Garlic LLM Runs Qwen3.5-9B on a Jailbroken PS4 at Up to 43 Tokens per Second
- Cargo
- Machine Learning
- Shipped
- Shipper
- Shane Downing
- Readers
- 917
Garlic LLM runs a quantized 9B model offline on PS4 hardware, using custom GPU kernels and speculative decoding to reach up to 43 tokens per second.

A 2013-era PlayStation 4 can run Alibaba’s Qwen3.5-9B locally, with Garlic LLM reporting 31 to 39 tokens per second for text chat and up to 43 tokens per second for code. Developer Leonardo Alegría’s project gets there by shrinking the model to fit the console’s shared memory and sending compute work directly to its GPU. It’s a striking demonstration of what carefully tuned inference can do on old hardware, though the reported speeds come without a specified prompt or measurement method.
Garlic LLM is an offline application for a jailbroken PS4. It requires GoldHEN, a homebrew enabler for exploited consoles, and Alegría tested it on a base or Slim PS4 running firmware 11.52. The free package is a 5.66GB download available on GitHub and Hugging Face. Installation needs about 6GB of free space, or 12GB if the package is first copied to the console’s internal storage. The application itself doesn’t contain an exploit.

Fitting a 9B model into PS4 memory
The PS4 has 8GB of shared memory for its CPU and GPU. Storing nine billion model parameters at eight bits each would require at least 9GB, before accounting for other inference data. Garlic LLM instead uses a mixed quantization scheme: most weights are stored at four bits, while the two largest matrices in each layer use three-bit weights, which average about 3.5 bits once scaling is included. The resulting model weights occupy roughly 4.5GiB, compared with about 4.9GiB if all weights used four bits.
The attention, or KV, cache is also quantized to four bits. That makes a 16K-token context fit, well below the model’s native 256K-token context. Speech features add to the memory footprint: the Parakeet dictation model takes 0.33GiB, while the Chatterbox voice models use 0.29GiB and 0.09GiB. Together, the listed components total 5.23GiB, or 5.62GB, close to the downloadable package’s 5.66GB size.
The name Garlic refers to the memory the PS4 GPU reads directly; the related memory type is called “Onion.” That distinction reflects the project’s design: the model weights stay in GPU-accessible memory while the CPU manages the application and prepares inputs. The constrained memory budget shapes both the quantization choices and the usable context length, so the headline model size alone doesn’t describe what the console can hold during inference.
Custom GPU work instead of a standard inference stack
Garlic LLM doesn’t appear to use llama.cpp or a Vulkan driver port. Instead, its matrix multiplication and other model operations run as OpenCL C kernels compiled with LLVM for AMD’s Graphics Core Next architecture. The app builds its own PM4 command buffers, the low-level instructions a graphics driver would typically prepare, and submits them to the GPU.
The processing path starts with the prompt being tokenized on the CPU. For each generated token, the application assembles GPU commands and runs the GCN kernels; the CPU then handles the interface, text display and audio. The project is built with the open-source OpenOrbis PS4 toolchain. This direct approach gives the developer control over the console’s GPU path, while also leaving the project responsible for work commonly handled by a driver and a general-purpose inference engine.
A related effort, PS5LM, runs llama.cpp natively on PS5 and accesses its AMD GPU through PS5_Vulkan, a port of the RADV driver. That gives the PS5 project a different software foundation. Garlic LLM’s hand-built route is tailored to the PS4’s GPU and system software, and the application does not support PS5.
Speculative decoding and speech features
Qwen3.5-9B includes a multi-token prediction head, which Garlic LLM uses to draft as many as two tokens ahead before the full model checks them. This form of speculative decoding can reduce the work needed to produce a sequence when the drafts are accepted. Alibaba’s vLLM guidance for the model also uses two speculative tokens. Alegría credits the feature with helping the model reach its reported chat speed on the older console.
Voice interaction adds more models and more trade-offs. Garlic LLM can produce spoken replies in seven languages using Chatterbox, with chat generation reported at about 22 tokens per second in this mode. Holding a controller button enables dictation through Nvidia’s Parakeet TDT 0.6B v3, which supports 25 European languages. The text-generation figures therefore shouldn’t be treated as the speed of a complete voice interaction: speech input and output add separate processing steps.
How to read the performance figures
The reported 31 to 39 tokens per second for chat and peak of 43 for code suggest that the console can produce useful short responses at a steady pace. At the midpoint of the chat range, 35 tokens per second, a 300-token answer would take a little under nine seconds to generate, excluding prompt processing and any speech work. The project doesn’t provide the prompt details or test method behind its speed figures, so comparisons should be treated as indicative rather than controlled benchmark results.
Temperature is set to 0.7 by default. Alibaba recommends that setting for general tasks with thinking turned off, and 1.0 when thinking is on; the project’s demo video instead uses 0, which makes output more predictable. The distinction matters when comparing generations, since sampling settings can affect both the wording and repeatability of responses.
For context, a PS5 reportedly ran the larger Qwen3.8-27B at about 20 tokens per second, while Gemma 4 E4B and Llama 3.2 3B reached 61 and 124 tokens per second on that console. Those results involve different models and hardware, so they don’t establish a direct efficiency ranking. They do show the familiar trade-off: a larger model may offer different capabilities, while smaller models can generate faster.
The PS4 Pro hasn’t been tested, and the developer is seeking user reports on GitHub. For now, Garlic LLM’s practical reach is limited to compatible jailbroken PS4 systems and its 16K-token context. Its clearest contribution is technical: a custom GPU path, aggressive quantization and speculative decoding can make a 9B model usable on hardware designed long before local language models became a common workload.