llamafile v0.10.5

llamafile v0.10.5 is out, and it tracks a much more recent llama.cpp, which lets it run two models people have been using lately: Ternary Bonsai 27B and Poolside's Laguna-S-2.1. Both already existed as GGUF weights, but older llamafile builds couldn't load them, because the bundled llama.cpp predated support for their architecture and quantization. This release also includes documentation fixes covering the help system, the different release binaries, and GPU support.

Two big-sounding models that run locally

Both new models are interesting for the same reason, and they get there two different ways. Each one sounds too large for a laptop, but isn't, because of how it's built.

Ternary Bonsai 27B is a compressed build of Qwen3.6-27B from PrismML. Its weights are ternary, so each one is constrained to {-1, 0, +1}, which works out to roughly 1.58 bits per weight instead of 16. The result is a 27B model that lands around 6GB on disk and is meant to run on a laptop while keeping most of the quality of the full-precision base. It's also multimodal, though the vision tower is only loaded when you actually pass it an image. Weights and details are on Hugging Face.

Laguna-S-2.1 is Poolside's open-weight coding model. It's listed at 118B parameters, which sounds like a server model, but it's a Mixture-of-Experts: 118B total, 8B active per token. Generation cost tracks the 8B that fire on each token, not the full 118B, so a quantized copy fits in consumer memory and runs at usable speeds. It's built for agentic coding and long sessions, with a large context window. Worth noting: the quantized checkpoints are configured for 256K context rather than the full 1M that the BF16 weights support.

Speed depends on your quantization and hardware, but both are fast enough to use interactively, which a year ago was not a given for a 27B or a 118B on a personal machine, meaning a laptop or desktop with enough RAM rather than a server or a rented GPU.

Why these models showed up now: the boring part

llamafile can only run the models its embedded llama.cpp understands, and llama.cpp moves fast, so the real bottleneck on model support in llamafile is keeping up with upstream. In v0.10.5, we ran three syncs in two weeks, advancing the embedded llama.cpp across three upstream builds, and improved the agent skill that drafts those sync PRs midway through, so later syncs needed fewer manual passes before merging. This turnaround will help us get you new models sooner after release instead of months.

This pins a tested checkpoint for llama.cpp support, not same-hour parity with an upstream that keeps moving. We expect to have fewer corrective commits and smaller diffs over the next few releases.

For a project whose whole value is packaging, this sync work matters more than the two models it happens to unlock. These models are proof it's working.

Packaging and docs

One packaging note: transcribefile, the self-contained speech-to-text executable built on transcribe.cpp that we introduced back in June, now ships as a release artifact thanks to a community contribution. Nothing new about the capability, you just no longer have to build the binary yourself to get local STT.

On docs, this release adds coverage of the new help system and CLI arguments. Community contributions cleaned up a couple of longstanding confusions too: the difference between llamafile, llamafile-thin, and the other release binaries, and current GPU support including the Vulkan backend. Thanks to everyone who filed and fixed these.

Get it

Grab v0.10.5 from the releases page. If something breaks, an issue or a discussion is the fastest way to reach us.