Expert Opinion

Building the Missing Half

Open models are ready for agents, but the infrastructure around them isn't. Otari is a gateway that restores what hosted APIs quietly handle, like web search and code execution, so you keep control of your providers, credentials, and sandboxes while your app sees the same tool interface.

David de la Iglesia Castro

Oct 13, 2026 — 2 min read

Back in July, I argued that open models are ready for agents but their APIs aren't. Migrating our agent app off a proprietary frontier API meant rebuilding a surprising amount of underlying infrastructure around the model itself. This piece dives into the part I didn't cover back then: what it actually takes to craft that missing half.

The goal

Our goal is “straightforward”: building on open models should feel just as seamless as using closed alternatives.

The hardest parts aren't the obvious feature gaps people talk about; they are the subtle, invisible functions hosted APIs quietly perform in the background that your code unconsciously relies on.

We built Otari as the gateway layer to bring those missing capabilities back, sitting neatly between your application and whatever model host you choose.

Why open

Consider web search as an example. When enabled in Claude, queries execute somewhere behind the scenes on a search engine operating under terms you never directly agreed to. You can't select the underlying provider, track what data gets retained, or swap services if domain results deteriorate. The same applies to code execution: user files run inside sandboxes you didn't configure, hosted in locations you can't verify.

With Otari, these remain your architectural decisions. Web queries route through services you explicitly configure using your own credentials and contracts. Code runs within sandbox environments you specify, governed by an open published protocol that anyone can implement. Your application experiences the exact same tool interface, but you retain full visibility and ownership.

What's next

While there are still things to implement and polish, we’ve already managed to do this in octonous.com, which has been an important milestone.

The remaining technical challenges might not be obvious if you don’t build around different frontier APIs: context compaction, prompt caching, and persistent memory that developers can inspect and migrate across models instead of locking them inside proprietary silos.

The key benchmark we evaluate is whether an agent designed for commercial frontier APIs can point directly at open models through Otari and operate flawlessly.

Feel free to open an issue in our repository on GitHub to share your findings.

Mistral Large 4 is live in Otari

Mistral Large 4 is live in Otari from day one. Route to Mistral's 1T-parameter MoE model with one string change, and keep the policies, traces and cost tracking you already run in Otari, Mozilla AI's open-source gateway and policy layer.

You Don’t Just Need Better AI. You Need Better Infrastructure.

Coding agents now edit repos, run tests, and open pull requests, but a rule in AGENTS.md is an instruction, not a constraint. As we delegate more, we need limits that hold even when model judgment fails, a record of every decision, and open infrastructure we own, so rules outlive any agent.

Benchmarking Local LLM Servers: llama.cpp, llamafile, LM Studio, and Ollama

Benchmarking Local LLM Servers evaluates llama.cpp, llamafile, LM Studio, and Ollama across Mac, Linux, and Steam Deck. The study reveals build flags and configurations drive up to 63% performance gains, whereas underlying engines perform similarly due to a shared llama.cpp core.

Introducing Agent Skills in Octonous

Agent Skills in Octonous offer a central, reusable library for your context, guidelines, and procedures. Written in standard Markdown, Skills end repetitive setup. Store team knowledge once, share it across your workspace, and seamlessly attach it to automated tasks or live conversations.