Technical Content
Benchmarking Local LLM Servers evaluates llama.cpp, llamafile, LM Studio, and Ollama across Mac, Linux, and Steam Deck. The study reveals build flags and configurations drive up to 63% performance gains, whereas underlying engines perform similarly due to a shared llama.cpp core.
Agent Skills in Octonous offer a central, reusable library for your context, guidelines, and procedures. Written in standard Markdown, Skills end repetitive setup. Store team knowledge once, share it across your workspace, and seamlessly attach it to automated tasks or live conversations.
Open source isn't a moral choice; it’s an operational strategy to prevent vendor lock-in. As transient AI models change, controlling the orchestration layer that routes and manages them becomes critical. Owning your infrastructure code is the only way to ensure true architectural sovereignty.
Expert Opinion
Discover how a Product Operations Manager uses Octonous to streamline daily workflows, from turning GitHub releases into newsletters to capturing competitor intel from Slack. Save hours on repetitive tasks and focus on the strategic work that really matters.
Product Release
llamafile v0.10.5 is out! Updated llama.cpp core support lets you run two huge models locally: the compressed 6GB Ternary Bonsai 27B and the fast 118B Laguna-S-2.1 coding MoE. This release also fixes docs and adds pre-built transcribefile speech-to-text binaries.
Expert Opinion
AI’s future isn’t about replacing human thinking, but enhancing it. As AI agents evolve from simple assistants into deeply connected systems, control, privacy, and personalisation will shape the next era of AI. Mozilla AI is building towards a more open and user-controlled future.
Expert Opinion
Frontier AI models are becoming more capable, but also more tightly coupled to proprietary state, memory and execution environments. This article explores what that means for developers building open, portable AI systems.
Technical Content
Why would you worry about them? LLMs promise something close to infinite capability, and worrying about the meter feels like someone else's problem. Ship the feature. Let the model think as long as it needs to, right? Right. Until the bill arrives. The shock Picture it: your best
Expert Opinion
Swapping LLMs sounds simple, but managing separate SDKs, credentials, and billing turns model evaluation into an infrastructure headache. Otari solves this with a unified gateway, decoupling application logic from providers so teams can seamlessly route traffic, test new models, and control spend.
Subscribe to get the latest news and ideas from our team
At ACM FAccT, we demonstrated why AI guardrails need the same scrutiny as models. Moving from static policies to context- and language-specific evaluations, our hands-on session proved that agentic guardrails equipped with tools like web search are vital for reliable, real-world deployment.
Open models have reached impressive levels of capability, but the surrounding platform often falls short. This blog explores the hidden infrastructure gaps developers face and why bridging them is essential for production-ready AI applications.
Discover how Octonous helps our team at Mozilla.ai cut through daily busywork, from automatically monitoring our libraries to staying on top of the latest developments in AI safety.
The Model’s the Easy Part - How to Get, and Keep, Value Here’s how I see the evolution of AI in enterprises over the last few years: * Autumn of 2022, the world thinks it’s going into a recession. IT budgets are frozen for 2023. * November 30, 2022: ChatGPT
If you are building LLM-powered applications today, you are probably managing multiple LLM providers, a pile of API keys, and your own logic for routing, budgets, and failovers. Accessing language models is no longer the challenge. Operating LLM infrastructure effectively is. Today, we are excited to launch Otari, the
Meet transcribe.cpp, a new open-source C/C++ speech-to-text inference library with portable, GPU-accelerated support for multiple STT models. Developed through Mozilla.ai's Builders in Residence program, it makes adding fast, local transcription to applications easier than ever.
A look at how we use Octonous inside mozilla.ai to reduce the everyday overhead of product work, from turning Slack feedback into GitHub issues to staying on top of product changes and finding context across the tools where work already happens.
Encoderfile now handles images. Starting with image classification, you can run vision models as a single executable — no Python runtime, no serving infrastructure, just a file path in and a label out.
Runaway agents? Provider outages? Discover why your AI stack needs an LLM control plane, not just a gateway, to handle production routing, budgets, and privacy.
AI coding sessions can feel like a black box. Route OpenCode through the Otari Gateway to track costs, token usage, and model activity in real time. Get budget controls and visibility across every session without changing a single line of application code.
Meet Otari, an open-source LLM gateway powered by any-llm, and Otari.ai, the hosted platform built on the same foundation. Run frontier or open-weights models through one API with usage tracking, budget controls, routing policies, observability, and team management.
Cloud AI pricing changed fast in 2026. This post looks at why more teams are moving back to local models, the tradeoffs behind tools like Ollama and LM Studio, and why portability and ownership are becoming bigger concerns for developers.
