📘+🐊 Blueprint Releases & Lumigator's Latest Features

Mozilla.ai

Mar 25, 2025 — 4 min read

Hello Developers & AI Enthusiasts,

Welcome to the first Mozilla.ai Update. We're thrilled to share the latest developments with you, including updates on our Blueprints and Lumigator toolkit. You'll also find community highlights and information about upcoming events where we can connect and collaborate.

Blueprint Releases

For the unfamiliar, Blueprints are customizable workflows that empower developers to build AI applications with open-source tools and models. Check out our newly revamped Blueprints Hub to see all the latest updates.

New Releases

Finetuning an LLM using Federated AI – This Blueprint demonstrates how to finetune LLMs using Flower, a framework for federated AI – an approach where models are trained collaboratively across multiple devices or servers without sharing raw data. It includes an example of fine-tuning the Qwen2-0.5B-Instruct model on the Alpaca-GPT4 dataset. Learn more about this blueprint here.
Open Street Maps AI Helper – This Blueprint helps you use object detection and image segmentation models to identify and map features in OpenStreetMap. It includes code for running inference on satellite images, validating results, and submitting contributions to OSM. Learn more about this blueprint here.
Bring your own timeline algorithm (BYOTA) – This blueprint introduces an approach to personal, local timeline algorithms that people can either run out-of-the-box or customize. It enables you to visualize, search, and re-rank posts from the fediverse without any of them leaving your computer. Learn more about this blueprint here.

Upcoming Blueprints

Keep an eye out for our upcoming blueprints in development:

Slides Q&A – Enabling you to query presentation slides using natural language.
OCR for Library Collections – Providing a workflow for accurate OCR on pdfs at scale (developed in collaboration with EleutherAI).
Surf Spot Finder – Enabling you to build an AI Agent to solve a specific task, for example, finding the best surf spot near you.
Document to Speech – Building on top of Document to Podcast, this blueprint transforms your written documents into natural-sounding audio.

Explore all our blueprints on our Blueprints Hub!

Lumigator Latest release (v0.1.2-alpha)

Lumigator is Mozilla.ai’s tool for evaluating language models, helping you compare different models, understand their strengths, and pick the right one for your needs. Here's what's new:

New Features

Easier Dataset Handling – You can now preview, search, and export datasets directly in the UI.
Better Security – Manage API keys and stored secrets more easily with the Secret Management API, making it simpler to connect external models or services.
Translation Evaluation – We’ve added BLEU and COMET, two widely used metrics for assessing AI-generated translations. While BLEU checks similarity to reference translations, COMET provides a more nuanced evaluation by considering contextual accuracy and fluency. Soon, this metric will be available in the UI as well.
LLM-as-a-Judge Metrics – Lumigator now offers G-Eval metrics for Summarization tasks, an AI-driven evaluation approach that functions like a human reviewer. It assesses the coherence, consistency, fluency, and relevance of generated responses, providing a more comprehensive evaluation.
More Models – Lumigator now supports any model with an OpenAI-compatible API, thanks to LiteLLM. This means you can evaluate local models, cloud models, or hosted APIs, whichever setup fits your needs.

Documentation Updates

New user guide on preparing datasets for summarization evaluation.
Improved README with details on model caching and secret management.
Updated API reference with the latest changes.

Infrastructure & CI Improvements

Model Caching – Faster startup times for first-time runs.
Better Local Dev Workflow – Improved Docker handling to prevent orphaned containers.
Deployment Enhancements – Updated Docker and Helm configurations for seamless multi-architecture support.

Try out the latest version and let us know what you think!

From the Blog

Stay up-to-date with our latest insights and deep dives:

Blueprint Deep Dive: Turn Documents into Podcasts Locally with Open-Source AI – Learn how to transform written content into audio using entirely open-source tools.
Evaluating DeepSeek V3 with Lumigator – See how we tested DeepSeek's latest model using our evaluation framework.
Running an open-source LLM in 2025 – Practical insights on deploying and using open-source language models today.

Looking for more? Check out our full blog archive for additional technical articles and insights.

Community Highlights

A community member extended our Document-to-Podcast blueprint with 'Markitdown' for document pre-processing, making it compatible with even more document types.
In addition, many community members are working on innovative new blueprints like a Deepfake Image Detector and a LLM Q&A Trainer. Keep an eye out as they are finalized and launched on the blueprint hub.

Events Recap & Upcoming

FOSDEM 2025 Highlights

The Mozilla.ai team had a fantastic time at FOSDEM 2025, connecting with many of you from the open-source community. We presented our latest products: Blueprints hub, for streamlined AI development workflows (watch the video) and Lumigator, for model evaluation (watch video here). The conversations at our stand confirmed what we've thought: There's a real hunger for trustworthy, transparent, and truly open AI solutions in the community. Thanks to everyone who stopped by to chat with us, your feedback and enthusiasm are what drive our work forward.

Blueprints Brainstorm Session (March 26th)

Join us on Wednesday, March 26th, at 1pm EST, on Discord, for a collaborative session where we'll explore new ideas, discuss potential improvements to existing blueprints, and help community members develop their concepts into full-fledged implementations.

Vertical AI Agents w/ Mozilla.ai (April 8th)

Join us at Cloudflare’s office in Lisbon to discuss AI Agents verticals, evaluation and practical deployments with the local community.

Get Involved

We're building a vibrant community around open-source AI, and we'd love to have you join us!

Contribute to Lumigator – Help improve our evaluation toolkit by refining evaluation methods, enhancing documentation, or developing new integrations. Check out our CONTRIBUTING.MD to get started.
Build a Blueprint – Have an idea for a new AI workflow? Join our Blueprint Brainstorm Session on March 26th, or share your concept in our Discord.
Connect With Us – Jump into the AI @ Mozilla Discord to share what you're working on, get help, and become part of our growing community.

Whether you're interested in evaluating models, creating new AI applications, or simply connecting with like-minded developers, there's a place for you in the Mozilla.ai ecosystem

Happy coding!

The Mozilla.ai Team