š„³ Marvin's turning 1.0!
It's the lightweight AI engineering framework that you'd probably build yourself (if you had time)
Today weāre releasing Marvin 1.0! This major release is the result of incredible feedback and opportunities to work closely with our developer community over the past few months. Weāre so grateful for their time, attention, and contributions, and this release is truly a community effort.
Check out the docs at askmarvin.ai, give Marvin a star on GitHub, or take 1.0 for a spin by upgrading now: pip install marvin -U
Hello, Marvin!
Marvin is a lightweight framework for building AI-powered software thatās reliable, scalable, and easy to trust. It is designed primarily for AI engineers: users who are deploying cutting-edge AI to build powerful new features and applications. Since our initial release, thousands of developers, engineers, and founders have adopted Marvin to integrate and build with Large Language Models (LLMs). Weāve been humbled by the warm reception.
Sometimes the most challenging part of working with generative AI is remembering that it's not magic; it's software. It's new, it's nondeterministic, and it's incredibly powerful, but it's still software: parameterized API calls that can trigger dependent actions (and just might talk like a pirate). Marvin's goal is to bring the best practices of building dependable, observable software to the frontier of generative AI. As the team behind Prefect, which does something very similar for data engineers, we've poured years of open-source developer tool experience (and a few hard-won lessons!) into Marvin's design.
Developer Experience
Above all else, Marvin is focused on a rock-solid developer experience. It's ergonomic and opinionated at every layer, but also incrementally adoptable so you can use it as much or as little as you like. Itās a Swiss Army Knife, not a kitchen sink. Itās familiar. It feels like the library youād write if you had the time: simple, accessible, portable LLM abstractions that you can quickly deploy in your application, whether youāre doing straightforward NLP or building a full-featured autonomous agent.
Marvin prioritizes a developer experience focused on speed and reliability. It's built with type-safety and observability as first-class citizens. Its abstractions are Pythonic, simple, and self-documenting. These core primitives let us build surprisingly complex agentic software without sacrificing control:
š§© AI Models for structuring text into type-safe schemas
š·ļø AI Classifiers for bulletproof classification and routing
šŖ AI Functions for complex business logic and transformations
š¤ AI Applications for interactive use and persistent state
If our mission sounds exciting to you, please join our community!
Ambient AI
One of our favorite reactions to Marvinās initial release came from Dr. Jim Fan at NVIDIA:
This idea really resonated with us, and Marvin is the first step on a journey to deliver āAmbient AI:ā omnipresent but unobtrusive autonomous routines that act as persistent translators for noisy, real-world data. Ambient AI makes unstructured data universally accessible to traditional software, allowing the entire software stack to embrace AI technology without interrupting the development workflow.
This ideal has the potential to unlock value analogous to the shift from analog to digital music, which revolutionized accessibility and search. Suddenly, vast song libraries were at our fingertips, ready to be explored with a few keystrokes no matter where we were. Ambient AIs can do the same for real-world data and interactions by making them suddenly available to traditional software, without compromise. That is how AI technology can become as pervasive and personalized as music is today.
To play its part, Marvin introduces a set of AI components that integrate seamlessly with the Software 1.0 stack. This lets developers focus on core functionality and delivering value to users, not turning knobs on an LLM. Our goal is to create a future where all software can work in concert with ambient AI receptors. This synergy brings simplicity and stability to AI engineering, allowing every application to effortlessly harness AI technologies and making AI a natural and integrated part of the software landscape.
AI Components
Marvinās core AI abstractions are all packaged as familiar, Pythonic interfaces. These form the building blocks of the Ambient AI framework.
š§© AI Models
AI Models are a drop-in replacement for Pydanticās BaseModel that bring structure to unstructured data. They can be instantiated from any string, allowing them to parse unstructured text into typesafe schemas. AI models are perfect for typed entity extraction, generating synthetic data, and standardization.
š·ļø AI Classifiers
AI Classifiers let you build a multi-label classifier with no code and no training data. They are fast and bulletproof because they use a clever logit bias trick to get results with just a single token.
šŖ AI Functions
AI functions look just like ānormalā functions, but donāt have any source code. Instead, they use AI to predict outputs based on their inputs and a description of the function. Theyāre perfect for complex NLP tasks like entity extraction and qualitative tasks like sentiment analysis. They can also be used for generating synthetic data according to a set of constraints.
š¤ AI Applications
AI Applications are optimized for interactive use cases, where users invoke the application multiple times to achieve a goal (or it loops autonomously!). In addition to tracking a history of user inputs, AI Applications maintain persistent state and are capable of making high-level plans, which keeps them reliably on track. This means that while they can implement āclassicā LLM demos like chatbots, coding assistants, and tool-using agents, they can also implement full-fledged applications for which chat (or unstructured text) might be a surprising interfaceā¦ at least not without a lot of extra coding! This includes role-playing games, technical writing, or even this to-do app:
Marvinās high-level abstractions are powered by a complete AI platform that users may choose to delve into, if itās helpful. At the lowest level, Marvin introduces a drop-in replacement for the openai
library that adds useful features like portable configuration, logging, and other useful utilities.
In addition, Marvin has a powerful prompt DSL inspired heavily by (and developed in collaboration with) our friend Jason Liuās OpenAI Function Call library. Rather than attempting to munge string templates together, Marvin embraces prompts-as-code and allows users to customize LLM interactions in a variety of ways.
What makes Marvin different?
There's no shortage of tools and libraries out there for integrating LLMs into your software. So what makes Marvin different? In addition to a relentless focus on incrementally-adoptable, familiar abstractions, Marvin embraces five pillars:
User-Centric Design
We built Marvin with you in mind. It's not just about what it does, but how it does it. Marvin is designed to be as user-friendly as possible, with a focus on an easy, intuitive experience. Whether you're a coding expert or just starting, Marvin works for you.
Flexibility
Marvin is built to adapt to your needs, not the other way around. Embracing Prefectās core principle of incremental adoption, you can use as much or as little of Marvin as you need. Need a full suite of LLM integration tools? Weāve got you covered. Only need a single component? Marvin can happily do that, too.
Community Driven
Marvin isn't just a tool, it's a community. We value feedback and collaboration from users like you, and Marvin reflects hard-won best practices from folks on the cutting edge of production AI engineering. We're always learning, iterating, and improving based on what the community learns.
Speed
We're serious about speed. We believe that getting started should be quick and easy, and every Marvin-specific keystroke is a penalty you pay for our design. That's why with Marvin, you can get up and running in no time and with almost no dependencies.
Open-Source
Marvin is fully open-source, which means it's not only free to use, but you're also free to modify and adapt as you see fit. The Prefect team has years of open-source experience and is fully committed to supporting Marvin as an open-source product. We believe in the power of collective intelligence, and we're excited to see what you can create.
The road to 1.0
Marvin was born in 2022 as an internal tool at Prefect, powering our early exploration of LLM technologies. In March 2023, we released the first open-source version of Marvin as a ābatteries-included libraryā for building AI-powered software. A few thousand users later, we discovered that users donāt actually want a library that prepackages everything. What they want are clear and effective tools for deploying AI in a manageable way. With that in mind, itās no surprise that AI Models and AI Functions have always been Marvinās most popular features.
Marvin 1.0 leans hard into this discovery by ensuring that every abstraction is rock solid and independent. Thatās how we hope to pursue the goal of truly ambient AI. In addition to a full rewrite, 1.0 introduces two new major components, AI Classifiers and AI Applications, that we expect to become bedrock for a new class of AI-enabled features. However, please note that some 0.x features didnāt meet the standard for inclusion in 1.0, so for some users this may be a breaking release.
If Marvin has a secret weapon, itās that we view its primary job as making AI technology, no matter how complex, available through interfaces that feel familiar to our users. Every Marvin component strives to be future-proof and incredibly useful. As a team, we are committed to making sure that Marvinās DX is solid and we take complete responsibility for continuously adapting it to the ever-changing AI landscape.
We hope you enjoy using Marvin as much as we do!
We still have a number of major features to announce, so please watch this space and @AskMarvinAI for more.
Happy engineering!