Mill: the factory that powers our products.
We never planned to build an agent runtime. We were building Fina, an AI analyst for financial reporting teams, and we had to ship a product while figuring out how to build the agent. So day by day we built a harness anyway. The loop, the streaming, the retries, the tool dispatch, the sandbox. None of it was finance but we needed all of it.
Then more products came and the question got simple. Are we building all of this again every time?
We listed what a new agent product actually needs, and kept all of it native in the runtime:
- An agent loop. Call the model, run the tools it asks for, feed the results back, repeat till it stops asking.
- An LLM provider layer. Streaming, retries, fallbacks, the part that absorbs provider failures.
- Composition. A way to plug in your tools and prompts.
- Compaction. Long runs outgrow the context window, so the transcript has to fold down without losing the thread. The product only wants the final output it can show a user.
- A sandbox. An environment where the agent runs the code it writes.
- Subagents. They keep their own work out of the parent's context window, and can fork the current transcript when they need what the parent already knows.
- Web search. Everyone needs it.
- Guardrails. Duplicate call detection, token budgets, stall detection.
So a product builder writes their tools and the execution behind them, a system prompt, and prompts for the natives, what compaction should conserve, how subagents behave. That's it. Everything else is the runtime's problem. We pulled all of this into one service and called it Mill. A mill is a shared machine, people bring raw material, the mill does the work, the output leaves with the owner.
Stateless, literally
Products bring state to Mill. Mill runs the loop. Products take the result home. Mill keeps no durable state.
One rule runs the whole design, and we mean it literally. Mill has no database. No chat table, no session store, no user model, it never even sees a user ID. Auth is one API key per product. A product opens a websocket and sends everything the turn needs, the full ordered message history, its tool schemas, a model alias. A turn is one product request: Mill runs the loop until the model stops asking, streams events back the whole time, and ends with a snapshot of the new messages. The product persists that snapshot and sends it back whole next turn. The moment the socket closes, Mill has forgotten you.
Sounds like a detail, it's actually the whole trick. Every "where should this live" question now has one answer, in the product. User identity, citations, artifacts, chat history, there is nowhere in Mill to put them so they can't leak in. Runtimes slowly absorb product concerns one PR at a time. A runtime with no storage can't.
It also makes multi-product actually work. Fina drives Mill with its full toolset, native subagents and its own system prompt. Fina-Excel sends messages, tools and a model name and ignores every optional field. The same deployed instance serves both. And you can spawn as many instances as you want and call any of them, no instance holds anything so it doesn't matter which Mill picks up your turn. The contract is versioned and additive only, so the thin product keeps working while the rich one grows.
The obvious objection is the wire. Sending the whole history every turn sounds wasteful, and it would be if the history were the expensive part. It isn't. The same transcript goes to the model either way, and the model call dominates both the cost and the latency. Prompt caching survives the trip too, because the provider keys the cache on the prefix, not on which of our processes sent it, so it genuinely doesn't matter which instance picks up your turn.
What it does cost is recovery. Stateless means no durable state, not no state during a turn. While the socket is open Mill holds its bookkeeping in memory, and when the socket closes that goes too, which makes a turn all or nothing. If a connection dies four minutes into a six minute run, the tool results from those four minutes die with it and the product retries from the last message it persisted. We think that's the right trade. Resumable turns need a durable turn log, a durable turn log is a database, and then the rule is gone. But it is a real trade, not a free one.
Two things people assume you lose. Debugging is the first, and it doesn't need storage either. A turn emits a trace as it runs, and because the product already holds the exact payload it sent, a bad turn is reproducible by replaying it. Identity is the second. Audit, rate limits, entitlements, all of it lives in the product, which is where the user actually exists. Mill authenticates the product, the product authenticates the person.
Give the agent a computer
Six of Mill's native tools are the workspace: read, ls, grep, write, edit, exec. Web search and subagents make eight. A harness with exec has no ceiling, coding agents proved that already. Simon Willison's line about Claude Code, a general agent disguised as a developer tool, is the whole idea, and so is the corollary, anything you can do by typing commands into a computer can be automated. Cowork and the Codex app are the labs saying it out loud, taking the coding harness, sandboxing it, and handing it to people who will never write code.
Exec is the reason. Need a PDF, an Excel model, a chart, a diff of two filings? The agent doesn't need a PDF integration, it writes the code that makes the PDF and runs it. Agents will always need a computer, so Mill treats one as a primitive.
And this doesn't break the stateless rule, because the sandbox is its own service. State lives in exactly two places, the product's database and the sandbox's revision store, and neither of them is Mill. The workspace is versioned, every write, edit or exec commits a revision, the product stores just the revision ID, and next turn Mill mounts it back. The agent gets a persistent computer and the runtime still remembers nothing.
The wiring per chat is simple. A workspace is just an opaque ID, Fina uses the chat ID. At the start of a turn the sandbox syncs itself to the chat's last revision, so every chat gets its own computer that picks up exactly where the last turn ended. Workspaces are namespaced per product as well, one product can never read another's files. And Mill still stores none of it, the revision ID lives in the product's DB next to the chat.
Composition, not features
Everything a product plugs in is plain data. Tools are JSON schemas. Mill runs its own natives in process and forwards every other name back over the same websocket as a callback, so it has no idea what any product tool does. Skills are not a Mill concept at all, a skill is product content loaded by a product tool, it just enters the chat as normal messages. Subagents are native but the product supplies their prompts, so children behave how the product wants. Compaction too, the product's prompt says what to conserve when the transcript folds. What differs per product is config: which natives are on, how compaction is tuned.
Every feature request gets one question, is this runtime or is this product? If it's product it stays out. That's what keeps Mill small. The loop is 1,292 lines. The in-process loop it replaced was 2,152, and only Fina could use it. Mill is about 4,700 lines all in, and that total buys the provider layer, the sandbox client, the guardrails and the native tools on top of the loop.
The factory
Mill went from first commit to Fina on prod in about a week. Everything since is hardening, and every fix lands for every product at once.
The line in the notebook where this started still holds. Mill: the factory that powers our products.
If you are building more than one AI product you are already building a runtime, just inside your first product where only that product can use it. Pull it out, keep it stateless, let the products carry their own state. The next product then is mostly a config change.
Mill is live on prod. Fina and Fina-Excel run on it, the financial reporting and revenue contracts products are on the way.
