Code Mode Without the Interpreter: Agents That Write Aggregation Pipelines

Standfirst Code mode gives an AI agent a code interpreter and lets it write real programs—one round trip instead of five, filtering where the data lives instead of in the context window. It also means running arbitrary code that a model wrote, which is a hard sell in a regulated product. This is an account of doing the same thing without the interpreter: the agent authors a MongoDB aggregation pipeline, a handler executes it server-side, and only the reshaped result crosses back. Two production systems, the query work that made them viable, and where the approach falls down.

The problem with fixed tools

Our document agent answers questions about the contracts a customer has uploaded. Which of these autorenews? What notice period does each one need? Where does a particular indemnity actually sit?

Figure 1. Fixed tools workflow with multiple sequential round trips and context accumulation.

The first version used ordinary MCP tools. One tool per operation, each returning one shape of result. That works right up until a question doesn't map onto a tool. "Which contracts auto-renew, and what notice do they need" becomes a sequence: list the documents, fetch the first one's sections, scan them, fetch the next one's, scan those, and on. Five findOne calls in a row, where anyone who knows the database would have written one query. Two things go wrong, and the second is worse.

The obvious one is round-trips. Every call is a full turn. The model waits, the result arrives, and it decides what to ask next. The subtle one is what those results do to the context window. To find three paragraphs, you pull back the sections of every document you looked at—hundreds of kilobytes, most of it irrelevant, now sitting in the window permanently, with the model reasoning over all of it for the rest of the conversation. It compounds with every call. Nothing failed. Each tool returned exactly what it promised. That is the problem. You can paper over both of those by writing more specific tools, but then you are in the business of anticipating every question anyone will ever ask, and the ones you didn't anticipate fall back to the primitive calls.

There is a third problem, and it is the one that has no fix at this layer. A legal answer is often not in a document. It is in the relationship between several. A folder might hold sixty files: a master agreement, two addenda, a side letter, a handful of schedules, and superseded drafts nobody deleted. The notice period in the master agreement is real until an addendum changes it. An indemnity may be capped by a schedule whose title says nothing about indemnities. A clause that looks decisive on its own is routinely narrowed, extended, or voided by another document in the same folder. So you cannot know which documents matter until you have looked across all of them. No index helps, because you are not retrieving a row—you are reconciling a set. With fixed tools, the only available strategy is to pull all sixty documents' sections into the context window and let the model compare them there. That is not an inefficiency to optimize away. It is the only move the toolset permits, and it is the move that fills the window.

Agent code mode, and what it costs

The pattern the industry has converged on is to let the model write code. TypeScript or Python, executed in a code interpreter, with the result of the run coming back rather than the raw data. It is a genuinely good answer to both problems. Filtering happens inside the program rather than inside the context window, and N round-trips collapse into one. The cost is that you are now executing arbitrary code that a model wrote. That means a sandbox, a filesystem policy, egress rules, resource limits, and a threat model covering all of it. Plenty of teams ship this and ship it well. In a product where the thing being processed is somebody's confidential contract, there is a lot of new surface to take on.

Figure 2. Agent code mode pattern using a code interpreter for arbitrary code execution.

Code mode with databases

So the agent writes a program—just not code. It authors a MongoDB aggregation pipeline. A handler on our side validates it, prepends a tenancy filter, executes it against that workspace's documents, and returns the result. The five sequential lookups become one pipeline: scope to the workspace, $unwind the sections, $match the section type, $project only the fields needed, $group back by document. One round trip. What comes back is a few hundred bytes shaped for the next reasoning step, rather than the corpus it was computed from.

Figure 3. Code mode with databases pattern using a validated aggregation pipeline instead of an interpreter.

The sixty-document case is the same shape, which is the point. One pipeline touches every file in the folder, pulls out only the sections that bear on the question, and returns them side by side with their source document attached—so the model receives the comparison rather than the material to assemble a comparison from. The cross-document work happens where the documents are. No interpreter. No sandbox process. No filesystem API. The program is JSON, and the runtime is the aggregation engine. That is the trade. You keep composition, filtering at the source, and transformation before the tokens are spent. You give up arbitrary execution. And you can afford to give it up, because the language is small enough to allow-list exhaustively.

Why this works with MongoDB in particular

This works for two reasons, and they are worth separating because only one of them is about the database.

The model already knows the language. Aggregation pipeline syntax is very well represented in the material these models were trained on. We are not teaching an agent a bespoke query DSL we invented and then hoping it generalizes. We are asking it to write something it has seen a great deal of. The failure modes are the ones any MongoDB developer would recognize, which also means they are the ones we know how to catch.

The document model and the security model happen to agree. Contracts are documents with nested sections, so $unwind and $project reach inside them directly. The same question against a relational schema needs joins—and cross-collection stages are exactly what we block, because a join is how a query escapes its tenancy scope. In a normalized schema, the thing we most need to forbid is the thing the query would most need to do. Here it isn't. That second point took us a while to notice, and it is the reason the allow-list can be as short as it is.

And the agent uses the framework well. This is the part I did not expect. It works in two passes: a first pipeline to see what is actually in the matter, then a second, targeted one written against what it found. The targeted pipeline is frequently large—fifty or sixty operators composed together—and it holds up. We instruct it to be token-efficient, and what that produces is the interesting bit. It reaches for $limit to paginate rather than pulling everything back. And it uses $addFields to compute the thing we actually asked for instead of returning the material to compute it from: concatenating fields into the string the answer needs, or assembling a boolean expression that comes back as true or false. That last habit is the whole argument in miniature. Asking whether any document in a folder contains an uncapped indemnity can return a boolean computed in the database, rather than the indemnity clauses of sixty documents for the model to read and compare. The comparison still happens. It just does not happen in the context window.

Nine stages

The permitted vocabulary is $match , $unwind , $project , $group , $sort , $limit , $skip , $count and $addFields. Every top-level key of every stage is checked, and a rejection returns a structured error naming the offending operator and listing the legal set. What that buys is concrete. $out and $merge are rejected, so the tool is read-only by construction rather than by convention. $lookup , $graphLookup, and $unionWith are rejected, so a query cannot reach another collection. Server introspection stages are rejected. Here is the part that surprised us. Restricting the language is not only a security control. It is a prompt engineering control. A nine-stage vocabulary is small enough to fit, with worked examples and anti-patterns, into a document that the model reads before every session. A forty-stage vocabulary is not. The property that makes the surface auditable is the same property that makes it learnable.

The filter the model cannot reach

Every pipeline gets a tenancy filter prepended before it runs. The workspace identifier is not a tool parameter—it is captured when the server is constructed for that conversation, so there is nothing for the model to supply, see or spoof. Because cross-collection stages are blocked, a query cannot get out from under it either. That filter also excludes documents that failed processing, since a failed document has no extracted text to reason over. It lives in one named constant, with a comment explaining why a different consumer elsewhere deliberately retains those same documents, so the interface can list them. Two consumers, two correct answers, one shared constant, so nobody re-inlines the wrong one. This is the strongest argument I have for executing server-side rather than running an interpreter. The invariant is enforced in one place, in reviewed TypeScript—not re-derived by a model in every program it writes.

Teaching it the language

We tried out of the box first. Models do write MongoDB. They do not write it well. The queries worked and were inefficient, and they were written against the MongoDB of a few years ago—newer operators simply were not in the picture. So we wrote the reference ourselves. A skill the agent must load before it writes a single stage: tool contracts, document types, the structure of a contract's sections, ready-to-adapt patterns, explicit antipatterns, and operator tables. It runs to 851 lines, and it is not optional. It is also not allowed to write against a remembered schema. Summary first, then details for the documents that matter, then a query using field paths taken from that response. The instruction is blunt: do not guess field names. Some of what is in that skill is scar tissue, and those are the best lines in it. One says: do not pass the global flag to $regexFindAll , it finds all matches already, and the flag will cause an error. That is a production failure distilled into a sentence, now read before every session. The skill is where post-mortems go to become prevention.

What it bought us: Document intelligence

On our evaluation suite, accuracy improved by roughly 20%. More useful than the percentage: cases that had never passed started passing. A number moving is easy to argue with; a test that was red going green is not. That figure needs a reference point, because on its own, a percentage means very little. In our experience, a well-judged prompt change or a better-written skill moves legal quality by five or six points. That is a good day's work. Twenty points, from changing how the agent gets its data rather than how it reasons about it, is not a return we normally see for that amount of effort.

From red to passing evals

To evaluate this capability specifically, we constructed a benchmark built around a challenging legal scenario: a fictitious matter encompassing over fifty files. It is the kind of massive corpus that taxes both a legal professional and an agentic legal solution. Working alongside our legal domain experts at Genie AI, we established nine strict validation parameters: identifying the exact contracting entity, determining notice windows accurately, avoiding cross-document clause misattributions, and six parallel constraints.

Scoring was strictly categorical: met, partially met, not met.

The baseline: Without MongoDB Search and code-mode aggregation authorship, the agent satisfied just two of the nine requirements.

With MongoDB Search in code-mode: The raw pipeline immediately doubled performance to four out of nine. Once we equipped the subagent with structured legal skill guidance tailored for contract analysis and better instruction on how to use MongoDB, success jumped to seven out of nine parameters.

Moving from a baseline calculation to over a 70% resolution rate was more than an incremental improvement; it reshaped how our architecture handles complex retrieval.

The comparison that told us most was not against another product. It was against the same model, with none of this around it. Hand a strong general model a contract and ask it to review, and it does a reasonable job. Give it structure, it can query, a language it already knows how to write, and a schema it does not have to guess at, and it does a substantially better one. The entire difference is in what surrounds the model. Finding the right clause used to be a conversation. It is now one pipeline of about sixty lines, written once and executed once. I should say plainly that we did not expect this. It started as a side project. It has been running in production for five months.

We didn't stop there

Investigating a production issue in an agentic system is hard in a specific way. The failure usually isn't an error. Nothing spikes. The system returns a fluent, complete answer, and the answer is wrong. We were running two observability platforms to cope — one for the AI behavior, one for the engineering — and neither answered the question we actually had, which was usually why it decided that, and whether the same thing was happening to anyone else. Dashboards answer questions you decided to ask when you built them. Those are not the questions worth asking about an agent.

So once the document work had been in production for a few months, we reused the strategy. Same pattern: the agent writes its own aggregations against the operational data and runs them, intersecting things we had never thought to connect in advance. It is agent-first, so there is no dashboard to design, build, review, and then maintain as the questions change — asking a new question is cheaper than adding a panel. And a full investigation now takes a couple of minutes rather than ten or twelve. That last part matters more than it sounds. At ten minutes, you investigate when something is clearly broken. At two, you investigate because you are curious. Those are different jobs, and the second one finds things. It only became viable after some query work, and the one that mattered most is worth stating plainly. A usage lookup was examining 34,053 documents to return a single workspace's rows. A partial index brought that down to 392 documents examined, and the query from 2.2 seconds to about 20 milliseconds.

That is the honest caveat on this whole pattern. A model-authored query can be perfectly correct and quietly expensive, and nothing in the result tells you. Past that, the problem of understanding our own data mostly dissolves. The agent runs large queries to explore bugs and patterns, and we stopped building things to look at data with.

What it does not do

Regex is not a semantic search. The retrieval quality floor is whether the agent thought of the right synonym, and security is embedded at multiple layers during the search. The prompt compensates by mandating synonym sweeps and a coverage check on queries claiming to be exhaustive, but the floor is still the floor. And $limit is a silent truncator. A query meant to find everything, which hits the limit, returns a subset that looks exactly like an answer. Both are real. In a legal review, an empty result is not an empty answer! Furthermore, our security framework treats every corpus as inherently untrusted; leveraging MongoDB Search enables us to enforce read-only execution and constrain query reach through $match filters that the model never sees.

The shape of the argument

The interesting decisions in agent systems are about what the model is allowed to do, not what it is asked to do. A bounded language, enforced outside the model, turned out to be better than a larger one plus a stern prompt — for safety, and unexpectedly for accuracy. You may not need the interpreter. If your data already lives somewhere with a composable query language the model has seen before, you may already have everything code mode was going to give you.