The MongoDB blog has made the case for bounded autonomy1, and cataloged the design patterns that produce safe agents2. Earlier entries in this series introduced the harness, the layer of code and configuration around the model that determines whether an agent works in production. A recent formal definition of the agent harness3 names four elements: an agent loop, a tool interface, context management, and control mechanisms. The control mechanisms are where the concerns in this post live. This entry covers the layer those design patterns sit on top of—the runtime loop itself, and the machinery that determines whether the loop survives contact with production. Framework, harness, orchestration, and platform name four distinct layers, and this post is about the third. Architects and platform builders evaluating that layer will find four questions at the end for auditing an existing one.
The loop is the part of an agent system that runs the model in a cycle. The agent receives a goal, the model decides what to do, a tool runs, the result comes back, and the model decides again—think, act, observe, repeat—until the task is done or the loop gives up. The names vary (agent loop, ReAct loop, control loop), but the shape is the same. Orchestration is the code that runs this loop: it dispatches the tool the model asked for, parses what comes back, decides whether to continue, and handles the cases where something goes wrong.
The design-pattern literature blurs a distinction here. A design pattern—supervisor-and-workers, plan-and-execute, reflection—describes the shape of the agent's reasoning. Orchestration is the code that executes that shape. Orchestration supplies the loop, the tool dispatch, and four kinds of machinery wrapped around it: retry logic for when a step fails, timeout logic for when a step runs too long, permission logic for when a step tries to do something it should not, and recovery logic for when the loop has to resume from a partial state.
Orchestration is also distinct from routing, though the two are often conflated. Routing decides which model or tool to use for a given step. Orchestration decides whether the step runs at all, what has to happen before it—fetching or aggregating memory, checking a permission—and what to do with the result. Either can be simple or sophisticated, but they answer different questions, and the orchestrator is where the context needed to observe and debug the run accumulates.
Figure 1. The runtime loop and its four concerns.
Most production agent failures live in that machinery. An agent that picks the right tool on a clean demo will still fail on iteration five in production when a tool times out, returns a malformed payload, or executes with the wrong user's permissions—and whether the agent recovers or cascades depends on the orchestration code, which is usually the least-designed part of the system.
How the orchestration loop dispatches and recovers tool calls
Orchestration starts with tool dispatch. When the model decides to call a tool, it does not run anything itself—it emits a structured request, and the orchestration layer turns that request into an actual call, waits for the result, and feeds the result back into the model's context. Tool dispatch is the seam between the model's reasoning and the outside world, and most of what goes wrong in an agent goes wrong at this seam. Dispatch failures get easier to reason about once the loop treats a tool call as a step in a distributed workflow rather than an ordinary function call: it can time out, partially succeed, or succeed while its response is lost, and the loop has to account for each.
Four concerns wrap the dispatch, and each assumes the previous one exists.
Retry handles the step that fails and could succeed if tried again. A tool returns a malformed response, an API rate-limits, or a downstream service is briefly unavailable. The orchestration layer decides whether to retry, how many times, with what backoff, and whether a retry is even safe, because retrying a call that already charged a credit card is a different decision than retrying a read. Retry logic that does not distinguish idempotent from non-idempotent calls is a common source of duplicated side effects.
Timeout handles the step that does not return. A tool hangs, a model call stalls, and an external job never completes. With no timeout, the loop waits indefinitely, and the agent appears frozen; with a naive timeout, the loop kills and retries a slow-but-valid call, producing the duplicate-side-effect problem that retry was supposed to prevent. Timeouts and retries are coupled, and orchestration resolves the coupling.
Permission handles the step that should not run. The model decides to call a tool the current user is not authorized to use, or to act on a record outside the user's scope. The check has to run after the model selects the tool but before the call executes, which means the orchestration layer needs a place to intercept the call and apply policy. Identity propagation and permission scoping are governed elsewhere in the stack, but the loop enforces them because the tool call is the interception point. This is a consistent gap across enterprise deployments: teams have working agents and working tool integrations, and no clean way to carry scoped identity through to the tool call so the agent can prove it is acting within a specific user's authorization.
Recovery handles the loop that has already partially run. An agent is six steps into a workflow when the process restarts, or it suspends waiting on a human approval that takes two days. Recovery is the logic that lets the loop resume from where it stopped instead of starting over, which depends on the agent's state being persisted somewhere durable. Orchestration consumes that persisted state without owning it. It decides when to checkpoint and how to resume.
Figure 2. The path a single tool call takes.
Retry, timeout, permission, and recovery all get harder as the tool surface grows. Trimming the tool surface raises success rates: fewer, better tools mean fewer chances for the model to pick the wrong one. Tool selection is the mechanism: the model chooses from the tools described in its context, and as the count climbs, selection accuracy falls. A single large tool catalog can consume a meaningful fraction of the context window before the agent has done anything; published measurements of GitHub's official MCP server15, which exposes roughly ninety tools, range from about 17,600 tokens of tool definitions to over 40,000 depending on the server version and the counting method. Teams often report the same pattern: accuracy degrades as the surface grows.
The Model Context Protocol standardizes how a tool describes itself to an agent and how the agent invokes it—a connectivity primitive, the “USB-C port” for tools, as its own documentation frames it. That standardization is real and useful. It does not extend to deciding which of several overlapping tools the agent should call, who the agent may call them on behalf of, or what happens when a call fails. Those remain the orchestration layer's problems.
Figure 3. What MCP covers, and what it leaves open.
MCP leaves the four-concern machinery to the orchestration layer. Anthropic's writing on managed agents5 describes routing tool calls through a credential proxy so that secrets and identity are handled by the platform rather than the agent—an acknowledgment that the protocol connects the tool but does not govern the call. When agents hand off to other agents, the gap widens: every handoff is another point where retry, timeout, permission, and recovery have to be re-established, and identity has to be re-scoped at each handoff. A boundary that skips that step passes the caller's scope downstream unchanged. A downstream agent acting with the wrong scope is a failure mode that teams consistently underestimate until the first cross-agent permission leak.
Figure 4. Agent-to-agent handoff as a tool call.
These failures are not random. The MAST taxonomy6, built from more than 1,600 annotated execution traces across seven frameworks, sorts agent failures into three groups: system-design issues, inter-agent misalignment, and task verification.
Meta's study of coding-agent trajectories7 sorts misbehaviors into specification drift, reasoning problems, and tool-call failures. The categories overlap, and they map onto the four concerns:
Figure 5. Documented failure modes, mapped to the four concerns.
That last row is where teams most often misdiagnose. A failure that presents as “the agent picked the wrong tool on turn five” usually traces back not to the dispatch logic but to what was in the model's context at the moment it chose. If the context at turn five is wrong, the tool call at turn five will look like an orchestration failure even when the dispatch logic is correct. Teams describe this consistently: the longer a session runs, the more the context blurs, and that is where wrong tool calls and hallucinations begin. But not every loop failure is a context failure. Non-idempotent retries, timeout behavior, schema drift, race conditions, authorization bugs, and state that was never persisted before a handoff all produce the wrong outcome with perfect context in front of the model. Reading the context window at the failing turn separates the two: if the right information was there, the fault is in the machinery.
Why production agent failures originate in the orchestration layer
OpenAI's account of building its Codex agent8 states the case plainly. A team of three engineers shipped roughly a million lines of code across some 1,500 pull requests in five months, with the agent writing essentially all of it. Their operating principle was to treat every agent failure as a question about the harness rather than the model: when the agent failed, they asked what capability was missing and how to make it legible and enforceable for the agent—not how to prompt the model differently. The reliability gains came from the machinery around the loop.
Meta's analysis of coding-agent misbehavior7 points the same way from the failure side. Across more than ten thousand trajectories, roughly thirty percent contained a misbehavior, and the large majority of single-step misbehaviors could be resolved with one corrective intervention at the right point in the loop. Many agent failures are recoverable, but only if the orchestration layer catches and corrects them.
The picture also holds when the harness changes and the model is held constant. A study that holds the model fixed and evolves only the harness9 lifts Terminal-Bench 2 pass@1 by about seven points across ten iterations, past a human-designed baseline, and the frozen harness transfers to other model families at lower token cost.
Agents are also being pointed at harder tasks. METR's time-horizon measurements10 express task difficulty in human-expert hours: the length of task, as timed by a human, that a frontier model completes at a 50% success rate. As of early 2026, that figure had reached roughly twelve hours for the strongest models, doubling somewhere between every four and seven months, though the estimates carry wide confidence intervals at the top end. The measure is task difficulty rather than agent runtime, but harder tasks mean more loop iterations, and every iteration is another retry, timeout, permission, and recovery decision that can go unhandled.
What teams actually run in production reflects this. Despite the framing of agents as autonomous, teams deploy them under tight constraints: the first large-scale study of production agents11 found that sixteen of twenty in-depth case studies ran agents inside structured workflows rather than open-ended autonomous planning, with a single case allowing unconstrained exploration, and that one only in a sandboxed environment. Those constraints are the orchestration layer doing its job, bounding where the model may decide. Tool-call failures are common enough that an agent making dozens of calls per task will hit them routinely. One small benchmark of a popular remote tool server17 saw seven of twenty-five runs fail under load. Retry and recovery sit on the main path.
The same picture shows up in aggregate across recent enterprise engagements. Teams say building an agent is the easy part, and that running, governing, and recovering it in production is the hard part. They raise identity propagation through tool calls more often than any other unsolved problem.
One failure from the field shows the shape. An agent appeared to finish, streaming its final answer to the user. The run hung anyway, because the runtime was still waiting on cleanup and an external service call that never completed cleanly. The fix was not a better prompt or a stronger model. It was making the surrounding workflow explicit: defining when the agent is actually done, bounding how long it waits on external calls, and making cleanup predictable.
How to audit your agent orchestration layer before production
What a well-designed platform should do
Orchestration is one of the six harness components this series covers, and a well-designed harness makes its orchestration layer the single place where the four concerns are enforced, rather than leaving them scattered through application code. The harness owns the tool call, and that is where it enforces all four concerns. The platform layer underneath the harness holds that enforcement across many agents and teams. It supplies durable execution, identity propagation, and per-action cost attribution once, rather than leaving each agent to re-implement them. Identity runs through both layers: carried into each tool call as a scoped credential so the agent provably acts within a specific user's authorization, including across agent-to-agent handoffs.
Identity, permission, and audit are the substance of structural governance, and that is the layer where most enterprise agent projects stall before production. Orchestration is where the property becomes concrete, because the tool call is where identity, permission, and auditability either hold or leak.
Figure 6. A production orchestration architecture.
What to audit now: four questions
For any orchestration layer, four questions separate one that handles the machinery from one that hopes to.
- Where is each concern enforced? If retry, timeout, permission, and recovery live in scattered try/except blocks across agent code rather than in the runtime, the system has no single place to enforce them.
- Is there a cost ceiling? A loop with no per-execution budget and no cap on concurrent invocations is one bad iteration away from a runaway bill—and at fleet scale, cost behaves as a reliability property.
- When something fails, who knows? A failed tool call with no audit record and no trace is a failure that the team rediscovers from a customer report. A system becomes debuggable when it records which agent did what, on whose behalf.
- What is the recovery path? When a step fails, or a process restarts mid-loop, does the agent resume from a checkpoint or start over—and if it resumes, against which version of the agent? Recovery from partial progress depends on the state that orchestration consumes but does not own.
An orchestration layer that answers all four enforces the four concerns rather than leaving them to chance. A “not yet” on anyone is an unhandled failure mode that the system will hit in production.
Treat the orchestration layer as production infrastructure
This post covered orchestration as the runtime loop and the four concerns wrapped around it. The next entry turns to the layer that feeds the loop: memory inside the harness, how an agent retains and retrieves the context it needs across turns and sessions, and why so much of what looks like an orchestration failure turns out, on inspection, to be a memory one.
SERIES: DESIGNING AN AGENTIC PLATFORM
Pt 0: Series Intro—The Infrastructure That Makes Agents Work
→ Pt 4: Orchestration & Tool Use (this post)
Pt 5: Memory Inside the Harness
Pt 6: Observability
Capstone: The Agent Platform Buyer's Checklist
REFERENCES
1. MongoDB, “The Case for Bounded Autonomy”
2. MongoDB, “Agentic Systems & Design Patterns”
3. Subramaniam et al., “What Makes a Harness a Harness”—arXiv 2606.10106, June 2026—constitutive definition: agent loop, tool interface, context management, control mechanisms
4. Model Context Protocol, “Security Best Practices” — 2026
5. Anthropic, “Scaling Managed Agents” — Apr 2026
6. MAST—Cemri et al., “Why Do Multi-Agent LLM Systems Fail?”—arXiv, NeurIPS 2025
7. Nanda et al. (Meta), “Wink: Recovering from Misbehaviors in Coding Agents”—arXiv 2602.17037, Feb 2026—~30% of trajectories contain a misbehavior; 10,000+ trajectories evaluated
8. OpenAI, “Harness Engineering: Leveraging Codex in an Agent-First World”—Feb 2026
9. “Agentic Harness Engineering” (AHE)—arXiv 2604.25850, Apr 2026—model fixed, harness evolved: Terminal-Bench 2 pass@1 69.7% → 77.0% over 10 iterations, beating the human-designed Codex-CLI harness (71.9%); frozen harness transfers +5.1–10.1 pts across model families at ~12% fewer tokens
10. METR, “Measuring AI Ability to Complete Long Tasks—Time Horizon 1.1”—Jan 2026
11. Pan et al., “Measuring Agents in Production”—arXiv 2512.04123—survey of 306 practitioners and 20 in-depth case studies across 26 domains; 16 of 20 case studies use structured workflows, 1 uses unconstrained exploration. Summarized in Cobus Greyling, “The AI Agent Reality Gap”—2026
Additional references
12. Anthropic, “Harness Design for Long-Running Applications”—Mar 2026
13. Inngest, “Your Agent Needs a Harness, Not a Framework”—2026
14. LangChain, “Frameworks, Runtimes, and Harnesses, Oh My!”—2026
15. StackOne, “MCP Token Optimization: 4 Approaches Compared”—2026—GitHub’s official MCP server (~94 tools) ≈ 17,600 tokens of tool definitions; per-tool 500–1,400
16. Anthropic, “Code Execution with MCP: Building More Efficient AI Agents”—2026—at scale, loading all tool definitions upfront pushes agents into hundreds of thousands of tokens; on-demand loading cut one example from 150K to 2K
17. Apideck, “Your MCP Server Is Eating Your Context Window”—2026—three servers (~40 tools) ≈ 55,000 tokens; Scalekit benchmark: ~28% failure on a popular remote tool server (7/25 runs)
18. deepset, “Harness Engineering: Engineering the System, Not the Model”—May 2026—failure-classification framework (context / constraint / verification / planning)
Authors and contributors
Mikiko Bazeley, Staff Developer Advocate, MongoDB. Mikiko writes about agentic AI systems, agent memory and context engineering, and the POC-to-production gap.
Ashish Kumar, Technical Fellow, MongoDB. Ashish joined MongoDB over two years ago through the acquisition of Grainite, a database startup he co-founded. Before that, he spent many years at Google, most recently responsible for Google’s native database suite — Bigtable, Spanner, Datastore, and Firestore — across Google’s own products and Google Cloud. His passion is large-scale distributed systems, and at MongoDB, he focuses on architectural improvements across the product stack.
Max Marcon, Director of Product Management, MongoDB. Max leads product for agentic AI initiatives at MongoDB. His mission is to ensure customers are successful with running agents at enterprise scale.
Charlie Xu, Senior Product Manager, Cloud Product, MongoDB. Charlie leads product thinking for agent-related initiatives, shaping their positioning and the customer research and architectural priorities.
Erik Beebe, Senior Staff Engineer, MongoDB. Erik joined MongoDB in 2023 to focus on stateful stream processing and, more recently, agentic AI technologies. Previously, he co-founded high-performance data startups Eventador Labs and ObjectRocket, and held engineering roles at eBay, PayPal, and Cloudera. He enjoys building reliable, event-driven distributed systems.
Ming He, Software Engineer, MongoDB. Ming has spent six years at MongoDB working across several areas of MongoDB Atlas, from cluster experiences to data lifecycle and developer-facing workflows. His interest is in the infrastructure patterns that help make AI applications reliable in real-world systems.