The Coordinator Problem in Multi-Agent Systems

When you run a single agent on a task, the failure modes are predictable. The agent either produces good output or it doesn't. You can debug it, prompt it differently, swap the model. The feedback loop is tight.

When you run five agents on a task that requires coordination, everything changes. You now have a distributed system problem masquerading as an AI problem. And if you try to debug it like an AI problem — adjusting prompts, tweaking temperatures, switching models — you will spend weeks going nowhere.

I've watched teams build multi-agent systems three or four times before they understand this. The first system collapses under a weird edge case. The second collapses under load. The third collapses when two agents disagree. By the fourth attempt, they finally start designing the coordination layer with the same rigor they'd apply to a message queue or a database schema.

This article is about what that coordination layer actually needs to look like.

The Three Failure Modes Nobody Talks About

Most multi-agent tutorials show you the happy path: Agent A does research, Agent B writes code, Agent C reviews it, done. What they don't show you is what happens when:

Agent B finishes before Agent A. The researcher is still gathering context, but the coder already started implementing based on incomplete requirements. Now you have a partially-built feature built on assumptions that the research contradicts. The artifacts are inconsistent and there's no clean way to reconcile them.

Two agents reach different conclusions from the same source. You ask a research agent and a verification agent to both analyze a codebase. They read the same files, in different orders, with different context windows. One concludes the authentication is Clerk-based. The other concludes it's custom JWT. Your coordinator agent now has to resolve this disagreement — but it has no principled way to decide who's right without re-reading the source.

An agent succeeds but produces output nobody expected. Your code-writing agent was supposed to produce a TypeScript function. Instead it produced the function plus a full module restructure plus three new files. Technically correct. Completely incompatible with what the review agent is set up to evaluate.

None of these are prompt engineering problems. They're coordination architecture problems. And you can't fix them by making your agents smarter.

State Is the Real Problem

The root cause of most multi-agent failures is implicit shared state. Agents are communicating through artifacts — files, strings, data structures — and assuming the other agents interpret those artifacts the same way they do.

In a traditional distributed system, you'd solve this with schemas, contracts, and typed interfaces. If service A sends a message to service B, there's a defined schema. Service B rejects anything that doesn't match. The contract is enforced.

In a multi-agent system, most people pass strings. Agent A writes a summary. Agent B reads the summary. Except Agent B was trained to expect a structured report with a specific format, and Agent A just wrote prose. Agent B does its best, makes assumptions, and produces output that looks plausible but is based on a misread.

The fix is obvious in retrospect but almost nobody does it from the start: define explicit state schemas between every agent handoff.

Not prompts that describe the expected format. Actual typed schemas — Zod in TypeScript, Pydantic in Python, whatever your stack uses. Every output an agent produces gets validated against a schema before it's passed to the next agent. If validation fails, the orchestrator handles the failure explicitly: retry, fall back, escalate, abort.

This adds friction. Your agents now have to produce structured output instead of free-form text. But it eliminates an entire class of silent failures where agents are operating on misunderstood inputs. Silent failures are the most dangerous kind in a system that's supposed to run autonomously.

The Supervisor-Worker Model, Done Right

The most common multi-agent pattern is supervisor-worker: one orchestrator agent that breaks down tasks and assigns them to specialist workers. This pattern works, but most implementations get one critical thing wrong.

They make the supervisor responsible for both planning and arbitration.

Planning is figuring out what needs to happen: what tasks, in what order, with what dependencies. Arbitration is resolving conflicts: when two workers disagree, when a worker produces unexpected output, when a dependency fails and downstream tasks need to be replanned.

If you give both responsibilities to the same agent, you get an agent that's constantly context-switching between forward-planning and backward-looking conflict resolution. Its context window fills with both the original plan and the accumulating history of disagreements and corrections. By the time you're three tasks into a complex workflow, the supervisor's context is a mess and its planning quality degrades.

The better architecture separates these into distinct layers:

The Planner takes the original task and produces a dependency graph: a structured list of subtasks with explicit dependencies. This is a one-time operation. The planner isn't involved in execution.

The Dispatcher reads the dependency graph and routes tasks to workers as their dependencies are satisfied. It's stateless logic — more like a scheduler than an agent. No LLM needed here.

The Workers execute their individual tasks and return structured outputs.

The Validator checks each output against its schema and the expectations defined in the plan. This is where mismatches surface.

The Arbitrator handles failures and conflicts — but only failures and conflicts. Its context is scoped to the specific disagreement at hand, not the entire history of the workflow.

This is more components. It's also dramatically more debuggable. When something goes wrong, you know exactly which layer failed. You're not hunting through a monolithic supervisor's context trying to figure out whether it was a planning failure or an execution failure or a validation failure.

Message Passing vs. Shared Artifact Stores

There are two ways to pass data between agents: message passing (agents send outputs directly to the next agent) and shared artifact stores (agents write to a common store, other agents read from it).

Message passing feels simpler. It's how most tutorials show it. But it creates a linear dependency chain: if any agent in the chain fails, everything downstream stalls. Retry logic gets complicated fast, because retrying one agent means you need to replay all its downstream effects.

Shared artifact stores decouple producers from consumers. Agent A writes a research summary to the store at research/v1/auth-analysis. Agent B reads it when it's ready to consume it. If Agent A needs to produce a corrected v2, it writes to research/v2/auth-analysis. Agent B can choose to use v1, wait for v2, or consume both and reconcile.

The artifact store approach maps well to how real codebases work. When you write code, you commit it to a repo. Other agents (reviewers, testers, documenters) pull from the repo when they're ready. The repo is the artifact store. This isn't an accident — it's a pattern that evolved to handle exactly the coordination problems that multi-agent systems face.

In practice, I'd use a combination: message passing for immediate, sequential handoffs where the dependency is tight, and artifact stores for any data that multiple agents need to access, or that might need to be updated without breaking consumers.

Content-addressed storage (where the identifier is based on the content hash, not a mutable name) eliminates a whole category of race conditions. If Agent B is reading research/abc123, it's guaranteed to get exactly what Agent A produced at that hash, even if Agent A has since produced a newer version.

Designing for the Inevitable Failures

Every agent will eventually produce bad output. Every tool call will eventually fail. Every external API your agents depend on will eventually be slow or unavailable. If you don't design your coordination layer to handle this explicitly, you'll get cascading failures that corrupt the entire workflow state.

Four patterns I've found essential:

Dead letter queues for failed outputs. When an agent produces output that fails validation, don't retry immediately or silently drop it. Route it to a dead letter store with the full context: what was the task, what was the input, what was the output, what was the validation error. This gives you a corpus of real failures to analyze and a hook for human escalation when the system is stuck.

Idempotent task execution. Every agent task should have a stable identifier. If an agent fails halfway through and gets retried, it should produce the same output as if it had succeeded the first time. This requires agents to be stateless — all relevant state comes from inputs, not from agent-internal memory accumulated during the run.

Explicit timeouts with escalation paths. An agent that's taking longer than expected should trigger an escalation, not just keep running. Define timeout budgets per task type. An LLM call to summarize a 500-line file shouldn't take more than 30 seconds. If it does, something is wrong. Escalate to the arbitrator, don't just wait.

Workflow checkpointing. For long-running workflows — anything that takes more than a few minutes — checkpoint the workflow state after each successfully validated task completion. If the whole system crashes, you can resume from the last checkpoint instead of starting over. This requires your state representation to be serializable, which is another argument for explicit schemas.

When Not to Use Multi-Agent Systems

The honest answer is: more often than people think.

Multi-agent systems add coordination overhead. That overhead is worth it when the task genuinely requires parallelism or specialization that a single agent can't provide. It's not worth it when you're using multiple agents because you've hit a context window limit or because your prompts are too complex for a single agent.

Hitting a context window limit is a compression problem, not a specialization problem. The solution is better context management — summarization, retrieval-augmented approaches, hierarchical compression — not spinning up more agents.

Prompts that are too complex for a single agent are almost always a sign that the task is underspecified. Break the task down better. Give the single agent a cleaner, more focused scope. Multi-agent systems built to compensate for underspecified tasks are just moving the ambiguity around, not resolving it.

The cases where multi-agent genuinely pays off:

• Tasks with clearly independent subtasks that can run in parallel without shared state
• Tasks that require fundamentally different expertise or tool access (one agent has web access, another has code execution, another has database read access)
• Tasks where you want adversarial verification — one agent produces, another critiques, you want genuine independence between them
• Tasks long enough that a single agent's coherence degrades (typically anything over 45-60 minutes of sequential operation)

Everything else? A single well-designed agent with good tool use and explicit context management will outperform a naive multi-agent setup and be half the debugging work.

The Coordination Layer Is Your Product

Here's what I've come to believe after building and watching others build these systems: the agent models are commodities. The coordination layer — the schemas, the state management, the failure handling, the arbitration logic — is where the actual intellectual work lives.

Any team can wire together Claude and GPT-4 and call the result a multi-agent system. The teams that build systems that actually work in production are the ones who treat the coordination layer as a first-class engineering problem. They spec it like they'd spec a distributed system. They test it with adversarial inputs. They instrument it so they can observe what's actually happening, not what they think is happening.

The agents are the workers. The coordination layer is the factory floor. You can hire the best workers in the world, but if the factory floor is a mess, nothing coherent comes out the other end.

Most agent system failures I've seen weren't model failures. They were infrastructure failures. The model did exactly what it was asked. What it was asked was incoherent, or the output had no place to go, or the failure had no handler.

Get the coordination right first. Then worry about which model to use.