Why do multi-agent systems fail more often than single agents?

Because reliability compounds against you and agents add steps rather than attempts. If each step succeeds 95 percent of the time, ten steps finish about 60 percent of the time and twenty about 36 percent, since the probabilities multiply. A planner plus three specialists plus a critic is a longer chain than one agent doing the work, plus coordination on top, so unless each specialist's narrower scope genuinely raises its per-step reliability the result is worse. Every handoff also loses context, because what one agent knows lives in its window and what it passes on is a summary, and the receiving agent cannot tell what it was not told.

When is a multi-agent design actually the right answer?

Three cases. Genuinely parallel subtasks over disjoint context, such as researching many sources independently, where fan-out buys latency and each worker keeps a small focused context. A hard security boundary, where the agent reading untrusted input must not be the agent holding credentials, because prompt injection targets exactly that combination. And, more weakly, routing between models of different cost and capability, which is really model selection rather than a team of agents. Giving each agent a job title is not on the list.

What is the single highest-value way to make an agent more reliable?

Remove steps. The step count is the exponent in the success arithmetic, so replacing four reasoning steps with one deterministic tool call removes four chances to fail and shortens the chain at the same time. That is why tool design usually beats prompt engineering on a production agent. The follow-ups are making tool results structured rather than prose, validating arguments against a schema before execution, and cutting the chain with checkpoints so a failure does not restart everything.

Why should an agent run be a durable workflow instead of a long-running process?

Because runs last long enough for deploys, instance recycling and network partitions to happen inside them, and losing a run at step eighteen means paying for eighteen steps and delivering nothing. Persisting state after each step means a crash resumes rather than restarts, waiting on a human or an external system costs nothing, retries follow a policy rather than an ad hoc loop, and the deterministic orchestration can be replayed to reproduce a failure. The thing to get right is idempotency: a resumed workflow must not re-execute a side effect that already happened.

How do you evaluate a multi-agent system?

On end-to-end task completion against a definition of done, using a fixed set of realistic goals drawn from real traces, run as a suite against every prompt and model change. Per-step accuracy is the metric that stays green while the product fails. Report where runs fail as well as how often, because consistent failure at one step is a fixable tool problem while uniform failure is a capability problem. And be explicit about whether you are measuring first-attempt success or best-of-several, because those are different claims and conflating them is how a demo becomes a production disappointment.

How do you stop prompt injection in an agent system?

By architecture, not by prompt wording. The attacker writes in the same channel as your instructions and can argue with a rule that says to ignore instructions in content. The controls that hold are structural: separate the agent that ingests untrusted content from the one holding credentials and let only validated structured output cross between them, scope credentials to the minimum per role and enforce that at the tool layer rather than in a prompt, require human confirmation for irreversible actions chosen by reversibility rather than model confidence, and sandbox code execution with no network and no ambient credentials.

How should cost be measured for an agent system?

Per completed task, not per call. A run that fails on the last step has paid for every step and delivered nothing, so failures are the expensive case, which inverts the usual intuition that errors are cheap. Each agent also carries its own accumulating context and is charged for it on every subsequent step, so a five-agent design is roughly five times the spend unless the work is genuinely disjoint. Budgets on steps, tokens, wall clock and spend should each terminate a run with a distinguishable reason, because which budget was hit is the diagnosis.

System design interview guide

Multi-Agent System Design Interview: The Reliability Math, When a Second Agent Helps, and Why Most Designs Should Be One Agent With Better Tools

in this guide

Design a Multi-Agent System, part by part

7 parts open to everyone · 3 with a free account · 3 with the course

You can read 7 of 13 parts. A free account opens the deep dive; the course opens all of it.

Requirementsopen
Scale estimatesopen

Why this question is asked

Because the enthusiastic answer and the correct answer point in opposite directions, which makes it a very efficient question. Multi-agent architectures are heavily marketed and easy to draw, so a candidate who has read about them produces a clean diagram of a planner, three specialists and a critic, and never mentions reliability, context loss, cost, latency or how they would debug it. A candidate who has operated one starts with the arithmetic, points out that every box on that diagram is a chance to fail, and argues for fewer boxes. It also tests a specific engineering instinct that transfers well beyond agents: recognising when a system's problem is variance rather than capability. The reason a long agent run fails is rarely that the model cannot do any individual step. It is that doing forty steps in a row without a mistake is a different problem from doing one step well, and the fixes for it are the ordinary ones, which is to say fewer steps, checkpoints, idempotency, retries at a level that makes sense, and a human in the loop at the points where being wrong is expensive. And it rewards candidates who will say a system should be simpler, which is a harder thing to say in an interview than proposing an impressive architecture.

This page is the free part.

The course goes deeper on the Multi-Agent System design

₹2,777 in India$49 everywhere elseonce, for the whole course

The AI Engineering course covers the Multi-Agent System design across a run of lessons, not one page. These 4 alone are about 197 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.

Live from the course bench

Before you design Multi-Agent, try the questions it rests on.

Real interview questions, live diagrams and measured lab results, pulled at random from the lessons. A new set every time you come back.

Two engineers sketching a system design on a whiteboard

where the work happens

Stuck on a card?

Ask the course tutor. It answers with a real diagram, drawn for your question.

Ask the tutor

Laying out the cards…

Requirements

Always clarify these in the first 5 minutes of the interview. Do not start drawing boxes until both lists are agreed.

Functional requirements

Accept a goal expressed in natural language and run a loop that plans, calls tools, observes results, and decides whether the goal is met
Call external tools with validated arguments and handle their failures, including timeouts, rate limits and malformed responses, without the loop losing the thread

Go deeper

This guide is built on these 6 lessons. Each one teaches one piece of the Multi-Agent System design above, step by step.

Practice with 212 AI Engineering lessons

Lifetime access for ₹2,777 in India or $49 elsewhere. Real labs with measured numbers, RAG, agents, evals and serving, and an AI tutor beside every lesson.

Get lifetime access

Two courses by the author of this page:

AI Engineering

212 lessons · 10 free to read

₹2,777 in India · $49 elsewhere, once

Get AI Engineering

You own this course

System Design

770 lessons · 18 free to read

₹1,777 in India · $49 elsewhere, once

Get System Design

You own this course

Every lesson in both courses has an AI tutor beside it. It reads the same lesson you are reading and answers your questions from it. Try the tutor

Save your progress and get the AI Engineering cheat sheet PDF

AI Engineering Cheat Sheet: what our labs measured, one rule per result. 12 pages. Free with an account.

A multi-agent system is the design candidates reach for when a single agent is not reliable enough, and it is usually the wrong move, because the thing making the single agent unreliable gets worse when you add more of them. The arithmetic is unforgiving and it is the first thing to put on the whiteboard. If a step succeeds 95 percent of the time, a ten-step task finishes about 60 percent of the time and a twenty-step task about 36 percent, because the probabilities multiply. Adding a second agent does not add a second chance, it adds more steps, and each handoff between agents is a fresh opportunity to lose context. There are real cases where separate agents are the right answer, and they have a specific shape: genuinely parallel subtasks over disjoint context, or a hard isolation boundary you need for security. Neither of those is the reason most multi-agent designs get proposed. This page is about telling them apart, because that judgement is what the interview is testing.

A multi-agent system is several language-model-driven loops that pass work between each other, and the interview question is almost never how to wire them up. It is whether you should. The core issue is compounding reliability: an agent that takes the right action 95 percent of the time completes a twenty-step task about 36 percent of the time, and splitting that work across agents does not reduce the number of steps, it adds coordination steps on top. Every handoff also loses information, because what one agent knows lives in its context window and what it passes on is a summary, so the receiving agent is working from a lossy copy of the situation. Those two effects are why a single agent with better tools beats a committee of agents in most real systems: replacing five reasoning steps with one well-designed tool call removes five chances to fail, and it is the highest-value move available. There are genuine exceptions and they have a recognisable shape. Parallel research over disjoint sources is one, because the subtasks do not need each other's context and the fan-out is a latency win. A hard security boundary is another, because an agent that handles untrusted input should not be the same agent that holds the credentials. Cost and latency both fan out with the number of agents, and so does the debugging surface: a failure in a five-agent system requires reconstructing five conversations to find where it went wrong, which makes tracing and durable execution structural requirements rather than nice-to-haves. The strongest interview answer starts with the arithmetic, proposes the single-agent design, names the specific conditions under which it would split, and treats the orchestration as a workflow problem with a model inside it rather than as a conversation between colleagues.

Roni Das

22 September 2026 · 30 min read

Where it shows up

Asked wherever agents are being shipped rather than demoed: the AI product teams at Anthropic, OpenAI, Google and Microsoft, the agent and automation startups, developer-tool companies building coding agents, and enterprise platform teams putting agents in front of internal systems. It appears for AI engineer, applied AI, and ML platform roles, and increasingly in senior backend interviews where the real question is whether you treat a non-deterministic component as an ordinary service. A common variant is design an AI agent that can handle customer support end to end, where the interviewer is waiting to see whether you propose five specialised agents without mentioning what happens when one of them is wrong.

Spec sheeta Multi-Agent System

01Compounding success over a chain of steps: 0.95^10 is about 0.60, and 0.95^20 is about 0.36 (arithmetic)
02Steps in a real agent task: a handful for a narrow task, tens for an open-ended one (illustrative)
03Context window consumed per step: grows with every tool result appended, so the later steps of a long run operate on a much fuller context (structural)
04Cost and latency fan-out: roughly linear in the number of agents for cost, and set by the slowest branch for latency (structural)

worked through below, with the maths

Data modelfree account

Deep dives (7)free account

Unlock with the course · $49One payment, lifetime access to every lesson.

Multi-Agent Systems, and When Not to Build One
One agent with good tools is the right default. When a team of agents actually helps, why adding agents usually lowers reliability, and the decision rule.
36 min · diagrams · code you run · quiz
Agent Reliability Math: pass@k, pass^k, and Why 95 Percent Fails
pass@k and pass^k measure opposite things. See the math that makes a 95 percent per-step agent fail most 20-step tasks, with a Python example.
80 min · diagrams · code you run · quiz
The Agent Loop: Why AI Agents Fail as Tasks Get Longer
An AI agent is a language model in a loop, and its reliability compounds: 95 percent per step over 20 steps is only 36 percent. Why agents fail on long tasks, and the levers that fight it.
42 min · diagrams · code you run · quizfree to read now
Durable Execution: An Agent Run Is a Workflow, Not a Request
A web request retries whole. An agent run cannot. Durable execution persists every step and recovers by replaying recorded results, not re-running non-deterministic model calls.
39 min · diagrams · code you run · quiz

all part of

AI Engineering

212 lessons · about 222 hours · 10 free to read

₹2,777in India, by UPI

$49everywhere else, by PayPal

Get the AI Engineering course Read 10 lessons free first

You own this course

Continue with Multi-Agent Systems, and When Not to Build One

pay once,
yours for life

Persist run state so a run survives a process restart, a deploy, or a multi-hour wait on an external system, and resumes rather than restarting

Decide when to stop: on success, on a step budget, on a cost budget, or on repeated failure, and report which of those happened

Escalate to a human at defined points, with enough context for the human to act, and resume afterwards

Where work genuinely fans out, run subtasks in parallel over disjoint context and merge the results, with a defined policy for what happens when one subtask fails

Trace every run end to end: every prompt, tool call, result and decision, tied to one run identifier across all participating agents

Enforce per-agent permissions, so an agent handling untrusted input cannot reach the tools that hold credentials or make irreversible changes

Non-functional requirements

Task completion rate, measured end to end on real tasks rather than per-step accuracy, because per-step accuracy is the number that looks fine while the system fails
Cost per completed task, not cost per call. A run that fails on step eighteen has cost eighteen steps and delivered nothing, so failures are the expensive case
Latency: agent runs are seconds to minutes, which is a different interaction model from a request, and the system should be designed around that rather than fighting it
Durability: a run in progress must survive an infrastructure event, because runs are long enough that infrastructure events happen during them
Determinism where it is available: the orchestration, retries and state transitions should be deterministic even though the model is not, so failures are reproducible
Isolation: a compromised or confused agent must be contained by permissions rather than by instructions, because instructions are exactly what an attacker manipulates
Observability: a trace has to reconstruct the whole run, because the failure is usually three steps before the visible error

Back-of-envelope scale estimates

Show your math. Pulling numbers from thin air signals you have not thought about the load.

Compounding success over a chain of steps

0.95^10 is about 0.60, and 0.95^20 is about 0.36 (arithmetic)

The number to put on the whiteboard first. If each step is independent and succeeds with probability p, the run succeeds with p to the power of the number of steps. A 95 percent step is excellent and a 36 percent task is a broken product, and the gap between those two statements is the entire problem. It also explains why the highest-value change is usually removing steps rather than improving the model.

Steps in a real agent task

a handful for a narrow task, tens for an open-ended one (illustrative)

Step count is the exponent, so it matters more than anything else in the arithmetic above. This is the argument for collapsing several reasoning steps into one deterministic tool: a tool that does the right thing every time replaces a step whose probability is less than one, and shortens the exponent at the same time.

Context window consumed per step

grows with every tool result appended, so the later steps of a long run operate on a much fuller context (structural)

Each observation is appended to the conversation, so a long run carries an increasingly large context, which raises cost per step and degrades the model's attention to the earliest instructions. This is one reason step quality is not constant across a run, and why the independence assumption in the arithmetic above is optimistic rather than pessimistic.

Cost and latency fan-out

roughly linear in the number of agents for cost, and set by the slowest branch for latency (structural)

Every agent is its own sequence of model calls with its own context, so five agents is roughly five times the token spend for the same task unless the work is genuinely disjoint. Parallel fan-out buys latency because branches run concurrently, but the run still waits for the slowest branch, and a retry in one branch delays everything.

Human escalation rate

a design parameter, not a measurement, and it should be set deliberately (illustrative)

How often the system asks a person is a choice about where the cost of being wrong is too high to absorb. Setting it to zero is what makes an agent look impressive in a demo and dangerous in production. Treating it as a tunable, with the threshold tied to the reversibility of the action, is the production answer.

High-level architecture

Start with one agent, because that is what most designs should be, and add the second only against a stated reason. The single agent is a loop: a model receives the goal and the history, chooses a tool call, the runtime executes it, the result is appended, and the loop repeats until a stop condition. The interesting engineering is not in the loop, it is around it. The runtime that hosts the loop should be a durable workflow rather than a long-lived process. An agent run lasts seconds to minutes, sometimes longer when it waits on a human or an external system, which is long enough that deploys, restarts and instance failures happen mid-run. Modelling the run as a workflow with persisted state means each step is a checkpoint, a crash resumes rather than restarts, and retries do not re-execute side effects that already happened. That framing also gives you determinism where it is available: the orchestration, retry policy and state transitions are deterministic even though the model's output is not, so a failed run can be replayed. Tools sit behind a layer that validates arguments against a schema before execution, applies the permissions of the run, and makes destructive operations idempotent with a key derived from the run and step. Most agent failures that reach production are tool failures wearing a model costume: an argument that was almost right, a retry that duplicated a side effect, a result the model misread because it was an unstructured blob. When the system does fan out, the shape that works is an orchestrator that owns the goal and the final answer, and workers that each receive a self-contained brief and return a structured result. The orchestrator holds the context; the workers do not talk to each other. Peer-to-peer topologies where agents converse are where the context loss and the runaway loops live, and they are much harder to bound and to debug. The natural fit for this shape is genuinely parallel work over disjoint sources, where each worker's context is small and independent. The cross-cutting requirement is one trace per run, spanning every agent, every prompt, every tool call and every decision, tied to a single run identifier. Without it a five-agent failure is unresolvable, because the visible error is usually several steps downstream of the actual mistake.

In a real interview, sketch this on the whiteboard before diving into any single box.

Core components

Walk through each service. The interviewer wants to hear what each one owns, not just the names.

Agent loop runtime

Owns one run: assembles the prompt from goal, history and available tools, calls the model, parses the chosen action, executes it, appends the observation, and repeats. It enforces the budgets, which are the only thing standing between a confused model and an unbounded bill: maximum steps, maximum tokens, maximum wall clock, and maximum spend. Every one of those should terminate the run with a distinguishable reason, because which budget was hit tells you what went wrong.

Durable execution layer

Persists run state after each step so a run survives restarts and can wait hours on an external system without holding a process. This is what turns an agent from a long request into a workflow. It also provides the replay property: because orchestration is deterministic, a failed run can be re-run against the recorded model outputs to reproduce the failure, which is the difference between debugging and guessing. Temporal-style workflow engines are the common implementation and the concepts transfer whichever one you pick.

Tool layer with schema validation and idempotency

Validates every argument against a schema before execution and rejects malformed calls back to the model as a structured error it can act on, rather than letting a bad argument reach a production system. Destructive tools take an idempotency key derived from run and step, so a retry after a timeout cannot charge a card twice. Return structured results rather than prose, because a tool that returns an unformatted blob pushes parsing back into the model and adds a failure mode.

Context manager

Decides what the model sees. Naively appending every observation grows the context until cost rises, the earliest instructions lose the model's attention, and eventually the window overflows mid-run. The manager summarises or drops old observations, keeps the goal and constraints pinned, and stores the full history outside the window so a summary can be expanded when needed. This component is usually missing from whiteboard designs and is usually the reason long runs degrade.

Orchestrator

Present only when the system genuinely fans out. It owns the goal, decomposes it into briefs that are self-contained, dispatches workers, and merges structured results into the final answer. Crucially it is the only thing that holds the whole picture; workers do not have peer channels. It also owns the failure policy for branches, which must be explicit: fail the run, retry the branch, or continue with a partial result and say so.

Worker agents

Each gets a narrow brief, a small set of tools, and its own short context, and returns a structured result. Narrow scope is the point: a worker with three tools and one job has a much higher per-step success rate than a generalist with thirty, which is the only way adding agents improves rather than degrades the arithmetic. If a worker needs to ask the orchestrator a question mid-task, the brief was not self-contained, and that is a design signal rather than a feature to add.

Permission and isolation boundary

Each agent runs with its own identity and its own scoped credentials, enforced by the tool layer rather than by instructions in a prompt. This is the one architectural reason to split agents that has nothing to do with capability: an agent that reads untrusted content should not be the agent that holds write access, because prompt injection targets exactly that combination. Sandboxing for code execution belongs here too.

Trace store and evaluation harness

One trace per run covering every agent, prompt, tool call, result and decision, keyed by run identifier. It serves two jobs: debugging a specific failure, and building an evaluation set from real runs so that a prompt or model change can be measured on end-to-end task completion rather than on vibes. Per-step accuracy is the metric that looks fine while the product fails, so the harness has to score whole tasks.

the deep dive

Data model

runs. One row per task, and the row a product metric is computed from. `terminated_by` is the field that makes failures diagnosable in aggregate: a system hitting its step…

free with an account

Every table for a Multi-Agent System, its keys, and which store fits it and why.

One tap with Google. Free, no card.

the deep dive

Deep dives

The arithmetic, first, before any architecture. If a step succeeds independently with probability p, a run of n steps succeeds with p to the power n. At p = 0.95, ten steps gives about 0.60 and twenty gives about…

free with an account

7 deep dives on the parts that break at scale: The arithmetic, first, before any architecture; Every handoff loses context, and the loss is invisible; When a second agent genuinely earns its place.

One tap with Google. Free, no card.

the deep dive

Trade-offs to discuss

One agent with better tools versus several specialised agents. One agent keeps the context intact, keeps the chain short, and is far easier to debug and evaluate, and for most tasks it wins outright. Several agents can help when…

free with an account

7 decisions an interviewer expects you to defend, each with the alternative and why it loses here.

One tap with Google. Free, no card.

for course owners

What the interviewer asks next

A research fan-out sends ten workers to ten sources, and three of them fail. What does the orchestrator do?

with the System Design course

4 follow-up questions with model answers, the interview scorecard for this question, and a mock interview on Design a Multi-Agent System with the AI tutor.

Unlock with the course · $49One payment, lifetime access to every lesson.

How a Multi-Agent System actually does it

The public literature on this has converged faster than most, and usefully it has converged towards restraint. Anthropic's engineering write-up on building effective agents argues for the simplest thing that works, distinguishes workflows with predetermined paths from agents that direct themselves, and recommends starting with a single well-scoped loop rather than a multi-agent architecture. Anthropic has also published a description of a multi-agent research system, and it is worth reading alongside the first piece precisely because it describes a case where fan-out did pay: parallel research over disjoint sources, which is the shape this page argues is genuine. Cognition published a piece called Don't Build Multi-Agents, whose core point is the context-loss one above: decisions made in one agent's context are invisible to another, and the resulting conflicts are hard to see and harder to fix. OpenAI's practical guide to building agents covers the same ground from the orchestration side, including guardrails and human escalation. For the durability half, Temporal's documentation is the clearest explanation of deterministic replay and why side effects need idempotency, and the concepts carry over to any workflow engine. Our own course chapter on this is called Multi-Agent Systems, and When Not to Build One, and the neighbouring chapter on agent reliability math is where the compounding arithmetic on this page comes from. What this page deliberately does not do is quote a benchmark number for how much better a multi-agent setup performs. Those figures are extremely sensitive to the task, the harness, the number of attempts allowed and the model version, and a number quoted without all four is not evidence.

Sources

Was this helpful?

the question most readers have next

The interviewer's next question: what breaks when the Multi-Agent System design runs at full scale?

A whiteboard answer gets you through the first round. The follow-ups are about the parts that fail under real load. The lessons below teach them one at a time, starting with "Multi-Agent Systems, and When Not to Build One".

Where this is used in industry

The real examples, each with its source, are on this page: How a Multi-Agent System actually does it.

The lessons that teach the next part

One agent with good tools is the right default. When a team of agents actually helps, why adding agents usually lowers reliability, and the decision rule.
Multi-Agent Systems, and When Not to Build OneAI Engineering course36 min
pass@k and pass^k measure opposite things. See the math that makes a 95 percent per-step agent fail most 20-step tasks, with a Python example.
Agent Reliability Math: pass@k, pass^k, and Why 95 Percent FailsAI Engineering course80 min
An AI agent is a language model in a loop, and its reliability compounds: 95 percent per step over 20 steps is only 36 percent. Why agents fail on long tasks, and the levers that fight it.
The Agent Loop: Why AI Agents Fail as Tasks Get Longerfree lesson42 min
A web request retries whole. An agent run cannot. Durable execution persists every step and recovers by replaying recorded results, not re-running non-deterministic model calls.
Durable Execution: An Agent Run Is a Workflow, Not a RequestAI Engineering course39 min

Stuck on a step? Every lesson has an AI tutor beside it. Ask it your question and it answers from the lesson you are reading. Try the tutor

Get the free AI Engineering cheat sheet (PDF)

Where this is taught

AI Engineering

212 lessons · about 222 hours · 10 free to read

₹2,777 in India by UPI$49 elsewhere by PayPal, once, for life

See the AI Engineering course

You own this course

Also by the same author

Frequently asked questions

India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.

Free PDF · 79 pages

25 real tech interview questions, answered in full

Five from each of our AI, GenAI, Inference, Maths for ML and Forward Deployed packs. Each with a diagram and a model answer. Free by email.

free tool

Check these numbers yourself

Put in your own guesses for a Multi-Agent System and see requests per second, storage, bandwidth and servers, with every sum written out.

Open the Capacity Estimator

770 lessons · about 286 hours · 18 free to read

₹1,777 in India by UPI$49 elsewhere by PayPal, once, for life

See the System Design course

Multi-Agent System Design Interview: The Reliability Math, When a Second Agent Helps, and Why Most Designs Should Be One Agent With Better Tools

Design a Multi-Agent System, part by part

Why this question is asked

The course goes deeper on the Multi-Agent System design

Before you design Multi-Agent, try the questions it rests on.