Multi-Agent Systems: What Breaks When You Add the Second Agent

The pitch for multi-agent architectures is intuitive. Decompose a hard task, give each piece to a specialised agent, let them collaborate. It mirrors how teams work, so it feels like the right shape.

In production the shape brings costs that single-call systems do not have, and most of them show up after the demo. It is worth knowing what they are before committing an architecture to them.

Errors compound rather than average

This is the arithmetic that decides most of these designs. If each agent in a chain is independently correct 95% of the time, a five-step pipeline is correct about 77% of the time. At ten steps you are near 60%.

Human teams do not degrade this way because people notice when the input they were handed is wrong. An agent given a plausible but incorrect intermediate result will usually proceed, confidently, and produce a well-formed answer built on it. There is no equivalent of a colleague saying "this number looks off".

The practical consequence: every handoff needs a check that can fail. Schema validation, a constraint the output must satisfy, a cheap deterministic recomputation. Chains without gates do not degrade gracefully; they produce confident nonsense.

Non-determinism makes evaluation expensive

A single call can be evaluated against a fixed set of inputs and expected outputs. A multi-agent system takes a different path on each run, so the same input can succeed and fail on consecutive executions.

That breaks the normal test discipline. You cannot assert on an exact trajectory, and asserting only on the final answer hides which step went wrong. Teams end up running each case many times and tracking a pass rate — which works, but means your test suite is now a statistical instrument with a meaningful cost per run.

Budget for that. An evaluation suite you cannot afford to run on every change is one you will stop running.

Cost and latency are multiplied, not added

Each agent re-reads context. In a conversation that accumulates state, later steps carry the transcript of earlier ones, so token consumption grows faster than the step count. A five-agent workflow can cost fifteen times a single call rather than five, and the latency is serial unless you have designed for parallelism.

This is survivable for a workflow that runs a hundred times a day and replaces an hour of skilled work. It is not survivable in an interactive path where a user is waiting, or at a volume where the unit economics have to work.

Debugging crosses a boundary that tooling is bad at

When a five-step run produces a wrong answer, the question is which step introduced the error, and the evidence is a long trace of natural language. Standard observability was built for structured events, not for reading transcripts.

Instrument from the first day: persist every prompt, every response, every tool call, with the run id that ties them together. Teams that add this after the first production incident spend the incident blind.

When the second agent earns its place

Three cases, consistently:

• Genuinely different tools or permissions. A step that queries production data and a step that drafts customer-facing text should not share credentials. Separation here is a security boundary, not an architectural preference.
• Generate-then-critique. A second pass that only checks, with the authority to reject, catches a class of error the generating pass is structurally blind to. This is the cheapest multi-agent pattern and usually the highest value.
• Long tasks needing parallelism. Independent subtasks that fan out and join. The win is wall-clock time, and it only holds when the subtasks really are independent.

When we would advise against it

If a single well-constructed call with the right tools available can do the job, use that. It is cheaper, faster, testable with ordinary fixtures, and debuggable by reading one exchange.

We have replaced more multi-agent pipelines with one call and a good tool definition than we have built from scratch. The usual cause is that the decomposition was modelled on how a human team is organised rather than on where the task actually needs a boundary. Organisational structure is not a system requirement.

A reasonable path

Start with one call. Add a critique pass when you can name the error class it catches. Split further only when a step needs different permissions or a genuinely different tool. At each addition, write down the check that runs at the handoff — if you cannot define one, you have added failure surface without adding safety.

Then measure the whole thing end to end, against the single-call baseline you started with, on cost, latency and pass rate. Keep that baseline runnable. It is the only honest way to know whether the complexity is paying for itself, and it is the first thing teams throw away.

State is the part that gets designed last

Single-call systems have no memory problem: the context is the request. Once agents hand off, something has to decide what each one is allowed to see, what carries forward and what gets discarded.

Passing everything is the default and it is what makes cost grow superlinearly. Passing too little produces agents that contradict each other because they are reasoning from different pictures. Neither failure is obvious in a demo with three steps and a short transcript.

Treat the handoff payload as an interface and specify it. A typed object with named fields is easier to validate, cheaper to pass, and makes the failure legible when a downstream step is missing something. Free-form transcript passing feels flexible and is the reason these systems become impossible to reason about at step seven.

There is also a security dimension. If one agent reads customer records and another drafts outbound text, the transcript is the path by which data crosses that boundary. Deciding what is in the payload is deciding what your permission split actually enforces.

We cover the surrounding economics in what RAG costs in production, and the delivery side in AI engineering.