Multi-Agent Systems: When They Help and When They Hurt

Your agent works, but it is slow, its context is a mess, and it keeps forgetting the start of the task. So you think: what if I add more agents? Sometimes that is exactly right. Other times it triples your bill and makes bugs impossible to find. Today we learn how to tell the difference.

• One agent or many?
• Coordination is not free

In the last lesson we gave a single agent memory and context management. Now we ask a bigger question: when should several agents work together, and how do we wire them up? By the end you will know five patterns, a cost model with worked numbers, and a small Python orchestrator with a trace.

• Tools, ReAct, memory
• Now: many agents

Think of a newsroom. A reporter gathers facts, an editor shapes the story, a fact-checker hunts for errors. Nobody holds the whole job in their head. Multi-agent systems borrow that idea: each agent gets its own prompt, its own tools, and its own context window, focused on one role.

• Divide work by role
• Each sees only what it needs

The main benefit is not magic intelligence. It is context isolation. A worker can read fifty pages of search results, then hand back a short summary. The parent never sees the noise. Smaller, cleaner contexts mean cheaper calls, less distraction, and often better answers on each sub-task.

• Separate context windows
• Less noise per agent

But coordination is never free. Every message between agents costs tokens and time. Every handoff is a chance to drop a detail, because a summary is lossy. With n agents that can talk to each other, the number of possible conversation links grows roughly with n squared. More agents means more places for things to go wrong.

• Every message costs tokens
• Every handoff can lose detail

Pattern one is orchestrator and worker. A lead agent reads the task, breaks it into subtasks, sends each to a worker, then synthesizes the results. Workers are often short-lived and focused. This is the most common pattern, and it is the one we will code later in this lesson.

• Plan, delegate, synthesize
• Workers return summaries

Pattern two is the supervisor, or router. Instead of splitting a task, it picks who handles it. A billing question goes to the billing agent, a code question goes to the coding agent. The router can be a small, cheap model, because classifying is easier than solving. Only one specialist runs per request.

• Classify, then route
• One specialist handles it

Pattern three is the pipeline. Agents run in a fixed order: research, then draft, then edit, then format. It is predictable and easy to debug, since you can inspect the output between every stage. The weakness is rigidity. If stage one makes a mistake, every later stage inherits it.

• Fixed stages in order
• Output feeds next input

Pattern four is debate or critic. One agent proposes an answer, another tries to find flaws, and they loop until the critic is satisfied or a round limit is hit. This helps when errors are easier to spot than to avoid, like bugs in code or gaps in an argument. Always cap the number of rounds.

• Generator proposes
• Critic finds flaws

Pattern five is parallel sub-agents with isolated contexts. If a task splits into independent pieces, like researching five companies, run five agents at the same time. Wall-clock time drops to roughly the slowest worker, not the sum. The pieces must truly be independent, otherwise they will duplicate work or contradict each other.

• Independent tasks at once
• Isolated contexts

To recap the five patterns. Orchestrator-worker splits and merges. Router picks one specialist. Pipeline chains stages. Critic loops generation against review. Parallel fans out independent work. Real systems mix them, for example a router in front, and an orchestrator with parallel workers behind it.

• Orchestrator: split and merge
• Router: choose one
• Pipeline, critic, parallel

How do agents actually communicate? Three common ways. Shared state, where everyone reads and writes a common store. Message passing, where agents send explicit messages to each other. And handoffs, where one agent transfers control, and the conversation, to another. Each trades simplicity against isolation.

• Shared state
• Message passing
• Handoffs

Shared state is like a whiteboard. Agents read what they need and write their results back. It is simple, and easy to inspect. The risk is that if every agent reads the whole board, you lose the isolation benefit, and two agents may overwrite each other. Give each agent a narrow view of the state.

• Blackboard everyone edits
• Easy, but contexts leak

With message passing, the parent decides exactly what a worker sees: a task description, constraints, and the output format. That clarity is a feature. With a handoff, a triage agent might pass the whole conversation to a specialist, who then replies to the user directly. Handoffs keep the parent out of the loop, which saves tokens.

• Explicit, typed messages
• Handoff transfers control

A worker starts with zero knowledge. It has not seen your conversation. So a good delegation message states the goal, the scope, what to ignore, the tools allowed, and the exact output format, ideally a length limit. Vague delegation like, look into this, is the number one cause of useless worker output.

• Goal and scope
• Output format and limits

Remember that a summary is compression, and compression loses information. If a worker finds a number, a URL, or a code path that matters, require it to be passed through exactly. A good trick is to have workers store large artifacts, like files, outside the context, and return only a short reference plus a few key facts.

• Compression drops detail
• Pass references, not blobs

Let us do the math with assumed numbers, not measured benchmarks. A research task needs nine tool steps. The system prompt is one thousand tokens. Each step adds fifteen hundred tokens of tool output to the context. Since an agent resends its whole context at every step, input tokens grow with each step.

• Assumed numbers, not benchmarks
• Task needs 9 tool steps

One agent first. At step k, the input is one thousand plus fifteen hundred times k minus one. Step one reads one thousand tokens, step nine reads thirteen thousand. Add them up: nine thousand for the system prompt copies, plus fifteen hundred times the sum zero through eight, which is thirty-six. That is fifty-four thousand. Total: sixty-three thousand input tokens.

• Step k reads 1,000 + 1,500(k-1)
• Total: 63,000 input tokens

Now an orchestrator and three workers, each doing three steps in its own context. A worker reads one thousand, twenty-five hundred, then four thousand: seven thousand five hundred tokens. Three workers give twenty-two thousand five hundred. The orchestrator plans with twelve hundred tokens, and synthesizes with twenty-seven hundred after reading three summaries. Total, about twenty-six thousand four hundred.

• Workers: 3 steps each
• Total: about 26,400 input tokens

Why did splitting help? Because cost per step depends on context length, and a single long context makes every later step expensive. Three short contexts beat one long one. In this assumed example, the multi-agent version uses about forty percent of the input tokens, and it can also run the workers in parallel to cut latency.

• Cost grows with context squared-ish
• Short contexts are cheap

Now flip the situation. Suppose the task is to analyze one twenty thousand token contract, and four agents each need the full text to check different clauses. Each reads it separately: eighty thousand tokens, before any reasoning. One agent reads it once: twenty thousand. When agents need the same big context, splitting multiplies cost by the number of agents.

• 4 agents each need the same document
• 80,000 vs 20,000 tokens

So when is one agent better? More often than hype suggests. Beyond cost, there is latency, because sequential handoffs add model calls. There is error propagation, since one bad summary poisons everything downstream. And there is debugging: failures now hide in the interactions between agents, not in a single transcript.

• Cost and latency
• Error propagation
• Debugging difficulty

Here is an illustrative calculation. Suppose each stage in a pipeline succeeds ninety percent of the time, and failures are not caught. Two stages give eighty-one percent. Four stages give point nine to the fourth, about sixty-six percent. Each additional agent is another chance to fail, unless you add checks between them.

• Per-stage success 90%
• Four stages: about 66%

A practical rule: start with one agent and a good set of tools. Split only when you can name the reason. Context overload, where tool output swamps the window. Independent subtasks that benefit from parallelism. Genuinely different tools or permissions per role. Or a review step that needs a fresh perspective. If you cannot name one, stay single.

• Start with one agent
• Split only for a clear reason

Now the code. We reuse the agent class from earlier lessons: it has a name, a system prompt, optional tools, and a run method that executes the ReAct loop and returns final text. We will create two instances. A researcher with a search tool, and a writer with no tools, whose only job is clear prose.

• Agent: name, system prompt, tools
• run(task) returns text

The orchestrator needs a delegate function. It takes a worker, a task string, and a trace list. It records the task, calls the worker's run method, records the result and its token usage, and returns the text. Because every call goes through this one function, we get a complete trace almost for free.

• Wrap worker call
• Log every step to a trace

The orchestrator itself is short. It asks the model for a plan of at most three research questions. For each question it delegates to the researcher, collecting short notes. Then it delegates to the writer, passing the notes and a format instruction. Notice the max delegations constant. It guarantees the loop can never run forever.

• Plan, research, write
• Hard limit on delegations

Running it on the goal, explain how vector indexes work, produces a trace. Orchestrator to researcher: question one. Researcher back: three lines of notes, nine hundred tokens used. Same for questions two and three. Then orchestrator to writer with the notes, and the writer returns the final text. When the answer is wrong, you read this trace and see exactly which handoff dropped the fact.

• Who asked whom, and why
• Tokens per call

The three research questions are independent, so we can run them concurrently. Use a thread pool or asyncio to launch all three delegate calls, then wait for the results. Latency drops from the sum of three workers to about the slowest one. Just remember to protect any shared state, and to append to the trace safely.

• Independent questions run together
• Use threads or asyncio

You do not have to write this from scratch. Several frameworks provide these patterns, including LangGraph, AutoGen, CrewAI, and the OpenAI Agents SDK, among others. They differ in abstractions, such as graphs, conversations, or roles, but under the hood they all do what we just did: call a model, pass messages, and manage state. Pick based on your needs, and understand the loop first.

• LangGraph, AutoGen, CrewAI
• OpenAI Agents SDK, others

Where does this show up in practice? Research tasks that fan out across many sources. Coding workflows where one agent writes and another reviews or runs tests. Customer support, where a router sends requests to specialists with different permissions. Notice the common thread: each case has a clear reason to separate roles, context, or access.

• Research and analysis
• Coding with review
• Customer support routing

Mistake one is over-engineering. A planner, a researcher, a writer, an editor, and a manager, all to summarize a web page. Each extra agent adds calls, latency, and failure modes. If one prompt and two tools solve it, stop there. Measure first, and add an agent only when a clear metric improves.

• Five agents for a simple task
• Complexity without gain

Mistake two is unclear roles. If two agents both think they own the final answer, they will duplicate effort or contradict each other. Write each role as a contract: what it receives, what it may do, what it must return, and what it must never do. Distinct tools and permissions make roles enforceable, not just polite suggestions.

• Overlapping responsibilities
• Conflicting instructions

Mistake three is infinite delegation. Agent A asks B for help, B asks A, and tokens burn until your budget is gone. Prevent it with hard limits: a maximum delegation depth, a maximum number of rounds, a total token budget, and a rule that workers cannot call the orchestrator. Always fail loudly when a limit is hit, with the trace attached.

• Workers that delegate back
• Loops burn tokens

One more: never comparing against a baseline. Before you ship a multi-agent system, run the same test set through a single agent. Track quality, cost, and latency for both. If the multi-agent version is not clearly better on something you care about, you have built a more expensive way to get the same answer.

• Compare against a single-agent baseline
• Track cost, latency, quality

Let us recap. Multi-agent systems split work across specialists with separate contexts. Five patterns cover most designs: orchestrator, router, pipeline, critic, and parallel workers. Agents communicate through shared state, messages, or handoffs. Splitting can cut cost when contexts are independent, and multiply it when they are shared. Start with one agent, split for a named reason, and always set limits.

• Isolation is the main benefit
• Coordination always costs
• Start single, split with reason

Every agent we built needs tools and data, and so far we wired each one by hand. In the next lesson we meet the Model Context Protocol, MCP, an open standard for connecting models to tools and context in a consistent way.

• A standard for tools and context
• Subscribe for the next lesson