Orchestrator-Workers and Multi-Agent Systems: When to Use More Than One Agent

When a task grows, it is tempting to add more agents: a manager, some specialists, maybe a reviewer. Sometimes that is exactly right. Often it just multiplies cost and confusion. This lesson covers the most common multi-agent design, orchestrator-workers, where a lead agent splits a task, hands the pieces to workers and combines what they return. You will learn when it earns its cost, how to write a handoff that works, how it fails, and how to build a small, tested version.

What orchestrator-workers means

Anthropic's guide to building effective agents describes it this way: a central model splits a task into subtasks, gives them to worker models, and combines their results. It suits complex tasks where you cannot know the subtasks in advance. Its examples are code changes that touch several files, where the number of files and the change in each depend on the task, and searches that gather and analyze information from several sources.

This looks like the parallel pattern in Chaining, routing and parallelization, with one key difference. There, you wrote the subtasks into your code. Here, the orchestrator decides the subtasks for each input.

Three ideas make it work:

1. The lead plans and combines. It holds the big picture and the final answer.
2. Each worker has its own context. A worker starts empty, does one job, and returns a short result. Its long search history never reaches the lead.
3. Workers can run in parallel. If subtasks are independent, they run at once.

What multi-agent systems are good for

Anthropic has written up how it built its own multi-agent research system. Some of what it reported, from its own tests:

• On Anthropic's internal research evaluation, a system with a Claude Opus 4 lead and Claude Sonnet 4 workers beat a single Claude Opus 4 agent by 90.2 percent.
• On a browsing benchmark, token usage alone explained about 80 percent of the variation in performance. More work spent on a problem tended to give better answers, and several workers with separate contexts can spend more than one context window allows.
• Running subagents and their tool calls in parallel cut research time by up to 90 percent on complex queries.

These figures are Anthropic's, from its own system and its own tests, in 2025. Treat them as a sign of what is possible, not as a promise for your task.

The strong fits are:

• Heavily parallel work, such as researching many companies or sources at once.
• More information than fits in one context window. Each worker reads its share and reports back briefly.
• Many complex tools, where giving each worker a smaller toolset keeps it focused.

What they cost

The same write-up gives the price. Agents used about four times the tokens of a chat interaction, and multi-agent systems about fifteen times. Anthropic's conclusion was that the task has to be valuable enough to justify that cost.

There are other costs that do not show up on the bill:

• More moving parts. More prompts to write, more places for a mistake to start.
• Harder debugging. Runs differ from one to the next, so you need tracing. See Observability and tracing.
• Coordination errors. Workers can duplicate each other, miss a gap, or contradict each other.

When a single agent is better

Anthropic notes that multi-agent systems are a poor fit where agents need the same shared context or have many dependencies on one another. It adds that most coding tasks have fewer truly parallel pieces than research does, and that agents are not yet strong at real-time coordination with each other.

Prefer one agent, or a plain workflow, when:

• The steps depend on each other, so there is little to run in parallel.
• Everyone needs the same full context.
• The task is small. A lookup does not need a team.
• You have not yet shown that a single agent with good tools fails at it.

A sensible order is: try a single call, then a workflow from Chaining, routing and parallelization, then a single agent with good tools, and only then several agents. Measure at each step, as in Evaluating AI agents.

The handoff is the hard part

Most multi-agent failures start with a vague instruction. A worker has no memory of your conversation. Whatever you want it to know must be in its brief.

Anthropic reports that each subagent needs an objective, an output format, and guidance on tools and sources, and that vague instructions led to workers repeating one another's searches. A good brief holds:

• An objective, one clear job.
• An output format, so the lead can combine results without guessing.
• Boundaries, such as what other workers cover, so work does not overlap.
• Tool and source guidance, where it helps.
• A size limit on the answer, so the lead's context does not fill up.

The return trip matters as much. A worker should send back findings, not its whole history. That keeps the lead's context small, which is the reason for having workers in the first place. See Context engineering.

Shared state

Workers usually do not share a context. If they need to share facts, choose one place:

• Through the lead. The lead passes facts into each brief. Simple and safe.
• A shared store, such as a file or database that workers read and write. Flexible, but two workers writing the same thing can clash, so give each its own output location or make writes safe.
• A memory tool, as in Maintaining agent memory. Anthropic's research system saved the lead's plan to memory because the lead's context could be cut off in long runs.

Start with "through the lead". Add shared storage only when you have a reason.

The code

This code is a small orchestrator. The plan, the work and the final answer are passed in as functions, so you can test the whole thing without the API. Wrappers at the bottom connect them to Claude, using the larger model to plan and combine, and the smaller one for the workers.

1import json
2from concurrent.futures import ThreadPoolExecutor
3
4MODEL = "claude-sonnet-5-5"
5WORKER_MODEL = "claude-haiku-5-5"
6
7
8def ask(client, prompt, system="", model=MODEL):
9    reply = client.messages.create(
10        model=model, max_tokens=1024, system=system,
11        messages=[{"role": "user", "content": prompt}],
12    )
13    return reply.content[0].text
14
15
16def parse_subtasks(text, max_subtasks=5):
17    """Model output is untrusted: keep only well-formed, distinct subtasks, and cap how many."""
18    try:
19        items = json.loads(text)
20    except ValueError:
21        return []
22    seen, subtasks = set(), []
23    for item in items if isinstance(items, list) else []:
24        if not isinstance(item, dict):
25            continue
26        objective, fmt = item.get("objective"), item.get("output_format")
27        if not (isinstance(objective, str) and objective.strip() and isinstance(fmt, str) and fmt.strip()):
28            continue
29        if objective.strip().lower() in seen:
30            continue
31        seen.add(objective.strip().lower())
32        subtasks.append({"objective": objective.strip(), "output_format": fmt.strip()})
33    return subtasks[:max_subtasks]
34
35
36def make_brief(task, subtask):
37    """A worker starts with an empty context, so the brief must say everything it needs."""
38    return (f"Overall task (for context only): {task}\n\n"
39            f"Your objective: {subtask['objective']}\n"
40            f"Return: {subtask['output_format']}\n"
41            "Do only this objective. Other workers cover the rest. Reply with findings only.")
42
43
44def orchestrate(task, plan, work, synthesize, max_subtasks=5, max_chars=2000, max_workers=4):
45    subtasks = parse_subtasks(plan(task), max_subtasks)
46    if not subtasks:
47        return {"ok": False, "error": "the plan had no usable subtasks", "subtasks": []}
48
49    def run(subtask):
50        try:
51            return work(make_brief(task, subtask))[:max_chars], None
52        except Exception as exc:
53            return None, f"{type(exc).__name__}: {exc}"
54
55    with ThreadPoolExecutor(max_workers=max_workers) as pool:
56        outcomes = list(pool.map(run, subtasks))
57    results = [(s["objective"], o[0]) for s, o in zip(subtasks, outcomes) if o[1] is None]
58    failed = {s["objective"]: o[1] for s, o in zip(subtasks, outcomes) if o[1] is not None}
59    if not results:
60        return {"ok": False, "error": "every worker failed", "failed": failed, "subtasks": subtasks}
61    return {"ok": True, "answer": synthesize(task, results, failed), "failed": failed, "subtasks": subtasks}
62
63
64def claude_plan(client):
65    system = ('Split the task into 2 to 5 independent subtasks. Reply with JSON only: '
66              '[{"objective": "...", "output_format": "..."}]. Each objective must be specific and not overlap another.')
67    return lambda task: ask(client, task, system)
68
69
70def claude_work(client):
71    return lambda brief: ask(client, brief, model=WORKER_MODEL)
72
73
74def claude_synthesize(client):
75    def synthesize(task, results, failed):
76        parts = "\n\n".join(f"## {obj}\n{text}" for obj, text in results)
77        note = f"\n\nThese subtasks failed and are missing: {list(failed)}" if failed else ""
78        return ask(client, f"Task: {task}\n\nWorker findings:\n{parts}{note}\n\nWrite the final answer. Say what is missing.")
79    return synthesize

The tests use a stand-in client that returns scripted replies. Nothing here was run against a live model.

How it protects you

• parse_subtasks distrusts the plan. The plan is model output. The function keeps only items that have both an objective and an output format, drops duplicates, and caps how many subtasks you accept. Anthropic describes early versions of its system spawning fifty workers for a simple query, and a cap is the cheapest defense.
• make_brief is a template. Every worker gets an objective, a return format and a boundary, so no brief is written ad hoc.
• Each worker is isolated. A worker that fails or times out does not stop the others. Its error is recorded.
• Output is cut to max_chars. A runaway worker cannot flood the lead.
• The combiner is told what failed. synthesize receives the failures, and the Claude version tells the model to say what is missing, so a partial result is not presented as a complete one.
• If nothing usable comes back, the function says so and does not invent an answer.

What it leaves out

For clarity this version runs one round: plan, work, combine. Anthropic's system lets the lead start more workers or change course after seeing results, and keeps a separate agent to attach citations. That is a loop around this code, and it needs the stopping rules from Self-critique and evaluator-optimizer so it does not run forever. It also runs workers in a batch and waits, which the same write-up notes limits steering and coordination.

Scale the effort to the task

The research write-up found that early versions did not judge how much effort a query deserved. Its guidance was to scale effort to complexity: one worker with a few tool calls for a simple fact lookup, a handful of workers for a comparison, and many for complex research. Build that into your planner prompt, and enforce it in code with the cap on subtasks.

Failure modes

1. Duplicated work. Two workers do the same job because the briefs overlap. Write boundaries.
2. Gaps. Nobody covers a part of the task. Ask the lead to check coverage when combining.
3. Too many workers. Cost explodes. Cap them.
4. Contradictions. Workers disagree and the lead picks one without saying. Ask it to flag conflicts.
5. Lost detail. A worker's summary drops something that mattered. Specify what must be kept.
6. Compounding errors. A wrong early result flows into later steps. Anthropic reports resuming from checkpoints instead of restarting when a run fails partway.
7. Hard to see inside. Without traces, you cannot tell which worker went wrong.
8. Prompt injection spreading. A worker that reads a web page can pass hostile text up to the lead. Treat worker output as untrusted data, as covered in Prompt injection and tool sandboxing.

Orchestrator-workers or handoff

Two layouts are easy to mix up:

• Orchestrator-workers. The lead stays in charge. Workers answer and return. The lead writes the final answer.
• Handoff. Control passes from one agent to another, and the second agent takes over the conversation. A support agent that passes a billing question to a billing agent is an example.

Both are used. Some frameworks use the word "handoff" for the second one. The names differ between tools, so check the one you use. Agent frameworks compared looks at them.

Mistakes beginners make

1. Starting with many agents before showing that one agent fails.
2. Vague briefs, such as "look into pricing".
3. No cap on the number of workers.
4. Returning whole histories from workers to the lead.
5. Believing worker output without checking it.
6. No handling for a failed worker.
7. Using several agents for steps that depend on each other.
8. Ignoring cost. Fifteen times the tokens needs a task that is worth it.
9. No tracing, so failures cannot be traced to a worker.
10. Never comparing against a single agent on the same tasks.

Quick recap

• In orchestrator-workers, a lead splits the task, gives each worker a brief, and combines their short results. The lead decides the subtasks for each input.
• It fits heavily parallel work, information beyond one context window, and tasks with many tools.
• It costs a lot. Anthropic reported about four times the tokens of chat for agents and about fifteen times for multi-agent systems.
• It is a poor fit for tightly dependent work, and a single agent or workflow is often better.
• A good brief has an objective, an output format, boundaries and a size limit. Workers return findings, not histories.
• Treat the plan and the worker outputs as untrusted. Cap the workers, isolate failures, and tell the combiner what is missing.
• Measure against a single agent before you commit to the extra cost.

Try it yourself

Take a task such as "compare three laptops for a student". Write the plan by hand as JSON with an objective and an output format for each subtask, and run it through parse_subtasks. Then break it on purpose: duplicate an objective, remove a format, and add fifty items, and see what survives. Next run orchestrate with fake workers where one raises an error, and read what the combiner receives. Finally, write a brief for one worker that has an objective, a format, a boundary and a size limit, and check that a stranger could do the job from the brief alone.

Next lesson: Human-in-the-loop agents.