Chaining, Routing and Parallelization: Three Core Workflow Patterns for AI Agents

Many useful AI systems are not free-roaming agents. They are workflows: a few model calls arranged in a fixed shape that you chose in advance. Three shapes cover a large share of real work. A chain passes work from one call to the next. A router sends each input to the right handler. Parallelization runs several calls at once. These come from Anthropic's guide to building effective agents, and they are the sensible first things to try before reaching for a full agent. You will learn what each does, when to pick it, and how to build it in tested Python.

Why start with workflows

As Workflows or agents explains, a workflow follows a path your code defines, while an agent decides its own path. Workflows are easier to test, cheaper to run and more predictable. If the shape of the task is known ahead of time, write that shape down. Keep the free-roaming agent for tasks where you truly cannot.

Pattern 1: prompt chaining

In a chain, each call works on the output of the one before. Anthropic's guide describes it as a good fit when a task splits cleanly into fixed steps, and notes the trade: you accept more delay in return for higher accuracy, because each call has an easier job. Examples it gives include writing an outline, checking the outline, and then writing the document from it.

The most useful addition is a gate: a plain code check between steps that asks "is this good enough to continue?" A gate can test that the output is not empty, has the right fields, stays under a length, or parses as JSON. If it fails, the chain stops early with a clear reason, and you skip the cost of the later steps.

Chaining helps because:

• Each prompt has one job, so it can be short and clear.
• You can test each step on its own.
• A problem shows up where it started, not three steps later.

It is worse when the steps cannot really be separated, or when the extra delay matters more than the accuracy gain.

Pattern 2: routing

A router looks at the input, picks a category, and sends it to a handler built for that category. The guide says this lets you write specialized prompts, because tuning one prompt for one kind of input can hurt another kind. It works best when the categories are distinct and can be classified accurately, whether by a model or by an ordinary classifier.

Two common uses:

• By input type. Refund requests, technical questions and general questions each get their own prompt and tools.
• By model size. Easy, common questions go to a small, cheap model. Harder ones go to a more capable model. This is a cost and speed optimization.

The router itself is a point of failure. If it labels an input wrongly, the wrong handler answers. Two protections:

1. A default. If the label is not one you expect, fall back to a safe general handler. Model output can be anything, so never assume it matches your list.
2. Test the classifier on real examples, as you would any component. See Evaluating AI agents.

Pattern 3: parallelization

Parallelization runs several model calls at the same moment and combines the results in code. Anthropic describes two forms.

Sectioning splits a task into independent parts that run side by side. Examples: one call answers a user while another screens the message for inappropriate content, or several calls each score a different quality of an output. You gain speed, and each call can focus on one concern.

Voting runs the same task several times and compares the answers. Examples: several reviews of code for a vulnerability, or several judgments on whether content is acceptable. A vote threshold lets you balance false alarms against misses. You gain confidence, because one odd answer is outvoted.

Sectioning is close to the dependency graph in Planner-executor, DAG agents and tree of thought, but simpler: the tasks are fixed in advance and none depends on another.

The code

Here are all four helpers. They take plain functions, so you can use them with any client and test them without the API.

1from collections import Counter
2from concurrent.futures import ThreadPoolExecutor
3
4SMALL, LARGE = "claude-haiku-5-5", "claude-sonnet-5-5"
5
6
7def ask(client, prompt, system="", model=SMALL):
8    reply = client.messages.create(
9        model=model, max_tokens=1024, system=system,
10        messages=[{"role": "user", "content": prompt}],
11    )
12    return reply.content[0].text
13
14
15def chain(steps, text, gate=None):
16    """Run steps in order. A gate can stop the chain by returning a problem string."""
17    outputs = []
18    for name, step in steps:
19        text = step(text)
20        outputs.append((name, text))
21        problem = gate(name, text) if gate else None
22        if problem:
23            return {"ok": False, "failed_at": name, "problem": problem, "outputs": outputs}
24    return {"ok": True, "result": text, "outputs": outputs}
25
26
27def route(text, classify, handlers, default):
28    """Send the input to one handler. An unknown label falls back to the default."""
29    label = classify(text)
30    if label not in handlers:
31        label = default
32    return label, handlers[label](text)
33
34
35def sections(text, tasks, max_workers=4):
36    """Run independent tasks on the same input in parallel. One failure does not hide the rest."""
37    def run(task):
38        try:
39            return task(text), None
40        except Exception as exc:
41            return None, f"{type(exc).__name__}: {exc}"
42    with ThreadPoolExecutor(max_workers=max_workers) as pool:
43        outcomes = list(pool.map(run, tasks.values()))
44    results = {n: o[0] for n, o in zip(tasks, outcomes) if o[1] is None}
45    errors = {n: o[1] for n, o in zip(tasks, outcomes) if o[1] is not None}
46    return results, errors
47
48
49def vote(task, text, n=3, min_agree=2):
50    """Run the same task n times and accept an answer only if enough runs agree."""
51    with ThreadPoolExecutor(max_workers=n) as pool:
52        answers = list(pool.map(lambda _: task(text), range(n)))
53    answer, count = Counter(answers).most_common(1)[0]
54    return (answer if count >= min_agree else None), count, answers

How each helper works

• ask is one call to the Messages API. It defaults to the small model and lets you choose the large one.
• chain runs the steps in order, calls the gate after each, and stops at the first problem. The result says which step failed and why, and keeps the outputs so far for debugging.
• route asks classify for a label. If the label is not in handlers, it uses the default. It returns the label as well as the answer, so you can log how traffic is split.
• sections runs the tasks in a thread pool, and catches an error per task. One failed task does not hide the others. Threads suit this work because each call mostly waits for the network.
• vote runs the same task n times and accepts the most common answer only if at least min_agree runs agree. Otherwise it returns None, which means "no confident answer", and you decide what to do.

Putting them to work

Here is how the helpers combine in a small support system. The classifier uses the small model, the refund and technical handlers use the large one, and an unknown label falls back to general.

1from workflows import LARGE, SMALL, ask, chain, route, sections, vote
2
3
4def support_reply(client, message):
5    def classify(text):
6        return ask(client, f"Answer with one word: refund, tech or general.\n\n{text}", model=SMALL).strip().lower()
7
8    handlers = {
9        "refund": lambda t: ask(client, t, "You handle refund requests politely.", LARGE),
10        "tech": lambda t: ask(client, t, "You answer technical support questions.", LARGE),
11        "general": lambda t: ask(client, t, "You answer general questions briefly.", SMALL),
12    }
13    return route(message, classify, handlers, default="general")
14
15
16def write_post(client, topic):
17    steps = [
18        ("outline", lambda t: ask(client, f"Write a 3 point outline about: {t}")),
19        ("draft", lambda t: ask(client, f"Write a short post from this outline:\n{t}")),
20    ]
21    gate = lambda name, text: "outline is empty" if name == "outline" and not text.strip() else None
22    return chain(steps, topic, gate)
23
24
25def is_safe(client, text):
26    prompt = f"Is this message safe to publish? Answer exactly yes or no.\n\n{text}"
27    return vote(lambda t: ask(client, prompt).strip().lower(), text, n=3, min_agree=2)

Both blocks are tested with a stand-in client that returns scripted replies, so they run without the API. Nothing here was run against a live model.

Choosing a pattern

The patterns combine. A router can send an input to a chain. A chain step can be a set of parallel checks. A voting step can sit at the end. Build each piece and test it before joining them.

Costs and cautions

• Chains add delay. Each step waits for the one before. That is the price of accuracy.
• Parallel calls burst. Ten calls at once can hit a rate limit that ten spread-out calls would not. Cap the workers, and add retries as in Errors, retries and rate limits.
• Voting multiplies cost. Three votes cost about three times as much. Use it where a wrong answer is expensive.
• Votes are not independent. If the same model makes the same mistake every time, voting repeats the mistake. Vary the prompt or the model when you can, and do not treat agreement as proof.
• Parallel is not free of shared state. Do not run parallel calls that write to the same place.
• Routing errors compound. A wrong label sends the input to a handler that was never built for it. Log the labels, and review a sample.

Mistakes beginners make

1. Building an agent when a chain would do.
2. A chain with no gates, so a bad early step ruins everything after it.
3. A router with no default, so an unexpected label crashes or picks nothing.
4. Trusting the classifier without testing it on real inputs.
5. Routing to a small model and never checking whether it answers well enough.
6. Unbounded parallelism that trips rate limits.
7. Letting one failed parallel call hide the others.
8. Voting with the same prompt and model and treating the result as independent evidence.
9. Treating a vote as certainty when it is only agreement.
10. Never measuring whether the added steps beat a single call.

Quick recap

• Workflows have a shape you choose in advance. They are easier to test and predict than free-roaming agents.
• Chaining gives each call one job and adds gates that stop bad output early, at the cost of delay.
• Routing sends each input to a specialized handler, or to a smaller or larger model. Always give it a default and test the classifier.
• Sectioning runs independent calls in parallel for speed and focus. Voting runs the same call several times for confidence.
• Parallel calls can hit rate limits, voting multiplies cost, and votes from the same model are not truly independent.
• Patterns combine, so build and test each one, then join them.

Try it yourself

Pick a task you do often, such as turning meeting notes into an email. First write it as a chain of two or three steps, with a gate that checks the output of each. Then add a router that sends short notes to the small model and long notes to the large one. Finally, wrap a risky judgment, such as "is this email safe to send?", in vote with three runs, and test what happens when the answers split. Use the stand-in client from the tests to script the replies.

Next lesson: Orchestrator-workers and multi-agent systems.