Planner-Executor, DAG Agents and Tree of Thought: Planning Architectures Explained

A ReAct loop decides one step at a time, looking at the latest result before choosing the next move. That works well for many tasks, and it is where you should start. But some jobs call for thinking ahead: a long task with many parts, a task where several steps can happen at once, or a puzzle where the first idea is often wrong. This lesson covers three architectures for those jobs. A planner and executor split deciding from doing. A DAG agent runs independent steps in parallel. Tree of thought explores several paths and keeps the best. You will learn when each helps, what it costs, and how to build the core of each in tested Python.

Why plan ahead at all

A step-by-step loop has costs that show up on bigger tasks:

• Wandering. With no overall plan, an agent can drift, repeat itself or forget part of the goal.
• Slow sequences. If three lookups do not depend on each other, doing them one after another wastes time.
• A growing history. Every step re-sends everything that came before, which costs tokens.
• No review point. A person cannot approve a direction that was never written down.

Planning attacks these. The price is that a plan can be wrong, or go stale when reality differs from what the planner expected. Every design below is a trade between those two.

Planner and executor

The simplest form splits the work in two:

• The planner reads the goal and writes a plan: a short list of steps.
• The executor carries out the steps, one by one, with tools.
• A replanner, which may be the planner again, looks at the results and changes the plan when something fails or turns out differently than expected.

This split has real advantages. The plan is a document, so a person can read it, edit it or approve it before anything happens, which fits the approval ideas in Human in the loop. The two roles can use different prompts, and even different models: a strong model to plan and a cheaper, faster one to execute simple steps, a choice that matters for cost and latency. And the executor's context stays small, because it works on one step at a time instead of carrying the whole task.

You have seen the human version of this. Claude Code Plan Mode: Think Before You Build in the Claude Code course lets Claude research a project and propose a plan before it changes anything, and you approve it first.

There is also a research idea about cost. The ReWOO paper, whose name stands for reasoning without observation, points out that a ReAct agent pauses after every tool call and then re-sends everything, which leads to redundant prompts. It proposes separating the reasoning from the tool observations, so the model plans first and the results are filled in afterwards. The paper's abstract reports a fivefold improvement in token efficiency and a 4 percent accuracy gain on a multi-step question benchmark, HotpotQA. Treat the figures as that paper's results on its own tests.

Plans are data: validate before you run

If the planner outputs a plan, your code should treat it as untrusted data. The model may invent a tool that does not exist, refer to a step that is not in the plan, or create a loop. So give plans a structure and check it before running anything. A plan can be a list of steps, each with an id, a tool, an input and the ids of the steps it depends on:

1[
2  {"id": "a", "tool": "fetch", "input": "Product A specs"},
3  {"id": "b", "tool": "fetch", "input": "Product B specs"},
4  {"id": "c", "tool": "compare", "input": "Which is better value?", "depends_on": ["a", "b"]}
5]

You can ask the planner to produce this with the structured output feature from Tool definitions and structured output, which makes the shape reliable. A planner prompt follows Prompting for agents: say what the tools are, ask for the fewest steps that do the job, and require the depends_on field. Even with a reliable shape, the content still needs checking, which validate_plan does in the code below.

DAG agents: run independent steps together

Look at the plan above. Steps a and b do not depend on each other, so they can run at the same time. Step c needs both, so it must wait. That structure is a directed acyclic graph, or DAG: nodes are steps, arrows are dependencies, and "acyclic" means there are no loops, so there is always a valid order.

The LLMCompiler paper turned this into a system. It notes that common function-calling methods reason and act for one function at a time, which can cause high latency and cost. Its design has three parts: a planner that formulates the plan of function calls, a task fetching unit that dispatches the tasks, and an executor that runs them in parallel. Compared with ReAct-style calling, the paper reports gains of up to 3.7 times in latency, up to 6.7 times in cost and up to about 9 percent in accuracy. Those are the paper's best-case figures across its benchmarks, not a promise for your tasks.

Here is the code for the core of this idea. It contains three things: a plan validator, a parallel runner, and the tree search used later in the lesson. First, read validate_plan and run_plan:

1from concurrent.futures import ThreadPoolExecutor
2from fractions import Fraction
3from itertools import combinations
4
5
6def validate_plan(steps, tools):
7    ids = [s["id"] for s in steps]
8    if len(set(ids)) != len(ids):
9        raise ValueError("duplicate step ids")
10    for s in steps:
11        if s["tool"] not in tools:
12            raise ValueError(f"step {s['id']} uses unknown tool {s['tool']}")
13        for d in s.get("depends_on", []):
14            if d not in ids:
15                raise ValueError(f"step {s['id']} depends on missing step {d}")
16    done, remaining = set(), list(steps)
17    while remaining:
18        ready = [s for s in remaining if all(d in done for d in s.get("depends_on", []))]
19        if not ready:
20            raise ValueError("the plan has a cycle")
21        done.update(s["id"] for s in ready)
22        remaining = [s for s in remaining if s["id"] not in done]
23    return steps
24
25
26def run_plan(steps, tools, max_workers=4):
27    results, failed, pending = {}, {}, {s["id"]: s for s in steps}
28    with ThreadPoolExecutor(max_workers) as pool:
29        while pending:
30            settled = results.keys() | failed.keys()
31            ready = [s for s in pending.values() if all(d in settled for d in s.get("depends_on", []))]
32            if not ready:
33                raise ValueError("the plan has a cycle")
34            futures = {}
35            for s in ready:
36                deps = s.get("depends_on", [])
37                if any(d in failed for d in deps):
38                    failed[s["id"]] = "skipped: a step it depends on failed"
39                    del pending[s["id"]]
40                else:
41                    futures[s["id"]] = pool.submit(tools[s["tool"]], s["input"], {d: results[d] for d in deps})
42            for sid, future in futures.items():
43                try:
44                    results[sid] = future.result()
45                except Exception as e:
46                    failed[sid] = f"{type(e).__name__}: {e}"
47                del pending[sid]
48    return results, failed
49
50
51def tree_search(start, propose, score, is_goal, breadth=5, max_depth=4):
52    frontier = [(start, [start])]
53    for _ in range(max_depth):
54        candidates = []
55        for state, path in frontier:
56            for nxt in propose(state):
57                if is_goal(nxt):
58                    return path + [nxt]
59                candidates.append((score(nxt), nxt, path + [nxt]))
60        candidates.sort(key=lambda c: c[0], reverse=True)
61        frontier = [(s, p) for _, s, p in candidates[:breadth]]
62        if not frontier:
63            return None
64    return None
65
66
67def propose_24(numbers):
68    options = []
69    for i, j in combinations(range(len(numbers)), 2):
70        a, b = numbers[i], numbers[j]
71        rest = [n for k, n in enumerate(numbers) if k not in (i, j)]
72        results = [a + b, a * b, a - b, b - a]
73        results += [a / b] if b else []
74        results += [b / a] if a else []
75        options += [tuple(sorted(rest + [r])) for r in results]
76    return options
77
78
79def score_24(numbers):
80    return -min(abs(n - 24) for n in numbers)
81
82
83def can_make_24(numbers):
84    if len(numbers) == 1:
85        return numbers[0] == 24
86    return any(can_make_24(n) for n in propose_24(numbers))
87
88
89def score_lookahead(numbers):
90    return 1 if can_make_24(numbers) else 0

validate_plan

It rejects plans with duplicate step ids, unknown tools, dependencies on steps that do not exist, and cycles. The cycle check repeatedly takes every step whose dependencies are already done. If at some point no step is ready but some remain, the rest depend on each other, and the plan can never finish. This is the standard way to find an order for a dependency graph, called a topological sort.

run_plan

It runs the plan in waves. Each pass it finds every step whose dependencies are finished, runs them all at the same time in a thread pool, waits for them, and then looks again. A tool is called as tool(input, deps), where deps holds the results of the steps it depends on, so later steps can use earlier results.

Two behaviors are worth reading closely:

• Failures do not stop the whole plan. If a step raises an error, it is recorded, the steps that depend on it are marked skipped, and independent steps still finish. The function returns both results and failed, so you can decide what to do next, for example give the failures to the planner and ask for a revised plan.
• Parallelism is real. The tests prove it with a Barrier that only opens when three steps are running at the same moment. They also prove that a dependent step starts only after its dependency finishes.

The runner has a known simplification: it waits for each whole wave before starting the next. A step whose dependencies finish early could start sooner. Replacing the wave loop with as_completed fixes that when it matters.

What to watch when you parallelize

• Rate limits. Ten steps at once is a burst. Keep max_workers modest, and remember from Errors, retries and rate limits that sudden jumps in traffic can trigger limits.
• Cost does not fall. Parallel steps finish sooner but use the same tokens. Savings come from fewer wasted steps, not from parallelism itself.
• Shared state. Steps that write to the same place can collide. Run steps that change things one at a time, as in The agent loop.
• Side effects. A plan can contain steps that send or delete things. Put approvals in the executor, not in the plan.

Claude's API can also run several tool calls from one reply in parallel, which you saw in earlier lessons. The DAG approach goes a step further, because the plan is explicit and the dependencies are known before anything runs.

Replanning

Plans meet reality. A step fails, a search returns nothing, or a result changes what should happen next. A rigid plan that ignores this is the classic failure of planner-executor designs. So add a replanning step: after a wave, or on a failure, give the planner the goal, the original plan, the results so far and the failures, and ask for a revised plan for what remains. Limit the number of replans, as you limit steps, so that an agent cannot replan forever. A simple rule: plan, execute, and replan at most a few times before reporting what it could not do.

Tree of thought: explore, judge, prune

Sometimes the problem is not ordering steps but finding which path works. For puzzles, tricky planning or code search, the first idea is often wrong, and a single left-to-right chain commits to it. Tree of thought, from the 2023 paper by Shunyu Yao and colleagues, extends chain of thought. The model considers several "thoughts" at each step, evaluates them, keeps the promising ones, and can look ahead or backtrack.

The paper tested it on tasks that need planning or search. On Game of 24, a puzzle where you combine four numbers with arithmetic to make 24, its abstract reports that GPT-4 with chain-of-thought prompting solved 4 percent of tasks, while tree of thought reached 74 percent.

The structure is a search with three ingredients:

1. Propose several next thoughts from each state.
2. Judge each one, for example by asking a model "is this sure, maybe or impossible?", or better, by running a check.
3. Keep the best few and expand them, discarding the rest.

tree_search in the code above is exactly this, in a form called beam search. At each depth it collects every proposal, scores them, keeps the top breadth, and returns as soon as any proposal satisfies is_goal. The functions propose_24 and score_24 below it are a small Game of 24 setup, where a thought is "combine two of the numbers", standing in for model calls.

The judge is the whole game

Running the tests taught a useful lesson, and it is worth showing. For the numbers 4, 9, 10 and 13, the answer is (4 - 10) which is -6, (9 - 13) which is -4, then -6 x -4 which is 24. Two different judges give very different results:

• A cheap judge that scores a state by how close any number is to 24 never found the answer, even when the search kept 200 candidates at each level. The right path goes through states such as (4, 6) that look far from 24, so the heuristic pruned it every time.
• A judge that looks ahead, using score_lookahead to check whether the remaining numbers can still reach 24, found it even with a beam of 1.

That second judge is cheating in a useful way. It solves the puzzle to judge it, so the search is nearly pointless. Real judges sit between the two. A model asked to estimate promise will be better than a naive heuristic and worse than an oracle, and each judgment costs a model call. The lesson is that a wider search does not rescue a poor judge. Invest in the judge: use a real check when you can, such as running tests on generated code, and treat a model's self-assessment as a noisy signal.

What tree of thought costs

Calls multiply. A breadth of 5 and a depth of 3 means dozens of proposals and judgments for one task. That is why it is for high-value problems where partial progress can be judged and a single guess often fails, not for everyday questions. Most agents never need it.

Choosing a design

Anthropic's advice from Workflows or agents applies strongly here: find the simplest solution and add complexity only when needed. If you can write the steps down in advance, you do not need a planner, and a plain workflow from Chaining, routing and parallelization is simpler. Reach for planning when the steps depend on the task and you want the benefits above, and measure whether it beat the simple loop on your own tasks, as in Designing tools that work.

Mistakes beginners make

1. Planning when a loop would do. Start with ReAct.
2. Running a plan without checking it. Validate tools, dependencies and cycles first.
3. A rigid plan, with no replanning when a step fails.
4. No limit on replans or steps.
5. Parallelizing steps that change shared state.
6. Ignoring rate limits when a wave fires many requests.
7. Skipping dependent-step handling, so one failure crashes everything or hides errors.
8. Tree search with a weak judge, and hoping a wider beam helps.
9. Using tree of thought for ordinary tasks, where the cost is not worth it.
10. Trusting a plan because it reads well. Plans are model output, and they can be wrong.

Quick recap

• A ReAct loop decides one step at a time. Planning helps on long tasks, parallel work and puzzles, at the price of a plan that can be wrong.
• A planner writes the steps, an executor carries them out, and a replanner revises when reality differs. The plan can be reviewed by a person.
• Treat a plan as untrusted data: validate tools, dependencies and cycles before running.
• A DAG of steps lets independent work run in parallel. Run in waves, pass dependency results along, and skip the steps that depend on a failure.
• Parallelism cuts waiting, not tokens, and it bursts against rate limits.
• Tree of thought proposes several thoughts, judges them, keeps the best and prunes the rest. A good judge matters more than a wide search.
• Choose the simplest design that works, and measure it against a plain loop.

Try it yourself

Write a plan for a task you do often, such as "compare two products and recommend one", as a list of steps with depends_on. Run it through validate_plan, then through run_plan with fake tools that sleep for a moment, and confirm that independent steps finish together. Break the plan on purpose with a cycle and a missing dependency, and read the error messages. Then run tree_search on the numbers 4, 9, 10 and 13 with the closeness judge and the lookahead judge, and try to find a better cheap judge that does not solve the puzzle.

Next lesson: Self-critique and evaluator-optimizer.