Errors, Retries and Rate Limits: Keep Your AI Agent From Crashing

Your agent from the last lesson works on a good day. The real world has bad days: the network drops, the API is busy, you hit a rate limit, a tool fails, or the model asks for something that does not exist. An agent makes many calls in a loop, so the chance that something goes wrong in one run is much higher than in a single prompt. This lesson shows you where each kind of failure should be handled, and finishes with a hardened version of your notes agent. After it, a hiccup in the network will not end your run, and a real problem will stop it with a clear message instead of a crash or a runaway bill.

Why agents need this more

Think about the numbers. If one call fails one time in a hundred, a ten-step agent fails roughly one run in ten. Add tools that read files or call other services, and there are more places to fail. The goal is not to prevent every failure. It is to handle each one in the right place:

• Temporary failures should be retried, quietly.
• Permanent failures should stop the run with a message that says what to fix.
• Tool failures should go back to the model, which can often recover on its own.
• Runaway behavior should be stopped by limits you set.

API errors: what the codes mean

When the API cannot do what you asked, it returns an HTTP status code and a JSON body with an error type and a message. Anthropic documents these codes for the Claude API:

The pattern is simple. Codes in the 400s (except 429) mean the request itself is the problem, so sending it again changes nothing. Rate limits, server errors and overload are temporary, so waiting and trying again is the right move.

Every error response also carries a request_id, and every response has a request-id header. Log it. If you ever need help from Anthropic's support, that ID lets them find the exact request. In the Python SDK, a successful response exposes it as message._request_id.

Two details are easy to miss:

• A 429 is not always a rate limit. If your organization reaches its monthly spend cap, the API also returns a 429, but without a retry-after header, and retrying will keep failing until access resumes. The error details carry the code enforced_spend_limit_reached so you can tell the two apart.
• A spend limit you set yourself returns a 400 instead, with a message that says when access resumes.

With streaming, one more case exists: the API can send a 200 and then report an error part-way through the stream as an event. Streaming code needs to watch for error events as well as failed requests.

Retries and backoff

When a failure is temporary, you retry. But how you retry matters. Retrying instantly, many times, makes a busy service busier. The standard technique is exponential backoff: wait a short time, then twice as long, then twice as long again.

Two refinements make it work well:

• Jitter. Add some randomness to each wait. If a hundred agents all fail at the same moment and all wait exactly two seconds, they all return at the same moment and fail again together. A random wait spreads them out.
• Respect retry-after. When a 429 includes a retry-after header, it tells you how many seconds to wait, and earlier retries will fail. Use it as a minimum.

The good news: the SDK already does this

The official Anthropic SDK retries transient failures for you. According to its documentation, connection errors, 408, 409, 429 and 5xx responses are retried two times by default, with a short exponential backoff, and the retry-after header is honored when present. You can change this with one setting:

1client = anthropic.Anthropic(max_retries=5, timeout=60.0)

Or for one call only:

1client.with_options(max_retries=5).messages.create(...)

timeout is in seconds. The default is 10 minutes, which is far too long for an agent step you are waiting on, so set one that fits your task. When a request times out, the SDK raises APITimeoutError, and by default it retries that twice as well. Very long non-streaming requests are better handled by streaming, as covered in Choosing and configuring a model.

For the Claude API, raising max_retries is usually all you need. Writing your own retry code is worth it when a provider's SDK does not retry for you, or when you want the same behavior on every provider, as in Function calling across providers. Here is a small, provider-neutral version:

1import random
2import time
3
4
5def with_backoff(call, retryable, tries=5, base=1.0, cap=30.0):
6    for attempt in range(tries):
7        try:
8            return call()
9        except retryable as e:
10            if attempt == tries - 1:
11                raise
12            delay = random.uniform(0, min(cap, base * 2**attempt))
13            hint = getattr(getattr(e, "response", None), "headers", {}).get("retry-after")
14            if hint:
15                delay = max(delay, float(hint))
16            time.sleep(delay)

Use it by passing a function that makes the call and the exception types worth retrying:

1reply = with_backoff(
2    lambda: client.messages.create(model=MODEL, max_tokens=1024, messages=messages),
3    retryable=(anthropic.APIConnectionError, anthropic.RateLimitError, anthropic.InternalServerError),
4)

Read the function once. It tries up to tries times. On the last failure it raises the error instead of hiding it. Between tries it waits a random time up to a ceiling that doubles each round and never passes cap. If the server sent retry-after, the wait is at least that long. If the SDK is also retrying, the two layers multiply, so use one or the other, not both.

A warning about tools. Retrying a model call is safe, because asking again has no side effects. Retrying a tool that changes something is not. If a "send email" tool times out after the email went out, a retry sends it twice. Retry read-only tools freely, and give tools that change things a way to detect repeats, such as a unique request key.

Rate limits

A rate limit is a cap on how much you can use the API in a period of time. For the Claude API, limits are measured as:

• RPM, requests per minute,
• ITPM, input tokens per minute,
• OTPM, output tokens per minute.

Each model class has its own numbers, and they depend on your usage tier. Look at your own limits on the Rate limits page of the Claude Console instead of relying on a table from a tutorial. When you exceed any of them, you get a 429 that says which one.

Some behaviors from Anthropic's documentation are worth knowing:

• Token bucket. Your capacity refills continuously up to the maximum, rather than resetting at fixed times. Short bursts can still trigger a limit. A rate of 60 requests per minute may be enforced as about one per second.
• Cached input is cheap for limits. For most models, tokens read from the prompt cache do not count toward ITPM. Only uncached input and tokens written to the cache do. Caching a long, repeated system prompt can lift your effective throughput a lot.
• max_tokens does not count against OTPM. Output limits count only the tokens actually generated.
• Ramp up gradually. A sudden jump in traffic can hit acceleration limits and produce 429 errors even below your normal limits. Raise your load step by step.

Why agents hit limits

A chat sends one request. An agent sends a request per step, and each one carries the whole growing history, as you learned in How LLMs work. So agents use input tokens quickly, especially when tool results are large. Run several agents at once and the numbers add up fast. The fixes are the ones you already know: keep tool results small, cache stable content, cap your steps, and limit how many agents run at the same time.

Watching your headroom

Every response includes headers that show where you stand. A few useful ones:

• retry-after: seconds to wait before retrying, on a 429.
• anthropic-ratelimit-requests-remaining: requests left before the limit.
• anthropic-ratelimit-input-tokens-remaining: input tokens left, rounded to the nearest thousand.
• anthropic-ratelimit-output-tokens-remaining: output tokens left.

Each also has -limit and -reset versions. To read headers in Python, use the raw response, then parse it back into the normal object:

1raw = client.messages.with_raw_response.create(
2    model=MODEL, max_tokens=1024, messages=messages
3)
4print(raw.headers.get("anthropic-ratelimit-input-tokens-remaining"))
5message = raw.parse()

An agent that sees headroom running low can slow down on its own before it ever sees a 429.

Tool errors: let the model recover

The model, not your code, decides what to do after a tool fails. So your job is to tell it clearly what went wrong. Claude's API has a field for this: set "is_error": true on the tool_result, and put the message in content.

Anthropic's advice is to write instructive error messages. "Failed" tells the model nothing. "There is no file called todo.tx. Call list_files to see the names" tells it exactly what to try next.

This also covers a model that calls a tool wrongly, such as leaving out a required input. Anthropic's documentation says that when a tool request is invalid, Claude usually retries two or three times with corrections before giving up and apologizing. For fewer such mistakes, add strict: true to the tool definition, which makes the model's inputs match your schema, and write clearer descriptions.

Truncated and unusual replies

Two cases need a check before you run any tool:

• max_tokens cut the reply off. The stop reason is max_tokens, and a tool request inside it may be incomplete. Do not run it. Raise the limit and ask again, or stop with a clear message.
• Other stop reasons. refusal, model_context_window_exceeded and pause_turn are not finished answers. The loop from Build your first agent already returns a message for any stop reason other than end_turn and tool_use.

Putting it together: the notes agent, hardened

Here are the changes to the agent from Build your first agent. First, the client gets retries and a timeout, and a failure limit joins the step limit:

1MAX_FAILURES = 3
2
3client = anthropic.Anthropic(max_retries=5, timeout=60.0)

Next, tools raise a ToolError with a helpful message, and run_tool returns the text and an error flag together:

1class ToolError(Exception):
2    pass
3
4
5def read_file(name):
6    path = (NOTES / name).resolve()
7    if path.parent != NOTES or not path.is_file():
8        raise ToolError(f"There is no file called {name}. Call list_files to see the names.")
9    return path.read_text()[:5000]
10
11
12def run_tool(name, args):
13    try:
14        if name == "list_files":
15            return list_files(), False
16        if name == "read_file":
17            return read_file(args["name"]), False
18        return f"Unknown tool: {name}", True
19    except ToolError as e:
20        return str(e), True
21    except KeyError as e:
22        return f"Missing required input: {e}", True
23    except Exception as e:
24        return f"{type(e).__name__}: {e}", True

Finally, the loop catches the errors that remain after the SDK's retries, passes is_error along, and counts failures in a row:

1def run_agent(task):
2    messages = [{"role": "user", "content": task}]
3    failures = 0
4
5    for step in range(1, MAX_STEPS + 1):
6        try:
7            response = client.messages.create(
8                model=MODEL,
9                max_tokens=1024,
10                system=SYSTEM,
11                tools=TOOLS,
12                messages=messages,
13            )
14        except anthropic.AuthenticationError:
15            return "Stopped: the API key was rejected. Check ANTHROPIC_API_KEY."
16        except anthropic.RateLimitError:
17            return "Stopped: still rate limited after retries. Try again later."
18        except anthropic.APIConnectionError:
19            return "Stopped: could not reach the API."
20        except anthropic.APIStatusError as e:
21            request_id = e.response.headers.get("request-id")
22            return f"Stopped: API error {e.status_code} (request {request_id})"
23
24        messages.append({"role": "assistant", "content": response.content})
25
26        if response.stop_reason == "end_turn":
27            return "".join(b.text for b in response.content if b.type == "text")
28        if response.stop_reason != "tool_use":
29            return f"Stopped early: {response.stop_reason}"
30
31        results = []
32        failed = False
33        for block in response.content:
34            if block.type == "tool_use":
35                print(f"step {step}: {block.name}({json.dumps(block.input)})")
36                output, is_error = run_tool(block.name, block.input)
37                failed = failed or is_error
38                print(f"        -> {'ERROR ' if is_error else ''}{output[:60]!r}")
39                results.append(
40                    {
41                        "type": "tool_result",
42                        "tool_use_id": block.id,
43                        "content": output,
44                        "is_error": is_error,
45                    }
46                )
47        messages.append({"role": "user", "content": results})
48
49        failures = failures + 1 if failed else 0
50        if failures >= MAX_FAILURES:
51            return f"Stopped: tools failed {MAX_FAILURES} times in a row."
52
53    return "Stopped: step limit reached"

Read it from the top. The order of the except lines matters: Python takes the first match, so the specific classes (AuthenticationError, RateLimitError) come before the general APIStatusError, which they inherit from. Every path out of the loop returns a plain sentence that says what happened, never a traceback and never silence. The request ID goes into the message so a failed run can be traced.

The failure counter is the guard that stops a stuck agent. If a tool fails once, the model usually corrects itself. If tools fail three steps in a row, something is wrong that more tries will not fix, and stopping is cheaper than looping to the step limit.

To see it work, ask the agent to read a file that does not exist. You should see an ERROR line followed by a recovery in the next step, and not a crash.

Mistakes beginners make

1. Catching every exception and carrying on. Hiding a permanent error such as a rejected key turns a clear failure into a confusing one.
2. Retrying errors that retrying cannot fix. A 400 or 401 will fail the same way every time.
3. Retrying instantly with no backoff. It makes a busy service worse and burns your own rate limit.
4. Stacking retry layers. An SDK that retries three times inside your loop that retries three times is nine attempts.
5. Retrying tools that change things without a way to detect a repeat.
6. Returning vague tool errors. Say what went wrong and what to try next.
7. Running a tool call from a reply that was cut off by max_tokens.
8. Leaving the 10-minute default timeout on an agent step.
9. Not logging the request ID, which is what support needs.
10. Counting only total steps, with no limit on consecutive failures.

Quick recap

• Temporary failures (429, 500, 504, 529, connection errors) get retried with backoff. Permanent ones (400, 401, 402, 403, 404, 413) get fixed or reported.
• The Anthropic SDK retries transient failures twice by default. Raise max_retries and set a realistic timeout instead of writing your own, unless you need the same behavior across providers.
• Backoff means waiting longer each time, with random jitter, and treating retry-after as a minimum.
• Rate limits count requests, input tokens and output tokens. Agents hit them because every step resends the history. Cache, trim and ramp up gradually.
• Return tool failures to the model with is_error and a message that says what to try next.
• Check the stop reason before acting, and end every failed run with a clear message and the request ID.
• Cap both the total steps and the failures in a row.

Try it yourself

Take your notes agent and break it on purpose, one way at a time. Ask it to read a file that is not there and watch the recovery. Set ANTHROPIC_API_KEY to a wrong value and confirm you get the key message instead of a traceback. Set MAX_FAILURES to 1 and MAX_STEPS to 2 and note which message you get. Then write a fake flaky() function that fails twice and succeeds on the third try, wrap it in with_backoff, and print how long each wait was. If you have more than one provider's SDK installed, work out which exceptions you would put in retryable for each.

Next lesson: Tool definitions and structured output.