Handling a 401 error in an LLM agent's tool call
Robust agent error handling.
Catch the tool error, return a structured observation to the LLM, and distinguish recoverable retries from terminal failures needing re-plan or escalation.
What's really being asked
Whether you treat tool failures as first-class feedback and can classify errors so the agent recovers intelligently instead of looping or crashing.
The full answer
CATCH AND STRUCTURE. The tool-execution layer wraps every call so exceptions and non-2xx responses are caught, never propagating as an uncaught crash. On failure it produces a structured observation containing the HTTP status, a concise error message, and optionally a hint about the likely cause and remediation. FEED BACK. This observation is appended to the agent's context as the result of its action, just like a successful result would be, so the LLM can reason over it in the next step. The prompt or scaffolding instructs the agent that tool calls may fail and that it should inspect the error and decide a recovery action. CLASSIFY THE ERROR. Crucially, a 401 Unauthorized is an authentication failure, not a transient glitch, so retrying the identical call is futile. The correct recovery is to refresh or re-fetch the credential or token, verify the auth configuration, try an alternative tool, or, if it cannot self-remediate, escalate to a human or report the blocker. Reserve retry-with-exponential-backoff for transient codes like 429 or 503. GUARDRAILS. Cap retries, enforce a maximum step count and budget, and add a circuit breaker so a persistently failing tool does not consume the loop.
The mistakes people make
Retrying a 401 repeatedly. Letting the raw exception crash the agent. Returning an unstructured stack trace the model cannot parse. Treating all errors identically with blind backoff. No loop or cost limit, allowing runaway retries.
What usually comes next
Which status codes warrant retry versus re-plan? How do you prevent infinite retry loops? How do you securely refresh a token mid-task? When should the agent give up and escalate?
A concrete example
The agent calls a CRM API and gets 401. The tool layer returns an observation: status 401 Unauthorized, likely expired token, action: refresh credentials. The agent reasons that retrying as-is will fail, invokes a refresh-token tool, then re-issues the original call with the new token and succeeds, all without crashing, and a step counter ensures it would escalate rather than loop if the refresh also failed.
Interview question
An LLM agent's API call returns 401 Unauthorized. Why is automatic retry-with-backoff the wrong default recovery?
- a.401 errors are always transient and resolve themselves
- b.Backoff is too slow for production latency budgets
- c.A 401 signals an authentication problem, so the identical call will keep failing; the fix is to refresh credentials or re-planCorrect
- d.Retrying would exceed the model's context window
Why? this is the answer
401 is a non-transient auth failure, so repeating the same request will not succeed; the agent should refresh the token, switch tools, or escalate. Backoff is appropriate for transient codes like 429 or 503.
Just read this? Test yourself on what you have been reading.
- #llm
- #agents
- #error-handling
- #tool-use
- #reliability
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on llm — each one lists the topics its interview covers.
See open roles