Reflection: Teaching LLM Agents to Learn from Mistakes

Why it exists

Standard agents often fail repeatedly on complex tasks because they lack a mechanism to analyze why they failed. They can get a low score but don't understand the root cause. Reflection was created to give agents a structured way to learn from their own history and iteratively improve their strategy.

The mental model

Imagine an agent talking to itself after a failure. It's not just getting a "wrong" score; it's generating a sentence explaining what was wrong. An Actor tries to solve a puzzle, the Evaluator says "You failed," and the Self-Reflection model says, "You failed because you tried the key on the wrong door. Next time, check all doors first." This advice is then remembered for the next attempt.

How it works

The process uses three distinct models. First, an Actor, often a Chain-of-Thought or ReAct agent, attempts a task and generates a trajectory of actions. Second, an Evaluator model scores this trajectory, providing a reward signal (e.g., a number or pass/fail). Third, a Self-Reflection model takes the trajectory and score as input and generates natural language feedback. This linguistic feedback is stored in the agent's long-term memory and added to the context for the next attempt, guiding the Actor toward a better solution.

When to use it

Use Reflexion for multi-step tasks where an agent can get it wrong on the first try and iterative improvement is valuable. This is common in complex decision-making, programming, and advanced reasoning problems where the solution path isn't obvious. It helps the agent avoid repeating prior mistakes by explicitly learning from them.

When not to use it

For simple, single-shot tasks where a correct answer can be found immediately, Reflexion is overkill. The overhead of the Actor-Evaluator-Reflector loop adds latency and cost. If the task doesn't benefit from iterative refinement, a simpler agent framework is more efficient.

One canonical example

An agent is tasked with writing a Python function. On its first try, the Actor generates code with a bug. The Evaluator runs the code against unit tests, which fail, providing a low score. The Self-Reflection model observes the failed test and buggy code, then generates the feedback: "The function failed because of an off-by-one error in the loop. The loop should iterate to n instead of n-1." This reflection is passed back to the Actor, which then generates corrected code in its next attempt.