Chain-of-Thought Prompting: Making LLMs 'Show Their Work'

WHY IT EXISTS Large language models often fail at tasks requiring multiple reasoning steps. They might guess an answer that seems plausible but is logically incorrect because they try to solve the problem in one go. Chain-of-Thought prompting was created to guide a model through a sequence of intermediate steps, improving its ability to solve complex problems correctly.

THE MENTAL MODEL Think of it like teaching a student to show their work on a math test. If you only ask for the final answer, they might make a hidden mistake. But if you require them to write down each step, they are more likely to follow the logic correctly and arrive at the right solution. CoT makes an LLM's reasoning process explicit, often leading to better outcomes.

HOW IT WORKS There are two main approaches. The simplest is zero-shot CoT, where you append a phrase like "Let's think step by step" to your prompt. This encourages the model to generate a reasoning chain on its own. The more robust method is few-shot CoT, where you provide one or more examples in the prompt that demonstrate the pattern: a question, a step-by-step analysis, and a final answer. The model then mimics this structure for your new query.

WHEN TO USE IT Use CoT for any task that requires complex, multi-step reasoning. It excels at arithmetic word problems, common-sense puzzles, and planning tasks where the solution isn't immediate and benefits from being broken down into smaller, logical parts. It's a go-to technique for improving the reliability of LLMs on analytical tasks.

WHEN NOT TO USE IT Avoid CoT for simple, factual recall questions like "What is the capital of Spain?". For these queries, it adds unnecessary latency and token cost without improving accuracy. A direct prompt is far more efficient. It can also be less effective for smaller models that lack the capability to follow complex reasoning chains, even when prompted.

ONE CANONICAL EXAMPLE A standard prompt might fail on a simple word problem. A CoT prompt guides it.

Standard Prompt: Q: A cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have? A: 29

Chain-of-Thought Prompt: Q: A cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have? A: Let's think step by step. The cafeteria started with 23 apples. They used 20, so they had 23 - 20 = 3 apples. Then they bought 6 more apples, so they have 3 + 6 = 9 apples. The answer is 9.