Chain-of-Thought: Making LLMs 'Show Their Work'
Chain-of-thought prompting makes an LLM 'show its work' by generating intermediate reasoning steps before the final answer. This simple few-shot technique dramatically improves performance on complex tasks like math word problems or commonsense questions, especially for very large models. The common footgun is applying it to smaller models, where it can actually degrade performance instead of helping, as the reasoning ability hasn't yet emerged.
The mental model
Instead of asking a large language model (LLM) for just the final answer, chain-of-thought (CoT) prompting instructs it to 'show its work' first. It's a technique for decomposing a complex problem into a series of intermediate reasoning steps, which the model generates as part of its response. This mimics how a human might work through a problem on paper, leading to more reliable and accurate results for complex tasks.
How it works
CoT is a simple prompting method, not a fine-tuning process. You provide the model with a few examples (few-shot exemplars) in the prompt itself. Each example demonstrates the desired format: a question followed by a detailed, step-by-step derivation of the answer, and then the final answer. When presented with a new, similar question, the model follows the pattern, generating its own chain of thought before concluding.
When to use it
• Arithmetic Reasoning: For multi-step math word problems, CoT allows the model to calculate intermediate values, significantly reducing errors. For instance, a 540B parameter model using just eight CoT examples achieved state-of-the-art accuracy on the GSM8K math benchmark. • Commonsense Reasoning: When a question requires connecting multiple pieces of general knowledge, CoT helps the model lay out the logical path. • Symbolic Reasoning: For tasks involving manipulating symbols or following a set of abstract rules, generating steps makes the process more robust.
When not to use it
• On Smaller Models: CoT is an emergent ability of scale. On smaller LLMs (e.g., under ~100B parameters), it often provides no benefit and can even degrade performance. • For Simple, Single-Step Tasks: If a question is a straightforward fact recall, forcing a chain of thought is unnecessary overhead that might introduce noise.
One canonical example
Imagine a math word problem. A standard prompt might look like this:
`Q: A juggler has 15 balls. He loses 6 and buys 2 new sets of 4 balls each. How many does he have? A: 17`
A chain-of-thought prompt would instead demonstrate the reasoning:
`Q: A juggler has 15 balls. He loses 6 and buys 2 new sets of 4 balls each. How many does he have? A: The juggler starts with 15 balls. He loses 6, so he has 15 - 6 = 9 balls. He buys 2 sets of 4, which is 2 * 4 = 8 new balls. He now has 9 + 8 = 17 balls. The answer is 17.`
By showing this pattern, you teach the model how to reason through the next problem it sees.
Interview question
Which scenario is LEAST suitable for applying Chain-of-Thought (CoT) prompting to improve a Large Language Model's performance?
- a.A. A complex arithmetic reasoning task requiring multiple calculations.
- b.B. A commonsense reasoning problem involving several logical steps.
- c.C. A task performed by a very large language model (e.g., 500B parameters).
- d.D. A task performed by a smaller language model (e.g., 50B parameters).Correct
Why? this is the answer
The card explicitly states that CoT is an emergent ability of scale and often provides no benefit or can even degrade performance on smaller LLMs (e.g., under ~100B parameters). The other options describe scenarios where CoT is highly effective, such as complex reasoning tasks and very large models.
Just read this? Test yourself on what you have been reading.
Read the original → arxiv.org
- #llm
- #prompt engineering
- #reasoning
- #emergent abilities
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on llm — each one lists the topics its interview covers.
See open roles