Zero-Shot, Few-Shot, and Chain-of-Thought Trade-offs

WHAT THIS TESTS This evaluates practical prompt-engineering judgment, specifically the ability to balance accuracy against token cost and latency for a multi-step reasoning task.

A GOOD ANSWER COVERS A zero-shot prompt just states the task. It uses the fewest tokens and has the lowest latency, but on genuine multi-step reasoning a base model often jumps to a wrong answer because nothing scaffolds the intermediate steps. A few-shot prompt prepends several worked examples. These demonstrations improve output format and accuracy and steer the model, but each example adds input tokens, raising prompt cost and slightly increasing prefill latency, and quality depends on example selection. A chain-of-thought prompt explicitly asks the model to reason step by step, optionally with few-shot reasoning exemplars. It substantially improves accuracy on arithmetic and logical multi-step problems because the model allocates computation to intermediate steps, but it generates many additional output tokens, which is the dominant cost and latency driver since decoding is sequential.

COMMON WRONG ANSWERS Declaring chain-of-thought always best, ignoring that for simple tasks it wastes tokens and adds latency for no gain. Confusing input-token cost from few-shot examples with the output-token cost of generated reasoning. Assuming reasoning helps small models as much as large ones; chain-of-thought benefits emerge mainly at scale.

LIKELY FOLLOW-UPS How does self-consistency, sampling multiple chains and voting, trade even more cost for accuracy? Why is output-token generation usually the latency bottleneck versus prompt length? When can you hide reasoning to control cost, or distill it? How does zero-shot chain-of-thought with a simple let us think step by step compare to few-shot CoT?

ONE CONCRETE EXAMPLE For a word problem requiring three arithmetic steps, a zero-shot prompt might output a fast but wrong single number. Adding a few worked examples helps formatting but the model may still skip steps. A chain-of-thought prompt makes it write out each step before the final answer, usually getting it right, but the response is several times longer, so it costs more output tokens and takes proportionally longer to return, a worthwhile trade only when the accuracy gain justifies the spend.