Self-consistency over chain-of-thought

WHAT THIS TESTS Whether you understand the mechanism behind self-consistency, the vote over answers not text, and can quantify its compute and latency footprint.

A GOOD ANSWER COVERS MECHANISM. Standard chain-of-thought greedily decodes one reasoning trace, so a single early misstep dooms the answer. Self-consistency instead samples multiple diverse reasoning paths using nonzero temperature, then extracts the final answer from each path and selects the most frequent answer by majority vote. The intuition is that there are many valid reasoning routes to a correct answer, so correct answers recur across samples while different errors disagree with each other, making the mode a robust estimate. Note the vote is over final answers, not over the prose. IMPLEMENTATION. Issue N generations of the same CoT prompt at a temperature like 0.5 to 0.7, run them in parallel rather than sequentially, parse a normalized answer from each, and aggregate by count, optionally weighting. Pick N to balance accuracy gains, which have diminishing returns, against cost. COSTS. Token and dollar cost scale roughly linearly with N since you run N full generations. If you parallelize, end-to-end latency stays close to a single call, but you consume N times the compute and reduce system throughput; sequential execution would multiply latency too.

COMMON WRONG ANSWERS Saying it averages or concatenates the reasoning text. Picking the longest or most confident chain instead of voting on answers. Assuming it is free or low cost. Claiming accuracy scales without limit in N. Forgetting that answer extraction and normalization are needed.

LIKELY FOLLOW-UPS Why does temperature matter for diversity? How do you set N given diminishing returns? How is this different from beam search? When is the extra cost not worth it for production latency budgets?

ONE CONCRETE EXAMPLE A math word problem is sampled five times. Three paths reach 42 via different arithmetic routes, one reaches 36, one reaches 40. Majority vote returns 42, overriding the two divergent errors that a single greedy chain might have produced. Running the five samples in parallel keeps latency near one call while costing about five times the tokens.