Criterion: the standard Rust benchmarking library

WHAT THIS TESTS It checks whether you know the real-world benchmarking tool in Rust and understand why statistically sound measurement matters more than a single timing.

A GOOD ANSWER COVERS Criterion.rs is the community standard. Unlike the built-in test::Bencher, which requires nightly and the unstable test feature, Criterion runs on stable Rust. It executes the benchmarked code over many iterations and samples, producing a statistical distribution rather than one number, and reports point estimates with confidence intervals. It persists results so each run is compared against a saved baseline, automatically reporting whether performance changed by a statistically significant margin, which enables regression detection in CI. It also performs warm-up, supports parameterized benchmarks over input sizes, and provides black_box to stop the optimizer from deleting work whose result is unused.

COMMON WRONG ANSWERS Naming the std time functions and a manual loop as sufficient. Forgetting that test::Bencher is nightly-only. Believing a single run gives a reliable number without accounting for variance, warm-up, or compiler elimination of dead code.

LIKELY FOLLOW-UPS What does black_box do and why is it needed? How would you wire Criterion into CI to fail on regressions? What is the difference between throughput and latency benchmarks?

ONE CONCRETE EXAMPLE Suppose you optimize a parsing function and want to confirm a speedup. With Criterion you define a benchmark that calls the parser inside black_box, run cargo bench to record a baseline, apply your change, and run again. Criterion reports something like a change with a confidence interval and labels it a significant improvement or regression, rather than leaving you to eyeball two raw numbers that might differ only due to noise. Saving baselines means the same comparison can run in CI to catch silent performance regressions on future commits.