Little's Law for capacity planning

WHAT THIS TESTS: Whether you understand the fundamental queueing relationship between concurrency, throughput, and latency, and can apply it to size a system.

A GOOD ANSWER COVERS: Little's Law states that for a stable system, the average number of items in the system, L, equals the average arrival rate, lambda, times the average time an item spends in the system, W. So L equals lambda times W. For a request-based service this becomes: average concurrency equals throughput (requests per second) times average response time (seconds). The power is that knowing any two gives the third. You can size the concurrency you need to hit a target throughput at a known latency, or compute the maximum throughput a fixed pool can sustain. Crucially it assumes a stable system observed over a long enough window, where arrivals roughly equal departures; it does not describe a system already saturating and queueing unboundedly.

COMMON WRONG ANSWERS: Confusing it with the unrelated claim that throughput equals one over latency, applying it to an overloaded unstable system where it breaks, or mixing up units between requests, seconds, and concurrency.

LIKELY FOLLOW-UPS: What happens as utilization approaches one? How does it relate to the universal scalability law? How do you measure W in practice? Why does the average suffice rather than the distribution?

ONE CONCRETE EXAMPLE: You need to serve five hundred requests per second and each request takes on average two hundred milliseconds, which is zero point two seconds. By Little's Law the required in-flight concurrency is five hundred times zero point two, equal to one hundred concurrent requests. So your thread pool or worker count, accounting for some headroom, must support around one hundred simultaneous requests; provision far fewer and requests queue, latency climbs, and the stability assumption breaks. The same formula lets you check that a one-hundred-worker pool serving two-hundred-millisecond requests tops out near five hundred requests per second.