Why does 200ms latency drop requests? Diagnose it.

WHAT THIS TESTS Whether you can connect a modest latency injection to a large queueing collapse through the math of bounded concurrency, rather than treating the impact as mysterious.

A GOOD ANSWER COVERS The core insight is Little's Law: the number of concurrent in-flight requests equals arrival rate multiplied by average response time. Adding 200ms per request increases each request's residence time, so at a fixed arrival rate the number of simultaneously in-flight requests climbs. If your service uses a bounded thread pool, connection pool, or concurrency limit, that ceiling is hit, new requests queue, the queue fills, and requests are dropped or time out. So a small latency bump can be amplified into request loss when concurrency is constrained. To diagnose, inspect thread-pool and connection-pool utilization and queue depth during the experiment, check whether downstream connection limits or a saturated client pool are the bottleneck, and look at configured timeouts, a too-long timeout lets slow calls hold resources longer. Critically, check for retries: if the client retries on the added latency or timeouts, it multiplies load and accelerates exhaustion. Use a trace of one slow request to see exactly where time and waiting accrue.

COMMON WRONG ANSWERS Blaming the downstream service for being slow, when the real cause is your own bounded concurrency. Ignoring Little's Law and treating the drop as random. Overlooking retry amplification. Assuming you need more downstream capacity rather than fixing pool sizing, timeouts, or load shedding.

LIKELY FOLLOW-UPS State Little's Law. How would retries worsen this. How do you fix it, bulkheads, timeouts, load shedding, or backpressure. What is the difference between a thread-per-request and async model here.

ONE CONCRETE EXAMPLE The service handles 500 requests per second with a 50ms baseline, so about 25 in-flight, well under a 100-thread pool. Inject 200ms and residence time becomes 250ms, pushing in-flight to roughly 125, exceeding the pool. Threads block waiting on the downstream, the request queue fills, and excess requests are dropped. A trace confirms threads parked on the downstream call, and pool-utilization graphs hit 100 percent, pointing the fix at concurrency limits, timeouts, and load shedding rather than the downstream itself.