Latency
47 bites tagged Latency: interview questions with model answers, and 60-second explainers.
Test yourself: Top 30 Latency interview questions →Multiple choice, with the correct answer and why it is correct on every question. Free, no sign-in.Why tail latency (p99) matters
Averages hide the worst experiences, and fan-out amplifies tails so most requests touch a slow path; causes include GC pauses, queueing, contention, and noisy neighbors.
Diagnose a degraded canary release
Check statistical significance versus baseline, confirm apples-to-apples comparison, isolate the cause via traces and logs, then weigh the regression against SLO budget.
First steps on a p99 latency page
Confirm scope and blast radius, check recent changes and deploys, then look at dependencies and resource saturation.
Writing SLIs in PromQL
Availability is good requests over total using rate and non-5xx counters; latency uses histogram_quantile over rate of buckets summed by le.
Prometheus histogram versus summary
Histograms expose bucket counts and let you compute aggregatable quantiles at query time; summaries compute fixed quantiles per instance that cannot be combined.
Histograms versus summaries for latency
Histograms store bucket counts and stay aggregatable with query-time approximate quantiles; summaries precompute quantiles per instance that cannot be averaged.
Essential tags for a request latency metric
Add method, route template, status code, and maybe service or region; they enable slicing while staying low-cardinality.
Diagnosing a healthy p50 but breaching p99
One percent of requests are slow, hurting power users and fan-out calls; investigate GC, locks, contention, cold caches, retries.
Proposing availability and latency SLIs for an auth API
Availability as the ratio of successful valid requests; latency as the fraction served under a threshold; measure at the edge from the user's view.
Why use latency percentiles over the average?
Averages hide the tail and are skewed by outliers, so most users can suffer while the mean looks fine; use percentiles; prioritize p99 when tail users are high-value or fan-out…
What are the Four Golden Signals?
Name latency, traffic, errors, and saturation; explain each briefly; describe measuring latency as a distribution separating success from failure.
Dynamic batching and the throughput-latency trade-off
The server groups concurrent requests into one batch to use the GPU fully, but larger batches and waiting to fill them raise per-request latency and time to first token.
Three techniques to cut LLM inference latency
Quantization shrinks weights with small accuracy risk, KV-cache plus continuous batching boost throughput, speculative decoding drafts tokens for lossless speedup.
Designing a production LLM summarization eval
A representative gold set, quality via human or LLM-as-judge plus faithfulness checks, and operational metrics like p95 latency and cost per request.
Pinning exclusive CPU cores to a pod
Set kubelet CPU Manager policy to static, make the pod Guaranteed QoS with integer CPU limits equal to requests, so it gets exclusive dedicated cores.
Diagnosing database latency layer by layer
Split total time into pool-wait, query execution, and ORM-generated query patterns; use metrics at each layer.
Tuning a database connection pool
Max size, min idle, connection and max-lifetime timeouts; size from cores and latency, not guesswork.
Minimizing model prediction latency end to end
Shrink the model via quantization, pruning, distillation, compilation; cut infra latency via caching, edge, faster runtimes.
Batch prediction versus online real-time prediction
Batch is scheduled bulk scoring, online is low-latency per-request scoring; contrast latency, freshness, cost; give a use case each.
Challenges deploying a model on edge hardware
Limited memory and compute cause latency, thermal and power limits, accuracy loss from compression, operator support gaps.
Serverless cold starts and how to mitigate them
A cold start is the latency to provision and initialize a fresh environment; mitigate with provisioned concurrency, smaller packages, and lighter runtimes.
Cold starts in serverless environments
A cold start is the delay to provision a fresh instance and initialize the runtime; mitigate with provisioned concurrency and by shrinking init work.
Accelerating uncacheable dynamic traffic globally
Terminate TLS at a nearby edge and ride the provider backbone via Global Accelerator or CDN dynamic acceleration; add edge compute; ultimately deploy multi-region.
Diagnosing latency with distributed tracing
Follow the trace ID across spans, compare per-span durations to find the slow hop, distinguish service time from network and queueing.
Get Latency bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.