Interview questions in Monitoring & SRE, page 3
Designing a cache health dashboard
Hit ratio, latency, memory and evictions, connections and saturation, with stat panels for current state and time-series for trends.
Scaling Prometheus for HA and volume
Shard scraping, add long-term object storage and global query via Thanos/Cortex/VictoriaMetrics, run redundant replicas for HA, with dedupe.
Writing SLIs in PromQL
Availability is good requests over total using rate and non-5xx counters; latency uses histogram_quantile over rate of buckets summed by le.
Loki versus Elasticsearch for logs
Loki indexes only labels and stores raw log chunks, cheap but needs label-scoped brute-force search; Elasticsearch full-text indexes content, fast arbitrary search but costly to store…
Symptom-based versus cause-based alerting
Symptom alerts fire on user impact like high error rate or latency; cause alerts fire on internal conditions like high CPU. Page on symptoms, treat causes as diagnostic.
First steps on a p99 latency page
Confirm scope and blast radius, check recent changes and deploys, then look at dependencies and resource saturation.
Fixing a noisy non-actionable alert
Review the data to see if it is ever actionable, then tune threshold or duration, re-target at user-facing symptoms, or delete it.
SLO-based alerting and error budgets
Define SLI/SLO, derive an error budget, alert on how fast you burn it using multi-window multi-burn-rate rules so fast burns page and slow burns ticket.
Anatomy of an effective runbook
Alert meaning and impact, diagnostic steps with dashboard and query links, concrete remediation with rollback, and escalation contacts.
Designing an escalation policy
Define timeout-based escalation tiers, multiple notification channels, secondary and management layers, coverage across time zones, and severity-based routing.
Measuring on-call health quantitatively
Track pages per shift especially off-hours, actionability rate, time-to-resolve, and load distribution; use trends to prioritize reliability work and protect against burnout.
Catching rare intermittent failures
Add high-cardinality traces and structured logs with exemplars, use tail-based sampling to keep failing traces, and alert on SLO burn rather than coarse averages.
Alert silencing versus alert inhibition
Silencing mutes by matcher for a window, inhibition suppresses lower alerts when a higher one fires, dependency-aware.
Triaging a 5xx error spike
Confirm impact and scope, declare and assign roles, check recent changes, stop the bleeding before root cause.
Primary goal and sections of a post-mortem
Goal is organizational learning and prevention, not blame; include summary, timeline, impact, root cause, action items.
Designing a SEV1-SEV4 severity matrix
Tie levels to user impact, scope, and SLO burn with concrete thresholds; map each to response and escalation.
Fast rollback versus targeted fix in an incident
Weigh time-to-recovery and blast radius against rollback side effects; prefer restoring service fast, gated by data like deploy diff and migration risk.
Conducting a blameless post-mortem in practice
Focus on systems not individuals, assume good intent, use neutral language and facilitation, end with owned action items.
Technical investments to reduce MTTR
Cut detection, diagnosis, and recovery time via observability, runbooks/automation, fast rollback, and resilient architecture.
Architecting a single source of truth for incidents
One curated view overlaying golden metrics, deploy/change events, and active alerts on a shared timeline, fed by standardized telemetry.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles