Everything in Monitoring & SRE, page 7
Explain the three pillars of observability
Metrics are cheap aggregated time series for detecting and alerting, logs are discrete event records for detail and context, traces follow one request across services to localize latency.
Error budget policy across dependent microservices?
Set per-service SLOs but anchor on user-facing journey SLOs, budget for dependency error via the multiplication of availabilities, attribute downstream-caused failures correctly, and use…
SRE vs traditional ops on a recurring alert?
Traditional ops repeatedly handles the alert manually and scales by adding people; SRE treats it as a bug, automates or eliminates the root cause, and scales sublinearly.
Risky launch with a near-empty error budget?
Contain blast radius via canary and feature flags, use progressive rollout tied to budget burn, negotiate explicit risk acceptance, add fast rollback.
What makes a blameless postmortem effective?
Cover summary, impact, timeline, root cause, action items; explain blameless means focusing on systemic causes so people report honestly.
How do you find and eliminate toil systematically?
Inventory and measure toil via time tracking or ticket analysis, prioritize by frequency times cost versus automation effort, automate highest-ROI items, measure success…
Error budget exhausted early: what now?
Invoke the error budget policy, shift focus from features to reliability, prioritize stability work, analyze what burned the budget.
How do you set SLOs for a service from scratch?
Identify critical user journeys, pick SLIs that reflect user happiness, measure current performance, set achievable targets with buffer, iterate.
How do SRE and DevOps relate?
DevOps is a broad culture of breaking down silos; SRE is one prescriptive way to do it with specific practices.
SLI vs SLO vs SLA: how do they relate?
SLI is a measured metric, SLO is the internal target for that metric, SLA is the external contract with consequences.
What defines toil, with an example?
Define toil as manual, repetitive, automatable, tactical, no-enduring-value, scaling-with-growth work; distinguish it from overhead; give a concrete example.
Normalization of Deviance
Normalization of deviance is the gradual process by which unsafe practices become accepted as normal because they have not yet caused a visible failure. Each tolerated shortcut lowers the bar, eroding safety margins until a catastrophe finally results.
Hiring for SRE
Hiring for SRE seeks engineers who blend software-development skill with systems and operations depth, plus strong debugging and incident temperament. The hardest part is finding people who can both write automation and reason about failure at scale under…
LitmusChaos
LitmusChaos is an open-source, Kubernetes-native chaos engineering platform that runs fault experiments as custom resources. It injects failures like pod kills, network latency, and resource stress to validate that services stay resilient under real-world…
Backpressure
Backpressure is a mechanism by which a slow consumer signals an upstream producer to slow down or stop, preventing unbounded queues and resource exhaustion. It keeps systems stable under overload by propagating capacity limits backward through a pipeline.
Performance Profiling
Profiling measures where a program actually spends its time and resources, attributing CPU cycles, memory, or wall-clock latency to specific functions or call paths. It replaces guesswork with data so optimization effort targets the real bottleneck.
Review of Reviews (Postmortem Metrics)
A meta-process that audits the quality and follow-through of postmortems themselves, tracking metrics like action-item completion, time-to-close, and recurrence of incidents.
The Second Story of an Incident
The first story blames human error and stops there; the second story asks why the action made sense to the person at the time and what systemic conditions enabled it. Seeking the second story is the heart of blameless, learning-oriented incident analysis.
Counterfactual Reasoning in Incident Analysis
Counterfactual reasoning asks what would have prevented or mitigated an incident: if this alert had existed, if this check had run. Used well it finds systemic gaps, but it is a trap when it implies a single person should have just acted differently in…
Actionable Remediation Items
Actionable remediation items are specific, owned, trackable follow-ups from a postmortem that prevent recurrence or improve response. Good ones are concrete and assigned with a due date; vague items like be more careful are non-actionable and predictably…
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles