Interview questions in Monitoring & SRE, page 2
What is an error budget?
Error budget is one minus the SLO, the allowed unreliability; it gates feature velocity versus reliability work.
Proposing availability and latency SLIs for an auth API
Availability as the ratio of successful valid requests; latency as the fraction served under a threshold; measure at the edge from the user's view.
Diagnosing a healthy p50 but breaching p99
One percent of requests are slow, hurting power users and fan-out calls; investigate GC, locks, contention, cold caches, retries.
Defining SLOs for a new critical service
Start from user journeys, pick SLIs, involve product, engineering, and business stakeholders, set realistic targets iteratively.
Handling planned maintenance in SLOs and error budgets
Either charge maintenance to the error budget or formally exclude it via a maintenance window; each trades honesty against fairness.
Setting SLOs across a dependency chain
Serial dependencies multiply, so each backend needs a tighter SLO than the target; mitigate with redundancy, caching, and graceful degradation.
Launching a risky feature with no error budget left
Bring burn history, what consumed the budget, blast radius, and rollback safety; offer mitigations like flags and canaries.
Fixing watermelon SLOs that hide unhappy users
Green-outside, red-inside SLOs come from server-side measurement, coarse aggregation, wrong events, and lax thresholds; fix by measuring at the client and using meaningful SLIs.
The three pillars of observability
Metrics show what and when via cheap aggregates, logs give detailed per-event context, traces show where across services.
Structured vs unstructured logging
Unstructured logs are free-text lines; structured logs are machine-readable key-value or JSON, enabling reliable parsing, filtering, and aggregation.
Essential tags for a request latency metric
Add method, route template, status code, and maybe service or region; they enable slicing while staying low-cardinality.
OpenTelemetry API, SDK, and Collector
API defines vendor-neutral instrumentation, SDK implements and exports it, Collector receives, processes, and routes telemetry to backends.
Trace context and propagation across services
Trace context bundles trace ID, span ID, and flags; propagated via headers like W3C traceparent so each service extracts and continues the trace.
Redesigning a high-cardinality request metric
Cardinality is unique label combinations; user_id and raw path are unbounded; redesign by dropping user_id and templating the path.
When to add custom OpenTelemetry instrumentation
Wrap domain logic in custom spans, attach business attributes like tenant or plan, answer revenue and per-customer questions.
Histograms versus summaries for latency
Histograms store bucket counts and stay aggregatable with query-time approximate quantiles; summaries precompute quantiles per instance that cannot be averaged.
Core components of Prometheus
Server scrapes targets found via service discovery, stores samples in a local time-series database, with Alertmanager and exporters as helpers.
High cardinality in time-series databases
Each combination is a separate series consuming memory and index; manage by avoiding unbounded labels and bucketing values.
Prometheus histogram versus summary
Histograms expose bucket counts and let you compute aggregatable quantiles at query time; summaries compute fixed quantiles per instance that cannot be combined.
Correlating logs, metrics, and traces
Propagate a trace and span ID through context, stamp it on logs and metric exemplars, then pivot metric to trace to logs.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles