Observability
124 bites tagged Observability — interview questions with model answers, and 60-second explainers.
What metrics and instrumentation prove your CI/CD feature saves DevOps time?
Propose pipeline duration and queue time as leading metrics and rollback frequency as lagging. Mapping a fuzzy value prop to technical proxies for engineer time. Citing build count without linking to minutes saved.
How would you instrument front-end and back-end to quantify reported slowness?
This tests translating vague complaints into end-to-end telemetry. A strong answer covers front-end Web Vitals, back-end traces, and segments by user or page to isolate pain points. A red flag is focusing only on server CPU or logs without real user timings.
How would you systematically diagnose high latency in an online inference service?
Check p90/p99 and TTFT to split queuing from compute; inspect queue depth, batch size, GPU, and benchmarks; check cache. Systems reasoning across serving stack.
How would you systematically debug an inference API latency breach?
This tests structured debugging across the full inference stack. A strong answer traces the request path from ingress to GPU, splits TTFT from token-generation latency, inspects queuing and batching, then applies targeted fixes.
Compare Go and Rust approaches to exposing profiling data
Contrast Go's pprof import with Rust crates or profilers, noting runtime versus OS-level sampling. Trade-offs between Go's pull model and Rust's push or attach models. Claiming Rust has a std-lib pull endpoint like Go.
How would you implement zero-downtime secrets rotation?
Inventory secrets and app caching; baseline monitoring; dual-phase rotation with overlapping secrets; verify before revoking old. rotating credentials without downtime.
How do you unify real-time CI/CD health across hundreds of microservices?
Canonical event schema over an event bus; domain-level SLI views; federated ownership with golden paths. designing event-driven observability at scale without a new monolith.
Design a system correlating CI/CD deployments with observability metrics
Tests event-driven correlation and temporal join patterns across distributed data. Strong answers outline: async deploy event ingestion, time-windowed metric joins with version tags, and handling clock skew.
How would you instrument CI/CD to measure a DORA metric accurately?
Tests mapping DORA definitions to pipeline events. A strong answer picks one metric, defines exact boundaries from merge to production, and correlates deployments with incidents.
How would you use distributed tracing to debug a deployment latency issue?
This tests causal request-path analysis beyond aggregate metrics. A strong answer filters traces by the new version, finds the exact regressed span, and compares it to a pre-deployment baseline.
What are the four Golden Signals for service health monitoring?
Tests whether you can name the four essential metrics—latency, traffic, errors, saturation—and explain why each matters for detecting user-facing regressions after a release, rather than drowning in infrastructure noise.
How can a service mesh facilitate canary or A/B testing?
Tests mesh-level traffic control decoupled from app releases. Strong answers name traffic shifting, request routing, ingress gateways, and telemetry-driven rollback. Red flag: citing mTLS alone and omitting observability automation.
What problem can a breaking API change cause during a rolling update?
Tests if you know rolling updates run mixed versions, so breaking API changes crash cross-traffic. Good answer: note old and new pods serve together, watch probes fail, and monitor 5xx spikes. Red flag: claiming Kubernetes isolates versions during rollout.
Prevent developer access to production secrets while preserving debuggability
Use dynamic short-lived credentials, break-glass with dual-control audit, and structured telemetry or synthetic transactions for debugging. Architecture keeping production plaintext secrets invisible to developers.
APM: Turning System Metrics into Business Meaning
APM turns system metrics into business meaning by tracking software performance and availability. It matters most when slowdowns threaten service levels. The footgun is gathering data without translating IT metrics into what the business actually cares about.
How do you root-cause a 20% revenue drop with no pipeline failures?
Reconcile against raw events, slice by dimension for silent gaps, audit schema drift. Incident leadership and validating data integrity before calling a downturn.
How would you design an automated data quality monitoring system?
Tests turning data quality into tiered checks for exec dashboards. Strong answers combine freshness, volume, schema, and distribution validation with severity-based paging. Red flag: static thresholds without noise reduction or business-impact triage.
Explain the difference between correlation and causation with a software example.
Tests whether you distinguish association from causation to avoid blaming production issues. A strong answer defines both concepts, names a confounding variable, and gives a software example with a common cause. Red flag: claiming correlation is causation.
Why prefer median and p95 over mean for API latency?
This tests statistical intuition for skewed distributions. A strong answer notes that median captures typical experience, p95 captures tail suffering, and mean hides outliers. A red flag is claiming mean alone is sufficient.
How do you root-cause bad data across microservices and Spark?
This tests structured debugging and observability for distributed pipelines. A strong answer isolates the break via lineage, validates schema and freshness per stage, and compares microservice outputs to Spark inputs.
How do you measure data platform ROI and track it?
Cite adoption, time to insight, downtime cost, and cost per workload; then describe cost tags and usage telemetry. Linking platform spend to business value and team health.
A key metric dropped 20%. How would you investigate?
This tests systematic diagnosis of critical issues. A great answer segments the drop (by region, platform), then traces data upstream from the dashboard to the source, correlating with technical metrics. A red flag is jumping to code before scoping the impact.
How would you measure P95 latency by geographic region?
Tests your ability to translate a business need into a concrete observability implementation. A good answer involves instrumenting the API with a histogram metric, adding a region label via GeoIP, and querying with `histogram_quantile`.
Why use median/p95 for API latency instead of the mean?
This tests if you understand how long-tail distributions make averages misleading for user experience. A good answer explains that median (p50) shows the typical user, while p95 captures the worst-case experience.
Get Observability bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.