Everything in Monitoring & SRE
Datadog Infrastructure Basic targets host health checks
Datadog describes a lighter monitoring tier for host metrics and selected processes on noncritical workloads.
Grafana 13.2 adds multiple notification policy trees
Grafana says multiple notification policies are generally available in Grafana-managed alerting with Grafana 13.2, enabling independently managed routing trees.
Federating reliability ownership to product teams
Build a self-service reliability platform (golden paths, paved roads), train teams and embed SLO/on-call practices, and govern with standards plus error budget…
Capacity planning for a distributed cache
Track hit ratio, memory and eviction rate, throughput/latency, and connections; correlate with growth to forecast when to add capacity before the hit ratio or evictions…
Capacity planning for distributed stateful systems
Account for replication and cross-region network, IOPS and disk throughput, connection limits, partition/shard balance, and N+1 region failover headroom; validate with load and chaos…
Designing a feature flagging service
Control plane (UI, store, targeting), SDKs that cache flags locally for zero-latency evaluation, streaming/CDN delivery for near-real-time updates, and stale-flag lifecycle tooling.
Managing a risky release with feature flags
Deploy code dark behind an off flag, enable for internal then small percentage, monitor metrics, ramp gradually, then remove the flag.
Automating actions on error budget burn
Use multi-window burn-rate alerts; on fast burn, auto-trigger actions like halting deploys, rolling back, or scaling, with escalating tiers.
Safeguards for automated remediation runbooks
Add rate limits and circuit breakers on restarts, escalate to humans after N attempts, log all actions, and check for cascading failure before acting.
Breaking tunnel vision during an incident
Call out the assumption, ask for disconfirming evidence, list parallel hypotheses, split responders to investigate them, and anchor on what changed and the data.
Key roles in incident response
Incident Commander coordinates and decides but does not fix; Communications Lead handles stakeholders; Operations/SMEs do hands-on diagnosis and remediation.
OpenTelemetry agent and gateway architecture
Agents run per-node for local collection and host enrichment; gateways are central, horizontally scaled pools for batching, tail sampling, and routing.
First-SRE 90-day plan at a startup
Listen and measure first, pick one high-impact service, define SLIs/SLOs and basic alerting, then show reduced toil or incidents to leadership.
Designing shallow vs deep health checks
Shallow checks confirm the process is alive; deep checks verify dependencies; use shallow for liveness/load-balancer routing and deep sparingly to avoid…
Rolling update vs blue-green deployment
Rolling replaces instances gradually with minimal extra capacity but mixes versions; blue-green runs two full environments for instant switch and rollback at double the cost.
Writing high-quality postmortem action items
Good action items are specific, assigned to an owner, prioritized, tracked to completion, and ideally prevent recurrence rather than just detect faster.
Calculating downtime for a 99.9% SLO
0.1% of 30 days is roughly 43 minutes of allowed downtime; healthy budget enables faster shipping while depletion slows or freezes deploys.
Designing an error budget policy
Define SLO and budget, tiered consequences as burn worsens, a feature freeze on exhaustion, and concrete earn-back criteria.
Keeping a postmortem blameless after an admission
Acknowledge the courage, redirect from who to why the system allowed it, ask what guardrails were missing.
Conducting a Production Readiness Review
Assess monitoring and alerting, capacity and load testing, failure modes and dependencies, on-call and runbooks, and rollback or release safety.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles