Skip to content
tezvyn:

All bites

The whole library, newest first. Filter by what you are here for, or pick a topic if you already know.

8664 bites

Page 23

Monitoring & SRE2 min read

Design a simple chaos experiment for a cache dependency?

Hypothesis that the service degrades gracefully when Redis is unavailable, monitor error rate, latency, DB load, and cache hit rate.

Monitoring & SRE2 min read

What is blast radius and how do you limit it?

Blast radius is the scope of users or systems an experiment can harm; limit it by targeting a small traffic percentage and by having an automated abort.

Monitoring & SRE1 min read

How does chaos engineering differ from other testing?

It experiments on real systems by injecting faults to test a steady-state hypothesis, versus verifying known behaviors like integration or load tests.

Monitoring & SRE2 min read

Reliability paved roads for an internal PaaS?

Built-in observability, safe deploys with health checks and rollback, sane timeouts/retries/limits, and SLO tooling.

Monitoring & SRE2 min read

Design a chaos experiment for a payment dependency?

Define a measurable steady state, hypothesize it holds when payments fail, limit blast radius to a small traffic slice, and auto-abort on SLO breach.

Monitoring & SRE1 min read

Replication and consistency for active-active regions?

Choose per data class between synchronous (low RPO, higher latency) and async replication, address write conflicts, and reason via CAP and PACELC.

Monitoring & SRE2 min read

Zero-downtime index migration on a hot table?

Build the index concurrently to avoid table locks, run off-peak with monitoring, and keep it reversible since dropping an index is cheap.

Monitoring & SRE1 min read

Mitigating risk from an unproven external dependency?

Timeouts and circuit breakers to fail fast, bulkheads to isolate resources, fallbacks or cached/degraded responses.

Monitoring & SRE1 min read

Reliability patterns for queue-based job processing?

Retries with backoff and jitter for transient faults, dead-letter queues plus a poison-message limit, idempotent handlers and visibility timeouts.

Monitoring & SRE1 min read

What is an SLO and how do you define API availability?

SLO is a target on an SLI, availability SLI is good requests over valid requests, you need labeled request counts over a window.

Monitoring & SRE1 min read

What telemetry must a new microservice ship with?

Metrics for trends like p99 latency and error rate, structured logs for per-request detail, traces for cross-service causality.

Monitoring & SRE2 min read

When horizontal scaling is the wrong fix

Adding instances fails when the bottleneck is a shared resource like one database, a lock, or a queue, so more instances just add contention; investigate where time is actually spent.

Monitoring & SRE1 min read

Why tail latency (p99) matters

Averages hide the worst experiences, and fan-out amplifies tails so most requests touch a slow path; causes include GC pauses, queueing, contention, and noisy neighbors.

Monitoring & SRE1 min read

Front-end performance budgets

A performance budget is an enforced limit on metrics like Core Web Vitals and bundle size, checked in CI to fail builds that regress.

Monitoring & SRE1 min read

Little's Law for capacity planning

L equals lambda times W, concurrency equals arrival rate times time in system; rearrange to size threads or concurrency for a target throughput and latency.

Monitoring & SRE1 min read

Capacity planning for annual traffic spikes

Forecast peak from history, load-test to find per-unit capacity, use autoscaling with pre-warming and headroom, and protect with caching and graceful degradation.

Monitoring & SRE1 min read

Diagnose database CPU saturation under load

Find the expensive queries via the database's stats, check for missing indexes and full scans, then fix with indexing, query rewrites, caching, or read replicas.

Monitoring & SRE1 min read

Load vs stress vs soak testing

Load tests expected traffic, stress pushes past limits to find the breaking point, soak runs sustained load for hours to expose leaks.

Monitoring & SRE1 min read

Golden signals for capacity planning

Monitor the four golden signals, latency, traffic, errors, and saturation, from day one, watching percentiles and saturation to forecast scaling.

Monitoring & SRE1 min read

Canary a shared downstream microservice

Route a slice of traffic to the canary via mesh rules, propagate context, and use distributed tracing to measure impact on upstream callers across the full path.