All bites
The whole library, newest first. Filter by what you are here for, or pick a topic if you already know.
8664 bites
Page 26
Writing SLIs in PromQL
Availability is good requests over total using rate and non-5xx counters; latency uses histogram_quantile over rate of buckets summed by le.
Scaling Prometheus for HA and volume
Shard scraping, add long-term object storage and global query via Thanos/Cortex/VictoriaMetrics, run redundant replicas for HA, with dedupe.
Designing a cache health dashboard
Hit ratio, latency, memory and evictions, connections and saturation, with stat panels for current state and time-series for trends.
Correlating logs, metrics, and traces
Propagate a trace and span ID through context, stamp it on logs and metric exemplars, then pivot metric to trace to logs.
Prometheus histogram versus summary
Histograms expose bucket counts and let you compute aggregatable quantiles at query time; summaries compute fixed quantiles per instance that cannot be combined.
High cardinality in time-series databases
Each combination is a separate series consuming memory and index; manage by avoiding unbounded labels and bucketing values.
Core components of Prometheus
Server scrapes targets found via service discovery, stores samples in a local time-series database, with Alertmanager and exporters as helpers.
Histograms versus summaries for latency
Histograms store bucket counts and stay aggregatable with query-time approximate quantiles; summaries precompute quantiles per instance that cannot be averaged.
When to add custom OpenTelemetry instrumentation
Wrap domain logic in custom spans, attach business attributes like tenant or plan, answer revenue and per-customer questions.
Redesigning a high-cardinality request metric
Cardinality is unique label combinations; user_id and raw path are unbounded; redesign by dropping user_id and templating the path.
Trace context and propagation across services
Trace context bundles trace ID, span ID, and flags; propagated via headers like W3C traceparent so each service extracts and continues the trace.
OpenTelemetry API, SDK, and Collector
API defines vendor-neutral instrumentation, SDK implements and exports it, Collector receives, processes, and routes telemetry to backends.
Essential tags for a request latency metric
Add method, route template, status code, and maybe service or region; they enable slicing while staying low-cardinality.
Structured vs unstructured logging
Unstructured logs are free-text lines; structured logs are machine-readable key-value or JSON, enabling reliable parsing, filtering, and aggregation.
The three pillars of observability
Metrics show what and when via cheap aggregates, logs give detailed per-event context, traces show where across services.
Fixing watermelon SLOs that hide unhappy users
Green-outside, red-inside SLOs come from server-side measurement, coarse aggregation, wrong events, and lax thresholds; fix by measuring at the client and using meaningful SLIs.
Launching a risky feature with no error budget left
Bring burn history, what consumed the budget, blast radius, and rollback safety; offer mitigations like flags and canaries.
Setting SLOs across a dependency chain
Serial dependencies multiply, so each backend needs a tighter SLO than the target; mitigate with redundancy, caching, and graceful degradation.
Handling planned maintenance in SLOs and error budgets
Either charge maintenance to the error budget or formally exclude it via a maintenance window; each trades honesty against fairness.
Defining SLOs for a new critical service
Start from user journeys, pick SLIs, involve product, engineering, and business stakeholders, set realistic targets iteratively.