Metrics
400 bites tagged Metrics — interview questions with model answers, and 60-second explainers.
Why is tracking team velocity as a KPI dysfunctional?
Tests if you know velocity is for planning, not performance. Explain it's easily gamed and measures output, not outcome. Propose metrics focused on value delivery and process improvement like cycle time.
Apply Little's Law to a Kanban system to optimize flow
Tests applying queuing theory to software delivery. Define Little's Law as WIP = Throughput × Cycle Time. Explain how reducing WIP limits directly shortens cycle time for a stable throughput.
CFD 'Code Review' band is widening. What does it mean?
This tests your ability to interpret a CFD and propose actions. A widening 'Code Review' band means work enters faster than it leaves. Diagnose the bottleneck (e.g., review quality, reviewer availability), then propose solutions.
Investigating Variable Sprint Velocity
This tests your ability to diagnose issues by connecting process metrics to technical health. A great answer hypothesizes technical causes (e.g., tech debt, flaky tests), identifies specific data for validation (e.g., cycle time, build logs), and avoids…
CFD shows a widening 'Testing' band. What does it mean?
This tests your ability to interpret process metrics and propose data-driven solutions. First, define the bottleneck: work enters testing faster than it leaves. Then, propose experiments to diagnose the cause before suggesting solutions.
Lead Time vs. Cycle Time in Kanban
Tests your understanding of core Kanban metrics for process improvement. Define Lead Time (request to delivery) and Cycle Time (work start to completion), noting Cycle Time is a subset. A red flag is confusing the two or being imprecise about start/end points.
How do you measure a new feature's success beyond bugs and uptime?
Tests if you connect engineering to business value. A great answer links success to the feature's original goals, proposes user behavior and business impact metrics, and names specific tools.
How would you A/B test a redesigned dashboard?
This tests your ability to translate a product goal into a technical plan. A good answer defines "engagement" with metrics, outlines the bucketing and instrumentation strategy, and discusses statistical significance.
Flow Efficiency: Are You Working or Waiting?
Flow efficiency measures the ratio of active work time to total lead time, revealing how much time tasks spend just waiting. Use it to diagnose why features take so long to ship. The biggest footgun is optimizing work speed when most delays hide in queues.
Throughput: Measuring What Gets Done
Throughput measures how many work items a team *finishes* in a time period, not how busy they are. It's used for forecasting future work and spotting bottlenecks. The footgun: never compare throughput between different teams, as item sizes and context vary.
Evidence-Based Management (EBM): Measure Value, Not Just Velocity
Evidence-Based Management (EBM) is like a fitness tracker for your organization, using data to guide decisions instead of gut feel. It helps you measure progress toward goals and improve outcomes.
CSAT: Measuring If You Met Customer Expectations
CSAT measures if your product met a customer's expectations for a specific interaction. It's used to evaluate experiences like a support call or purchase, giving a direct pulse check on service quality.
Goal-Question-Metric: Measure What Matters, Not What's Easy
GQM is a top-down framework for defining metrics. You start with a Goal, ask Questions to clarify it, then define Metrics to answer them. This avoids the common trap of collecting vanity metrics that don't reflect true software quality or business goals.
Counter Metrics: Guardrails for Your Goals
Counter metrics are the guardrails for your primary goal, preventing you from optimizing one number at the expense of user experience. If you increase ad impressions for revenue, track user retention to ensure you aren't just driving users away with spam.
The HEART Framework for Measuring UX
The HEART framework provides a structure for measuring user experience on large-scale web applications. It helps teams define user-centered metrics to track progress towards goals and make data-driven decisions.
AARRR 'Pirate' Metrics: A Funnel for What Really Matters
The AARRR framework is a five-stage funnel (Acquisition, Activation, Retention, Referral, Revenue) that tracks the user journey. It helps product teams focus on metrics that directly impact business health, not vanity metrics like social media likes.
Vanity vs. Actionable Metrics: Measure What Matters
Vanity metrics look impressive but don't inform decisions (e.g., total downloads). Actionable metrics tie to business goals and guide your next move (e.g., conversion rate). This helps product teams focus on real growth, not just impressive-looking charts.
Unit Economics: Is Each Customer Profitable?
Unit economics asks if you make or lose money on a single customer or sale. It's used in SaaS to compare customer lifetime value (LTV) to acquisition cost (CAC). The footgun is defining the 'unit' poorly, hiding that each new customer costs you money.
Customer Lifetime Value (LTV): Predicting Future Customer Profit
LTV predicts the total net profit a customer will generate over their entire relationship with you. It guides how much to spend on acquiring customers (CAC) and helps identify your most valuable segments. The footgun: LTV is profit, not revenue.
TAM, SAM, SOM: Sizing Your Market Opportunity
TAM, SAM, and SOM are nested filters for market size. TAM is the total demand, SAM is the segment you can serve, and SOM is what you can realistically capture. It's how you go from 'everyone' to 'our first 1,000 users'.
Mean Time To Repair (MTTR): Measuring Your Fix Velocity
MTTR measures how quickly your team can fix a problem once active work begins. It's the 'wrench time' of incident response, not total outage duration. SREs track it to gauge runbook and diagnostic effectiveness.
Mean Time to Acknowledge (MTTA): Your First Response Clock
MTTA measures the time from an alert firing to a human acknowledging it. It's about reaction speed, not fix time. On-call teams use this to ensure issues are seen quickly, minimizing downtime.
Prometheus Exemplars: Link Your Metrics to Traces
Exemplars are like footnotes for your metrics, linking a data point like a latency spike directly to a specific trace ID. This lets you jump from a 'what' on a dashboard to the 'why' in your tracing system.
Cardinality: The Hidden Cost of Time-Series Metrics
Cardinality is the number of unique label combinations in your metrics. High cardinality, from labels like user IDs, is the silent killer of monitoring systems like Prometheus, exploding memory and cost. The footgun is adding a label with unbounded values.
Get Metrics bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.