Metrics
400 bites tagged Metrics — interview questions with model answers, and 60-second explainers.
Why use p50/p95 over mean for API response times?
Tests your grasp of statistical distributions for UX metrics. A good answer explains how outliers skew the mean, while percentiles (p50, p95) better represent typical and worst-case user experiences.
How would you debug a sudden drop in a key metric?
Tests your systematic debugging of complex data systems. First, validate the drop against a source of truth and segment the data. Then, hypothesize causes like misattribution or data loss.
How would you measure the ROI of a data analytics platform?
This tests your ability to connect platform engineering to business value. A great answer quantifies ROI via cost savings, revenue generation, and risk reduction, then details the systems (e.g., cost tagging, metadata tracking) needed.
How would you design an analytics strategy for a marketplace?
Tests your ability to balance conflicting needs in a complex system. A great answer covers core health (liquidity, match rate), side-specific KPIs (buyer satisfaction, seller utilization), and unit economics (take rate).
Explain the North Star Metric and propose one for a product
Tests your ability to link product strategy to a single metric reflecting customer value and business growth. Define the NSM, propose one for a product like Spotify, and justify it. A red flag is picking a vanity metric like DAU or a pure business metric.
How would you diagnose a flat feature adoption KPI?
This tests your ability to create a diagnostic plan from a single lagging metric. A great answer outlines a funnel (Awareness > Activation > Usage), segments users, and combines quantitative data with qualitative feedback.
Pitfalls of 'Conversion Rate' as a North Star Metric
This tests your ability to see beyond a single metric and understand its second-order effects. A strong answer identifies pitfalls like lower AOV, then proposes counter-metrics (AOV, return rate) and guardrail metrics (page load time).
Translate 'increase engagement' into a technical measurement plan
This tests your ability to translate vague business goals into concrete metrics. First, clarify the goal with the PM. Then, propose specific, measurable proxy metrics (e.g., DAU/MAU, session length). Finally, outline the instrumentation plan.
Evidence-Based Management: Metrics Over Gut Feel
EBM replaces 'I think' with 'I know because the data shows...' It's about making decisions using evidence, not just intuition. Use it to prioritize features with user data or optimize processes by measuring cycle time. The footgun is metric fixation.
RFM Analysis: Find Your Best Customers
RFM analysis segments customers by scoring their Recency, Frequency, and Monetary value. This helps identify your best customers (high RFM), those at risk (low R/F), and new high-spenders.
GSM: Connect Your Goals to Real Metrics
The GSM framework turns fuzzy goals into concrete numbers by linking what you want (Goal), to observable behaviors (Signal), to a specific measurement (Metric). It's used to define KPIs for new features. The footgun is choosing easy-to-measure vanity metrics.
Using Flow Metrics to Coach for Predictability
Tests if you use data for coaching, not coercion. Define Lead/Cycle Time, explain how stable times (p85) create predictability, and contrast this with Velocity's flaws. The red flag is treating any metric as a target instead of a diagnostic tool for the team.
How do you use a cycle time scatterplot to set an SLE?
This tests your ability to use data, not feelings, to manage stakeholder expectations. Explain the scatterplot, identify the outlier, calculate the 85th percentile, and propose a data-backed Service Level Expectation (SLE).
What's the relationship between Cycle Time, WIP, and Throughput?
This tests your grasp of Little's Law. A great answer defines Cycle Time, WIP, and Throughput, states the formula (Cycle Time = WIP / Throughput), and explains why lowering WIP reduces cycle time.
Burn-up vs. Burn-down Charts: What's the Difference?
Tests your understanding of project tracking metrics and their implications. A burn-down shows work remaining vs. time. A burn-up shows work completed vs. total scope, making it better for visualizing scope creep. A red flag is just describing the lines.
How do you provide a probabilistic forecast for completing 25 stories?
This tests your ability to move beyond single-date estimates to probabilistic forecasting. A good answer outlines using historical cycle times in a Monte Carlo simulation to generate a range of dates with confidence levels.
How would you set a WIP limit for code review?
This tests your ability to use data to diagnose a bottleneck and facilitate a solution. A great answer gathers flow metrics, proposes an initial limit based on team size (e.g., N/2), and frames it as an experiment.
Team Velocity Dropped for 3 Sprints. How Do You Diagnose?
This tests your ability to use data for diagnosis, not blame. A good answer gathers quantitative (cycle time, unplanned work) and qualitative data, then presents hypotheses to the team.
How do you measure your effectiveness as a Scrum Master?
Tests your ability to link coaching to business outcomes, not just team output. A great answer quantifies impediment removal, team health improvements, and stakeholder satisfaction, avoiding vague claims or simply citing team velocity.
Distinguish Throughput from Velocity in agile planning
This tests your grasp of outcome (Throughput) vs. effort (Velocity) metrics. Define both: Throughput is item count/time, Velocity is points/sprint. Contrast them by explaining Throughput measures actual delivery, not estimates.
How would you coach a team with fluctuating velocity?
This tests your ability to use metrics for coaching, not just reporting. A good answer reframes the goal to predictability, investigates both qualitative and quantitative data, and proposes experiments. A red flag is treating velocity as a performance metric.
How would you advocate for decentralizing deployment approvals?
This tests your ability to drive organizational change with data. A great answer frames the problem using a decision framework (e.g., SAFe), proposes a phased pilot, and defines metrics like Cycle Time and Change Failure Rate to prove value.
How would you create a probabilistic forecast for a backlog?
This tests your grasp of probabilistic forecasting over single-date estimates. A good answer explains using historical throughput to run a Monte Carlo simulation, then presenting a range of dates with confidence levels (e.g., 50%, 85%).
When does a task's Cycle Time begin and end?
This tests your practical grasp of process metrics. Define Cycle Time as starting when active work begins ('In Progress') and ending when 'Done' (shippable). Contrast it with Lead Time (request to delivery). A red flag is confusing the two or being too vague.
Get Metrics bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.