Metrics
400 bites tagged Metrics — interview questions with model answers, and 60-second explainers.
Bar Chart vs. Line Chart for Market Share Comparison?
This tests your grasp of visualization fundamentals for categorical vs. time-series data. A bar chart is correct for comparing discrete categories (companies) at one point in time. A line chart wrongly implies a trend. Red flag: justifying a line chart.
When is a pie chart an appropriate visualization?
Tests your grasp of data viz principles for part-to-whole data. A good answer defines this use case (e.g., market share), then lists pitfalls like too many slices or comparing multiple pies. A red flag is defending pie charts for complex data.
Why use median/p95 for API latency instead of the mean?
This tests if you understand how long-tail distributions make averages misleading for user experience. A good answer explains that median (p50) shows the typical user, while p95 captures the worst-case experience.
Sudden metric drop, no recent deployments. What's the cause?
This tests your ability to debug data discrepancies beyond code, focusing on the analytics pipeline. First, distinguish data loss from misattribution. Then, check processing delays and hidden data sources. A red flag is not segmenting data first.
How would you measure the ROI of a data analytics platform?
This tests your ability to connect platform metrics to business value. A good answer defines KPIs for adoption, performance, and cost, then links them to business impact.
Design a KPI Strategy for a Two-Sided Marketplace
Tests your ability to balance a complex ecosystem. A great answer defines KPIs for liquidity (search-to-fill), transaction economics (take rate), and true health (net revenue over GMV).
Explain the North Star Metric and propose one for a product
Tests your ability to connect user value to business outcomes. A great answer defines the NSM, proposes one for a product (e.g., Spotify), and justifies how it links customer value to business success. A red flag is picking a vanity metric like DAU or revenue.
How would you diagnose why a new feature isn't being adopted?
This tests your ability to diagnose a flat KPI. A great answer outlines a funnel (awareness, activation, usage) and combines quantitative data with qualitative insights from session replays. A red flag is proposing solutions without a diagnostic plan.
Pitfalls of 'Conversion Rate' as a North Star Metric
Tests your ability to see beyond a single metric. A good answer identifies how optimizing conversion can hurt revenue or UX, and proposes guardrails like Average Order Value, support tickets, and return rates.
Translate 'increase engagement' into a technical measurement plan
This tests your ability to translate a vague business goal into a structured, measurable technical plan. Clarify the goal with the PM, define a primary metric and supporting metrics, then create an instrumentation spec.
Throughput vs. Velocity in Agile Planning
This tests your grasp of flow vs. estimation metrics. Define throughput as a count of delivered items and velocity as a sum of estimated points. Throughput measures actual output, making it better for forecasting. Red flag: claiming velocity is more accurate.
How do you handle wildly fluctuating team velocity?
Tests if you know velocity is for team planning, not a manager's KPI. A good answer reframes the goal to predictability, investigates root causes with the team (e.g., story sizing, unplanned work), and proposes experiments.
Advocating to decentralize a deployment approval board
Tests your ability to influence change with data. A good answer frames deployments as frequent, time-critical decisions ideal for decentralization, proposes a phased rollout with metrics like cycle time, and defines new guardrails.
How would you create a probabilistic forecast for 40 stories?
This tests your ability to use statistical methods for forecasting. A great answer explains how to use historical throughput in a Monte Carlo simulation to generate a probability distribution of completion dates, not a single point estimate.
When Does Cycle Time Begin and End?
This tests your grasp of process metrics. Define Cycle Time as starting when active work begins ('In Progress') and ending when it's 'done' (code complete/merged), not when the ticket was created. A red flag is confusing this with customer-facing Lead Time.
Why is team velocity a poor KPI for agile success?
Tests your grasp of agile principles and Goodhart's Law. Explain velocity is for forecasting, not performance. Detail dysfunctions like point inflation and ignoring quality. Propose outcome-focused alternatives like cycle time.
How would you apply Little's Law to a Kanban system?
This tests your ability to use metrics for process improvement. A great answer defines Little's Law for Kanban (Cycle Time = WIP / Throughput), explains how reducing WIP shortens cycle times, and gives a numerical example.
Diagnosing a Widening Code Review Band in a CFD
This tests data interpretation and problem-solving. A great answer defines the bottleneck, hypothesizes causes (capacity, PR size), proposes diagnostics, then offers solutions. A red flag is blaming individuals or jumping to a single, unverified solution.
Investigating Variable Sprint Velocity: Technical Root Causes
This tests your ability to diagnose team issues with data, not anecdotes. Propose technical hypotheses like flaky tests or merge conflicts and link them to metrics like CI/CD failure rates or PR cycle time. A red flag is blaming individuals or poor estimation.
Diagnosing a Widening CFD 'Testing' Band
Tests your ability to interpret a CFD and propose data-driven experiments. A widening 'Testing' band means work enters faster than it leaves. Diagnose with experiments (e.g., tracking test failures, environment downtime) before proposing solutions.
How would you measure the success and impact of a new feature?
This tests your ability to connect engineering work to business value. A strong answer defines success metrics upfront, instruments code for quantitative data like adoption rates, and gathers qualitative feedback.
How would you A/B test a redesigned dashboard?
Tests translating a vague goal ('more engagement') into a concrete engineering plan. A good answer defines key metrics first, then outlines user bucketing, instrumentation, and statistical analysis.
A key metric dropped 15%. How do you investigate?
This tests systematic debugging of business metrics. A great answer first validates the data itself, then checks for recent changes (deploys, features), and finally segments the drop to isolate the cause. A red flag is immediately assuming a product bug.
Visualize two continuous and one categorical variable?
Tests your ability to map data to visual encodings. A great answer starts with a scatter plot, then adds the categorical data using color, shape, or faceting, explaining the tradeoffs. A red flag is suggesting a 3D chart, which is difficult to read.
Get Metrics bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.