Metrics
400 bites tagged Metrics — interview questions with model answers, and 60-second explainers.
MRR and ARR: Two Lenses on Recurring Revenue
MRR and ARR are two lenses on recurring revenue. MRR tracks short-term movement and immediate performance while ARR shows long-term scale. SaaS companies need both to guide present tactics and future strategy.
KPI Tree: Linking Product Changes to Business Outcomes
A KPI tree traces how changes ladder up to outcomes, giving metrics connective tissue. Build one when dashboards and experiments conflict and you need a defensible line from features to impact. Let it stay static and it becomes disconnected.
What experiment artifacts and metrics do you present to decide shipping?
This tests structured experiment discipline. A strong answer covers the pre-approved design doc, scorecard results for primary goal and guardrail metrics plus secondary breakdowns versus the hypothesis, and duration context.
What differentiates leading and lagging indicators for subscriptions?
Leading indicators forecast; lagging indicators confirm. Subscriptions: activation rate leads, MRR lags. Predictive vs historical metrics in growth. Using raw signups as leading without proven revenue link.
Implement a 10% onboarding tutorial feature flag and track success
Tests lightweight experiment design and metric selection beyond vanity numbers. Strong answers cover user-ID bucketing, conversion tracking, and guardrail metrics like drop-off. Red flag: manual toggles or click-through without downstream outcomes.
WAU is flat despite positive A/B tests; why and how to diagnose
This tests distinguishing real impact from statistical artifacts. Strong answers cite false positives from low base rates, peeking, novelty, and local-global mismatches. Diagnose with long-term holdouts, audits, and causal bridges.
Design an A/B test for loss aversion versus gain framing at checkout
Tests whether you can isolate framing effects from checkout confounders. Strong answers detail user-level randomization, event logging, and guardrail metrics like revenue per visitor. Red flag: a conversion-only analysis with no unit of diversion defined.
What counter metrics track health of weekly active users?
Tests whether you can spot growth-at-all-costs blind spots. A strong answer pairs WAU with 7-day retention, sessions per user, and error rate, mapping each to churn, shallow engagement, or bugginess.
HEART Framework: Five Metrics for Product Health
HEART is a vital-signs panel for product health, not a single score. Use it to pick metrics for feature launches so you track user value, not vanity numbers. Measuring all five dimensions when only one matters creates dashboard bloat and analysis paralysis.
What metrics measure design system success and adoption?
Tests if you connect design system health to delivery speed and consistency. Strong answers hit adoption rate, component usage, design-to-code parity, and accessibility compliance tied to team behavior.
Quantify UI Debt Before It Compounds
UI debt quantification turns messy interfaces into measurable cost. Teams track component adoption, override rates, and design-dev drift to prioritize refactors. The footgun is treating every inconsistency as debt, ignoring the business value of shipping fast.
NPS for Design Systems: Smoke Alarm, Not Scorecard
NPS for design systems is a smoke alarm for team trust, not a feature scorecard. Poll consuming teams quarterly to catch sentiment drops before adoption stalls. Never benchmark against consumer SaaS; internal tools face forced usage and different expectations.
How would you validate that search users are more likely to purchase?
This tests correlation versus causation in product analytics. A strong answer defines the purchase window, matches searchers to similar non-searchers, and picks a statistical test. A red flag is running a t-test without controlling for user intent or time.
Design a system to measure technical content deflection
Tests causal attribution of content on tickets. Answers must unify identity, correlate topics with NLP, compute deflection as self-service resolutions over total intent, and validate outcomes.
What metrics prove documentation business impact and how do you instrument?
Tests tying docs to revenue and costs. Strong answers present acquisition (dev signups, quickstart rate), adoption (endpoint breadth, trial conversion), and deflection (support tickets) via event instrumentation and CRM joins.
What CTA metric wins an A/B test and how to log it?
This tests connecting instrumentation to business outcomes via a click metric and tracked event. An answer picks click or conversion rate, fires an event with variant ID, and notes uniqueness. A red flag is using views without linking the event to the button.
How would you collect metrics and KPIs for your Internal Developer Platform?
This tests product-thinking: treating developers as customers, not captive users. Strong answers cover adoption (golden-path usage), developer experience (deploy speed, NPS), and business value. Red flag: tracking CPU or uptime without linking to adoption.
Design a system correlating CI/CD deployments with observability metrics
Tests event-driven correlation and temporal join patterns across distributed data. Strong answers outline: async deploy event ingestion, time-windowed metric joins with version tags, and handling clock skew.
How would you instrument CI/CD to measure a DORA metric accurately?
Tests mapping DORA definitions to pipeline events. A strong answer picks one metric, defines exact boundaries from merge to production, and correlates deployments with incidents.
What is the difference between a metric and a KPI?
Tests strategic vs operational measurement discernment. Answer: KPIs track critical goals; metrics track processes. Page views are a metric; conversion rate is the KPI. Red flag: calling all data KPIs or using page views as success proof.
How do you root-cause a 20% revenue drop with no pipeline failures?
Reconcile against raw events, slice by dimension for silent gaps, audit schema drift. Incident leadership and validating data integrity before calling a downturn.
What data do you need and what steps build a WAU dashboard?
Tests defining a metric, modeling events, and wiring them into a BI tool. A strong answer names the feature event, sets a rolling 7-day window, counts distinct users by period, and configures the BI layer. Red flag: jumping to charts before defining active.
Explain cohort retention and write a pseudo-query for May signups
Tests cohort retention vs aggregate DAU and SQL self-joins for Week 1, 2, and 4 retention from May signups. Strong answers define cohorts by signup date, use datediff, and left-join activity. Red flag: using calendar week instead of relative signup date.
Describe client-side events and properties to track Export to CSV usage
This tests telemetry design for async actions. A strong answer defines three custom events—click, success, failure—with properties like location, file_size, error_code, and user_id, fired at the right lifecycle moments.
Get Metrics bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.