Metrics
400 bites tagged Metrics — interview questions with model answers, and 60-second explainers.
How would you visualize three years of monthly revenue?
This tests your grasp of time-series visualization and data integrity. A strong answer picks a line chart, insists on a zero-based Y-axis and clear labels, and adds context like seasonality.
How do you shift analytics from growth to profitability?
This tests your ability to translate business strategy into technical changes. A great answer connects profitability drivers to specific updates in event taxonomy, data models, and dashboards. A red flag is ignoring core financial metrics like LTV and CAC.
How would you define and calculate Weekly Active Users (WAU)?
This tests your product sense and technical precision in defining a core business metric. A great answer defines 'active' with specific user actions, outlines the SQL/event-based calculation, and discusses pitfalls like bots and background events.
When is A/B testing not feasible, and what is an alternative?
Tests your grasp of causal inference when randomization isn't possible. A great answer names a scenario (like a regional launch), proposes Difference-in-Differences (DiD), and explains its core 'parallel trends' assumption.
Calculate Sample Size for a 2% A/B Test Lift
This tests if you connect statistical inputs to business goals. A good answer defines baseline rate, minimum detectable effect (MDE), and power, then explains MDE as a cost/benefit trade-off.
What does a p-value of 0.03 mean in an A/B test?
This tests your practical grasp of statistical significance. A good answer defines p-value (probability of the result if the null hypothesis is true), explains that p=0.03 is significant vs. alpha=0.05, and concludes you can reject the null.
Propose a multi-touch attribution model and its data pipeline
Tests your grasp of attribution models and their data engineering needs. Propose a rule-based model (e.g., time-decay), outline the data pipeline for it, and acknowledge privacy-driven signal loss. A red flag is ignoring the challenge of identity resolution.
Find leading indicators for long-term churn
This tests your ability to translate a business problem into a data investigation. A strong answer defines churned vs. retained cohorts, hypothesizes key early behaviors, and compares their frequency to find a leading indicator.
Calculate Daily Active Users (DAU) with SQL
This tests product sense and SQL fundamentals. Define 'active' with a core product action, describe the event data needed, then write a COUNT(DISTINCT user_id) query. A red flag is writing SQL before defining the business logic for 'active'.
Client-Side vs. Server-Side Event Tracking
Tests your grasp of data integrity and architectural trade-offs. A great answer defines both, favors server-side for reliability (avoids ad-blockers), but notes client-side's richness for UI events. A red flag is presenting them as equal choices.
Why don't analytics and backend user counts match?
This tests your systematic debugging and understanding that 'user' has different definitions. A good answer first defines 'user' in each system, then investigates tracking implementation, timing differences, and filtering.
Client-side vs. Server-side Event Tracking: When and Why?
This tests your grasp of data reliability and security trade-offs. A good answer defines both, contrasts reliability (ad blockers) vs. implementation ease, and uses a critical event like "Payment Processed" to justify server-side's accuracy.
Architect a Multi-Touch Attribution System
This tests your grasp of modern data challenges like signal loss. A good answer discusses data ingestion, identity resolution, and model trade-offs. A red flag is focusing only on the algorithm and ignoring the data pipeline's fragility.
What is the 'multiple comparisons problem' in A/B testing?
Tests your grasp of statistical pitfalls in large-scale A/B testing. Define the problem (inflated false positives), explain the business risk (wasted effort), and propose a mitigation like Bonferroni correction.
Visualizing Load Time vs. Session Duration with a Third Variable
Tests your ability to visualize correlation and add dimensions. A great answer suggests a scatter plot for the initial relationship, then uses color to segment by the categorical third variable (network type).
Cohort Analysis for a New Onboarding Flow
Tests applying analytics to measure impact. Define a cohort, then compare a pre-launch (Dec) vs. post-launch (Jan) acquisition cohort, tracking retention over time. A red flag is using aggregate metrics, which hide the true impact of the change.
How would you visualize a complex, multi-stage user funnel?
Tests product sense and data viz literacy. A good answer proposes a Sankey/Alluvial diagram to show non-linear flows, explains how it visualizes drop-off and re-entry, and notes the data needs. A red flag is just suggesting a better standard funnel chart.
Design an analytics event payload for a button click
This tests your data modeling for analytics. A good answer includes the event name, user ID, and timestamp, then adds contextual and user properties. A red flag is forgetting the user ID or suggesting dynamic property names, which breaks segmentation.
Describe AARRR and apply it to B2B vs. B2C analytics
Tests applying the AARRR framework to different business models. Define AARRR, then apply to B2B SaaS (account-level activation) vs. a B2C game (user-level virality). Red flag: using generic metrics that ignore the context of B2B sales vs. B2C usage.
What's the difference between a metric and a KPI?
This tests your ability to connect technical measurements to strategic business outcomes. A great answer defines both, notes KPIs are a subset of metrics tied to goals, and gives a concrete example like page views (metric) vs. conversion rate (KPI).
Design a Centralized Metrics Layer
This tests your grasp of data governance and creating a single source of truth. A good answer defines a semantic layer between the data warehouse and BI tools, centralizing metric definitions in code.
Instrumenting a New User Interaction for Analytics
Tests your grasp of the full data lifecycle. A good answer covers event definition, client-side implementation, the backend pipeline, and end-to-end verification.
How would you build a weekly active user dashboard?
This tests your ability to translate a business request into a technical plan. A good answer defines "active," identifies necessary data (user ID, timestamp, event), outlines the data modeling, and explains the BI tool implementation.
How do you measure impact while accounting for the novelty effect?
Tests your ability to design experiments that isolate long-term effects. A good answer proposes a long-running A/B test, analyzing user cohorts by join date to see if initial lift decays. A red flag is ignoring the novelty effect and suggesting a short test.
Get Metrics bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.