Intermediate everything in Analytics & Metrics, page 8
Explain the North Star Metric and propose one for a product
Tests your ability to connect user value to business outcomes. A great answer defines the NSM, proposes one for a product (e.g., Spotify), and justifies how it links customer value to business success. A red flag is picking a vanity metric like DAU or revenue.
How would you diagnose why a new feature isn't being adopted?
This tests your ability to diagnose a flat KPI. A great answer outlines a funnel (awareness, activation, usage) and combines quantitative data with qualitative insights from session replays. A red flag is proposing solutions without a diagnostic plan.
Pitfalls of 'Conversion Rate' as a North Star Metric
Tests your ability to see beyond a single metric. A good answer identifies how optimizing conversion can hurt revenue or UX, and proposes guardrails like Average Order Value, support tickets, and return rates.
Design a Privacy-Compliant Analytics Architecture
This tests your ability to balance data utility with strict privacy controls. A great answer outlines a central governance layer, dynamic masking, and purpose-based access tied to auditable logs.
A key metric dropped 15%. How do you investigate?
This tests systematic debugging of business metrics. A great answer first validates the data itself, then checks for recent changes (deploys, features), and finally segments the drop to isolate the cause. A red flag is immediately assuming a product bug.
SARIMA vs. LightGBM for Forecasting with External Variables
Tests your grasp of practical trade-offs in model selection. A strong answer contrasts SARIMA's interpretability with LightGBM's power for handling many non-linear variables, covering performance and implementation costs.
Visualize two continuous and one categorical variable?
Tests your ability to map data to visual encodings. A great answer starts with a scatter plot, then adds the categorical data using color, shape, or faceting, explaining the tradeoffs. A red flag is suggesting a 3D chart, which is difficult to read.
How would you visualize three years of monthly revenue?
This tests your grasp of time-series visualization and data integrity. A strong answer picks a line chart, insists on a zero-based Y-axis and clear labels, and adds context like seasonality.
Define idempotency in data processing and give an example
Tests your grasp of distributed systems reliability. Define idempotency (N>1 runs = 1 run), explain its role in fault-tolerant retries, and provide a concrete example using transaction IDs. A red flag is confusing it with immutability.
How do you shift analytics from growth to profitability?
This tests your ability to translate business strategy into technical changes. A great answer connects profitability drivers to specific updates in event taxonomy, data models, and dashboards. A red flag is ignoring core financial metrics like LTV and CAC.
How would you define and calculate Weekly Active Users (WAU)?
This tests your product sense and technical precision in defining a core business metric. A great answer defines 'active' with specific user actions, outlines the SQL/event-based calculation, and discusses pitfalls like bots and background events.
When is A/B testing not feasible, and what is an alternative?
Tests your grasp of causal inference when randomization isn't possible. A great answer names a scenario (like a regional launch), proposes Difference-in-Differences (DiD), and explains its core 'parallel trends' assumption.
Calculate Sample Size for a 2% A/B Test Lift
This tests if you connect statistical inputs to business goals. A good answer defines baseline rate, minimum detectable effect (MDE), and power, then explains MDE as a cost/benefit trade-off.
Design a Schema Validation System for Analytics Events
This tests your ability to balance data quality, performance, and developer experience. A good answer defines a central 'Tracking Plan,' enforces it on the client for feedback and the server for integrity, and quarantines failed events.
Calculate MRR from a subscriptions table using SQL
Tests your ability to translate a business metric (MRR) into a precise SQL query. A great answer filters for active subscriptions this month and sums their prices, correctly amortizing annual plans.
What does a p-value of 0.03 mean in an A/B test?
This tests your practical grasp of statistical significance. A good answer defines p-value (probability of the result if the null hypothesis is true), explains that p=0.03 is significant vs. alpha=0.05, and concludes you can reject the null.
How do you handle timezones for a daily global sales report?
This tests your understanding of time data modeling and business requirements. A good answer stores events in UTC with a timezone identifier, then converts to the business's chosen 'day' at query time. A red flag is storing local time without context.
How would you implement Change Data Capture (CDC)?
Tests your grasp of data replication trade-offs. A great answer compares log-based CDC (low impact, complete) with query-based methods (higher impact, misses deletes), and recommends log-based CDC for its minimal production impact.
Propose a multi-touch attribution model and its data pipeline
Tests your grasp of attribution models and their data engineering needs. Propose a rule-based model (e.g., time-decay), outline the data pipeline for it, and acknowledge privacy-driven signal loss. A red flag is ignoring the challenge of identity resolution.
Transform a time series for a supervised learning model?
This tests your ability to reframe a time series problem for tabular models. A great answer explains creating features from lags, rolling windows, and calendar data, then emphasizes using a time-aware validation split. A red flag is forgetting validation.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles