Visualizing a correlation with a third variable
WHAT IT TESTS: Bivariate viz plus confound awareness. OUTLINE: A scatter plot with a trend line shows the relationship; encode network type by color or facets to expose a lurking variable. RED FLAG: Treating the correlation as causal.
Cohort analysis for an onboarding change
WHAT IT TESTS: Cohort reasoning and clean framing. OUTLINE: A cohort groups users by a shared start trait; compare pre and post Jan-1 signup cohorts on retention by age. RED FLAG: Comparing calendar periods instead of cohort age, confounding seasonality.
Visualizing a non-linear funnel with re-entry
WHAT IT TESTS: Matching visualization to data shape. OUTLINE: A linear funnel cannot show branching or re-entry; a Sankey diagram encodes flow volume, splits, and leaks as proportional ribbons. RED FLAG: Defaulting to a bar funnel or pie chart.
Enforcing GDPR Right to be Forgotten at scale
WHAT IT TESTS: Privacy engineering across stores. OUTLINE: Map the subject's data, then crypto-shred the lake, DELETE in the warehouse, and evict caches via an auditable, idempotent workflow. RED FLAG: Assuming one DELETE suffices everywhere.
Designing a useful button_click event payload
WHAT IT TESTS: Event instrumentation design. OUTLINE: Include identity, timestamp, and context plus properties like button id, screen, and state; govern with a naming convention. RED FLAG: A bare event name, or ad hoc field names per event.
Designing a warehouse model for feature adoption
WHAT IT TESTS: Dimensional modeling skill. OUTLINE: Star schema with a feature-usage fact table at a defined grain, surrounded by user, feature, date, and device dimensions. RED FLAG: One giant wide table or modeling without defining the grain.
Diagnosing model degradation over time
WHAT IT TESTS: MLOps maturity around drift. OUTLINE: Name it model drift, split data vs concept drift; diagnose by comparing distributions and ruling out pipeline bugs; fix via monitoring and retraining. RED FLAG: Blind retraining before diagnosis.
Stationarity in time series and why ARIMA needs it
WHAT IT TESTS: whether you know stationarity means stable statistical properties over time. OUTLINE: constant mean/variance/autocovariance; ARIMA's coefficients assume them; test with the ADF test and ACF plots; achieve it via differencing or log transforms.
Describe the architecture of a generic A/B testing framework
WHAT IT TESTS: system design with statistical safety. ANSWER OUTLINE: hash-based user bucketing, config service, pre-registered metrics, and confidence intervals on dashboards. RED FLAG: request-level randomization or skipping power analysis.
Explain event schemas and why schema registries matter at scale
This tests schema evolution and data contracts in distributed systems. A good answer defines schemas as contracts, explains that a registry enforces compatibility to block breaks, and lists pain like pipeline failures. Never treat schemas as optional docs.
How would you instrument and query P95 API latency by region?
This tests white-box latency instrumentation and safe cardinality for percentile aggregation. Strong answer: emit histograms by region, query P95 with histogram_quantile or a log percentile, and keep trace IDs in logs only.
Which classical baseline model handles weekly seasonality and upward trend?
Tests matching model structure to data characteristics. Name Holt-Winters triple exponential smoothing; map its level, trend, and seasonal equations to weekly period. Red flag: jumping to SARIMA without explaining why ETS is the natural baseline.
How do you measure forecast accuracy and compare MAE to RMSE?
This tests out-of-sample validation and how MAE and RMSE weight errors. A strong answer demands a train-test split, defines both, and notes RMSE punishes outliers more while MAE is more robust. A red flag is citing in-sample fit instead of held-out error.
Trade-offs between pre-aggregated and raw event data for dashboards
WHAT IT TESTS: Balancing latency, cost, and freshness in analytics. ANSWER OUTLINE: Pre-aggregations trade freshness for speed; raw queries preserve flexibility but spike cost and latency under load.
Compare data warehouses and data lakes. How does a lakehouse merge benefits?
Tests schema tradeoffs. Warehouses enforce ACID for BI but cost more; lakes store raw cheaply but lack governance. Lakehouses add ACID metadata on object storage to unify ML and BI.
Propose a North Star Metric for a product you know
WHAT IT TESTS: Can you isolate the one metric capturing user value that predicts business health. A GOOD ANSWER COVERS: definition; your product's metric; how value drives retention and revenue.
Apply AARRR to B2B SaaS vs B2C mobile game analytics
This tests mapping AARRR to instrumentation across business models. A strong answer contrasts B2B account activation and expansion against B2C session-zero funnels and whale monetization. Red flag: same metrics ignoring account hierarchies and ad attribution.
What is the difference between a metric and a KPI?
Tests strategic vs operational measurement discernment. Answer: KPIs track critical goals; metrics track processes. Page views are a metric; conversion rate is the KPI. Red flag: calling all data KPIs or using page views as success proof.
Design column-level data lineage from source to dashboard
WHAT IT TESTS: Metadata architecture tracing columns through heterogeneous batch and SQL engines. ANSWER OUTLINE: Propose AST extractors for Spark and dbt, a graph DB for column edges, and an API for impact analysis.
How do you root-cause a 20% revenue drop with no pipeline failures?
WHAT IT TESTS: Incident leadership and validating data integrity before calling a downturn. ANSWER OUTLINE: Reconcile against raw events, slice by dimension for silent gaps, audit schema drift.