Analytics &amp; Metrics

How would you determine if Feature X causally drives higher retention?

Tests causal inference intuition for product metrics. Great answers propose a randomized holdback or instrumental variable, control for user intent, and estimate a local average treatment effect.

What user segments do you check first after a 10% DAU drop?

Validate by time, platform, and geography; then slice by new vs returning, channel, and feature usage to isolate the bleeding cohort.

How would you visually represent statistical uncertainty in a chart?

Awareness that plotted points are perceived as exact truths. Replace isolated bars with intervals showing point estimate uncertainty; add hypothetical outcome plots to make values tangible. Offering p-values or raw means without visualizing uncertainty range.

How do you build a performant visualization for millions of time-series points?

Tests end-to-end data reduction: backend bucket downsampling like LTTB preserves visual shape, frontend uses level-of-detail rendering and viewport culling. Red flag: naive every-Nth sampling that drops peaks or sending raw millions to the browser.

Explain pre-attentive attributes and give three examples

This tests whether you know preattentive attributes are decoded in <200ms to guide attention freely. Name three such as color hue, size, and motion; then encode one variable in a dense scatter plot so targets pop out. Never call this decoration or color all.

Stakeholder claims feature usage drives retention. How do you validate?

Probe confounders, reverse causality, selection bias; propose diff-in-diff or propensity scores; plot lagged usage vs retention.

How would you structure a user engagement dashboard for PMs?

Tests narrative sequencing of metrics across abstraction layers. Strong answer: DAU headline for health, retention cohorts for pattern diagnosis, feature adoption funnels for root-cause drill-down.

Compare five companies' market share: bar or line chart?

This tests categorical vs. temporal encoding. Pick a bar chart because companies are discrete categories, not a time series; line charts falsely imply sequence or trend. Calling a line chart acceptable is a red flag.

When is a pie or donut chart appropriate?

Tests judgment of part-to-whole encoding. Answer: use for few categories with clear dominance, cite a share scenario like device traffic, and name angle-comparison difficulty and 3D distortion as pitfalls.

How do you manage event schema evolution without breaking reports?

Backward-compatible serialization, nullable new fields, raw versus modeled layers, versioned schemas, and consumer alerts.

What are the challenges of grouping by a high-cardinality dimension?

Tests columnar storage internals and query engine scalability. A strong answer covers memory pressure from giant hash tables, destroyed compression ratios, and massive result-set overhead.

How do duplicate events bias COUNT(*) and daily login reports?

Tests idempotency in streaming analytics. COUNT(*) overcounts; fix with unique event ID dedup via idempotent writes or COUNT(DISTINCT id), plus daily partition reconciliation. Red flag: SELECT DISTINCT * without a stable key or no reporting safeguard.

Why is star schema preferred over 3NF for analytics?

Tests your grasp of the read-performance trade-off in analytical schemas. A great answer names fact and dimension tables, emphasizes fewer joins for aggregations, and cites simpler SQL and faster query plans.

Slow dashboard querying a large fact table: first three checks?

This tests systematic diagnosis of fact-table query latency. A strong answer checks the execution plan and indexing, evaluates partitioning and data model fit, and inspects caching or pre-aggregation.

Why prefer median and p95 over mean for API latency?

This tests statistical intuition for skewed distributions. A strong answer notes that median captures typical experience, p95 captures tail suffering, and mean hides outliers. A red flag is claiming mean alone is sufficient.

Design a scalable data governance framework balancing autonomy and control

Self-serve platform with domain products, auto-catalog, schema contracts, and policy-as-code access in CI/CD.

How do you root-cause bad data across microservices and Spark?

This tests structured debugging and observability for distributed pipelines. A strong answer isolates the break via lineage, validates schema and freshness per stage, and compares microservice outputs to Spark inputs.

How would you build a Customer golden record across fragmented systems?

This tests master data management discipline for distributed, conflicting records. A strong response covers identity resolution, survivorship rules, merge architecture, lineage, and feedback loops.

Design a CDC pipeline that handles schema evolution gracefully

Tests designing resilient CDC pipelines against schema drift. A strong answer covers schema registries with versioning, backward-compatible serialization, and automated compatibility checks.