Propose a multi-touch attribution model and its data pipeline
This tests your ability to choose a practical data model under real-world constraints. Propose a time-decay or position-based model, then describe the data pipeline: event collection, identity resolution, and aggregation. A red flag is ignoring signal loss.
Calculate Daily Active Users (DAU) with SQL
This tests your ability to translate a business metric into a precise technical definition and query. A good answer defines "active," specifies the event data needed (user_id, timestamp, event_name), and uses COUNT(DISTINCT user_id).
Build a SQL query for a multi-step conversion funnel
Tests your ability to translate a product question into robust SQL. A great answer uses CTEs or left joins to count users at each step, defining the attribution model (e.g., first-touch) and time windows. A red flag is a naive query that double-counts users.
Trade-offs: Bundled Analytics vs. a Warehouse-Native Stack?
This tests your grasp of modern data stack trade-offs: cost, data governance, and flexibility. Discuss the pros of warehouse-native (unified data, lower cost, security) vs. the cons (loss of specialized UI, implementation complexity).
Architect a Multi-Touch Attribution System
Tests your grasp of data pipeline trade-offs under real-world signal loss. A great answer outlines the pipeline (ingest, store, model), contrasts last-touch (simple state) vs.
Design a User Onboarding Funnel Analysis System
This tests translating a business need into a data model and query. First, define the cohort. Then, use CTEs to find the first timestamp for each event per user. Finally, count users at each step.
Implement an A/B test for a new checkout flow
Tests your grasp of the full A/B testing lifecycle. A great answer outlines a feature flag system, consistent user bucketing via hashing a stable user ID, and an analytics query grouping by variant. A red flag is suggesting simple client-side randomization.
How do you handle the multiple comparisons problem in A/B testing?
Tests your grasp of statistical risk in experimentation. Explain how multiple tests inflate false positives, then describe mitigations like Bonferroni correction or limiting concurrent tests. A red flag is suggesting total test isolation, which is impractical.
Visualize Correlation Between Load Time and Session Duration
Tests your ability to choose the right chart for correlation and layer in additional variables. A great answer starts with a scatter plot (load time vs. session duration), then uses color to represent the network type.
.png&w=3840&q=75)
Set up a cohort analysis for a new onboarding flow
This tests your ability to design a clean experiment to measure product impact. A great answer defines control/treatment cohorts by acquisition date (before/after Jan 1st), picks a specific metric like W1 retention, and compares them.
How to visualize a complex, multi-stage customer funnel?
Tests your ability to choose the right visualization for non-linear user flows. Propose a Sankey or Alluvial diagram to show flow volume, drop-off, and re-entry. A red flag is suggesting multiple simple charts that fail to show the paths between stages.
Enforce GDPR's Right to be Forgotten Across a Complex Architecture
This tests your design of a verifiable, async deletion workflow. A strong answer proposes a central index metastore, an orchestrated workflow (e.g., Step Functions) for deletion, and an auditing layer.
Apply the AARRR framework to B2B SaaS vs. B2C mobile games
Tests translating the AARRR framework into concrete analytics for different business models. Define AARRR, then contrast B2B SaaS (account-level activation) with B2C games (user-level engagement). Red flag: using identical metric definitions for both contexts.
What is the difference between a metric and a KPI?
This tests your ability to connect technical measures to business outcomes. Define metrics as operational data and KPIs as the subset tied to critical goals.
A key metric dropped 20%. How would you investigate?
This tests systematic diagnosis of critical issues. A great answer segments the drop (by region, platform), then traces data upstream from the dashboard to the source, correlating with technical metrics. A red flag is jumping to code before scoping the impact.
Design an A/B Testing Framework
This tests your ability to design a scalable system with statistical rigor for non-experts. A great answer outlines config management, deterministic user hashing, a data pipeline for metrics, and a results UI that simplifies stats.
Propose a technical architecture for a centralized Metrics Layer.
This tests your grasp of data governance and semantic layers. A great answer outlines a system with a central definition store (e.g., YAML in Git), a query engine, and an API, ensuring all teams get consistent metric results.
Design a Real-Time Analytics Pipeline for Mobile Events
This tests your grasp of low-latency streaming architectures. A good answer outlines ingestion (SDK to Kafka/Kinesis), real-time processing (Flink/Spark), and sinking to a fast OLAP database (Druid/ClickHouse). A red flag is proposing a batch-based ETL design.
Design a data model for tracking feature adoption
This tests your grasp of data warehouse star schemas for analytics. Outline a fact table for events and dimension tables for users and features, explaining how this structure enables fast, ad-hoc cohort analysis for a product manager.
Explain cohort analysis and write a retention query
Tests your grasp of user behavior metrics. First, define a cohort (e.g., May signups). Then, explain why it's better than aggregate retention. Finally, outline a query grouping users by signup week and checking for activity. A red flag is a flawed query.