Intermediate interview questions in Growth & Experimentation
Describe RICE scoring and architect data pipelines for Reach and Effort
Define RICE; automate Reach via event streams with time windows; automate Effort from PM tool estimates with calibration.
How do you design allocation logic to minimize concurrent A/B test interactions?
Compare mutual exclusion with layered randomization via layers and reservations; stress isolation vs throughput.
How would you structure an event schema for funnel and cohort analysis?
This tests analytical event modeling. Use an immutable log with one row per event; attach context user_id, session_id, campaign; use a wide atomic table plus extensible contexts for funnel and cohort queries without joins.
Describe the SQL and data model for weekly cohort retention
Join users and activity, compute week offset per user, group by cohort and offset for retention.
Propose a strategy to enforce a consistent analytics event schema
Schema registry with CI validation, typed SDK wrappers blocking bad builds, plus ingestion-time rejection.
How do you instrument client and server to debug payment drop-offs?
Tests designing telemetry that distinguishes intent from errors across the stack. Strong answers use visibility pings for abandonment, validation events for client errors, and gateway status codes for backend failures.
What causes client order_completed events to diverge from backend records?
Tests end-to-end event reliability. Separate client failures (network, ad blockers, duplicates) from backend gaps (idempotency, validation, races) and propose timestamped join analysis. Red flag: blaming users or fixing before measuring gap direction.
Explain user identity stitching across devices and SDK roles
Tests event-level identity resolution. A strong answer covers anonymousId per device, the identify call binding anonymous events to userId, and backend merge handling out-of-order events. Red flag: claiming cookies alone stitch mobile and web.
Design an A/B test for loss aversion versus gain framing at checkout
Tests whether you can isolate framing effects from checkout confounders. Strong answers detail user-level randomization, event logging, and guardrail metrics like revenue per visitor. Red flag: a conversion-only analysis with no unit of diversion defined.
Design a variable daily-login reward system with anti-gaming controls
Tests server-side reward probabilities and idempotency in distributed systems. Strong answers cover: configurable weights, idempotent tokens with DB unique constraints, rolling windows, and server-side grants.
Design the data model and backend for a 7-day trial at scale
Tests state machine design for time-bound entitlements at scale. A strong answer covers: an idempotent enrollment API, a trial ledger with timezone-aware expiration, and an event-driven expiration pipeline.
Explain RICE scoring and its Confidence factor
Score equals Reach times Impact times Confidence divided by Effort; Confidence discounts uncertain estimates; ground it in evidence tiers.
Develop a testable hypothesis for a 40% email verification drop-off
This tests structured hypothesis formation under uncertainty. Strong answers: segment the 40% drop by device and latency; build a Customer Theory from data; isolate one lever; draft a four-part MECLABS hypothesis. Red flag: skipping diagnosis to guess fixes.
When increasing CTA clicks, what side-effects and guardrails should you consider?
This tests balancing growth with business safety. Name guardrails like retention, revenue, fraud, load time; set NI thresholds pre-launch; and include SRM checks. A red flag is treating guardrails as optional success metrics instead of hard stop limits.
Combine qualitative and quantitative data for hypotheses
Quant reveals what and where, qual reveals why, then triangulate into a falsifiable hypothesis with a metric.
Explain statistical power, MDE, and sample size impact
Power is the chance of detecting a true effect; MDE is the smallest lift worth measuring; raising power or shrinking MDE inflates sample size.
How would you implement a last-touch attribution model for user signups?
Tests your ability to translate marketing concepts into warehouse SQL. A strong answer covers UTM/pageview events, sessionized tables, and a windowed join for the last touch within 30 days of signup.
Architect an A/B test for paid-ad signup flows
Tests pre-auth bucketing and funnel attribution. Hash a stable anonymous ID for fast assignment; stream events via Kafka into hourly aggregates; run t-tests on signup rates. Red flag: assigning after signup starts or DB lookups per assignment.
How would you validate that early Project creation drives retention?
Tests causal rigor on behavioral predictors. Good answer: define D30 retention and the 24-hour treatment; pull timestamps and covariates; cohort-compare with propensity matching; show lift with confidence intervals and propose an A/B nudge.
How would you design resumable multi-step onboarding state management?
This tests cross-device onboarding resume. A strong answer uses debounced server sync for cross-device resume with localStorage fallback, covers anonymous users, and handles conflicts. Red flag: pure client or server storage ignoring offline gaps or privacy.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles