Interview questions in Growth & Experimentation, page 6
Design a referral feature's lifecycle and races
A referral entity with explicit states, a unique constraint on the invited user, and atomic transactions plus idempotency to prevent double credits.
Run concurrent experiments without interference
Independent non-interacting tests can share traffic through orthogonal layers; interacting ones need mutual exclusion in a shared layer.
Design a contamination-safe pricing experiment
User-level price tests leak via fairness perception, so use geo holdouts or time-based cohorts where everyone in a unit sees one price.
Prevent conflicting experiments with layers
Group conflicting experiments into one layer so a user's per-layer bucket maps to at most one of them; orthogonal layers can overlap.
Increase experiment velocity for non-engineers
Server-driven config, feature flags, and a self-serve UI let non-engineers ship copy or layout variants instantly; add guardrails and metric checks.
Architect a configurable, goal-based onboarding flow
Capture the goal, let the backend return a server-driven flow definition mapping goal to steps and content, render generic components on the client.
Why repeatedly extending a test inflates false positives
Repeatedly checking and extending until significance is p-hacking via optional stopping, which inflates the false-positive rate; fix with fixed sample sizes or sequential…
Handle interaction effects on a shared page
Combined variants may produce effects neither has alone; use mutual exclusion for likely interactions, orthogonal designs with interaction monitoring otherwise.
Attribute a mobile install to a desktop ad
Deterministic matching via a shared login is accurate but needs auth on both ends; probabilistic fingerprinting scales without login but is noisy and privacy-fraught.
Set up a client-side button color A/B test
Stable hashing of a persistent ID into buckets, conditional rendering of the variant, exposure plus click event logging.
Design a centralized experimentation service
A config/assignment API, deterministic SDK-side bucketing, and a separate exposure-logging pipeline.
Experiment design under network effects
Cluster-level randomization, graph or geo clustering to contain spillover, and analysis at the cluster unit.
Client-side vs server-side event tracking
Client captures UI intent but loses data to ad blockers and tampering; server is trustworthy for transactions but blind to UI interactions.
Design an analytics event schema
Consistent object-action naming, snake_case, typed properties with units, and shared context like user, session, timestamp.
Migrate a breaking analytics schema change
Dual-write both fields during overlap, backfill history, migrate consumers, then deprecate the old field.
Design a streak-saver mechanism
Timezone-aware streak state, capped saver inventory with replenish rules, and A/B testing forgiveness against retention plus guardrails.
Components of a testable A/B hypothesis
A specific change, a predicted directional effect on one primary metric, a rationale, and a measurable success threshold.
Communicate experiment results and check guardrails
Explain the lift and confidence in plain business terms, report a confidence interval not just a point, and verify guardrails before shipping.
Capture UTM params and attribute on signup
Parse UTMs on landing, persist them in a cookie tied to an anonymous ID, then stamp them onto the account at signup.
Design a unique referral code system
A unique DB constraint as the source of truth, generation via random retry or an encoded counter, and collision handling.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles