Intermediate interview questions in Growth & Experimentation, page 4
Components of a testable A/B hypothesis
A specific change, a predicted directional effect on one primary metric, a rationale, and a measurable success threshold.
Communicate experiment results and check guardrails
Explain the lift and confidence in plain business terms, report a confidence interval not just a point, and verify guardrails before shipping.
Capture UTM params and attribute on signup
Parse UTMs on landing, persist them in a cookie tied to an anonymous ID, then stamp them onto the account at signup.
Design a unique referral code system
A unique DB constraint as the source of truth, generation via random retry or an encoded counter, and collision handling.
Explain deferred deep linking flow
Capture link payload server-side at click, route to the store, then match the new install to the click on first launch to route the user.
Detect fraudulent app installs
Click-to-install timing distributions, device and IP fingerprints, post-install engagement, and attribution anomalies.
Implement a welcome-message A/B test
Deterministic hash of a stable ID for sticky assignment, conditional rendering of the personalized variant, and exposure plus click tracking keyed to the same ID.
Cross-platform stateful onboarding sync
Store onboarding state server-side keyed to the user, expose idempotent step-completion APIs, and push updates to other clients.
Dynamic personalized onboarding architecture
A segmentation pipeline, a serving layer choosing task order per segment, an experimentation engine, and a feedback loop measuring activation.
Cold-start to personalized feed transition
Start with popularity or onboarding-declared interests, collect implicit signals like dwell and clicks, then blend toward personalized as confidence grows.
Peeking in A/B tests and how to mitigate it
Peeking is checking significance repeatedly and stopping at the first significant result, which inflates false positives; mitigate with fixed sample sizes or sequential…
Build a SaaS churn prediction model
Define churn and the prediction window, engineer usage-trend and tenure features, try logistic regression then gradient-boosted trees, and evaluate on class-imbalanced metrics.
Sensitivity analysis on a growth model
Perturb each input by a normalized amount, measure the change in the long-term output, and use elasticities or global methods to rank drivers.
Designing a self-serve experimentation platform
SDK with sane defaults, automated pre-launch validation, sample-ratio and guardrail-metric checks.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles