Everything in Growth & Experimentation
Designing a self-serve experimentation platform
SDK with sane defaults, automated pre-launch validation, sample-ratio and guardrail-metric checks.
Sensitivity analysis on a growth model
Perturb each input by a normalized amount, measure the change in the long-term output, and use elasticities or global methods to rank drivers.
Build a SaaS churn prediction model
Define churn and the prediction window, engineer usage-trend and tenure features, try logistic regression then gradient-boosted trees, and evaluate on class-imbalanced metrics.
Peeking in A/B tests and how to mitigate it
Peeking is checking significance repeatedly and stopping at the first significant result, which inflates false positives; mitigate with fixed sample sizes or sequential…
Cold-start to personalized feed transition
Start with popularity or onboarding-declared interests, collect implicit signals like dwell and clicks, then blend toward personalized as confidence grows.
Dynamic personalized onboarding architecture
A segmentation pipeline, a serving layer choosing task order per segment, an experimentation engine, and a feedback loop measuring activation.
Cross-platform stateful onboarding sync
Store onboarding state server-side keyed to the user, expose idempotent step-completion APIs, and push updates to other clients.
Implement a welcome-message A/B test
Deterministic hash of a stable ID for sticky assignment, conditional rendering of the personalized variant, and exposure plus click tracking keyed to the same ID.
Detect fraudulent app installs
Click-to-install timing distributions, device and IP fingerprints, post-install engagement, and attribution anomalies.
Explain deferred deep linking flow
Capture link payload server-side at click, route to the store, then match the new install to the click on first launch to route the user.
Design a unique referral code system
A unique DB constraint as the source of truth, generation via random retry or an encoded counter, and collision handling.
Capture UTM params and attribute on signup
Parse UTMs on landing, persist them in a cookie tied to an anonymous ID, then stamp them onto the account at signup.
Communicate experiment results and check guardrails
Explain the lift and confidence in plain business terms, report a confidence interval not just a point, and verify guardrails before shipping.
Components of a testable A/B hypothesis
A specific change, a predicted directional effect on one primary metric, a rationale, and a measurable success threshold.
Design a streak-saver mechanism
Timezone-aware streak state, capped saver inventory with replenish rules, and A/B testing forgiveness against retention plus guardrails.
Migrate a breaking analytics schema change
Dual-write both fields during overlap, backfill history, migrate consumers, then deprecate the old field.
Design an analytics event schema
Consistent object-action naming, snake_case, typed properties with units, and shared context like user, session, timestamp.
Client-side vs server-side event tracking
Client captures UI intent but loses data to ad blockers and tampering; server is trustworthy for transactions but blind to UI interactions.
Experiment design under network effects
Cluster-level randomization, graph or geo clustering to contain spillover, and analysis at the cluster unit.
Design a centralized experimentation service
A config/assignment API, deterministic SDK-side bucketing, and a separate exposure-logging pipeline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles