Everything in Growth & Experimentation, page 2
Set up a client-side button color A/B test
Stable hashing of a persistent ID into buckets, conditional rendering of the variant, exposure plus click event logging.
Attribute a mobile install to a desktop ad
Deterministic matching via a shared login is accurate but needs auth on both ends; probabilistic fingerprinting scales without login but is noisy and privacy-fraught.
Handle interaction effects on a shared page
Combined variants may produce effects neither has alone; use mutual exclusion for likely interactions, orthogonal designs with interaction monitoring otherwise.
Why repeatedly extending a test inflates false positives
Repeatedly checking and extending until significance is p-hacking via optional stopping, which inflates the false-positive rate; fix with fixed sample sizes or sequential…
Architect a configurable, goal-based onboarding flow
Capture the goal, let the backend return a server-driven flow definition mapping goal to steps and content, render generic components on the client.
Increase experiment velocity for non-engineers
Server-driven config, feature flags, and a self-serve UI let non-engineers ship copy or layout variants instantly; add guardrails and metric checks.
Prevent conflicting experiments with layers
Group conflicting experiments into one layer so a user's per-layer bucket maps to at most one of them; orthogonal layers can overlap.
Design a contamination-safe pricing experiment
User-level price tests leak via fairness perception, so use geo holdouts or time-based cohorts where everyone in a unit sees one price.
Run concurrent experiments without interference
Independent non-interacting tests can share traffic through orthogonal layers; interacting ones need mutual exclusion in a shared layer.
Design a referral feature's lifecycle and races
A referral entity with explicit states, a unique constraint on the invited user, and atomic transactions plus idempotency to prevent double credits.
Why consistent user IDs matter in experiments
The user id seeds deterministic bucketing and ties events to one person across devices; achieve it via authenticated ids and anonymous-to-known stitching.
Trace an event from click to analysis
Client SDK captures and batches, a collection endpoint ingests, a stream and ETL enrich and load into a warehouse for analysis.
Communicate forecast uncertainty with prediction intervals
A point estimate hides risk; produce a prediction interval via model error, simulation, or scenarios, and state assumptions.
When user-level A/B tests get contaminated
Network or marketplace spillover violates SUTVA, so randomize by cluster (geo, group, time) and analyze at that level.
Build an opportunity-sizing model before building
Locate the affected funnel step, estimate addressable population times a bounded conversion lift times value per user, then sanity-check against a realistic ceiling.
Combine qualitative and quantitative data for hypotheses
Quant reveals what and where, qual reveals why, then triangulate into a falsifiable hypothesis with a metric.
Explain RICE scoring and its Confidence factor
Score equals Reach times Impact times Confidence divided by Effort; Confidence discounts uncertain estimates; ground it in evidence tiers.
Instrument a first-full-song activation event
Define 'full song' server-side, emit a typed event with user, song, and context, dedupe the first-time flag.
Architect an experimentation dashboard for culture
Searchable experiment repository, structured hypotheses, results regardless of outcome, and cross-team discovery.
Resurrection Campaign
A resurrection campaign is a targeted effort to win back dormant or churned users by re-engaging them with relevant value, often via email or push. It matters because reactivating known users is usually cheaper than acquiring new ones.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles