Intermediate everything in Growth & Experimentation, page 3
What experiment artifacts and metrics do you present to decide shipping?
This tests structured experiment discipline. A strong answer covers the pre-approved design doc, scorecard results for primary goal and guardrail metrics plus secondary breakdowns versus the hypothesis, and duration context.
Compare PostgreSQL versus a columnar warehouse for raw event data
Contrast row vs column storage, compression, and scan speed; note Postgres suits OLTP and point lookups while columnar stores excel at aggregations.
Should we build or buy an A/B testing platform?
This tests build-vs-buy judgment for experimentation infrastructure. Strong answers cover build for warehouse metrics and cache control; buy for proven stats and front-end speed. Red flag: answering with cost alone or assuming in-house is always superior.
Design an assignment service for an experimentation platform
This tests deterministic hashing and independence in randomization. A strong answer lists user ID, experiment salt, and allocations; uses salted hash for consistency; and warns that separate exposure and bucket hashes with mod can cause sample ratio mismatch.
How would you incorporate external events into a time-series forecast?
This tests exogenous regressor design. A strong answer proposes binary or continuous regressors for holidays and campaigns in Prophet or ARIMA, then validates lift via backtesting. A red flag is dropping outlier days or applying post-hoc manual adjustments.
How would you model a 10% Day-1 retention improvement's impact on LTV?
Tests whether you model retention as a survival curve, not a single point. A strong answer builds a cohort curve, propagates D1 lift to D30/D90 via decay, and sums revenue. Red flag: claiming 10% D1 gain equals 10% LTV growth without curve assumptions.
Walk me through a magic link login system and its security considerations
Tests auth architecture and threat modeling for passwordless flows. Strong answers map request-token-email-verify-session, then harden with short expiry, single-use tokens, rate limits, and device binding.
How would you enforce a 3-project freemium limit and handle upgrades?
Tests entitlement and growth tradeoffs. Strong answers use API-level enforcement, atomic checks to prevent concurrent overages, soft upsell prompts preserving context, and async billing webhooks. Red flag: UI checks or limits in the projects table.
Design referral tracking from invite to conversion
Tests data modeling for multi-stage conversion tracking. Strong answers separate codes from conversion events, model status transitions, and enforce unique constraints. Weak answers merge invite and reward into one table or store denormalized counts on users.
What are the key architectural differences between freemium and free trial models?
Contrast tiered entitlements against trial timers and revocation; discuss free-user overhead and conversion tracking.
How would you instrument an application to calculate Customer Lifetime Value?
Tests whether you tie revenue and cost events to persistent identity and model cohort retention. Outline: track purchases, CAC, and churn with persistent IDs; project value via retention curves.
Design a system to handle subscription renewals
States include active, past_due, and cancelled; transitions are payment success, dunning exhaustion, and grace expiry; use idempotent webhooks.
How do you ensure consistent A/B test variants across sessions and devices?
Tests stable identity and delayed attribution. Fix: one stable user ID for SDK bucketing, persisted across devices via login or first-party cookies, attached to every conversion event. Never use per-device randomization or third-party cookies.
Compare fan-out-on-write vs fan-out-on-read for an activity feed
Tests whether you tie feed architecture to read/write ratios and follower distribution. Strong answers contrast push O(1) reads with celebrity storms against pull O(1) writes with read amplification, then propose a hybrid threshold.
Outline architecture for a weekly email digest of unread notifications
This tests batch processing and scheduled delivery at scale. Pre-aggregate unread counts, shard digest jobs across a distributed scheduler, and cache unsubscribes for fast filtering. Never scan the notifications table at send time for millions of users.
Design an A/B test for a Buy Now button
This tests experiment plumbing: deterministic bucketing, sticky storage, and logging. A strong answer covers user-ID hashing, cookie persistence, and impression-plus-conversion events.
Design a role-based personalized onboarding system
Tests separation of content and logic for scalable personalization. Strong answer: CMS-backed rule engine, multi-channel delivery, event-driven triggers, and per-segment metrics. Red flag: hardcoding role-specific UI components in the client.
How would you design resumable multi-step onboarding state management?
This tests cross-device onboarding resume. A strong answer uses debounced server sync for cross-device resume with localStorage fallback, covers anonymous users, and handles conflicts. Red flag: pure client or server storage ignoring offline gaps or privacy.
How would you validate that early Project creation drives retention?
Tests causal rigor on behavioral predictors. Good answer: define D30 retention and the 24-hour treatment; pull timestamps and covariates; cohort-compare with propensity matching; show lift with confidence intervals and propose an A/B nudge.
Architect an A/B test for paid-ad signup flows
Tests pre-auth bucketing and funnel attribution. Hash a stable anonymous ID for fast assignment; stream events via Kafka into hourly aggregates; run t-tests on signup rates. Red flag: assigning after signup starts or DB lookups per assignment.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles