More in Growth & Experimentation — page 5
Compare fan-out-on-write vs fan-out-on-read for an activity feed
Tests whether you tie feed architecture to read/write ratios and follower distribution. Strong answers contrast push O(1) reads with celebrity storms against pull O(1) writes with read amplification, then propose a hybrid threshold.
Outline architecture for a weekly email digest of unread notifications
This tests batch processing and scheduled delivery at scale. Pre-aggregate unread counts, shard digest jobs across a distributed scheduler, and cache unsubscribes for fast filtering. Never scan the notifications table at send time for millions of users.
Design an A/B test for a Buy Now button
This tests experiment plumbing: deterministic bucketing, sticky storage, and logging. A strong answer covers user-ID hashing, cookie persistence, and impression-plus-conversion events.
Describe the data model and backend logic for a daily login bonus.
This tests streak state machines and calendar edge cases. A strong answer stores last_login_utc and streak_count, uses UTC day buckets, resolves timezones per user tz, and needs no leap-year logic. RED FLAG: Naive 24-hour windows break during DST shifts.
Describe the end-to-end data flow for tracking a 'Share' button click
WHAT IT TESTS: Full analytics pipeline design. ANSWER OUTLINE: Payload carries event type, user ID, timestamp, device, content; client batches with retry; backend validates and lands in a partitioned store. RED FLAG: Raw PII, no timestamps, no dedup.
Long-term onboarding holdback: technical and data integrity challenges
This tests the engineering cost of year-long holdbacks in growth. A strong answer covers feature-flag entropy, pipeline drift, survivorship bias, and counterfactual validity. Red flag: treating the holdback as static config that never rots.
Design a role-based personalized onboarding system
Tests separation of content and logic for scalable personalization. Strong answer: CMS-backed rule engine, multi-channel delivery, event-driven triggers, and per-segment metrics. Red flag: hardcoding role-specific UI components in the client.
How would you design resumable multi-step onboarding state management?
This tests cross-device onboarding resume. A strong answer uses debounced server sync for cross-device resume with localStorage fallback, covers anonymous users, and handles conflicts. Red flag: pure client or server storage ignoring offline gaps or privacy.
How would you validate that early Project creation drives retention?
Tests causal rigor on behavioral predictors. Good answer: define D30 retention and the 24-hour treatment; pull timestamps and covariates; cohort-compare with propensity matching; show lift with confidence intervals and propose an A/B nudge.
How do you determine if a user is 'new' for a setup guide?
This tests whether you separate account age from user state for onboarding. Good answers compare created_at (brittle) with a persistent flag (idempotent) and consider milestones. A red flag is using a timestamp as a permanent new proxy without managing reruns.
How would you instrument a 4-step onboarding wizard?
WHAT IT TESTS: Event schema design for funnel analysis beyond page views. ANSWER OUTLINE: Track Step Started and Step Completed with step_index and flow_variant; tie via distinct_id. RED FLAG: Only pageviews without step IDs, forcing brittle URL funnels.
Design a programmatic SEO system for 1 million landing pages
Tests data infrastructure thinking, not content generation. Covers one-row-one-page schema, template rendering with edge caching, hierarchical routing, and crawl-budget controls via sitemaps. Red flag: AI bulk writing without structured data or caching.
Architect an A/B test for paid-ad signup flows
Tests pre-auth bucketing and funnel attribution. Hash a stable anonymous ID for fast assignment; stream events via Kafka into hourly aggregates; run t-tests on signup rates. Red flag: assigning after signup starts or DB lookups per assignment.
How would you implement a last-touch attribution model for user signups?
Tests your ability to translate marketing concepts into warehouse SQL. A strong answer covers UTM/pageview events, sessionized tables, and a windowed join for the last touch within 30 days of signup.
What is the multiple comparisons problem and how to correct?
This tests your grasp of family-wise error inflation across many tests. A strong answer defines the problem, contrasts per-comparison and family-wise error, and names corrections like Bonferroni or FDR.
Explain statistical power, MDE, and sample size impact
TESTS: This checks whether you grasp the power-MDE-sample-size trade-off. OUTLINE: Power is the chance of detecting a true effect; MDE is the smallest lift worth measuring; raising power or shrinking MDE inflates sample size.
What is a p-value in A/B tests and what does threshold mean?
WHAT IT TESTS: your grasp of frequentist testing mechanics. ANSWER OUTLINE: p-value is the chance of this or more extreme result if the null is true; the 5% threshold is the accepted false-positive rate. RED FLAG: saying it is the probability the variant wins.
How do you set up a button color A/B test?
Tests whether you can define a valid experiment, not just a color change. A strong answer covers hypothesis, randomization, primary metric, sample size, and significance threshold.
WAU is flat despite positive A/B tests; why and how to diagnose
This tests distinguishing real impact from statistical artifacts. Strong answers cite false positives from low base rates, peeking, novelty, and local-global mismatches. Diagnose with long-term holdouts, audits, and causal bridges.
How would you structure your growth team's experimentation portfolio?
WHAT IT TESTS: Capital allocation across risk classes in growth. ANSWER OUTLINE: 3 asset classes (iterative 30-70%, tech investments, big bets 20-40%), use expected value per week, and evolve the mix. RED FLAG: Gut voting or flat effort without expected ROI.