Growth &amp; Experimentation

How would you adapt a growth model for network effects and k-factor?

Define K as invites x conversion; K over 1.0 explodes, yet K over 0.7 with fast cycle time still compounds; anchor at peak delight.

Describe a strategy for reconciling different forecasts into one robust prediction

Tests synthesis of heterogeneous models into a consensus forecast. Strong answers diagnose divergence drivers first, then weight by track record or uncertainty, and output a distribution. Red flag: blind averaging without understanding why models disagree.

Trace an event from click to analysis

Client SDK captures and batches, a collection endpoint ingests, a stream and ETL enrich and load into a warehouse for analysis.

How would you implement a simple feature flag system?

Tests runtime config separation and distributed hygiene. Outline: control service, datastore, API, SDK, and update mechanism; local evaluation with cached state; short-lived flags. Red flag: static config files requiring redeployment are not feature flags.

Why consistent user IDs matter in experiments

The user id seeds deterministic bucketing and ties events to one person across devices; achieve it via authenticated ids and anonymous-to-known stitching.

Design an assignment service for an experimentation platform

This tests deterministic hashing and independence in randomization. A strong answer lists user ID, experiment salt, and allocations; uses salted hash for consistency; and warns that separate exposure and bucket hashes with mod can cause sample ratio mismatch.

Should we build or buy an A/B testing platform?

This tests build-vs-buy judgment for experimentation infrastructure. Strong answers cover build for warehouse metrics and cache control; buy for proven stats and front-end speed. Red flag: answering with cost alone or assuming in-house is always superior.

Compare PostgreSQL versus a columnar warehouse for raw event data

Contrast row vs column storage, compression, and scan speed; note Postgres suits OLTP and point lookups while columnar stores excel at aggregations.

Design a system to reduce large client-side experiment payload size

Tests edge evaluation and payload compression. Use server-side pre-evaluation or edge nodes sending only assigned variants; compact bucketing indexes or Bloom filters; lazy-load noncritical experiments. Never do full client-side evaluation of every flag rule.

Design a pre-aggregation architecture for low-latency experiment results

Tests OLAP-at-scale trade-offs. Strong answers design streaming rollups into a real-time OLAP store, use partial cubes for high-cardinality dimensions, and retain raw events.

How do you mitigate peeking in experiment infrastructure?

Lock results behind minimum samples; auto-correct via sequential testing; hide early metrics and require stop approval.

Lifecycle of a feature flag experiment from creation to cleanup

Tests operational rigor across the full flag lifecycle. A strong answer covers six stages: SDK instrumentation with event tracking, phased rollout, monitored experiment, ship/kill decision, and code cleanup.

What experiment artifacts and metrics do you present to decide shipping?

This tests structured experiment discipline. A strong answer covers the pre-approved design doc, scorecard results for primary goal and guardrail metrics plus secondary breakdowns versus the hypothesis, and duration context.

How do you systematically manage and pay down experiment debt?

Tests sustainable velocity through experiment lifecycle hygiene. Strong answers cover isolated experiment directories, TTLs on feature flags, and recurring cleanup sprints. Red flag: banning experiments or treating all experiment code as permanent.

How do you prevent concurrent onboarding and navigation experiments from polluting results?

This tests experiment isolation via layer-based traffic allocation. A strong answer covers hashing users into independent layers with one variant per layer, and assigning each experiment to a distinct layer.

Compare server-side and client-side experimentation architectures

This tests rendering-layer architecture. A strong answer contrasts server-side zero-flicker and algorithm tests against client-side marketer agility and SEO safety, mapping each to release cycles. A red flag is claiming one approach dominates every dimension.

Describe cluster or switchback randomization for network-effect A/B tests.

Tests SUTVA violation, cluster-switchback tradeoffs. Outline: cluster (geo, teams) or switchback (time slices) isolation; cover pipeline changes, 10x inflation, and correlated error. Red flag: user-level randomization with post-hoc fixes or ignoring spillover.

Design a referral system: data models, APIs, attribution, self-referral prevention

Tests data modeling with fraud guardrails and idempotent rewards. Cover: Users with nullable referred_by, ReferralEvents state table, async ledger attribution, and device-fingerprint self-referral blocks. Red flag: bare integer credit with no audit trail.

Build a system to measure viral coefficient and attribute invites to signups

Tests if you can map K=i×c to logged events and resilient pipeline. Good answers define invite_sent, click, signup events with referral tokens; sketch stream joins; and flag cross-device and organic attribution gaps. Red flag: assuming perfect attribution.