Design a centralized experimentation service

WHAT THIS TESTS This evaluates platform design: separating the control plane that defines experiments from the data plane that assigns and logs, and keeping bucketing identical across heterogeneous clients.

A GOOD ANSWER COVERS Three planes. A control plane with a config API where teams define experiments, variants, traffic allocations, and targeting rules, persisted in a versioned store. A client SDK shared across web, iOS, and Android that fetches that config and performs deterministic local bucketing: the same hashing algorithm and salt everywhere so a given identity buckets identically regardless of platform. Identity resolution must be explicit, mapping anonymous IDs to authenticated IDs and choosing the unit of randomization. A separate exposure-logging pipeline streams events asynchronously to a warehouse for analysis, decoupled from assignment so logging failures never block rendering. APIs: getAssignment local, getConfig over network with caching, and logExposure fire and forget.

COMMON WRONG ANSWERS A synchronous assignment endpoint hit on every page render, adding latency and a hard dependency. Different hash functions per platform, so the same user splits across variants. Mixing exposure logging into the assignment call, coupling availability. No identity stitching, so a user double counts before and after login.

LIKELY FOLLOW-UPS How do you guarantee identical bucketing across SDK versions? How do you handle mutually exclusive experiments and layers? How do you avoid sample ratio mismatch from caching stale config?

ONE CONCRETE EXAMPLE A team defines checkout-flow-v2 with 10 percent treatment. The config syncs to all SDKs. On iOS the SDK hashes the user ID with the layer salt, lands in treatment, renders the new flow, and asynchronously emits an exposure event tagged with experiment, variant, and SDK version. The same user on web computes the identical bucket and sees the identical flow.