Consistent A/B bucketing across sessions and devices

WHAT THIS TESTS The interviewer wants to see that you treat bucketing as a deterministic function of stable identity and that you anticipate how caching can poison experiments. It rewards reasoning about idempotency and cache keys, not vague mention of cookies.

A GOOD ANSWER COVERS Use a deterministic hash of a stable identifier and the experiment key, for example, hash of user ID plus experiment salt, then mod into buckets. Because hashing is pure, the same user always lands in the same variant on any device or session with no lookup required. For logged-in checkout, the account ID is the natural anchor, which solves cross-device consistency that a device cookie cannot. For anonymous users you anchor on a persistent ID and accept they may rebucket after login, so you reconcile at sign-in. On caching: any shared cache, CDN, or page cache must incorporate the variant into the cache key, otherwise a cached response built for variant A is served to a variant B user, corrupting both the experience and the metrics. Vary on the experiment dimension explicitly.

COMMON WRONG ANSWERS Assigning randomly on each request, which gives different variants per session. Storing assignment only in a device cookie, breaking cross-device consistency. Forgetting the cache key, leading to cross-variant contamination. Persisting every assignment in a database when a deterministic hash avoids that entirely.

LIKELY FOLLOW-UPS How do you handle a user who is anonymous then logs in mid-experiment. How do you ramp traffic from one percent to fifty without rebucketing existing users. How do you avoid hash collisions across overlapping experiments.

ONE CONCRETE EXAMPLE For the checkout test you compute bucket equals hash(userId plus 'checkout-2024') mod 100; buckets zero through forty-nine see the new flow. A logged-in user on phone and laptop hashes identically, so both devices show the new flow. Your CDN caches the checkout page, so you add the variant to the Vary or cache key; without it, the first cached render would be served to everyone, silently breaking the experiment and inflating the wrong variant's numbers.