Design an assignment service for an experimentation platform

What's really being asked

This question evaluates whether you understand the mechanics of trustworthy online experimentation at scale. The interviewer cares about three things: deterministic assignment so the same user always sees the same variant, uniform randomness so treatment effects are unbiased, and independence between traffic allocation and bucketing so you do not accidentally introduce sample ratio mismatch. It also checks if you know what data must flow into the service to make these guarantees possible.

The full answer

First, the data model. The service needs a stable user identifier, an experiment salt that is unique per experiment, the target exposure rate, and the variant weights. Second, consistency. A senior candidate explains that you hash the concatenation of the user ID, experiment salt, and a fixed suffix to produce a deterministic number. Because the inputs never change, the user always lands in the same bucket across devices and sessions. Third, randomness and independence. You must use a high-quality hash function and avoid modulo bias. The Wish case shows that a two-step FNV hash with modulo 100 created a hidden dependency between the exposure draw and the bucket draw, producing periodic non-uniformity and SRM. A better approach is to use a single high-quality hash output and partition the space, or use a cryptographic hash like SHA-256 and map it with integer division over the full 64-bit or 128-bit range rather than naive modulo. Fourth, validation. Mention monitoring for sample ratio mismatch as a guardrail.

The mistakes people make

A common red flag is proposing a random number generator without anchoring to the user identity; this breaks consistency across sessions and devices. Another is suggesting two independent hash calls with modulo to decide exposure and then bucket without recognizing that modulo can couple the distributions. Candidates who ignore the experiment salt also fail because reusing salts across experiments causes correlated assignments. Finally, forgetting to mention SRM detection signals a lack of production experience with experimentation platforms.

What usually comes next

The interviewer may ask how you would handle mutually exclusive experiments, how to support device-level versus user-level bucketing, or how to roll out an experiment gradually without reassigning existing users. They might also dig into how you would validate uniformity at scale, or what you would do if an SRM alarm fires after launch.

A concrete example

Suppose you have an experiment with a 10 percent exposure rate and two variants split 50-50. You concatenate user_id, experiment_salt, and a constant string, then compute a SHA-256 hash. You interpret the first 64 bits as an integer in the range 0 to 2^64 minus 1. If the integer is less than 0.1 times 2^64, the user is exposed. Within that exposed set, you use the next 64 bits to assign control or treatment based on whether the value is below or above 0.05 times 2^64. Because the hash space is uniform and you avoid modulo, the exposure decision and the bucket decision remain independent.