Why consistent user IDs matter in experiments

What's really being asked

This checks whether you understand identity as the foundation of both assignment and measurement in experimentation. The interviewer wants the two roles the id plays and a practical grasp of stitching anonymous and authenticated identities.

The full answer

The user id serves two purposes. First, assignment: the platform hashes the id with the experiment key to deterministically bucket the user into a variant, so the same id always resolves to the same experience without storing a giant assignment table. Second, attribution: all the user's events carry the id, so conversions can be tied back to the variant they actually saw. Consistency across devices and sessions is essential because if the same person gets a different id on their phone versus laptop, they may be assigned different variants, see a flickering experience, and be counted as two users, biasing results and breaking the deterministic guarantee. Common strategies: when a user is authenticated, use the durable account id everywhere. When anonymous, generate and persist an anonymous id in local storage or a cookie so it survives sessions on that device. On login or signup, stitch the prior anonymous id to the account id, merging the histories so pre-login behavior is attributed correctly and assignment stays stable.

The mistakes people make

Using a session id, so a returning user is treated as new and may flip variants. Using only a device id, so cross-device users are split. Never stitching anonymous to authenticated, losing pre-login events and causing reassignment at login. Believing random per-request assignment is fine, which destroys consistency.

What usually comes next

What happens at login if the anonymous user was in variant A but the account is in variant B? Define a deterministic precedence and stitch carefully. How do you handle logged-out users who later create accounts? How do privacy rules constrain cross-device identity?

A concrete example

A shopper browses anonymously on mobile with a persisted anonymous id and is bucketed into the new-checkout variant. They later log in on a laptop. The platform stitches the anonymous id to their account id, so the laptop session resolves to the same variant, their earlier browsing events attach to the account, and they count as one user, keeping the experiment clean.