User Bucketing: The Engine of A/B Testing

Why it exists

To run a valid A/B test, you must divide your users into groups and show each group a different variation of a feature. Crucially, each user must consistently see the same variation on every visit. User bucketing is the mechanism that solves this problem of consistent, random assignment.

The mental model

Think of user bucketing as a deterministic hashing function. It takes a stable user identifier and an experiment identifier as inputs and always produces the same output: the specific variation (A, B, or C) that user should see. It's a lottery where you know the outcome in advance for any given player.

How it works

A common method is to combine a stable user ID with an experiment ID or a salt. This combined string is then fed into a non-cryptographic hash function like MurmurHash. The output is a number which is then mapped into a predefined range, typically 0-9999. This range is then partitioned according to your experiment's traffic allocation. For a 50/50 A/B test, users whose hash falls in the 0-4999 range get variation A, while users in the 5000-9999 range get variation B. This process is deterministic, fast, and requires no state storage.

When to use it

User bucketing is the core mechanism for any form of experimentation or controlled rollout. Use it for A/B tests, multivariate tests, multi-armed bandits, and phased feature flag rollouts. It's fundamental to measuring the impact of changes on user behavior.

When not to use it

Avoid this for changes that are not experiments, such as critical bug fixes that must go to 100% of users immediately. The concept is also less relevant for purely aesthetic changes where you don't intend to measure user impact, though it's still best practice to use a feature flag system.

One canonical example

An e-commerce site wants to test a new checkout flow for 20% of users. A user with ID 'user-xyz-123' logs in. The system hashes the string 'user-xyz-123' plus the experiment ID 'new-checkout-flow-v2'. The result maps to the number 8134 in the 0-9999 range. Since the test is for 20% of users (e.g., range 8000-9999), this user is bucketed into the new checkout flow. Every subsequent visit, the same hash is calculated, and the user consistently sees the new flow, ensuring valid data for the experiment.