How would you track which headline wins in an A/B test?

WHAT THIS TESTS: The interviewer wants to see if you can trace a single user action through a distributed system. They care that you understand variant assignment, rendering mechanics, event instrumentation, and data integrity. This is not about naming a tool like Optimizely or Google Optimize; it is about whether you can explain how the control and treatment are served and how causation is preserved from impression to click.

A GOOD ANSWER COVERS four layers in order. First, assignment: explain that users must be bucketed deterministically, typically by hashing a stable identifier like a user ID or first-party cookie so the same user always sees the same variant and sessions do not cross-contaminate. Second, serving: describe server-side rendering where the application backend embeds the chosen headline directly in the HTML, or acknowledge client-side rendering where a JavaScript snippet modifies the DOM after load. If you mention the client-side path, you must note the flicker effect described in the canonical reference, where the original headline renders for 200 to 500 milliseconds before the swap, potentially skewing behavior. Third, instrumentation: the click event must carry metadata including the experiment ID, variant ID, and a timestamp, fired to an analytics collector via a beacon or XHR request that is buffered against page unloads. Fourth, analysis: raw events are deduplicated and aggregated in a warehouse to compute click-through rate per variant, with a mandatory sanity check that the assignment split matches the traffic allocation, for example verifying that a 50-50 test actually delivers within a few percentage points of half the impressions to each arm.

COMMON WRONG ANSWERS include suggesting that A/B testing is just a frontend CSS change with no persistent user bucketing, which destroys session consistency. Another red flag is omitting the network path for click events, as if the browser magically knows which variant won. Candidates who propose client-side visual editors without discussing anti-flicker techniques or the performance cost of hiding the page until the script executes also signal shallow experience. Finally, ignoring sample ratio mismatch, where one variant accidentally receives 60 percent of traffic due to a bug, shows weak experimental rigor.

LIKELY FOLLOW-UPS: How would you handle bot traffic diluting the results? What happens if the analytics request fails before the user navigates away? How do you prevent a headline test from hurting SEO if the variants are served at the same URL? When is it worth moving from client-side to server-side assignment for a content site?

ONE CONCRETE EXAMPLE: Imagine a news article at slash-article-slash-123. The server receives a request and checks a cookie named headline_exp_42. If the cookie is absent, the edge worker hashes the anonymous user ID modulo 2 to pick variant A or B, sets the cookie with a 30-day TTL, and returns HTML containing the assigned headline. The page includes a data attribute on the headline link recording the variant. When the user clicks, a JavaScript handler sends a POST to slash-events with the article ID, variant, and a click timestamp. A nightly batch job counts clicks and impressions per variant, filters out internal IP ranges, and checks that the impression split is 49-51 rather than 40-60 before declaring a winner.