How do you implement a CTA A/B test and attribute conversions?

WHAT THIS TESTS: This question evaluates whether you can design a rigorous technical experiment pipeline rather than just changing UI copy. Interviewers want to see that you understand randomization, exposure logging, and causal attribution. The CTA is critical, so the solution must avoid bias from flicker, network race conditions, or inconsistent bucketing across sessions.

A GOOD ANSWER COVERS: A good answer hits four things in order. First, deterministic bucketing: map each user to a variant using a stable identifier such as a user ID or device ID hashed with the experiment key, or use an experimentation platform like Optimizely or LaunchDarkly. This keeps the same user in the same bucket across sessions. Second, serving mechanism: render the correct copy server-side or via a synchronous feature flag evaluation before the DOM paints to prevent flicker or layout shift. Third, exposure event: fire an event when the variant is actually shown, logging experiment ID, variant ID, timestamp, and user ID. This creates the denominator for your conversion rate and lets you detect sample ratio mismatch. Fourth, attribution: attach the experiment and variant IDs as metadata on every downstream conversion event, or join conversion events to exposure events in your warehouse using user ID and timestamp windows. This closes the loop from impression to outcome.

COMMON WRONG ANSWERS: Common wrong answers include using a random number on every page load, which splits users unpredictably and breaks session consistency. Another red flag is relying solely on URL parameters without logging an exposure event, because users who do not see the CTA can still land on a tagged URL and pollute the funnel. Suggesting purely client-side DOM manipulation after page load is also weak because it introduces flicker and can miss users with ad blockers or slow networks.

LIKELY FOLLOW-UPS: Interviewers often push deeper with these follow-ups. How do you handle logged-out users? You might use a first-party cookie or device ID, but acknowledge the trade-off of cookie deletion. What if the conversion happens days later on another device? You might discuss identity resolution or intent-to-treat analysis. How do you know the result is statistically significant? You should mention powering the test for a minimum detectable effect and using a fixed horizon rather than peeking. How do you prevent interaction effects when multiple experiments run simultaneously? You might mention mutually exclusive layers or orthogonal hashing.

ONE CONCRETE EXAMPLE: Imagine an e-commerce checkout page with a CTA reading Buy Now versus Complete Purchase. When the user lands on the page, the edge or application server hashes their user ID with the experiment key exp_cta_2024_q3 and computes a bucket from zero to ninety-nine. Users zero through forty-nine see Buy Now; fifty through ninety-nine see Complete Purchase. The server injects the chosen string into the HTML before sending the response. On render, the frontend sends an exposure event to the analytics pipeline with experiment exp_cta_2024_q3 and variant buy_now. When the user clicks the button, the click event carries the same experiment and variant metadata. At the end of the week, an analyst joins exposure events to click events on user ID and confirms that Complete Purchase lifted conversions by four percent with a p-value below zero point zero five.