Multi-armed bandit vs A/B testing for headlines

WHAT THIS TESTS: It probes whether you understand adaptive experimentation architecture and how a bandit fundamentally differs from a fixed-horizon A/B test, not just in outcome but in mechanism.

A GOOD ANSWER COVERS: Architecturally, each headline is an arm with reward statistics, impressions and clicks, kept in a fast store like Redis. On each request a serving layer asks the bandit policy which arm to show; the policy balances exploit, favoring the arm with the best estimated click-through, against explore, occasionally showing others to keep learning. Common policies are epsilon-greedy, Upper Confidence Bound, and Thompson sampling. After the user acts, a feedback path updates that arm's reward state, so allocation continuously shifts toward the winners. You need low-latency reads of arm state, an asynchronous update pipeline, and careful handling of delayed or sparse rewards.

THE DIFFERENCE FROM A/B: A traditional A/B test fixes traffic at an even split for the entire run regardless of early results, then makes a single decision at a predetermined sample size, deliberately spending traffic on the losing variant to get clean statistics. A bandit instead reallocates during the test, reducing regret by sending most traffic to the leader while still exploring, so it earns more clicks during learning but yields messier classical statistics and can prematurely starve a slow-burn winner.

LIKELY FOLLOW-UPS: Explain how Thompson sampling actually picks an arm. How do you handle rewards that arrive with a delay. When would you still prefer a clean A/B test over a bandit. How do you avoid prematurely killing a good arm and cope with non-stationarity in click-through over time.

ONE CONCRETE EXAMPLE: Five headlines launch with equal exposure. Within hours Thompson sampling notices arm C's higher click-through and routes most impressions there while still sampling the others occasionally. If a fresh arm later improves, exploration catches it and traffic shifts again, whereas a traditional A/B test would have shown all five headlines equally right up until its fixed predetermined end date regardless of early signal.