Build a simple A/B test for a headline
basic A/B test mechanics.
assign each visitor a sticky bucket, serve the matching headline variant, log impressions and conversions per variant.
re-randomizing on every load so a user sees both variants.
WHAT THIS TESTS: It checks that you know the essential client and server pieces of an A/B test and can explain why each one exists rather than just naming a tool.
A GOOD ANSWER COVERS: Three parts. First, assignment: when a visitor arrives, randomly bucket them into A or B and persist that choice, typically in a cookie or tied to a user id, so the same person consistently sees the same headline on return visits, which is essential for valid measurement. Second, serving: render the assigned variant. Doing it server-side avoids a flash of the wrong headline, while doing it client-side is simpler but can flicker noticeably. Third, tracking: log an impression event recording the variant shown and a conversion event when the user takes the goal action, sending these to an analytics or events backend. With per-variant impression and conversion counts you compute each variant's conversion rate and run a significance test before declaring a winner.
COMMON WRONG ANSWERS: Re-randomizing on every page load, so a user sees A then B and the data is polluted. Not persisting assignment at all. Forgetting to log impressions, which leaves no denominator for the conversion rate. Calling a winner from a small sample without any statistical check.
LIKELY FOLLOW-UPS: How do you ensure sticky assignment survives across sessions and devices. How do you avoid the flicker problem with client-side rendering. What sample size or significance level do you need before deciding. How do you split traffic fifty-fifty reliably and without bias.
ONE CONCRETE EXAMPLE: A new visitor with no cookie is randomly assigned variant B, stored in a cookie; the server renders headline B; the client logs an impression for B; later the user signs up and a conversion event for B is logged. Aggregating across users, B converts at six percent versus four percent for A, and a significance test confirms whether that lift is real before you roll variant B out to everyone.
Read the original → optimizely.com
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.