tezvyn:

Build a simple A/B test for a headline

AI-drafted, machine-checkedSource: interviewbeginner
WHAT IT TESTS

basic A/B test mechanics.

OUTLINE

assign each visitor a sticky bucket, serve the matching headline variant, log impressions and conversions per variant.

RED FLAG

re-randomizing on every load so a user sees both variants.

WHAT THIS TESTS: It checks that you know the essential client and server pieces of an A/B test and can explain why each one exists rather than just naming a tool.

A GOOD ANSWER COVERS: Three parts. First, assignment: when a visitor arrives, randomly bucket them into A or B and persist that choice, typically in a cookie or tied to a user id, so the same person consistently sees the same headline on return visits, which is essential for valid measurement. Second, serving: render the assigned variant. Doing it server-side avoids a flash of the wrong headline, while doing it client-side is simpler but can flicker noticeably. Third, tracking: log an impression event recording the variant shown and a conversion event when the user takes the goal action, sending these to an analytics or events backend. With per-variant impression and conversion counts you compute each variant's conversion rate and run a significance test before declaring a winner.

COMMON WRONG ANSWERS: Re-randomizing on every page load, so a user sees A then B and the data is polluted. Not persisting assignment at all. Forgetting to log impressions, which leaves no denominator for the conversion rate. Calling a winner from a small sample without any statistical check.

LIKELY FOLLOW-UPS: How do you ensure sticky assignment survives across sessions and devices. How do you avoid the flicker problem with client-side rendering. What sample size or significance level do you need before deciding. How do you split traffic fifty-fifty reliably and without bias.

ONE CONCRETE EXAMPLE: A new visitor with no cookie is randomly assigned variant B, stored in a cookie; the server renders headline B; the client logs an impression for B; later the user signs up and a conversion event for B is logged. Aggregating across users, B converts at six percent versus four percent for A, and a significance test confirms whether that lift is real before you roll variant B out to everyone.

Read the original → optimizely.com

Get five bites like this every day.

Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.