Why A/B-only, no upfront research, costs engineers more

WHAT THIS TESTS The interviewer wants a nuanced argument that respects A/B testing while showing why it cannot replace upfront research, framed in costs an engineering org cares about.

A GOOD ANSWER COVERS Make the core distinction: A/B testing is an optimization tool that compares variants of something you have already decided to build, answering which version performs better within a chosen direction. It cannot tell you whether that direction is right, whether the feature solves a real user problem, or what to build in the first place, because it can only test what already exists. Skipping upfront generative research therefore risks building the wrong thing well. The engineering cost is concrete: full implementation effort spent on a feature that fails, then expensive iteration cycles or a complete teardown, plus the tech debt and maintenance burden of abandoned code paths and feature flags. There is also a validity cost: A/B tests need sufficient traffic and run time to reach significance, so for low-traffic features or big bets you may never get a trustworthy signal, and you cannot A/B test a direction you never considered. Upfront research is comparatively cheap insurance that de-risks the expensive build phase.

COMMON WRONG ANSWERS Dismissing A/B testing as useless, which is wrong and makes you look dogmatic. Arguing only on user-empathy grounds without the cost-of-rework angle the PM will feel. Ignoring that A/B testing has statistical prerequisites. Implying research replaces experimentation rather than complementing it.

LIKELY FOLLOW-UPS When is A/B-first actually fine. How much upfront research is enough. How do generative research and experimentation work together. How do you quantify the cost of building the wrong thing.

ONE CONCRETE EXAMPLE A team skips research and A/B tests two checkout button colors, optimizing a flow users fundamentally distrust because of a hidden fee. The test improves clicks marginally while the real problem, surfaced trivially by five interviews, goes unaddressed, and engineering later rebuilds the whole fee-disclosure flow, far more costly than the upfront study would have been.