Build an opportunity-sizing model before building
quantifying upside before investing.
locate the affected funnel step, estimate addressable population times a bounded conversion lift times value per user, then sanity-check against a realistic ceiling.
WHAT THIS TESTS This probes whether you can decide if an idea is worth building before building it. Opportunity sizing is a back-of-envelope model that bounds upside; the interviewer wants a structured estimation using real funnel data, explicit assumptions, and a sense of ceilings rather than hand-waving.
A GOOD ANSWER COVERS First, name the top-line metric and the specific funnel step the idea influences. Pull current data: how many users reach that step per period, the current conversion or retention rate there, and the value of a converted or retained user. The addressable population is users who hit the affected step, not the whole base. Estimate a plausible improvement to the step's rate, anchored to evidence such as prior experiments or industry benchmarks, and explicitly bounded; a change almost never lifts conversion to one hundred percent. Compute incremental impact as addressable population times the rate improvement times value per user. Express the result as a range, an optimistic and a conservative scenario, with the assumptions listed. Cross-check against the theoretical ceiling: even a perfect fix cannot exceed the gap between current and ideal conversion for that segment.
COMMON WRONG ANSWERS Assuming the idea perfectly fixes the problem for every user. Applying the lift to the entire user base instead of the affected segment. Producing a single confident number with no range or stated assumptions. Ignoring downstream effects, such as a step lift that does not flow through to revenue. Using a benchmark from an unrelated context.
LIKELY FOLLOW-UPS How do you choose the lift assumption? Triangulate prior tests and benchmarks, and show sensitivity. How do you account for cannibalization or novelty effects? Discount the optimistic case. When is the opportunity too small to pursue? When even the optimistic case is below the build cost threshold.
ONE CONCRETE EXAMPLE A team considers redesigning a payment step where 200k users per month reach it and 70 percent convert. The maximum headroom is the 30 percent who fail. Benchmarks and a prior test suggest a redesign might recover a quarter of those, so roughly 15k extra conversions monthly. At an average order value, that yields a revenue range with conservative and optimistic bounds, which the team weighs against the engineering cost before committing.
Read the original → shopify.engineering
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.