How would you structure your growth team's experimentation portfolio?

WHAT THIS TESTS: This question tests whether you view engineering time as a finite capital pool that must be allocated across investments with different risk and return profiles. The interviewer wants to see portfolio thinking rather than a flat backlog of ideas. They are looking for familiarity with expected value calculations, the ability to balance short term output against long term optionality, and the judgment to shift that balance as the product and market mature.

A GOOD ANSWER COVERS: A strong response names the three asset classes from growth portfolio theory and gives their typical success rates. First, iterative experiments in established areas usually win 30-70% of the time and provide steady returns. Second, investments in tech or user experience do not always move metrics immediately but increase future velocity or protect brand reputation. Third, big bets such as new channels or onboarding overhauls succeed only 20-40% of the time yet can transform the trajectory of the business. The answer should then describe a rigorous selection process that replaces democratic brainstorming votes with an expected value formula per engineering week, specifically probability of success multiplied by incremental impact divided by engineering cost. It should also note that a small gain at the top of a core funnel usually beats fixing a broken edge flow, and that the portfolio mix should evolve over time as the product matures and the highest leverage iterative opportunities are exhausted.

COMMON WRONG ANSWERS: Candidates often say they would let the team vote on ideas or prioritize whatever feels most exciting, which ignores opportunity cost and confuses activity with impact. Another red flag is treating all experiments as equal without distinguishing between iterative optimizations and strategic bets. Some candidates also forget tech and UX investments entirely, focusing only on experiments that directly move a metric this quarter and leaving the team slower or the brand degraded over time.

LIKELY FOLLOW-UPS: An interviewer might ask how you would estimate the probability of success for a project that has no historical precedent. They could also ask what percentage of the portfolio you would allocate to big bets in year one versus year four, or how you would defend a UX investment that slightly hurts a growth metric. Be ready to discuss how you would build a tooling investment that compounds experiment velocity.

ONE CONCRETE EXAMPLE: Suppose your team wants to improve email reactivation. An iterative experiment might test a new subject line that you expect to lift open rates by 5%; with a 60% chance of success and one week of work, the expected value is high. A tech investment might be building automated copy localization so future email experiments launch in ten markets instead of one; it has no immediate user gain but doubles experiment surface area for the next year. A big bet might be replacing the entire email preference center to create a new personalized send-time feature; it takes six weeks and has a 30% chance of a 15% lift, but if it works it opens a durable new growth loop. You would score all three with the same expected value per week metric and fund the highest ROI mix.

Source: jwegan.com