Framing ad-load tradeoffs: revenue versus retention

WHAT THIS TESTS The interviewer wants to see a rigorous framework for a classic tension, immediate revenue versus long-term user health, rather than a one-sided answer.

A GOOD ANSWER COVERS Start by defining metrics on both sides. The primary metric captures the intended gain, ad revenue per user. Equally important are guardrail or counter metrics that detect the feared harm: retention, session frequency, time spent, churn, and direct satisfaction signals. Then design an experiment that can actually observe long-term effects, since revenue lifts appear immediately but dissatisfaction shows up as slow churn. That means running the test long enough, and often keeping a long-term holdback group that never sees the extra ads so you can measure the delayed retention gap. Finally, put both effects in a common currency. Convert added revenue and any retention or engagement loss into expected lifetime value, so the decision compares net long-term value rather than just the upfront revenue bump.

COMMON WRONG ANSWERS Recommending the change because short-term revenue rises, ignoring delayed churn. Running the experiment too briefly to catch retention damage. Treating revenue and satisfaction as incomparable and deciding by gut. Looking only at average effects and missing harm concentrated in your most valuable users.

LIKELY FOLLOW-UPS How long should it run? Long enough to span the retention horizon where churn manifests, often weeks, with a holdback for ongoing measurement. How do you combine metrics? Use an overall evaluation criterion or LTV model rather than many disconnected metrics. How do you protect power users? Segment results and check whether high-value cohorts churn disproportionately. What about novelty or ad-blindness effects? Account for the change settling over time.

ONE CONCRETE EXAMPLE You A/B test the higher ad load with revenue per user as the primary metric and 30-day retention, sessions per week, and churn as guardrails, plus a 5 percent long-term holdback. After several weeks the treatment shows a 4 percent revenue lift but a measurable retention decline concentrated in heavy users. Translating both into lifetime value, the churn-driven loss exceeds the revenue gain, so the framework recommends against the rollout, or in favor of a gentler ad increase, a conclusion that pure short-term revenue would have hidden.