Proxy Metrics: Estimate Long-Term Impact Now

Why it exists

The most important business metrics, like annual retention or customer lifetime value, take a long time to measure. This creates a conflict: you need to move fast and ship features, but you also need to ensure those features positively impact long-term goals. Waiting a year to validate every change is not an option.

The mental model

A proxy metric is like a weather forecast for your key performance indicators. Instead of waiting three months to see if it actually rained (the true metric), you use a model that analyzes current atmospheric data (early user behavior) to predict the probability of rain (the long-term outcome). The forecast isn't reality, but it's a fast, actionable estimate that's better than guessing.

How it works

You build a machine learning model that predicts a long-term outcome using short-term data. For example, the model might take a user's activity in their first 7 days to predict their 90-day retention. In an A/B test, you feed data for each user into this model, and the model's output—the predicted retention—becomes your proxy metric. Critically, your statistical analysis must account for the model's prediction error (its Mean Squared Error, or MSE) to generate accurate confidence intervals and p-values.

When to use it

Use a proxy metric in experimentation when your primary success metric materializes too slowly for a typical testing window. It's ideal for connecting short-term product changes to long-term value, allowing you to optimize for goals like annual revenue, user churn, or lifetime value without multi-month feedback loops.

When not to use it

Do not use a proxy metric if you cannot build a high-quality, unbiased predictive model. If the model's inputs are influenced by experiment assignment, it suffers from data leakage, or it's systematically biased, your results will be misleading. Never rely on a proxy metric alone; always measure directly observable mechanism metrics alongside it.

One canonical example

An e-commerce company wants to test a feature to increase customer lifetime value (LTV). Waiting 12-24 months to measure true LTV is impractical. Instead, they use a proxy metric: a model that predicts LTV based on a user's actions in their first 14 days (e.g., purchase frequency, cart size, categories browsed). This allows them to run a two-week experiment and get a statistically sound estimate of the feature's impact on long-term LTV.