More in Analytics & Metrics — page 23
Net Promoter Score (NPS): A Snapshot of Customer Loyalty
NPS distills customer loyalty into one number by asking, "How likely are you to recommend us?" It's a quick pulse check for product teams. The footgun is treating the score as a diagnosis; it tells you *what* customers feel, but not *why*.
Customer Acquisition Cost: What It Costs to Win a Customer
Customer Acquisition Cost (CAC) is the total price you pay in sales and marketing to get one new customer. Businesses use it to see if their model is viable by comparing it to customer lifetime value (LTV). The footgun is forgetting to include all costs.
Ensemble Forecasting: Predicting with a Crowd of Models
Instead of one 'best guess,' ensemble methods generate many forecasts to map the range of possibilities. This is crucial for complex systems like weather prediction, where a single model is misleadingly precise.
LSTMs: Giving Neural Networks a Long-Term Memory
LSTMs give neural networks a selective memory, letting them remember important information over long sequences. This is key for language translation or time-series forecasting where old context matters.
Granger Causality: Forecasting, Not Causing
Granger Causality tests if one time series can forecast another, not if it causes it. It's used in econometrics to see if money supply changes predict inflation. The footgun is the name itself: it only shows predictive power, not true cause-and-effect.
ARIMA: Forecasting by Modeling Autocorrelation
ARIMA models forecast a time series by learning its "memory"—how past values influence the next. It's used for forecasting sales or server load where patterns are driven by internal dynamics.
Prophet: Automated Time Series Forecasting
Prophet treats forecasting like decomposing a signal, breaking a time series into trend, seasonality, and holiday effects. It excels at predicting business metrics with strong seasonal patterns, like web traffic.
Time Series Cross-Validation
Unlike standard cross-validation that shuffles data, time series cross-validation respects the arrow of time. It evaluates a forecasting model by training only on past data to predict a future point, then rolling forward. The footgun is using random k-folds.
ACF vs. PACF: A Time Series Signal and Its Echo
Autocorrelation (ACF) measures a time series's total correlation with its past, including indirect echoes. Partial Autocorrelation (PACF) isolates the direct correlation by removing intermediate effects. They help select forecasting model parameters.
Holt-Winters Exponential Smoothing
Forecast time-series data by giving more weight to recent events. Holt-Winters smoothing is used to analyze metrics and make predictions by accounting for user-defined assumptions like seasonality.
Exponential Smoothing: Weighting Recent Data More Heavily
Exponential smoothing creates forecasts from a weighted average of past data, where weights decay exponentially for older points. It's used for quick, reliable predictions like inventory demand. The footgun: simple versions ignore trends and seasonality.
Stationarity: A Time Series's Stable Personality
A time series is stationary if its statistical personality, like its average and spread, remains constant over time. Many forecasting models require this stability to work correctly.
Moving Average: Smoothing Out the Noise in Data
A moving average smooths out noisy data by replacing each point with the average of its neighbors, revealing the underlying trend. It's used to track stock prices or server metrics without overreacting to short spikes.
Propensity Score Matching: Mimicking an A/B Test
Propensity Score Matching (PSM) mimics a randomized trial with observational data by finding a "statistical twin" for each subject. It's used to estimate a feature's impact when a true A/B test isn't possible. The footgun is assuming it removes all bias.
Instrumental Variables: Isolating True Cause and Effect
Instrumental Variables (IV) isolate true causality when a key variable is tainted by hidden factors. An IV is a "clean" substitute that influences your cause without directly touching your effect, letting you measure the true relationship, free from…
Regression Discontinuity Design (RDD)
RDD finds a natural experiment at a cutoff point, like a test score threshold for a scholarship. By comparing people just above and below the score, you can estimate the program's effect. The footgun is assuming this local effect applies to everyone.
Isolating Impact with Difference-in-Differences (DiD)
Difference-in-Differences (DiD) isolates an intervention's true effect by comparing a treatment group's change over time to a control group's. This reveals if a new feature truly boosted engagement, not just rode a general upward trend.
Multi-Armed Bandit: The Explore vs. Exploit Trade-off
A multi-armed bandit algorithm balances exploring new options with exploiting the current winner, like a gambler trying slot machines to find the best payout.
Sample Ratio Mismatch (SRM): When Your A/B Test Is Broken
Sample Ratio Mismatch (SRM) means your A/B test's traffic split is broken, violating random assignment. For example, a 50/50 split results in a statistically significant imbalance.
The Novelty Effect: When New Isn't Always Better
The Novelty Effect is a temporary metric spike from a feature's newness, not its inherent value. It often appears in A/B tests for high-frequency products, inflating short-term metrics. The footgun is mistaking this initial excitement for a long-term win.