Statistics
196 bites tagged Statistics — interview questions with model answers, and 60-second explainers.
Probability Distributions: Mapping Odds to Outcomes
A probability distribution is a map of all possible outcomes and their chances. It's used to model everything from coin flips to customer churn. The footgun is assuming a simple bell curve when reality is often skewed or unpredictable.
Causal Inference: Proving Cause, Not Just Correlation
Causal inference goes beyond correlation to ask "did X *cause* Y?". It's used to prove a new feature drove engagement or a drug improved outcomes. The main footgun is mistaking association for causation, ignoring confounding variables.
Sequential Testing: Stop When You Have Enough Data
Instead of running a test for a fixed duration, sequential testing lets you peek at data and stop as soon as you have a statistically significant winner. It's used in A/B testing to ship features faster.
Multi-Armed Bandits: Balancing Exploration and Exploitation
A multi-armed bandit algorithm balances exploiting a known winner with exploring new options to find a better one. It's used to dynamically test website headlines or ad copy, automatically shifting traffic to the best performer to maximize conversions.
Multivariate Testing (MVT): Finding the Best Combination
Multivariate testing (MVT) finds the best combination of elements, not just the best single version. It tests multiple headlines, images, and buttons at once to see how they interact. The main footgun is needing massive traffic for statistically valid results.
A/B Testing: Calculating Your Sample Size
Calculating A/B test sample size ensures your results are statistically significant, not just random noise. It's vital before testing anything from button colors to headlines. The biggest footgun is stopping a test early when a variant looks like it's winning.
Control vs. Variation: The Core of A/B Testing
To know if a change works, compare it against the original. The original is your 'control,' and the new version is your 'variation.' This is the core of A/B testing, used to test new button colors or email subjects.
ROC Curve and AUC: Measuring Classifier Performance
An ROC curve visualizes a classifier's trade-off between catching true positives and flagging false ones across all thresholds. It's used to evaluate models like medical diagnostics or spam filters.
Normalizing Flows: Shaping Simple Distributions into Complex Ones
Normalizing flows transform a simple probability distribution, like a Gaussian, into a complex one, like images. This allows generative models to explicitly calculate the probability of any data point. The footgun is that the transformation must be invertible.
Marketing Mix Modeling (MMM): Isolating Marketing's Impact on Sales
Marketing Mix Modeling (MMM) statistically links aggregate marketing efforts to sales outcomes over time. It's used to determine the ROI of past campaigns, like a TV ad blitz. The main footgun: the model is only as good as the historical data you feed it.
Ensemble Forecasting: Predicting with a Crowd of Models
Instead of one 'best guess,' ensemble methods generate many forecasts to map the range of possibilities. This is crucial for complex systems like weather prediction, where a single model is misleadingly precise.
Granger Causality: Forecasting, Not Causing
Granger Causality tests if one time series can forecast another, not if it causes it. It's used in econometrics to see if money supply changes predict inflation. The footgun is the name itself: it only shows predictive power, not true cause-and-effect.
ARIMA: Forecasting by Modeling Autocorrelation
ARIMA models forecast a time series by learning its "memory"—how past values influence the next. It's used for forecasting sales or server load where patterns are driven by internal dynamics.
ACF vs. PACF: A Time Series Signal and Its Echo
Autocorrelation (ACF) measures a time series's total correlation with its past, including indirect echoes. Partial Autocorrelation (PACF) isolates the direct correlation by removing intermediate effects. They help select forecasting model parameters.
Holt-Winters Exponential Smoothing
Forecast time-series data by giving more weight to recent events. Holt-Winters smoothing is used to analyze metrics and make predictions by accounting for user-defined assumptions like seasonality.
Stationarity: A Time Series's Stable Personality
A time series is stationary if its statistical personality, like its average and spread, remains constant over time. Many forecasting models require this stability to work correctly.
Moving Average: Smoothing Out the Noise in Data
A moving average smooths out noisy data by replacing each point with the average of its neighbors, revealing the underlying trend. It's used to track stock prices or server metrics without overreacting to short spikes.
Propensity Score Matching: Mimicking an A/B Test
Propensity Score Matching (PSM) mimics a randomized trial with observational data by finding a "statistical twin" for each subject. It's used to estimate a feature's impact when a true A/B test isn't possible. The footgun is assuming it removes all bias.
Instrumental Variables: Isolating True Cause and Effect
Instrumental Variables (IV) isolate true causality when a key variable is tainted by hidden factors. An IV is a "clean" substitute that influences your cause without directly touching your effect, letting you measure the true relationship, free from…
Regression Discontinuity Design (RDD)
RDD finds a natural experiment at a cutoff point, like a test score threshold for a scholarship. By comparing people just above and below the score, you can estimate the program's effect. The footgun is assuming this local effect applies to everyone.
Isolating Impact with Difference-in-Differences (DiD)
Difference-in-Differences (DiD) isolates an intervention's true effect by comparing a treatment group's change over time to a control group's. This reveals if a new feature truly boosted engagement, not just rode a general upward trend.
Multi-Armed Bandit: The Explore vs. Exploit Trade-off
A multi-armed bandit algorithm balances exploring new options with exploiting the current winner, like a gambler trying slot machines to find the best payout.
Sample Ratio Mismatch (SRM): When Your A/B Test Is Broken
Sample Ratio Mismatch (SRM) means your A/B test's traffic split is broken, violating random assignment. For example, a 50/50 split results in a statistically significant imbalance.
Average Treatment Effect (ATE): Isolating the Impact of a Change
The Average Treatment Effect (ATE) isolates an intervention's true impact by comparing the average outcome of a treated group to a control group. It's used in A/B tests and policy evaluations. The footgun is assuming causation without true randomization.
Get Statistics bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.