Describe two methods for generating prediction intervals
This tests your grasp of uncertainty quantification. A great answer contrasts an analytical method (assuming normal errors, using multipliers like 1.96 for 95%) with a simulation method (bootstrapping residuals).
What's really being asked
This question assesses your ability to move beyond simple point forecasts and quantify uncertainty, a critical skill for senior roles where decisions are risk-adjusted. It tests if you understand that different methods for generating prediction intervals rely on different assumptions (e.g., normality of residuals) and have different computational costs. The interviewer is looking for a nuanced understanding of why a simple +/- 10% is naive and how to do better.
The full answer
A strong answer describes two distinct approaches. First, the analytical or parametric method. This involves assuming the forecast errors are uncorrelated and follow a specific distribution, typically a Normal distribution. The interval is calculated as the point forecast plus or minus a multiplier times the estimated standard deviation of the forecast error. For a 95% interval, this multiplier is 1.96. Second, a simulation-based or non-parametric method like bootstrapping. This method does not assume a specific distribution for the errors. Instead, it involves repeatedly resampling from the set of historical residuals and adding them to the point forecasts to generate many possible future sample paths. The prediction interval is then determined by taking the percentiles (e.g., the 2.5th and 97.5th percentiles for a 95% interval) from the distribution of these simulated outcomes.
The mistakes people make
A major red flag is confusing prediction intervals with confidence intervals. A prediction interval is a range for a future observation, while a confidence interval is a range for an unobservable model parameter (like a regression coefficient). Prediction intervals are therefore always wider than confidence intervals. Another weak answer is only describing the analytical method without being able to offer an alternative, or vaguely mentioning "adding some buffer" without statistical justification. Simply stating a fixed percentage (e.g., "we use a +/- 10% band") without tying it to the model's error variance is a junior-level answer.
What usually comes next
"When would you choose bootstrapping over the analytical method?" (Answer: When residuals are not normally distributed, or when the model is complex and an analytical formula for the forecast variance is intractable). "How do the assumptions of these two methods differ?" (Answer: The analytical method assumes a specific error distribution, while bootstrapping only assumes the historical residuals are representative of future errors). "How does the forecast horizon h affect the width of the prediction interval?" (Answer: The interval gets wider as h increases because uncertainty accumulates over time).
A concrete example
For a model forecasting sales, let's say our point forecast for next month is 10,000 units. Using the analytical method, if the standard deviation of our 1-step-ahead forecast errors is 500 units, a 95% prediction interval would be 10,000 ± 1.96 * 500, which is [9,020, 10,980]. Using bootstrapping, we would take our historical residuals, simulate 10,000 possible future outcomes, and find the 2.5th percentile (e.g., 8,950) and the 97.5th percentile (e.g., 11,050) of those simulations to form our interval. The bootstrapping interval might be asymmetric if the error distribution is skewed.
Interview question
A model's residuals are found to be heavily skewed. Which method is most appropriate for generating a robust prediction interval for its forecasts?
- a.Calculating a confidence interval for the model's coefficients to determine the range of the forecast.
- b.The analytical method, using a standard multiplier like 1.96 on the error standard deviation.
- c.Applying a fixed percentage buffer, like +/-10%, determined from the model's historical mean absolute error.
- d.Bootstrapping the residuals to simulate future outcomes, as this does not assume a normal error distribution.Correct
Why? this is the answer
The analytical method assumes normally distributed errors, which is violated when residuals are skewed. Bootstrapping is a non-parametric method that does not require this assumption, making it more robust.
Just read this? Test yourself on what you have been reading.
Read the original → otexts.com
- #forecasting
- #statistics
- #uncertainty
- #modeling
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on forecasting — each one lists the topics its interview covers.
See open roles