Random split vs walk-forward validation in forecasting
Awareness of temporal leakage.
Random splits leak future data into training; walk-forward validation rolls the origin ahead, testing only on later observations.
Claiming random splits work for time-series.
What's really being asked
This question probes whether you understand that time-series data is not independent and identically distributed. In forecasting, the order of observations carries information; a standard random split severs that order and allows the model to peek at the future during training. Interviewers want to see that you treat temporal structure as a first-class constraint rather than an afterthought.
The full answer
First, the mechanism of a random split: it shuffles all observations and partitions them into train and test without regard to time, which means a model can learn patterns from 2024 and be tested on 2022. Second, the mechanism of walk-forward validation: you start with a minimum training set of early observations, forecast the very next period, record the error, then expand the training set by one observation and repeat. Third, the core principle of no future data in training: every training set must contain only observations that occurred prior to the test observation. Fourth, the practical output: you average forecast accuracy across all test periods to get a robust estimate of out-of-sample performance. Fifth, optional nuance about expanding versus sliding windows: an expanding window keeps all past data, while a sliding window uses a fixed history length, which matters for non-stationary series.
The mistakes people make
Claiming that cross-validation always requires shuffling, so random splits are fine for time series. Saying that walk-forward validation is only about getting more test folds rather than preventing leakage. Asserting that tree-based models are immune to temporal leakage because they do not assume linearity. Suggesting that you can fix the problem by blocking contiguous chunks randomly without preserving the forward order.
What usually comes next
How would you handle multiple forecast horizons, such as predicting one step ahead versus twelve steps ahead? When would you prefer a sliding window over an expanding window? How do you deal with seasonality when your minimum training set is shorter than one full seasonal cycle? What metrics do you use when errors are not symmetric, and how does that change your cross-validation setup?
A concrete example
Imagine you are forecasting daily electricity demand. A random split might place a hot summer Tuesday in the training set and a cool autumn Tuesday in the test set. The model learns the summer peak and appears to predict the autumn day well because it has memorized similar day-of-week patterns from other seasons, but it has never been forced to forecast forward from autumn into winter. Walk-forward validation starting in January would require you to forecast February 1st using only January data, then February 2nd using January through February 1st, and so on through December. Your final accuracy reflects true sequential prediction rather than interpolation.
Interview question
Which statement best explains why random train-test splits are unreliable for evaluating forecasting models?
- a.They destroy seasonal patterns, making the model unable to learn true cyclical behavior.
- b.They allow the model to train on data from time periods that occur after the test set observations.Correct
- c.They prevent evaluation across multiple test folds, producing a less stable accuracy estimate.
- d.They reduce the total amount of training data available compared to walk-forward methods.
Why? this is the answer
Random splits ignore chronological order, allowing the model to learn from future observations and producing an unrealistic performance estimate. The most tempting distractor confuses the purpose of walk-forward validation: its primary goal is preventing temporal leakage, not simply increasing the number of test folds.
Just read this? Test yourself on what you have been reading.
Read the original → otexts.com
- #time series
- #cross-validation
- #forecasting
- #model evaluation
- #temporal leakage
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on time series — each one lists the topics its interview covers.
See open roles