Time Series Cross-Validation

The mental model

Standard cross-validation assumes data points are independent, so it shuffles them randomly into folds. This breaks for time series data, which has a strict chronological order. You can't use knowledge from Tuesday to predict Monday. Time series cross-validation respects this "arrow of time" by ensuring the model is only ever trained on past data to predict future data, mimicking how it would operate in the real world.

How it works

This procedure is often called "evaluation on a rolling forecasting origin." Instead of random folds, you create a series of sequential splits. The process is as follows: first, you select an initial block of data for the first training set. Second, you train your model and use it to forecast the very next data point, which serves as your first test set (of size one). Third, you calculate the forecast error. Fourth, you expand the training set to include the point you just tested on. You then repeat this process, rolling the "origin" forward one step at a time, forecasting the next point, and adding it to the training set. The final evaluation metric is the average of the forecast errors across all these steps. To ensure reliable forecasts, the earliest observations are typically not used as test sets, as the initial training data would be too sparse.

When to use it

Use time series cross-validation whenever you are evaluating a forecasting model. This is the correct way to estimate a model's out-of-sample performance on any data with a temporal component, such as daily sales figures, hourly server loads, or minute-by-minute stock prices. It provides a much more realistic estimate of how the model will perform on new, unseen data compared to naive train-test splits or standard k-fold CV.

When not to use it

This method is specifically for time-ordered data. For problems where the data points are independent and have no inherent sequence, such as classifying images or predicting customer attributes from a static dataset, standard k-fold cross-validation is more appropriate and computationally efficient. Using time series CV on non-temporal data provides no benefit and adds unnecessary complexity.

One canonical example

Imagine you have 12 months of user activity data and want to forecast month 13. To evaluate your model, you might start with an initial training set of the first 6 months. Fold 1: Train on months 1-6, predict month 7. Fold 2: Train on months 1-7, predict month 8. Fold 3: Train on months 1-8, predict month 9. ...and so on, up to Fold 6, which trains on months 1-11 to predict month 12. You would then average the forecast errors from these 6 folds to get a robust performance estimate before using all 12 months of data to train your final model.