Point-in-Time Correctness: Avoiding Data Leakage in ML

WHY IT EXISTS In machine learning, we need to train models on historical data. But this data often lives in different tables, updated by different pipelines at different times. A simple join won't work because the timestamps rarely align perfectly. We need a way to assemble a snapshot of the world as it was at a specific moment for each training example.

THE MENTAL MODEL Think of it as creating a "flashback" for a movie. For a specific event (the "label," like a user churning), you need to show the state of the world just before it happened. You can't use information that arrived later. A point-in-time correct join gathers all the feature values (user's last login, number of support tickets) that were current as of the churn timestamp, ensuring no spoilers.

HOW IT WORKS It's a temporal join, often implemented as an ASOF LEFT JOIN in SQL. You start with a table of labels, each with a primary key and a timestamp (e.g., user_id, event_timestamp). You then join this with your feature tables. For each row in the label table, the join finds the row in the feature table with the most recent timestamp that is less than or equal to the label's timestamp. This guarantees you only retrieve feature values that were available at that point in time.

WHEN TO USE IT This is the standard for creating training or batch inference datasets for supervised machine learning. It's essential when your features are precomputed and stored in a feature store, especially when different feature groups have different update cadences. Use it to ensure historical accuracy and model reproducibility.

WHEN NOT TO USE IT For real-time online inference, you typically just fetch the absolute latest feature values via a direct lookup, not a historical join. It's also less critical if all your data is generated in a single, perfectly synchronized transaction, though this is rare in complex systems.

ONE CANONICAL EXAMPLE To predict customer churn, you have a table of churn events with user_id and churn_timestamp. You also have feature tables for user activity (e.g., logins_last_7_days) and purchases, updated at different intervals. For a user who churned on May 10th at 3 PM, a point-in-time correct join fetches the feature values valid just before that moment. It prevents data from May 11th, or even May 10th at 4 PM, from leaking into that user's training row.