Feature Governance: Taming Your ML Inputs

WHY IT EXISTS Without a central system, the feature logic used to train a model often differs from the logic used for live predictions. This mismatch, called training-serving skew, causes silent model failures. Feature governance exists to prevent this by creating a single source of truth and stopping teams from constantly reinventing the same feature engineering logic.

THE MENTAL MODEL A feature store is the technical implementation of feature governance. Think of it as a Git repository for your data features. Instead of checking in code, you check in the definitions and logic for creating features, like 'average transaction value over the last 7 days'. Everyone pulls from this central, versioned source, ensuring consistency across all models and environments.

HOW IT WORKS A system for feature governance, typically a feature store, ingests raw data and runs transformation pipelines to create features. It stores these features in two ways: an offline store (e.g., a data lakehouse) for training on large historical datasets, and a low-latency online store (e.g., Redis) for serving features to live models. It provides APIs for data scientists to discover and use features for training, and for production services to fetch features at inference time.

WHEN TO USE IT Use it when multiple teams or models share common data sources. If fraud, marketing, and recommendation teams all need a customer's purchase frequency, one governed feature should serve all of them. It is critical for ensuring consistency between batch training and real-time inference environments, which is a common source of bugs.

WHEN NOT TO USE IT For a solo data scientist working on a single, self-contained project with static data, a full-blown feature governance system is overkill. If your features are simple, rarely change, and are only used by one model, the overhead of maintaining a feature store is not justified. It is designed to manage shared, evolving assets at scale.

ONE CANONICAL EXAMPLE A fraud detection model needs features like 'number of transactions in the past week' and 'location of recent purchases'. A feature store implements the governance. When training the model, a data scientist pulls historical values for millions of users from the offline store. When a live transaction occurs, the production service queries the online store for the same features for that specific user, guaranteeing the model sees data in the exact same format it was trained on.