Fairness Metrics: Quantifying AI's Impact on People

Why it exists

Models learn from data, and real-world data often contains historical and societal biases. Without explicit checks, an AI model can easily learn and amplify these biases, leading to systems that systematically discriminate against certain groups, a phenomenon known as disparate impact. Fairness metrics were created to detect and quantify this algorithmic bias before it causes real-world harm.

The mental model

Think of fairness metrics as a set of statistical lenses for examining your model's decisions. Each lens highlights a different potential type of unfairness. One lens might check if the model's outcomes are the same across groups, while another checks if its error rates are the same. You cannot look through all lenses at once; you must choose which view of fairness is most important for your specific application, as they are often in tension.

How it works

Fairness metrics compare a model's predictions across different subgroups, which are defined by sensitive attributes like race, gender, or age. There are two main families. First, group fairness metrics focus on statistical properties of a whole group. Demographic Parity is one such metric; it is satisfied if the proportion of positive outcomes (e.g., loan approvals) is the same for every subgroup. Second, individual fairness metrics, like Counterfactual Fairness, ask: would the decision for a specific person change if only their sensitive attribute were different, and all else remained identical?

When to use it

Use fairness metrics whenever a model's output has a meaningful impact on people's lives, especially in regulated domains like finance, employment, housing, and criminal justice. They are essential tools for auditing models before deployment, monitoring them in production, and demonstrating compliance with anti-discrimination laws. They force a clear, explicit conversation about what "fairness" means for your product.

When not to use it

Fairness metrics are not a silver bullet. They are less critical for low-stakes applications, like a model that recommends movies. More importantly, relying on a single metric without understanding its trade-offs is dangerous. For instance, strictly enforcing Demographic Parity (equal outcomes) might be inappropriate if one group is, on average, more qualified for a specific role. The choice of metric is a socio-technical decision, not just a technical one, and requires careful consideration of context.

One canonical example

A university uses an AI to screen applications. To be fair, they use the Demographic Parity metric, ensuring the percentage of admitted applicants from Group A is the same as from Group B. However, if Group B's applicants are, on average, more qualified, the model must either reject qualified applicants from Group B or accept unqualified applicants from Group A to meet the parity goal. This shows how one definition of fairness can conflict with merit or other fairness definitions.