Bias Mitigation Algorithms: Correcting Unfair AI

WHY IT EXISTS: Algorithmic bias is a systematic and repeatable harmful tendency in AI systems to create unfair outcomes, privileging one category over another. Since models learn from real-world data that contains historical and societal biases, they will naturally reproduce and even amplify those unfair patterns unless explicitly corrected. Bias mitigation algorithms exist to provide that correction.

THE MENTAL MODEL: Think of a bias mitigation algorithm as a set of corrective lenses for a model that's looking at the world through a distorted, biased lens (the training data). There is no single "perfectly clear" lens. Instead, you choose a specific corrective lens that addresses a particular type of distortion. You are actively choosing which definition of fairness to optimize for, such as equal opportunity or equal outcome.

HOW IT WORKS: Mitigation techniques generally fall into three categories. First, pre-processing methods adjust the training data itself by re-weighting, re-sampling, or generating new data to balance representation across groups. Second, in-processing methods modify the model's learning algorithm by adding a fairness constraint to its objective function, forcing it to trade some accuracy for less disparity in outcomes. Third, post-processing methods adjust the model's predictions after it's been trained, for example by setting different decision thresholds for different demographic groups to achieve a desired fairness metric.

WHEN TO USE IT: Use these algorithms when deploying models in high-stakes domains where biased outcomes can cause significant harm, such as in hiring, credit scoring, criminal justice, and healthcare. They are essential when you need to comply with legal or ethical standards of fairness and can clearly define the fairness metric you are aiming for in your specific context.

WHEN NOT TO USE IT: Avoid applying these techniques without a deep understanding of the context and the different mathematical definitions of fairness. There are inherent trade-offs; for example, optimizing for equal outcomes across groups might conflict with ensuring that similarly qualified individuals are treated similarly, regardless of group. Applying a mitigation algorithm without this awareness can lead to unintended negative consequences or "fairness gerrymandering."

ONE CANONICAL EXAMPLE: A company's hiring model, trained on past employee data, shows a bias against applicants from a certain demographic. To mitigate this, the company could use a pre-processing technique to up-sample resumes from that group in the training set. Alternatively, they could use an in-processing method that penalizes the model during training if it produces different interview recommendation rates for different demographic groups.