Counterfactual Explanations: How to Change a Model's Mind
A counterfactual explanation finds the smallest input change that flips a model's prediction. It's used to give actionable feedback, like telling a user what to change to get a loan approved.
Why it exists
Black-box models make decisions, but often don't explain them. For high-stakes predictions, like a loan application, simply saying 'rejected' is not enough. Counterfactuals were developed to provide actionable explanations that tell a user not just 'why', but 'how' they might achieve a different outcome.
The mental model
Think of it as answering the question: 'If X hadn't happened, Y wouldn't have happened.' For a model, this becomes: 'What is the smallest change I can make to the input features to flip the prediction from Y to Z?' It's a search for a nearby, hypothetical reality where the outcome is more favorable.
How it works
As a model-agnostic method, it treats the model as a black box, interacting only with its inputs and outputs. It takes an instance (e.g., a rejected loan application) and systematically probes the model by changing feature values. The goal is to find the combination of changes with the smallest 'cost'—the minimal adjustment—that results in the desired prediction, such as 'approved'. This search identifies the most efficient path across the model's decision boundary.
When to use it
Use counterfactuals to explain individual predictions to end-users who need actionable feedback. They are ideal for regulated domains like finance or HR where you must justify decisions. For example, 'Your application was rejected. However, if your income were $5,000 higher, it would have been approved.' This provides clear, understandable recourse.
When not to use it
Do not use a counterfactual to prove real-world causality. The explanation reveals the model's logic, which might be based on spurious correlations in the training data, not fundamental truths. An explanation is about the model's world, not the real world. Acting on the advice might change the model's output on a subsequent run, but it doesn't guarantee the same outcome in reality.
One canonical example
A bank's machine learning model rejects Peter's loan application. Peter wants to know why. A counterfactual explanation would not just list feature importances; it would provide a specific scenario: 'If you had one fewer credit card and had not defaulted on a loan five years ago, your application would have been approved.' This gives Peter concrete, albeit model-specific, steps for the future.
Interview question
What is the core objective of a counterfactual explanation?
- a.To rank features by their global importance to the model's overall predictions.
- b.To identify the minimal input adjustments needed to alter a specific model prediction.Correct
- c.To establish the true causal links between input variables and real-world outcomes.
- d.To decompose a model's output into contributions from each input feature for transparency.
Why? this is the answer
Counterfactual explanations specifically aim to find the smallest, most actionable changes to an instance's features that would result in a different model prediction, providing 'how-to' advice. Option C is incorrect because the card explicitly states counterfactuals should not be used to prove real-world causality, only to explain the model's logic.
Just read this? Test yourself on what you have been reading.
Read the original → christophm.github.io
- #ml
- #xai
- #explainability
- #model-agnostic
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles