Measuring fairness in an embedding-based text classifier
Operationalizing fairness metrics on text.
Define group definitions, apply demographic parity and equalized odds, and address noisy labels and implicit group membership.
What's really being asked
Whether you can select fairness metrics that capture distinct definitions, reason about their tradeoffs, and confront the messiness of measuring groups in unstructured text.
The full answer
Define the protected groups and the favorable outcome first. Then pick metrics that disagree by design. Demographic parity asks whether the positive-prediction rate is equal across groups, ignoring true labels. Equalized odds asks whether true positive and false positive rates are equal across groups, conditioning on the ground truth. Equal opportunity is the weaker variant requiring only equal true positive rates. These can be mathematically incompatible, so you must state which harm you prioritize. For embeddings specifically, note that the representation may encode demographic proxies even when the raw text does not mention them, so a classifier can discriminate via correlated features. Report metrics with confidence intervals because group sample sizes are often small.
The mistakes people make
Naming only accuracy gap. Citing one metric and treating it as universal fairness. Assuming clean protected-attribute labels are available. Ignoring that demographic parity and equalized odds generally cannot both hold when base rates differ.
What usually comes next
Which metric do you choose when base rates differ across groups and why? How do you obtain group labels without collecting sensitive data? How do embeddings leak protected attributes? How would you mitigate once you measure a gap?
A concrete example
A resume-screening classifier predicts interview yes or no from embedded text. You infer candidate groups from a labeled audit set. Demographic parity shows group A gets positive predictions at 40 percent versus group B at 25 percent. But equalized odds reveals group B also has a lower true positive rate, meaning qualified B candidates are rejected. You report both, flag that they conflict because base rates differ, and recommend equal opportunity as the operative target, plus an embedding audit to find the leaking proxy features.
Interview question
Two teams measure the same text classifier: one reports it satisfies demographic parity, another says it fails equalized odds. How is this possible?
- a.Demographic parity only applies to balanced datasets
- b.Equalized odds is a stricter version of demographic parity, so parity should imply it
- c.One team made a calculation error, since the metrics are equivalent
- d.The metrics capture different fairness notions and can conflict when group base rates differCorrect
Why? this is the answer
Demographic parity equalizes positive rates regardless of labels, while equalized odds conditions on ground truth; when base rates differ they are generally incompatible. They are distinct, not nested, definitions.
Just read this? Test yourself on what you have been reading.
Read the original → aclanthology.org
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on llm — each one lists the topics its interview covers.
See open roles