Member-only story
Model Fairness in Machine Learning: Metrics, Audits & Mitigation
The algorithm was supposed to help identify patients who needed extra care. But at the same risk score, Black patients were considerably sicker than White patients.
That was the central finding of a 2019 study by Ziad Obermeyer and colleagues. The system predicted healthcare spending rather than illness. Because spending on Black patients was lower despite comparable health needs, cost became a misleading stand-in for need. The researchers estimated that correcting the disparity would increase Black patients’ representation in the additional-care group from 17.7% to 46.5% in their analysis.
The problem was deeper than an unfortunate threshold. The model was answering a different question from the one its users needed answered.
That is where useful fairness work begins: with the decision, the people affected, and the meaning of the target. Metrics come next.
My view is that a fairness audit should help a team decide what to change, what evidence is missing, and whether deployment is justified. Producing a reassuring score is a much smaller achievement.
Before choosing a metric, identify the harm
“Is this model fair?” is too broad to guide an investigation.
