MESD: A Risk-Sensitive Metric for Explanation Fairness Across Intersectional Subgroups
Researchers Gideon Popoola and John Sheppard have introduced Multi-category Explanation Stability Disparity (MESD), a new procedural fairness metric for machine learning. Unlike traditional outcome-oriented metrics like Demographic parity, which only measure statistical consistency in predictions, MESD evaluates whether models use systematically different reasoning across demographic groups. This approach addresses 'fairness gerrymandering,' where models appear fair on individual attributes but exhibit disparities in intersectional subgroups, such as combinations of race and gender. The metric integrates label-aware aggregation, empirical-Bayes shrinkage for small group stability, and Conditional Value-at-Risk (CVaR) weighting to highlight worst-case disparities. The authors also propose a multi-objective optimization framework (UEF) using NSGA-II to jointly optimize utility, outcome fairness, and procedural fairness. Evaluations on three benchmark datasets against four state-of-the-art methods demonstrate that MESD reveals procedural disparities invisible to standard metrics. The work contributes to procedural justice theory and offers implications for regulatory compliance and intersectional equity in AI systems.
Wire timeline
MESD: A Risk-Sensitive Metric for Explanation Fairness Across Intersectional Subgroups
Researchers Gideon Popoola and John Sheppard have introduced Multi-category Explanation Stability Disparity (MESD), a new procedural fairness metric for machine learning. Unlike traditional outcome-oriented metrics like Demographic parity, which only measure statistical consistency in predictions, MESD evaluates whether models use systematically different reasoning across demographic groups. This approach addresses 'fairness gerrymandering,' where models appear fair on individual attributes but exhibit disparities in intersectional subgroups, such as combinations of race and gender. The metric integrates label-aware aggregation, empirical-Bayes shrinkage for small group stability, and Conditional Value-at-Risk (CVaR) weighting to highlight worst-case disparities. The authors also propose a multi-objective optimization framework (UEF) using NSGA-II to jointly optimize utility, outcome fairness, and procedural fairness. Evaluations on three benchmark datasets against four state-of-the-art methods demonstrate that MESD reveals procedural disparities invisible to standard metrics. The work contributes to procedural justice theory and offers implications for regulatory compliance and intersectional equity in AI systems.
cs.AI updates on arXiv.org