Predictive Uncertainty-based Bias Mitigation in Ranking

Social biases that are often prevalent in training data and hence are learned by the model can cause societal harms, by misrepresenting certain groups, and by enforcing stereotypes. Mitigating such biases demands algorithms that balance the trade-off between maximized utility for the user with fairness objectives, which incentivize unbiased rankings. In this talk we will demonstrate that using the predictive uncertainty of the ranking model with respect to its scores can aid in keeping the loss in user utility to a minimum, while achieving fairer ranked lists. Our proposed approach, PUFR re-ranks the documents based on whether they are biased and the score distribution, which we obtain through Laplace approximation. We compare PUFR with existing baselines for bias mitigation with respect to the utility-fairness trade. We show that an uncertainty-based approach can provide an intuitive and flexible trade-off that outperforms all baselines without additional training requirements.