research-article

Public Access

Measuring Fairness in Ranked Outputs

Authors:

Ke Yang,

Julia StoyanovichAuthors Info & Claims

SSDBM '17: Proceedings of the 29th International Conference on Scientific and Statistical Database Management

Article No.: 22, Pages 1 - 6

https://doi.org/10.1145/3085504.3085526

Published: 27 June 2017 Publication History

PDF eReader

Abstract

Ranking and scoring are ubiquitous. We consider the setting in which an institution, called a ranker, evaluates a set of individuals based on demographic, behavioral or other characteristics. The final output is a ranking that represents the relative quality of the individuals. While automatic and therefore seemingly objective, rankers can, and often do, discriminate against individuals and systematically disadvantage members of protected groups. This warrants a careful study of the fairness of a ranking scheme, to enable data science for social good applications, among others.

In this paper we propose fairness measures for ranked outputs. We develop a data generation procedure that allows us to systematically control the degree of unfairness in the output, and study the behavior of our measures on these datasets. We then apply our proposed measures to several real datasets, and detect cases of bias. Finally, we show preliminary results of incorporating our ranked fairness measures into an optimization framework, and show potential for improving fairness of ranked outputs while maintaining accuracy.

The code implementing all parts of this work is publicly available at https://github.com/DataResponsibly/FairRank.

References

[1]

Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. ProPublica (May 23 2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

Google Scholar

[2]

Danielle K. Citron and Frank A. Pasquale. 2014. The Scored Society: Due Process for Automated Predictions. Washington Law Review 89 (2014). http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2376209

Google Scholar

[3]

Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. 2012. Fairness through awareness. In Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8-10, 2012. 214--226.

Digital Library

Google Scholar

[4]

Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2016. On the (im)possibility of fairness. CoRR abs/1609.07236 (2016). http://arxiv.org/abs/1609.07236

Google Scholar

[5]

Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20, 4 (2002), 422--446.

Digital Library

Google Scholar

[6]

Joshua A. Kroll, Joanna Huey, Solon Barocas, Edward W. Felten, Joel R. Reidenberg, David G. Robinson, and Harlan Yu. 2017. Accountable Algorithms. University of Pennsylvania Law Review 165 (2017). http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2765268

Google Scholar

[7]

M. Lichman. 2013. UCI Machine Learning Repository. (2013). http://archive.ics.uci.edu/ml

Google Scholar

[8]

Julia Stoyanovich, Sihem Amer-Yahia, and Tova Milo. 2011. Making interval-based clustering rank-aware. In EDBT 2011, 14th International Conference on Extending Database Technology, Uppsala, Sweden, March 21-24, 2011, Proceedings. 437--448.

Digital Library

Google Scholar

[9]

Julia Stoyanovich and Ellen P. Goodman. 2016. Revealing Algorithmic Rankers. Freedom to Tinker (August 5 2016).

Google Scholar

[10]

Florian Tramèr, Vaggelis Atlidakis, Roxana Geambasu, Daniel J. Hsu, Jean-Pierre Hubaux, Mathias Humbert, Ari Juels, and Huang Lin. 2015. Discovering Unwarranted Associations in Data-Driven Applications with the FairTest Testing Toolkit. CoRR abs/1510.02377 (2015). http://arxiv.org/abs/1510.02377

Google Scholar

[11]

Richard S. Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. 2013. Learning Fair Representations. In International Conference on Machine Learning. http://jmlr.org/proceedings/papers/v28/zemel13.html

Digital Library

Google Scholar

[12]

Indre Zliobaite. 2015. A survey on measuring indirect discrimination in machine learning. CoRR abs/1511.00148 (2015). http://arxiv.org/abs/1511.00148

Google Scholar

Cited By

View all

Martens MDe Wolf RDe Marez L(2024)Trust in algorithmic decision-making systems in health: A comparison between ADA health and IBM Watson.Cyberpsychology: Journal of Psychosocial Research on Cyberspace10.5817/CP2024-1-518:1Online publication date: 1-Feb-2024
https://doi.org/10.5817/CP2024-1-5
Campbell FStoyanovich JMoskovitch Y(2024)Rodeo: Making Refinements for Diverse Top-K QueriesProceedings of the VLDB Endowment10.14778/3685800.368587017:12(4341-4344)Online publication date: 8-Nov-2024
https://doi.org/10.14778/3685800.3685870
Schumacher TLutz MSikdar SStrohmaier M(2024)Properties of Group Fairness Measures for RankingsACM Transactions on Social Computing10.1145/3674883Online publication date: 27-Aug-2024
https://dl.acm.org/doi/10.1145/3674883
Show More Cited By

Index Terms

Measuring Fairness in Ranked Outputs
1. Information systems
  1. Data management systems
2. Social and professional topics
  1. Computing / technology policy

Recommendations

A Nutritional Label for Rankings
SIGMOD '18: Proceedings of the 2018 International Conference on Management of Data

Algorithmic decisions often result in scoring and ranking individuals to determine credit worthiness, qualifications for college admissions and employment, and compatibility as dating partners. While automatic and seemingly objective, ranking algorithms ...
Airtime Fairness for IEEE 802.11 Multirate Networks

Under a multi rate network scenario, the IEEE 802.11 DCF MAC fails to provide air-time fairness for all competing stations since the protocol is designed for ensuring max-min throughput fairness and the maximum achievable throughput by any station gets ...
MithraLabel: Flexible Dataset Nutritional Labels for Responsible Data Science
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Using inappropriate datasets for data science tasks can be harmful, especially for applications that impact humans. Targeting data ethics, we demonstrate MithraLabel, a system for generating task-specific information about a dataset, in the form of a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SSDBM '17: Proceedings of the 29th International Conference on Scientific and Statistical Database Management

June 2017

373 pages

ISBN:9781450352826

DOI:10.1145/3085504

General Chair:
Alok Choudhary,
Program Chair:
Kesheng Wu,
Publications Chair:
Bin Dong

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Northwestern University: Northwestern University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation
US-Israel Binational Science Foundation

Conference

SSDBM '17

SSDBM '17: 29th International Conference on Scientific and Statistical Database Management

June 27 - 29, 2017

IL, Chicago, USA

Acceptance Rates

Overall Acceptance Rate 56 of 146 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

202
Total Citations
View Citations
3,385
Total Downloads

Downloads (Last 12 months)552
Downloads (Last 6 weeks)67

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Martens MDe Wolf RDe Marez L(2024)Trust in algorithmic decision-making systems in health: A comparison between ADA health and IBM Watson.Cyberpsychology: Journal of Psychosocial Research on Cyberspace10.5817/CP2024-1-518:1Online publication date: 1-Feb-2024
https://doi.org/10.5817/CP2024-1-5
Campbell FStoyanovich JMoskovitch Y(2024)Rodeo: Making Refinements for Diverse Top-K QueriesProceedings of the VLDB Endowment10.14778/3685800.368587017:12(4341-4344)Online publication date: 8-Nov-2024
https://doi.org/10.14778/3685800.3685870
Schumacher TLutz MSikdar SStrohmaier M(2024)Properties of Group Fairness Measures for RankingsACM Transactions on Social Computing10.1145/3674883Online publication date: 27-Aug-2024
https://dl.acm.org/doi/10.1145/3674883
Campbell FSilberstein AStoyanovich JMoskovitch Y(2024)Query Refinement for Diverse Top-k SelectionProceedings of the ACM on Management of Data10.1145/36549692:3(1-27)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654969
Chen ZZhang JHort MHarman MSarro F(2024)Fairness Testing: A Comprehensive Survey and Analysis of TrendsACM Transactions on Software Engineering and Methodology10.1145/365215533:5(1-59)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3652155
Li YYang YCao JLiu STang HXu GBaeza-Yates RBonchi F(2024)Toward Structure Fairness in Dynamic Graph Embedding: A Trend-aware Dual Debiasing ApproachProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671848(1701-1712)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671848
Alkhathlan MCachel KShrestha HHarrison LRundensteiner E(2024)Balancing Act: Evaluating People’s Perceptions of Fair Ranking MetricsProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659018(1940-1970)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659018
Cachel KRundensteiner E(2024)PreFAIR: Combining Partial Preferences for Fair Consensus Decision-makingProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658961(1133-1149)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658961
Cachel KRundensteiner ESerra ESpezzano F(2024)Wise Fusion: Group Fairness Enhanced Rank FusionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679649(163-174)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679649
Xu SHou DPang LDeng JXu JShen HCheng XHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Invisible Relevance Bias: Text-Image Retrieval Models Prefer AI-Generated ImagesProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657750(208-217)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657750
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

A Nutritional Label for Rankings

Airtime Fairness for IEEE 802.11 Multirate Networks

MithraLabel: Flexible Dataset Nutritional Labels for Responsible Data Science

Comments

Information

Published In

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations