Unlawful Proxy Discrimination: A Framework for Challenging Inherently Discriminatory Algorithms
Pages 1850 - 1860
Abstract
Emerging scholarship suggests that the EU legal concept of direct discrimination - where a person is given different treatment on grounds of a protected characteristic - may apply to various algorithmic decision-making contexts. This has important implications: unlike indirect discrimination, there is generally no ‘objective justification’ stage in the direct discrimination framework, which means that the deployment of directly discriminatory algorithms will usually be unlawful per se. In this paper, we focus on the most likely candidate for direct discrimination in the algorithmic context, termed inherent direct discrimination, where a proxy is inextricably linked to a protected characteristic. We draw on computer science literature to suggest that, in the algorithmic context, ‘treatment on the grounds of’ needs to be understood in terms of two steps: proxy capacity and proxy use. Only where both elements can be made out can direct discrimination be said to be ‘on grounds of’ a protected characteristic. We analyse the legal conditions of our proposed proxy capacity and proxy use tests. Based on this analysis, we discuss technical approaches and metrics that could be developed or applied to identify inherent direct discrimination in algorithmic decision-making.
References
[1]
Case AC 1155. 1989. R. v Birmingham City Council Ex p. Equal Opportunities Commission (No.1).
[2]
Case Eq. L.R. 406. 2012. Royal Bank of Scotland Plc v Morris.
[3]
Jeremias Adams-Prassl, Reuben Binns, and Aislinn Kelly-Lyth. 2023. Directly Discriminatory Algorithms. The Modern Law Review 86, 1 (2023), 144–175. https://doi.org/10.1111/1468-2230.12759
[4]
Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2023. Fairness and machine learning: Limitations and opportunities. MIT Press.
[5]
Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. California law review (2016), 671–732. https://doi.org/10.2139/ssrn.2477899
[6]
Barry Becker and Ronny Kohavi. 1996. Adult. UCI Machine Learning Repository. https://doi.org/10.24432/C5XW20
[7]
Reuben Binns, Jeremias Adams-Prassl, and Aislinn Kelly-Lyth. 2023. Legal Taxonomies of Machine Bias: Revisiting Direct Discrimination. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT ’23). Association for Computing Machinery, New York, NY, USA, 1850–1858. https://doi.org/10.1145/3593013.3594121
[8]
Liam Kofi Bright, Daniel Malinsky, and Morgan Thompson. 2016. Causally Interpreting Intersectionality Theory. Philosophy of Science 83, 1 (2016), 60–81. https://doi.org/10.1086/684173
[9]
Case 1 AC 50. 2000. Nagarajan v London Regional Transport.
[10]
Case 2 AC 751. 1990. James v Eastleigh Borough Council.
[11]
Case C-177/88. 1990. Elisabeth Johanna Pacifica Dekker v Stichting Vormingscentrum voor Jong Volwassenen (VJVCentrum) Plus. ECLI:EU:C:1990:383.
[12]
Case C-236/09. 2011. Association Belge des Consommateurs Test-Achats and Others. ECLI:EU:C:2011:100.
[13]
Case C-258/17. 2019. E.B. v Versicherungsanstalt öffentlich Bediensteter BVA. ECLI:EU:C:2019:17.
[14]
Case C-267/06. 2008. Tadao Maruko v Versorgungsanstalt der deutschen Bühnen. ECLI:EU:C:2008:179.
[15]
Case C-54/07. 2008. Centrum voor gelijkheid van kansen en voor racismebestrijding v Firma Feryn NV. ECLI:EU:C:2008:397.
[16]
Case C-668/15. 2017. Jyske Finans A/S v Ligebehandlingsnævnet, acting on behalf of Ismar Huskic. EU:C:2017:278.
[17]
Case C-83/14. 2015. CHEZ Razpredelenie Bulgaria AD v Komisia za zashtita ot diskriminatsia. ECLI:EU:C:2015:170.
[18]
Case EWCA Civ 280. 2018. Mruke v Khan.
[19]
Case ICR 1450. 2009. Ahmed v Amnesty International.
[20]
Case UKSC 11. 2011. Patmalniece v Secretary of State for Work and Pensions.
[21]
Case UKSC 27. 2017. Essop v Home Office (UK Border Agency).
[22]
Case UKSC 40. 2017. R (on the application of Coll) v Secretary of State for Justice.
[23]
Case UKSC 49. 2018. Lee v Ashers Baking Co Ltd.
[24]
Cases C-804/18 and C-341/19. 2021. IX v WABE eV (C-804/18) and MH Müller Handels GmbH v MJ (C-341/19). ECLI:EU:C:2021:594.
[25]
Silvia Chiappa. 2019. Path-Specific Counterfactual Fairness. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (Jul. 2019), 7801–7808. https://doi.org/10.1609/aaai.v33i01.33017801
[26]
Hugh Collins and Tarun Khaitan. 2018. Foundations of Indirect Discrimination Law. Hart Publishing.
[27]
Council of European Union. 2000. Racial Equality Directive. Council Directive 2000/43/EC of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin. Official Journal L 180 (2000), 22–26.
[28]
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. 2021. Retiring adult: New datasets for fair machine learning. Advances in neural information processing systems 34 (2021), 6478–6490.
[29]
Wouter Duivesteijn, Ad J Feelders, and Arno Knobbe. 2016. Exceptional Model Mining: Supervised descriptive local pattern mining with complex target concepts. Data Mining and Knowledge Discovery 30 (2016), 47–98.
[30]
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference. 214–226.
[31]
Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 259–268.
[32]
European Union Agency for Fundamental Rights and Council of Europe. 2018. Handbook on European non-discrimination law. Publications Office of the European Union.
[33]
Sandra Fredman. 2022. Discrimination law. Oxford University Press.
[34]
Alex Goldstein, Adam Kapelner, Justin Bleich, and Emil Pitkin. 2015. Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. Journal of Computational and Graphical Statistics 24, 1 (2015), 44–65.
[35]
Philipp Hacker. 2018. Teaching fairness to artificial intelligence: Existing and novel strategies against algorithmic discrimination under EU law. Common Market Law Review 55, Issue 4 (Aug. 2018), 1143–1185. https://doi.org/10.54648/cola2018095
[36]
Alex Hanna, Emily Denton, Andrew Smart, and Jamila Smith-Loud. 2020. Towards a critical race methodology in algorithmic fairness. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* ’20). Association for Computing Machinery, New York, NY, USA, 501–512. https://doi.org/10.1145/3351095.3372826
[37]
Franciso Herrera, Cristóbal José Carmona, Pedro González, and María José Del Jesus. 2011. An overview on subgroup discovery: foundations and applications. Knowledge and information systems 29 (2011), 495–525.
[38]
Boyang Hu, Qicheng Lin, Yao Zheng, Qiben Yan, Matthew Troglia, and Qingyang Wang. 2019. Characterizing location-based mobile tracking in mobile ad networks. In 2019 IEEE Conference on Communications and Network Security (CNS). IEEE, 223–231.
[39]
Lily Hu and Issa Kohler-Hausmann. 2020. What’s sex got to do with machine learning?. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* ’20). Association for Computing Machinery, New York, NY, USA, 513. https://doi.org/10.1145/3351095.3375674
[40]
Atoosa Kasirzadeh and Andrew Smart. 2021. The Use and Misuse of Counterfactuals in Ethical Machine Learning. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (Virtual Event, Canada) (FAccT ’21). Association for Computing Machinery, New York, NY, USA, 228–236. https://doi.org/10.1145/3442188.3445886
[41]
Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. 2017. Avoiding discrimination through causal reasoning. Advances in neural information processing systems 30 (2017).
[42]
Issa Kohler-Hausmann. 2018. Eddie Murphy and the dangers of counterfactual causal thinking about detecting racial discrimination. Nw. UL Rev. 113 (2018), 1163.
[43]
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. Advances in neural information processing systems 30 (2017).
[44]
Daniel Malinsky. 2018. Intervening on structure. Synthese 195, 5 (2018), 2295–2312.
[45]
Razieh Nabi and Ilya Shpitser. 2018. Fair Inference on Outcomes. Proceedings of the AAAI Conference on Artificial Intelligence 32, 1 (Apr. 2018). https://doi.org/10.1609/aaai.v32i1.11553
[46]
Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. 2016. Causal inference in statistics: A primer. John Wiley & Sons.
[47]
Anya ER Prince and Daniel Schwarcz. 2019. Proxy discrimination in the age of artificial intelligence and big data. Iowa L. Rev. 105 (2019), 1257.
[48]
Lauren N Ross. 2023. What is social structural explanation? A causal account. Noûs (2023).
[49]
P Senanayake, CL Jayawardena, and JDSU Jayakodi. 2018. Accuracy of smartphone location services for geo-tagged data collection: A field study. Annu. Sessions of IESL (2018), 447–451.
[50]
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Workshop Track Proceedings. http://arxiv.org/abs/1312.6034
[51]
Rory W Spanton and Olivia Guest. 2022. Measuring Trustworthiness or Automating Physiognomy? A Comment on Safra, Chevallier, Grèzes, and Baumard (2020). https://doi.org/10.48550/ARXIV.2202.08674
[52]
Michael Carl Tschantz. 2022. What is Proxy Discrimination?. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT ’22). Association for Computing Machinery, New York, NY, USA, 1993–2003. https://doi.org/10.1145/3531146.3533242
[53]
Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2021. Why fairness cannot be automated: Bridging the gap between EU non-discrimination law and AI. Computer Law & Security Review 41 (2021), 105567.
[54]
Elizabeth Anne Watkins, Michael McKenna, and Jiahao Chen. 2022. The four-fifths rule is not disparate impact: a woeful tale of epistemic trespassing in algorithmic fairness. arxiv:2202.09519 [cs.CY]
[55]
Hilde Weerts, Miroslav Dudík, Richard Edgar, Adrin Jalali, Roman Lutz, and Michael Madaio. 2023. Fairlearn: Assessing and Improving Fairness of AI Systems. Journal of Machine Learning Research 24, 257 (2023), 1–8. http://jmlr.org/papers/v24/23-0389.html
[56]
Hilde Weerts, Raphaële Xenidis, Fabien Tarissan, Henrik Palmer Olsen, and Mykola Pechenizkiy. 2023. Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT ’23). Association for Computing Machinery, New York, NY, USA, 805–816. https://doi.org/10.1145/3593013.3594044
[57]
Raphaële Xenidis and Linda Senden. 2019. EU non-discrimination law in the era of artificial intelligence: Mapping the challenges of algorithmic discrimination. General Principles of EU law and the EU Digital Order (Kluwer Law International, 2020) (2019), 151–182.
[58]
Junzhe Zhang and Elias Bareinboim. 2018. Fairness in Decision-Making — The Causal Explanation Formula. Proceedings of the AAAI Conference on Artificial Intelligence 32, 1 (2018). https://doi.org/10.1609/aaai.v32i1.11564
Index Terms
- Unlawful Proxy Discrimination: A Framework for Challenging Inherently Discriminatory Algorithms
Recommendations
Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree
FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and TransparencyConcerns regarding unfairness and discrimination in the context of artificial intelligence (AI) systems have recently received increased attention from both legal and computer science scholars. Yet, the degree of overlap between notions of algorithmic ...
Prestimulus oscillations in the alpha band of the eeg are modulated by the difficulty of feature discrimination and predict activation of a sensory discrimination process
Recent work has demonstrated that the occipital-temporal N1 component of the ERP is sensitive to the difficulty of visual discrimination, in a manner that cannot be explained by simple differences in low-level visual features, arousal, or time on task. ...
Timing of Target Discrimination in Human Frontal Eye Fields
Frontal eye field (FEF) neurons discharge in response to behaviorally relevant stimuli that are potential targets for saccades. Distinct visual and motor processes have been dissociated in the FEF of macaque monkeys, but little is known about the visual ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2024
2580 pages
ISBN:9798400704505
DOI:10.1145/3630106
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- ERC Starting Grant ?iManage?
Conference
FAccT '24
FAccT '24: The 2024 ACM Conference on Fairness, Accountability, and Transparency
June 3 - 6, 2024
Rio de Janeiro, Brazil
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 378Total Downloads
- Downloads (Last 12 months)378
- Downloads (Last 6 weeks)73
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in