Democratizing Algorithmic Fairness

Machine learning algorithms can now identify patterns and correlations in (big) datasets and predict outcomes based on the identified patterns and correlations. They can then generate decisions in accordance with the outcomes predicted, and decision-making processes can thereby be automated. Algorithms can inherit questionable values from datasets and acquire biases in the course of (machine) learning. While researchers and developers have taken the problem of algorithmic bias seriously, the development of fair algorithms is primarily conceptualized as a technical task. In this paper, I discuss the limitations and risks of this view. Since decisions on “fairness measure” and the related techniques for fair algorithms essentially involve choices between competing values, “fairness” in algorithmic fairness should be conceptualized first and foremost as a political question and be resolved politically. In short, this paper aims to foreground the political dimension of algorithmic fairness and supplement the current discussion with a deliberative approach to algorithmic fairness based on the accountability for reasonableness framework (AFR).

1. For an overview of major approaches to assess the values embedded in information technology, see Brey (2010).

2. The media have reported many cases of (potential) harm from algorithmic decision-making, but the racial bias in the COMPAS recidivism algorithm reported by ProPublica (Angwin et al. 2016; Angwin and Larson 2016), along with Northpointe’s (now renamed to “equivant”) response to ProPublica’s report (Dieterich et al. 2016), have arguably generated the most discussion. The COMPAS recidivism algorithm has since become the paradigmatic case for research on algorithmic bias, with various research citing it as their motivation or using it as a benchmark. Also, see O’Neil (2016) for an accessible discussion of other cases of algorithmic bias.

3. For a recent overview of the current approaches to algorithmic fairness and different techniques to achieve algorithmic fairness, see Lepri et al. (2018) and Friedler et al. (2019).

4. This is not to claim that the presumed ideas of fairness are unreasonable or idiosyncratic. In fact, some researchers have explicitly referred to the social or legal understandings of fairness in constructing their fairness measures. Still, it is the researchers’ choice to rely on a specific understanding of fairness, but not the others, for their fairness measures, and their choice is rarely informed by the public. I shall return to this point in my discussion of the AFR-based framework.

5. For example, Corbett-Davies et al.’s (2017) analysis of the COMPAS recidivism algorithm refers to three definitions of fairness, i.e., statistical parity, conditional statistical parity, and predictive equality. Berk et al.’s (2018) review of fairness in criminal justice risk assessments refers to six definitions of fairness, i.e., overall accuracy equality, statistical parity, conditional procedure accuracy equality, conditional use accuracy equality, treatment equality, and total fairness. Mitchell and Shadlen’s (2017) recent summary includes 19 definitions of fairness, and a recent talk by Arvind Narayanan (2018) has increased the number of definitions to 21.

6. National or international legislation against discrimination may supply the meaning of fairness to researchers and developers for their design and implementation of algorithms. However, there are two potential shortcomings in grounding the “fairness” in fair algorithms on national and international legislation. Firstly, the capacity of algorithms to identify patterns and correlations may engender new types of discrimination that are not based on common protected features, e.g., races and genders. Accordingly, the existing legislation is likely to be insufficient. Secondly, national and international legislation is often difficult and slow to change. Therefore, the idea of “fairness” in algorithmic fairness is likely to be conservative if it is based on the legislation. Of course, national and international legislation remains important to algorithmic fairness for identifying common types of discrimination.

7. For instance, the reason to opt for a specific definition of fairness is often left unarticulated or implicit in the research, except for a few notable exceptions in which researchers and developers acknowledge or reflect on the normative ground of their choice of definition(s). See, e.g., Dwork et al. (2012) and Lipton et al. (2018).

8. It is not entirely accurate to describe the incompatibility among different definitions of fairness as “the impossibility theorem.” There are indeed situations where some of the definitions of fairness in question can be satisfied simultaneously, but these situations are highly unrealistic, e.g., when we have perfect predictor or trivial predictor that is either always-positive or always-negative (Miconi 2017).

9. This is not intended to be a knock-down argument against viewing algorithmic fairness primarily as a technical challenge. However, as I have argued the focus on technical tasks can lead to a less critical attitude towards one’s idea of “fairness,” it is more likely that researchers and developers who see algorithmic fairness primarily as a technical challenge are less sensitive to the contentious nature of the definition of fairness.

10. There is an important distinction between actualized harm and risk of harm to be made in the discussion on the fair distribution of risk, see Hayenhjelm (2012) and Hayenhjelm and Wolff (2012). The debate on risk and distributive justice is out of the scope here, but my argument only relies on the assumption that the distribution of risk and benefit is, in fact, an issue of fairness.

11. Here, the claim about unfairness could at least be grounded on (i) a consequentialist perspective and (ii) a rights-based perspective. From the consequentialist perspective, the unfairness is due to a reduction of overall social good, whereas from the rights-based perspective, individuals have prima facie rights not to be exposed to a risk of harm (see Hayenhjelm and Wolff 2012).

12. In this respect, the increasing number of researchers being more explicit about the values and normative grounds of various definitions of fairness is a welcoming trend in the research on algorithmic fairness (see, e.g., Dwork et al. (2012); Friedler et al. (2016), Berk et al. (2018), Narayanan (2018)).

13. Hansson (2006) has forcibly questioned the applicability of (informed) consent in non-individualistic contexts. Here, the discussion is by no means an argument for the role of (informed) consent in justifying the imposition of risk by algorithms, but it is merely an example of the kind of ethical issues that may arise.

14. If one considers every use of algorithmic decision-making to be morally impermissible, then concerns over fairness in algorithms will cease to exist. The project of achieving fair algorithms presupposes some uses of algorithms to be morally permissible.

15. However, even if there is no disagreement among different groups of stakeholders, I take it that the AFR-inspired framework I outline can enhance the “fairness” of the decision.

16. My discussion only requires there to be at least some choices that are equally justifiable and thereby leading to the requirement for justifying one justifiable choice over another equally justifiable choice.

17. For Rawls, the fact of reasonable pluralism amount to “a pluralism of comprehensive religious, philosophical, and moral doctrines […] a pluralism of incompatible yet reasonable comprehensive doctrines” (Rawls 1993, p. xvi).

18. Rawls argues that despite there are differences in reasonable comprehensive doctrines, individuals in the society could still achieve mutual agreement on a political conception of justice through overlapping consensus, that is, individuals subscribe to different comprehensive doctrines can agree on the political conception of justice with their own reasons and from their own moral points of view (cf. Rawls 1993, p. 134). Yet, the agreement on the political conception of justice is necessarily thin, and thus, it is insufficient to supply fine-grained normative principles to settle substantive value-related issues, e.g., prioritizing the interests of different groups of stakeholders (cf. Daniels 1993).

19. Daniels and Sabin first proposed AFR in Daniels and Sabin (1997), and Daniels has since defended and applied AFR on various healthcare issues with Sabin and other colleagues. Note that this paper is not an exposition of AFR, and I shall not attempt to survey the extensive discussion on AFR. My discussion of AFR refers primarily to Daniels and Sabin (2008), which incorporate the earlier works on AFR and present the most systematic account of it. However, I shall also refer to earlier works on AFR when I consider them to be more relevant on a specific point under discussion.

20. The formulation of the four conditions I quoted is slightly different from the one presented in Daniels and Sabin (2008, p. 45). I refer to this formulation because it is explicitly targeted at the problem of priority-setting, and, as I point out, the choice of fairness measure and balance between fairness and accuracy can be viewed as a priority-setting problem.

21. Veale and Binns (2017) rightly point out that there are practical difficulties for private organizations to explicate the consequences of an algorithm and its distributional implications, for private organizations may not, or even are not, allowed to possess and process relevant data for such endeavors. I think, however, the responses Veale and Binns provided in their paper can resolve the practical difficulties. In this paper, I cannot discuss their responses in detail, but the proposed responses are compatible with the AFR-inspired framework I develop in here.

22. It is useful to caution that both Badano’s Full Acceptability condition and Daniels and Sabin’s Relevance condition risk over-intellectualized public deliberation and thereby excluding views and voices that are not presented in a rational, argumentative form. Similarly, implicit in the Full Acceptability condition, the importance of achieving consensus, which, in turn, can lead to a suppression of differences. In response to the two concerns, it is useful to explore whether Young’s (2000) communicative democracy can broaden the inclusion of views and voices by introducing other modes of communication in public deliberation, e.g., greeting, rhetoric, and narrative; and, whether Young’s ideal of differentiated solidarity based on mutual respect and caring but not mutual identification can avoid the suppression of differences (Young 2000, pp. 221–228).

23. The more fundamental questions for the AFR-based framework, therefore, are about (i) the normative and practical viability of deliberative democracy and (ii) the proper scope of it. In other words, a more comprehensive account of the AFR-based framework requires one to defend deliberative democracy as a better alternative than other forms of democracy and to work out the institutional arrangements where individuals’ views and voices can be adequately communicated. It must also specify whose views and voices are to be included, e.g., citizens vs. non-citizens in the democratic society, and what questions are open for democratic deliberation, e.g., national security issues. Debates on theoretical and practical aspects of deliberative democracy have generated an enormous amount of research that I cannot summarize in this paper, but I shall acknowledge the significant role deliberative democracy in normatively grounding my AFR-based framework. For a review of the prospect of deliberative democracy, see Curato et al. (2017).

24. Binns (2018b) is an important exception to this claim, where he explores the phenomenon of algorithmic accountability in terms of the democratic ideal of public reason. While there are affinities between my discussion and Binns’ account, there are two important differences. Firstly, I attempt to demonstrate the political dimension in the problem of algorithmic fairness is due to its internal features, particularly the impossibility theorem and the inherent trade-off between fairness and accuracy. Secondly, I attempt to offer a specific approach to ground decision-makers’ accountability with Daniels and Sabin’s AFR.

25. The other requirements listed in the report are related to “Accuracy, Validity, and Bias,” i.e., “Requirement 1: training datasets must measure the intended variables,” “Requirement 2: bias in statistical models must be measured and mitigated,” and “Requirement 3: tools must not conflate multiple distinct predictions” and to “Human-Computer Interface Issues,” i.e., “Requirement 4: predictions and how they are made must be easily interpretable,” “Requirement 5: tools should produce confidence estimates for their predictions,” and “Requirement 6: users of risk assessment tools must attend trainings on the nature and limitations of the tools.”

26. This is not the only possible mapping of the four conditions with the policy goals of AIAs and requirements in the report by Partnership on AI. The aim of this exercise is to demonstrate the affinity of the AFR-based framework with major ethical and governance principles.

27. For example, see Ney and Verweij (2015) for an excellent discussion of different methods to engage the public and to accommodate the normative principles and values of different, conflicting worldviews in relation to wicked problems, but also see Hagendijk and Irwin (2006) for a discussion about the difficulties for public deliberation and deliberative democracy in science and technology policies.

Authors and Affiliations

1. Department of Informatics, Universität Hamburg, Vogt-Kölln-Straße 30, 22527, Hamburg, Germany

   Pak-Hang Wong

Corresponding author

Correspondence to Pak-Hang Wong.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Reprints and permissions