Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3351095.3372867acmconferencesArticle/Chapter ViewAbstractPublication PagesfacctConference Proceedingsconference-collections
research-article

Implications of AI (un-)fairness in higher education admissions: the effects of perceived AI (un-)fairness on exit, voice and organizational reputation

Published: 27 January 2020 Publication History

Abstract

Algorithmic decision-making (ADM) is becoming increasingly important in all areas of social life. In higher education, machine-learning systems have manifold uses because they can efficiently process large amounts of student data and use these data to arrive at effective decisions. Despite the potential upsides of ADM systems, fairness concerns are gaining momentum in academic and public discourses. The criticism largely focuses on the disparate effects of ADM. That is, algorithms may not serve as objective and fair decision-makers but, rather, reproduce biases existing within the respective training data. This study adopted a different approach by focusing on individual perceptions of fairness. Specifically, we looked at two different dimensions of perceived fairness: (i) procedural fairness and (ii) distributive fairness. Using cross-sectional survey data (n = 304) from a large German university, we tested whether students' assessments of fairness differ with respect to algorithmic vs. human decision-making (HDM) within the higher education context. Furthermore, we investigated whether fairness perceptions have subsequent effects on three different outcome variables, which are hugely important for universities: (1) exit, (2) voice, and (3) organizational reputation. The results of our survey suggest that participants evaluated ADM higher than HDM in terms of both procedural and distributive fairness. Concerning the subsequent effects of fairness perceptions, we find that (1) distributive fairness as well as procedural fairness perceptions have a negative impact on the intention to protest against an ADM system, whereas (2) only procedural fairness perceptions negatively affect the likelihood of exiting. Finally, (3) distributive fairness, but not procedural fairness perceptions have a positive effect on organizational reputation. For universities aiming to implement ADM systems, it is crucial, therefore, to take possible fairness issues and their further implications into account.

References

[1]
Mustafa Acikkar and Mehmet Fatih Akay. 2009. Support vector machines for predicting the admission decision of a candidate to the School of Physical Education and Sports at Cukurova University. Expert Syst. Appl. 36, 3 (April 2009), 7228--7233.
[2]
Theo Araujo, Natali Helberger, Sanne Kruikemeier, and Claes H. de Vreese. 2019. In AI We Trust? Usefulness, Fairness, and Risk Perceptions About Automated Decision-Making by Artificial Intelligence Within Media, Justice, and Health. In ICA Conference.
[3]
Theo Araujo, Claes De Vreese, Natali Helberger, Sanne Kruikemeier, Julia van Weert, Nadine Bol, Daniel Oberski, Mykola Pechenizkiy, Gabi Schaap, and Linnet Taylor. 2018. Automated Decision-Making Fairness in an AI-driven World: Public Perceptions, Hopes and Concerns. Amsterdam.
[4]
Mohsen Attaran, John Stark, and Derek Stotler. 2018. Opportunities and challenges for big data analytics in US higher education: A conceptual model for implementation. Ind. High. Educ. 32, 3 (2018), 169--182.
[5]
Janine Baleis, Birte Keller, Christopher Starke, and Frank Marcinkowski. 2019. Cognitive and Emotional Responses to Fairness in AI - A Systematic Review. Düsseldorf. Retrieved from https://www.phil-fak.uni-duesseldorf.de/fileadmin/Redaktion/Institute/Sozialwissenschaften/Kommunikations_und_Medienwissenschaft/KMW_I/Working_Paper/Baleis_et_al.__2019__Literatur_Review.pdf
[6]
Solon Barocas and Andrew D. Selbst. 2016. Big Data's Disparate Impact. 104 Calif. Law Rev. 671, 2016 (2016).
[7]
Reuben Binns. 2018. Fairness in Machine Learning: Lessons from Political Philosophy. Proc. Mach. Learn. Res. 81, (December 2018), 1--11. Retrieved from http://arxiv.org/abs/1712.03586
[8]
Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. "It's Reducing a Human Being to a Percentage." In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18, 1--14.
[9]
Jeng-Fung Chen and Quang Hung Do. 2014. Training Neural Networks to Predict Student Academic Performance: A Comparison of Cuckoo Search and Gravitational Search Algorithms. Int. J. Comput. Intell. Appl. 13, 01 (March 2014), 1450005.
[10]
Jason A. Colquitt. 2001. On the dimensionality of organizational justice: A construct validation of a measure. J. Appl. Psychol. 86, 3 (June 2001), 386--400.
[11]
Ben Daniel. 2015. Big Data and analytics in higher education: Opportunities and challenges. Br. J. Educ. Technol. 46, 5 (2015), 904--920.
[12]
Lina Dencik, Arne Hintz, Joanna Redden, and Emiliano Treré. 2019. Exploring Data Justice: Conceptions, Applications and Directions. Information, Commun. Soc. 22, 7 (June 2019), 873--881.
[13]
Morton Deutsch. 1975. Equity, Equality, and Need: What Determines Which Value Will Be Used as the Basis of Distributive Justice? J. Soc. Issues (1975).
[14]
Manuela Ekowo and Iris Palmer. 2016. The Promise and Peril of Predictive Analytics in Higher Education. A Landscape Analysis. October (2016).
[15]
Federal Statistical Office. 2019. Education, research and culture - Institutions of higher education. Retrieved from https://www.destatis.de/EN/Themes/Society-Environment/Education-Research-Culture/Institutions-Higher-Education/_node.html#sprg266648
[16]
Nina Grgic-Hlaca, Elissa M. Redmiles, Krishna P. Gummadi, and Adrian Weller. 2018. Human Perceptions of Fairness in Algorithmic Decision Making. In Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18, 903--912.
[17]
Nina Grgic-Hlaca, Muhammad Bilal Zafar, Krishna P Gummadi, and Adrian Weller. 2018. Beyond Distributive Fairness in Algorithmic Decision. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.
[18]
Higher Education Compass. 2019. Degree Programmes. Retrieved December 5, 2019 from https://www.hochschulkompass.de/en/degree-programmes/application-admission/admissions-procedure.html
[19]
Albert O. Hirschmann. 1970. Exit, Voice and Loyalty. Responses to Decline in Firms, Organizations and States. Harvard University Press, Cambridge.
[20]
Anna Lauren Hoffmann. 2019. Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. Information, Commun. Soc. 22, 7 (June 2019), 900--915.
[21]
Nicolas Huck. 2019. Large data sets and machine learning: Applications to statistical arbitrage. Eur. J. Oper. Res. 278, 1 (2019), 330--342.
[22]
Ben Hutchinson and Margaret Mitchell. 2019. 50 Years of Test (Un)fairness. In Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT* '19, 49--58.
[23]
Fei Jiang, Yong Jiang, Hui Zhi, Yi Dong, Hao Li, Sufeng Ma, Yilong Wang, Qiang Dong, Haipeng Shen, and Yongjun Wang. 2017. Artificial intelligence in healthcare: past, present and future. Stroke Vasc. Neurol. 2, 4 (December 2017), 230--243.
[24]
Birte Keller, Janine Baleis, Christopher Starke, and Frank Marcinkowski. 2019. Machine Learning and Artificial Intelligence in Higher Education: A State-of-the-Art Report on the German University Landscape. Düsseldorf. Retrieved from https://www.phil-fak.uni-duesseldorf.de/fileadmin/Redaktion/Institute/Sozialwissenschaften/Kommunikations-_und_Medienwissenschaft/KMW_I/Working_Paper/Keller_et_al.__2019__-_AI_in_Higher_Education.pdf
[25]
Jakko Kemper and Daan Kolkman. 2018. Transparent to whom? No algorithmic accountability without a critical audience. Information, Commun. Soc. (June 2018), 1--16.
[26]
Min Kyung Lee. 2018. Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data Soc. 5, 1 (June 2018), 205395171875668.
[27]
Gerald S. Leventhal. 1980. What should be done with equity theory? New approaches to the study of fairness in social relationships. In Social exchange: Advances in theory and research, K.J. Gergen, M.S. Greenberg and R.H. Willis (eds.). Springer, Boston, 27--55.
[28]
Jennifer M. Logg, Julia A. Minson, and Don A. Moore. 2019. Algorithm appreciation: People prefer algorithmic to human judgment. Organ. Behav. Hum. Decis. Process. (2019).
[29]
Frank Marcinkowski and Christopher Starke. 2019. Wann ist Künstliche Intelligenz (un-)fair? Ein sozial wissenschaftliches Konzept von KI-Fairness [When is Artificial Intelligence (Un-)Fair? A Social Science Concept of AI-Fairness. In Politik in der digitalen Gesellschaft. Zentrale Problemfelder und Forschungsperspektiven [Politics in the Digital Society. Important Problems and Research Perspectives], Jeanette Hofmann, Norbert Kersting, Claudia Ritzi and Wolf J. Schünemann (eds.). transcript, Bielefeld, 269--288.
[30]
Paul K. McClure. 2018. "You're Fired," Says the Robot. Soc. Sci. Comput. Rev. 36, 2 (April 2018), 139--156.
[31]
Neema Mduma, Khamisi Kalegele, and Dina Machuve. 2019. A Survey of Machine Learning Approaches and Techniques for Student Dropout Prediction. Data Sci. J. 18, 1 (April 2019).
[32]
Leonard J Ponzi, Charles J Fombrun, and Naomi A Gardberg. 2011. RepTrak™ Pulse: Conceptualizing and Validating a Short-Form Measure of Corporate Reputation. Corp. Reput. Rev. 14, 1 (May 2011), 15--35.
[33]
Donghee Shin and Yong Jin Park. 2019. Role of fairness, accountability, and transparency in algorithmic affordance. Comput. Human Behav. 98, (September 2019), 277--284.
[34]
Megha Srivastava, Hoda Heidari, and Andreas Krause. 2019. Mathematical Notions vs. Human Perception of Fairness. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining - KDD '19, 2459--2468.
[35]
Linnet Taylor. 2017. What is data justice? The case for connecting digital rights and freedoms globally. Big Data Soc. 4, 2 (December 2017), 205395171773633.
[36]
Eric J. Topol. 2019. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25, 1 (January 2019), 44--56.
[37]
Sahil Verma and Julia Rubin. 2018. Fairness definitions explained. In Proceedings of the International Workshop on Software Fairness - FairWare '18, 1--7.
[38]
Bernd W. Wirtz, Jan C. Weyerer, and Carolin Geyer. 2019. Artificial Intelligence and the Public Sector---Applications and Challenges. Int. J. Public Adm. 42, 7 (May 2019), 596--615.
[39]
Tan Fee Yean and Ab Aziz Yusof. 2016. Organizational Justice: A Conceptual Discussion. Procedia - Soc. Behav. Sci. 219, (May 2016), 798--803.
[40]
Olaf Zawacki-Richter, Victoria I. Marin, Melissa Bond, and Franziska Gouverneur. 2019. Systematic review of research on artificial intelligence applications in higher education - where are the educators? Int. J. Educ. Technol. High. Educ. 16, 39 (December 2019).
[41]
Jonathan Zittrain. 2019. The Hidden Costs of Automated Thinking. The New Yorker. Retrieved July 29, 2019 from https://www.newyorker.com/tech/annals-of-technology/the-hidden-costs-of-automated-thinking

Cited By

View all
  • (2024)Exploration of AI Precision in Higher EducationAI-Driven Learning and Engagement in Higher Education10.4018/979-8-3693-4074-5.ch011(273-298)Online publication date: 20-Sep-2024
  • (2024)Addressing Issues and Challenges Using AI in PharmacyAdvances in Computational Intelligence for the Healthcare Industry 4.010.4018/979-8-3693-2333-5.ch002(22-41)Online publication date: 7-Jun-2024
  • (2024)Assessing Disparities in Predictive Modeling Outcomes for College Student Success: The Impact of Imputation Techniques on Model Performance and FairnessEducation Sciences10.3390/educsci1402013614:2(136)Online publication date: 29-Jan-2024
  • Show More Cited By

Index Terms

  1. Implications of AI (un-)fairness in higher education admissions: the effects of perceived AI (un-)fairness on exit, voice and organizational reputation

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
        January 2020
        895 pages
        ISBN:9781450369367
        DOI:10.1145/3351095
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 27 January 2020

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. algorithmic decision making
        2. artificial intelligence
        3. distributive fairness
        4. exit
        5. higher education systems
        6. procedural fairness
        7. reputation
        8. voice

        Qualifiers

        • Research-article

        Funding Sources

        Conference

        FAT* '20
        Sponsor:

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)729
        • Downloads (Last 6 weeks)77
        Reflects downloads up to 09 Nov 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Exploration of AI Precision in Higher EducationAI-Driven Learning and Engagement in Higher Education10.4018/979-8-3693-4074-5.ch011(273-298)Online publication date: 20-Sep-2024
        • (2024)Addressing Issues and Challenges Using AI in PharmacyAdvances in Computational Intelligence for the Healthcare Industry 4.010.4018/979-8-3693-2333-5.ch002(22-41)Online publication date: 7-Jun-2024
        • (2024)Assessing Disparities in Predictive Modeling Outcomes for College Student Success: The Impact of Imputation Techniques on Model Performance and FairnessEducation Sciences10.3390/educsci1402013614:2(136)Online publication date: 29-Jan-2024
        • (2024)Bridging the Human-Automation Fairness Gap: How Providing Reasons Enhances the Perceived Fairness of Public Decision-MakingSSRN Electronic Journal10.2139/ssrn.4819145Online publication date: 2024
        • (2024)Does Artificial Intelligence Help or Hurt Gender Diversity? Evidence from Two Field Experiments on Recruitment in TechSSRN Electronic Journal10.2139/ssrn.4764343Online publication date: 2024
        • (2024)Trade-offs in AI assistant choice: Do consumers prioritize transparency and sustainability over AI assistant performance?Big Data & Society10.1177/2053951724129021711:4Online publication date: 29-Oct-2024
        • (2024)Perceiving AI intervention does not compromise the persuasive effect of fact-checkingNew Media & Society10.1177/14614448241286881Online publication date: 21-Oct-2024
        • (2024)For Me or Against Me? Reactions to AI (vs. Human) Decisions That Are Favorable or Unfavorable to the Self and the Role of Fairness PerceptionPersonality and Social Psychology Bulletin10.1177/01461672241288338Online publication date: 24-Oct-2024
        • (2024)Algorithm Bias and Perceived Fairness: A Comprehensive Scoping ReviewProceedings of the 2024 Computers and People Research Conference10.1145/3632634.3655848(1-9)Online publication date: 29-May-2024
        • (2024)Explainable Artificial Intelligence for Academic Performance Prediction. An Experimental Study on the Impact of Accuracy and Simplicity of Decision Trees on Causability and Fairness PerceptionsProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658953(1031-1042)Online publication date: 3-Jun-2024
        • Show More Cited By

        View Options

        Get Access

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media