Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3351095.3372867acmconferencesArticle/Chapter ViewAbstractPublication PagesfacctConference Proceedingsconference-collections
research-article

Implications of AI (un-)fairness in higher education admissions: the effects of perceived AI (un-)fairness on exit, voice and organizational reputation

Published: 27 January 2020 Publication History

Abstract

Algorithmic decision-making (ADM) is becoming increasingly important in all areas of social life. In higher education, machine-learning systems have manifold uses because they can efficiently process large amounts of student data and use these data to arrive at effective decisions. Despite the potential upsides of ADM systems, fairness concerns are gaining momentum in academic and public discourses. The criticism largely focuses on the disparate effects of ADM. That is, algorithms may not serve as objective and fair decision-makers but, rather, reproduce biases existing within the respective training data. This study adopted a different approach by focusing on individual perceptions of fairness. Specifically, we looked at two different dimensions of perceived fairness: (i) procedural fairness and (ii) distributive fairness. Using cross-sectional survey data (n = 304) from a large German university, we tested whether students' assessments of fairness differ with respect to algorithmic vs. human decision-making (HDM) within the higher education context. Furthermore, we investigated whether fairness perceptions have subsequent effects on three different outcome variables, which are hugely important for universities: (1) exit, (2) voice, and (3) organizational reputation. The results of our survey suggest that participants evaluated ADM higher than HDM in terms of both procedural and distributive fairness. Concerning the subsequent effects of fairness perceptions, we find that (1) distributive fairness as well as procedural fairness perceptions have a negative impact on the intention to protest against an ADM system, whereas (2) only procedural fairness perceptions negatively affect the likelihood of exiting. Finally, (3) distributive fairness, but not procedural fairness perceptions have a positive effect on organizational reputation. For universities aiming to implement ADM systems, it is crucial, therefore, to take possible fairness issues and their further implications into account.

References

[1]
Mustafa Acikkar and Mehmet Fatih Akay. 2009. Support vector machines for predicting the admission decision of a candidate to the School of Physical Education and Sports at Cukurova University. Expert Syst. Appl. 36, 3 (April 2009), 7228--7233.
[2]
Theo Araujo, Natali Helberger, Sanne Kruikemeier, and Claes H. de Vreese. 2019. In AI We Trust? Usefulness, Fairness, and Risk Perceptions About Automated Decision-Making by Artificial Intelligence Within Media, Justice, and Health. In ICA Conference.
[3]
Theo Araujo, Claes De Vreese, Natali Helberger, Sanne Kruikemeier, Julia van Weert, Nadine Bol, Daniel Oberski, Mykola Pechenizkiy, Gabi Schaap, and Linnet Taylor. 2018. Automated Decision-Making Fairness in an AI-driven World: Public Perceptions, Hopes and Concerns. Amsterdam.
[4]
Mohsen Attaran, John Stark, and Derek Stotler. 2018. Opportunities and challenges for big data analytics in US higher education: A conceptual model for implementation. Ind. High. Educ. 32, 3 (2018), 169--182.
[5]
Janine Baleis, Birte Keller, Christopher Starke, and Frank Marcinkowski. 2019. Cognitive and Emotional Responses to Fairness in AI - A Systematic Review. Düsseldorf. Retrieved from https://www.phil-fak.uni-duesseldorf.de/fileadmin/Redaktion/Institute/Sozialwissenschaften/Kommunikations_und_Medienwissenschaft/KMW_I/Working_Paper/Baleis_et_al.__2019__Literatur_Review.pdf
[6]
Solon Barocas and Andrew D. Selbst. 2016. Big Data's Disparate Impact. 104 Calif. Law Rev. 671, 2016 (2016).
[7]
Reuben Binns. 2018. Fairness in Machine Learning: Lessons from Political Philosophy. Proc. Mach. Learn. Res. 81, (December 2018), 1--11. Retrieved from http://arxiv.org/abs/1712.03586
[8]
Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. "It's Reducing a Human Being to a Percentage." In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18, 1--14.
[9]
Jeng-Fung Chen and Quang Hung Do. 2014. Training Neural Networks to Predict Student Academic Performance: A Comparison of Cuckoo Search and Gravitational Search Algorithms. Int. J. Comput. Intell. Appl. 13, 01 (March 2014), 1450005.
[10]
Jason A. Colquitt. 2001. On the dimensionality of organizational justice: A construct validation of a measure. J. Appl. Psychol. 86, 3 (June 2001), 386--400.
[11]
Ben Daniel. 2015. Big Data and analytics in higher education: Opportunities and challenges. Br. J. Educ. Technol. 46, 5 (2015), 904--920.
[12]
Lina Dencik, Arne Hintz, Joanna Redden, and Emiliano Treré. 2019. Exploring Data Justice: Conceptions, Applications and Directions. Information, Commun. Soc. 22, 7 (June 2019), 873--881.
[13]
Morton Deutsch. 1975. Equity, Equality, and Need: What Determines Which Value Will Be Used as the Basis of Distributive Justice? J. Soc. Issues (1975).
[14]
Manuela Ekowo and Iris Palmer. 2016. The Promise and Peril of Predictive Analytics in Higher Education. A Landscape Analysis. October (2016).
[15]
Federal Statistical Office. 2019. Education, research and culture - Institutions of higher education. Retrieved from https://www.destatis.de/EN/Themes/Society-Environment/Education-Research-Culture/Institutions-Higher-Education/_node.html#sprg266648
[16]
Nina Grgic-Hlaca, Elissa M. Redmiles, Krishna P. Gummadi, and Adrian Weller. 2018. Human Perceptions of Fairness in Algorithmic Decision Making. In Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18, 903--912.
[17]
Nina Grgic-Hlaca, Muhammad Bilal Zafar, Krishna P Gummadi, and Adrian Weller. 2018. Beyond Distributive Fairness in Algorithmic Decision. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.
[18]
Higher Education Compass. 2019. Degree Programmes. Retrieved December 5, 2019 from https://www.hochschulkompass.de/en/degree-programmes/application-admission/admissions-procedure.html
[19]
Albert O. Hirschmann. 1970. Exit, Voice and Loyalty. Responses to Decline in Firms, Organizations and States. Harvard University Press, Cambridge.
[20]
Anna Lauren Hoffmann. 2019. Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. Information, Commun. Soc. 22, 7 (June 2019), 900--915.
[21]
Nicolas Huck. 2019. Large data sets and machine learning: Applications to statistical arbitrage. Eur. J. Oper. Res. 278, 1 (2019), 330--342.
[22]
Ben Hutchinson and Margaret Mitchell. 2019. 50 Years of Test (Un)fairness. In Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT* '19, 49--58.
[23]
Fei Jiang, Yong Jiang, Hui Zhi, Yi Dong, Hao Li, Sufeng Ma, Yilong Wang, Qiang Dong, Haipeng Shen, and Yongjun Wang. 2017. Artificial intelligence in healthcare: past, present and future. Stroke Vasc. Neurol. 2, 4 (December 2017), 230--243.
[24]
Birte Keller, Janine Baleis, Christopher Starke, and Frank Marcinkowski. 2019. Machine Learning and Artificial Intelligence in Higher Education: A State-of-the-Art Report on the German University Landscape. Düsseldorf. Retrieved from https://www.phil-fak.uni-duesseldorf.de/fileadmin/Redaktion/Institute/Sozialwissenschaften/Kommunikations-_und_Medienwissenschaft/KMW_I/Working_Paper/Keller_et_al.__2019__-_AI_in_Higher_Education.pdf
[25]
Jakko Kemper and Daan Kolkman. 2018. Transparent to whom? No algorithmic accountability without a critical audience. Information, Commun. Soc. (June 2018), 1--16.
[26]
Min Kyung Lee. 2018. Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data Soc. 5, 1 (June 2018), 205395171875668.
[27]
Gerald S. Leventhal. 1980. What should be done with equity theory? New approaches to the study of fairness in social relationships. In Social exchange: Advances in theory and research, K.J. Gergen, M.S. Greenberg and R.H. Willis (eds.). Springer, Boston, 27--55.
[28]
Jennifer M. Logg, Julia A. Minson, and Don A. Moore. 2019. Algorithm appreciation: People prefer algorithmic to human judgment. Organ. Behav. Hum. Decis. Process. (2019).
[29]
Frank Marcinkowski and Christopher Starke. 2019. Wann ist Künstliche Intelligenz (un-)fair? Ein sozial wissenschaftliches Konzept von KI-Fairness [When is Artificial Intelligence (Un-)Fair? A Social Science Concept of AI-Fairness. In Politik in der digitalen Gesellschaft. Zentrale Problemfelder und Forschungsperspektiven [Politics in the Digital Society. Important Problems and Research Perspectives], Jeanette Hofmann, Norbert Kersting, Claudia Ritzi and Wolf J. Schünemann (eds.). transcript, Bielefeld, 269--288.
[30]
Paul K. McClure. 2018. "You're Fired," Says the Robot. Soc. Sci. Comput. Rev. 36, 2 (April 2018), 139--156.
[31]
Neema Mduma, Khamisi Kalegele, and Dina Machuve. 2019. A Survey of Machine Learning Approaches and Techniques for Student Dropout Prediction. Data Sci. J. 18, 1 (April 2019).
[32]
Leonard J Ponzi, Charles J Fombrun, and Naomi A Gardberg. 2011. RepTrak™ Pulse: Conceptualizing and Validating a Short-Form Measure of Corporate Reputation. Corp. Reput. Rev. 14, 1 (May 2011), 15--35.
[33]
Donghee Shin and Yong Jin Park. 2019. Role of fairness, accountability, and transparency in algorithmic affordance. Comput. Human Behav. 98, (September 2019), 277--284.
[34]
Megha Srivastava, Hoda Heidari, and Andreas Krause. 2019. Mathematical Notions vs. Human Perception of Fairness. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining - KDD '19, 2459--2468.
[35]
Linnet Taylor. 2017. What is data justice? The case for connecting digital rights and freedoms globally. Big Data Soc. 4, 2 (December 2017), 205395171773633.
[36]
Eric J. Topol. 2019. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25, 1 (January 2019), 44--56.
[37]
Sahil Verma and Julia Rubin. 2018. Fairness definitions explained. In Proceedings of the International Workshop on Software Fairness - FairWare '18, 1--7.
[38]
Bernd W. Wirtz, Jan C. Weyerer, and Carolin Geyer. 2019. Artificial Intelligence and the Public Sector---Applications and Challenges. Int. J. Public Adm. 42, 7 (May 2019), 596--615.
[39]
Tan Fee Yean and Ab Aziz Yusof. 2016. Organizational Justice: A Conceptual Discussion. Procedia - Soc. Behav. Sci. 219, (May 2016), 798--803.
[40]
Olaf Zawacki-Richter, Victoria I. Marin, Melissa Bond, and Franziska Gouverneur. 2019. Systematic review of research on artificial intelligence applications in higher education - where are the educators? Int. J. Educ. Technol. High. Educ. 16, 39 (December 2019).
[41]
Jonathan Zittrain. 2019. The Hidden Costs of Automated Thinking. The New Yorker. Retrieved July 29, 2019 from https://www.newyorker.com/tech/annals-of-technology/the-hidden-costs-of-automated-thinking

Cited By

View all

Index Terms

  1. Implications of AI (un-)fairness in higher education admissions: the effects of perceived AI (un-)fairness on exit, voice and organizational reputation

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
        January 2020
        895 pages
        ISBN:9781450369367
        DOI:10.1145/3351095
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 27 January 2020

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. algorithmic decision making
        2. artificial intelligence
        3. distributive fairness
        4. exit
        5. higher education systems
        6. procedural fairness
        7. reputation
        8. voice

        Qualifiers

        • Research-article

        Funding Sources

        Conference

        FAT* '20
        Sponsor:

        Upcoming Conference

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)653
        • Downloads (Last 6 weeks)80
        Reflects downloads up to 05 Mar 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2025)Higher Education Act for AI (HEAT-AI): a framework to regulate the usage of AI in higher education institutionsFrontiers in Education10.3389/feduc.2025.150537010Online publication date: 20-Feb-2025
        • (2025)Employees’ perceptions of the fairness of AI-based performance prediction featuresCogent Business & Management10.1080/23311975.2025.245611112:1Online publication date: 22-Jan-2025
        • (2025)Fairness for machine learning software in educationJournal of Systems and Software10.1016/j.jss.2024.112244219:COnline publication date: 1-Jan-2025
        • (2025)Harnessing AI for sustainable higher education: ethical considerations, operational efficiency, and future directionsDiscover Sustainability10.1007/s43621-025-00809-66:1Online publication date: 13-Jan-2025
        • (2024)AI in K-12 EducationImproving Student Assessment With Emerging AI Tools10.4018/979-8-3693-6170-2.ch007(169-212)Online publication date: 29-Nov-2024
        • (2024)Exploration of AI Precision in Higher EducationAI-Driven Learning and Engagement in Higher Education10.4018/979-8-3693-4074-5.ch011(273-298)Online publication date: 20-Sep-2024
        • (2024)Addressing Issues and Challenges Using AI in PharmacyAdvances in Computational Intelligence for the Healthcare Industry 4.010.4018/979-8-3693-2333-5.ch002(22-41)Online publication date: 7-Jun-2024
        • (2024)THE IMPACT OF PUBLIC RELATIONS ON STRENGTHENING EDUCATIONAL INSTITUTIONS’ REPUTATIONS: KING KHALID UNIVERSITY AS A MODELJournal of Southwest Jiaotong University10.35741/issn.0258-2724.59.4.359:4Online publication date: 2024
        • (2024)Assessing Disparities in Predictive Modeling Outcomes for College Student Success: The Impact of Imputation Techniques on Model Performance and FairnessEducation Sciences10.3390/educsci1402013614:2(136)Online publication date: 29-Jan-2024
        • (2024)Fair and Transparent Student Admission Prediction Using Machine Learning ModelsAlgorithms10.3390/a1712057217:12(572)Online publication date: 13-Dec-2024
        • Show More Cited By

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media