Inclusive Portraits: Race-Aware Human-in-the-Loop Technology
Article No.: 15, Pages 1 - 11
Abstract
AI has revolutionized the processing of various services, including the automatic facial verification of people. Automated approaches have demonstrated their speed and efficiency in verifying a large volume of faces, but they can face challenges when processing content from certain communities, including communities of people of color. This challenge has prompted the adoption of "human-in-the-loop" (HITL) approaches, where human workers collaborate with the AI to minimize errors. However, most HITL approaches do not consider workers’ individual characteristics and backgrounds. This paper proposes a new approach, called Inclusive Portraits (IP), that connects with social theories around race to design a racially-aware human-in-the-loop system. Our experiments have provided evidence that incorporating race into human-in-the-loop (HITL) systems for facial verification can significantly enhance performance, especially for services delivered to people of color. Our findings also highlight the importance of considering individual worker characteristics in the design of HITL systems, rather than treating workers as a homogenous group. Our research has significant design implications for developing AI-enhanced services that are more inclusive and equitable.
References
[1]
[n. d.]. 2022 Biometric Technology Rally - The Maryland Test Facility. https://mdtf.org/Rally2022.
[2]
[n. d.]. FRVT 1:1 Verification - NIST. https://pages.nist.gov/frvt/html/frvt11.html.
[3]
Ghaliya Al Farsi, Jasiya Jabbar, and Ragad M Tawafak. 2019. A Review on models of human face verification techniques. In 2019 International Conference on Fourth Industrial Revolution (ICFIR). IEEE, 1–5.
[4]
Adriana Alvarado Garcia, Juan F Maestre, Manuhuia Barcham, Marilyn Iriarte, Marisol Wong-Villacres, Oscar A Lemus, Palak Dudani, Pedro Reynolds-Cuéllar, Ruotong Wang, and Teresa Cerratto Pargman. 2021. Decolonial Pathways: Our Manifesto for a Decolonizing Agenda in HCI Research and Design. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1–9.
[5]
Neetu Anand, Tripti Sharma, and Kumar Gaurav. 2022. Face Recognition for Examinee Verification.International Journal of Next-Generation Computing 13, 3 (2022).
[6]
Mohd Anwar, Hilda Goins, Tianyang Zhang, and Xiangfeng Dai. 2022. Human Elements in Machine Learning-Based Solutions to Cybersecurity. In Smart and Intelligent Systems. CRC Press, 137–148.
[7]
Okereke Eze Aru and Ihekweaba Gozie. 2013. Facial verification technology for use in ATM transactions. American Journal of Engineering Research (AJER) 2, 5 (2013), 188–193.
[8]
Lee David Baker. 1994. The role of anthropology in the social construction of race, 1896-1954. Temple University.
[9]
TV Bakunova, EA Trofimova, and EV Lapteva. 2019. Biometrics as a method of information security in the banking sector digitalization. In International Scientific and Practical Conference on Digital Economy (ISCDE 2019). Atlantis Press, 929–934.
[10]
Stephen Balaban. 2015. Deep learning and face recognition: the state of the art. Biometric and surveillance technology for human and activity identification XII 9457 (2015), 68–75.
[11]
Agathe Balayn, Christoph Lofi, and Geert-Jan Houben. 2021. Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems. The VLDB Journal 30, 5 (2021), 739–768.
[12]
Michael J Bamshad and Steve E Olson. 2003. Does race exist. Scientific American 289, 6 (2003), 78–85.
[13]
Shaowen Bardzell, Jeffrey Bardzell, Jodi Forlizzi, John Zimmerman, and John Antanitis. 2012. Critical design and critical theory: the challenge of designing for provocation. In Proceedings of the Designing Interactive Systems Conference. 288–297.
[14]
Ruha Benjamin. 2019. Race after technology: Abolitionist tools for the new jim code. Social forces (2019).
[15]
Cynthia L Bennett, Cole Gleason, Morgan Klaus Scheuerman, Jeffrey P Bigham, Anhong Guo, and Alexandra To. 2021. “It’s Complicated”: Negotiating Accessibility and (Mis) Representation in Image Descriptions of Race, Gender, and Disability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–19.
[16]
Austin Blanton, Kristen C Allen, Timothy Miller, Nathan D Kalka, and Anil K Jain. 2016. A comparison of human and automated face verification accuracy on unconstrained image sets. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 161–168.
[17]
Joy Adowaa Buolamwini. 2017. Gender shades: intersectional phenotypic and demographic evaluation of face datasets and gender classifiers. Ph. D. Dissertation. Massachusetts Institute of Technology.
[18]
Jacqueline G Cavazos, P Jonathon Phillips, Carlos D Castillo, and Alice J O’Toole. 2020. Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?IEEE transactions on biometrics, behavior, and identity science 3, 1 (2020), 101–111.
[19]
Yiqun T Chen, Angela DR Smith, Katharina Reinecke, and Alexandra To. 2023. Why, when, and from whom: considerations for collecting and reporting race and ethnicity data in HCI. In CHI Conference on Human Factors in Computing Systems.
[20]
Chun-Wei Chiang, Anna Kasunic, and Saiph Savage. 2018. Crowd coach: Peer coaching for crowd workers’ skill growth. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1–17.
[21]
Rodney D Coates. 2004. Race and ethnicity: Across time, space and discipline. Brill.
[22]
Patricia A Curtin and T Kenn Gaither. 2005. Privileging identity, difference, and power: The circuit of culture as a basis for public relations theory. Journal of Public Relations Research 17, 2 (2005), 91–115.
[23]
Roxana Daneshjou, Mary P Smith, Mary D Sun, Veronica Rotemberg, and James Zou. 2021. Lack of transparency and potential bias in artificial intelligence data sets and algorithms: a scoping review. JAMA dermatology 157, 11 (2021), 1362–1369.
[24]
Steven Dow, Anand Kulkarni, Brie Bunge, Truc Nguyen, Scott Klemmer, and Björn Hartmann. 2011. Shepherding the crowd: managing and providing feedback to crowd workers. In CHI’11 Extended Abstracts on Human Factors in Computing Systems. 1669–1674.
[25]
Paul Du Gay, Stuart Hall, Linda Janes, Anders Koed Madsen, Hugh Mackay, and Keith Negus. 2013. Doing cultural studies: The story of the Sony Walkman. Sage.
[26]
Virginia Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.
[27]
Huaying Fang, Qin Hui, Julie Lynch, Jacqueline Honerlaw, Themistocles L Assimes, Jie Huang, Marijana Vujkovic, Scott M Damrauer, Saiju Pyarajan, J Michael Gaziano, 2019. Harmonizing genetic ancestry and self-identified race/ethnicity in genome-wide association studies. The American Journal of Human Genetics 105, 4 (2019), 763–772.
[28]
Rachel S Fleischer. 2020. Bias In, Bias Out: Why Legislation Placing Requirements on the Procurement of Commercialized Facial Recognition Technology Must Be Passed to Protect People of Color. Public Contract Law Journal 50, 1 (2020), 63–89.
[29]
Claudia Flores-Saviaga, Shangbin Feng, and Saiph Savage. 2022. Datavoidant: An AI System for Addressing Political Data Voids on Social Media. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2 (2022), 1–29.
[30]
Claudia Flores-Saviaga, Ricardo Granados, Liliana Savage, Lizbeth Escobedo, and Saiph Savage. 2020. Understanding the complementary nature of paid and volunteer crowds for content creation. Avances en Interacción Humano-Computadora1 (2020), 37–44.
[31]
Claudia Flores-Saviaga, Yuwen Li, Benjamin Hanrahan, Jeffrey Bigham, and Saiph Savage. 2020. The challenges of crowd workers in rural and urban America. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 8. 159–162.
[32]
Claudia Flores-Saviaga and Saiph Savage. 2021. Fighting disaster misinformation in Latin America: the# 19S Mexican earthquake case study. Personal and Ubiquitous Computing 25 (2021), 353–373.
[33]
Claudia Flores-Saviaga, Saiph Savage, and Dario Taraborelli. 2016. Leadwise: using online bots to recruite and guide expert volunteers. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion. 257–260.
[34]
Md Tahmid Hasan Fuad, Awal Ahmed Fime, Delowar Sikder, Md Akil Raihan Iftee, Jakaria Rabbi, Mabrook S Al-Rakhami, Abdu Gumae, Ovishake Sen, Mohtasim Fuad, and Md Nazrul Islam. 2021. Recent advances in deep learning techniques for face recognition. IEEE Access (2021).
[35]
Mitchell L Gordon, Michelle S Lam, Joon Sung Park, Kayur Patel, Jeff Hancock, Tatsunori Hashimoto, and Michael S Bernstein. 2022. Jury learning: Integrating dissenting voices into machine learning models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–19.
[36]
Stuart Hall. 1997. Representation, meaning and language. Representation: Cultural representations and signifying practices (1997), 15–64.
[37]
Foad Hamidi, Morgan Klaus Scheuerman, and Stacy M Branham. 2018. Gender recognition or gender reductionism? The social implications of embedded gender recognition systems. In Proceedings of the 2018 chi conference on human factors in computing systems. 1–13.
[38]
Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, and Jeffrey P Bigham. 2018. A data-driven analysis of workers’ earnings on Amazon Mechanical Turk. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–14.
[39]
John Hartley. 2012. Communication, cultural and media studies: The key concepts. Routledge.
[40]
Thomas Huang, Ziyou Xiong, and Zhenqiu Zhang. 2005. Face recognition applications. In Handbook of Face Recognition. Springer, 371–390.
[41]
Yu-Hsuan Huang and Homer H Chen. 2020. Face recognition under low illumination via deep feature reconstruction network. In 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2161–2165.
[42]
Andreas Ihl, Kim Simon Strunk, and Marina Fiedler. 2018. The influence of utilitarian and hedonic motivation on success in crowd work. (2018).
[43]
Niharika Jain, Alberto Olmo, Sailik Sengupta, Lydia Manikonda, and Subbarao Kambhampati. 2022. Imperfect ImaGANation: Implications of GANs exacerbating biases on facial data augmentation and snapchat face lenses. Artificial Intelligence 304 (2022), 103652.
[44]
Byoungjun Jeon, Boseong Jeong, Seunghoon Jee, Yan Huang, Youngmin Kim, Gee Ho Park, Jungah Kim, Maierdanjiang Wufuer, Xian Jin, Sang Wha Kim, 2019. A facial recognition mobile app for patient safety and biometric identification: Design, development, and validation. JMIR mHealth and uHealth 7, 4 (2019), e11472.
[45]
Shivani Kapania, Ding Wang, and Alex Taylor. 2023. A hunt for the Snark: Annotator Diversity in Data Practices. ACM Conference on Human Factors in Computing Systems (CHI) (2023).
[46]
Vijay Keswani, Matthew Lease, and Krishnaram Kenthapadi. 2022. Designing closed human-in-the-loop deferral pipelines. arXiv preprint arXiv:2202.04718 (2022).
[47]
Lucy Kimbell. 2011. Rethinking design thinking: Part I. Design and culture 3, 3 (2011), 285–306.
[48]
Alfred Kobsa and Jörg Schreck. 2003. Privacy through pseudonymity in user-adaptive systems. ACM Transactions on Internet Technology (TOIT) 3, 2 (2003), 149–183.
[49]
Joshua A Kroll. 2022. ACM TechBrief: Facial Recognition.
[50]
Sophie Lebrecht, Lara J Pierce, Michael J Tarr, and James W Tanaka. 2009. Perceptual other-race training reduces implicit racial bias. PloS one 4, 1 (2009), e4215.
[51]
Christopher Libby and Jesse Ehrenfeld. 2021. Facial recognition technology in 2021: masks, bias, and the future of healthcare. Journal of Medical Systems 45, 4 (2021), 1–3.
[52]
Jennifer Mankoff, Gillian R Hayes, and Devva Kasnitz. 2010. Disability studies as a source of critical inquiry for the field of assistive technology. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility. 3–10.
[53]
Liam M Mayron, Yasser Hausawi, and Gisela Susanne Bahr. 2013. Secure, usable biometric authentication systems. In International Conference on Universal Access in Human-Computer Interaction. Springer, 195–204.
[54]
Michael McLaughlin and Daniel Castro. 2020. The Critics Were Wrong: NIST Data Shows the Best Facial Recognition Algorithms Are Neither Racist Nor Sexist. https://itif.org/publications/2020/01/27/critics-were-wrong-nist-data-shows-best-facial-recognition-algorithms/.
[55]
Ahmed M Megreya and Markus Bindemann. 2013. Individual differences in personality and face identification. Journal of Cognitive Psychology 25, 1 (2013), 30–37.
[56]
Ahmed M Megreya and Markus Bindemann. 2015. Developmental improvement and age-related decline in unfamiliar face matching. Perception 44, 1 (2015), 5–22.
[57]
Ahmed M Megreya, Markus Bindemann, and Catriona Havard. 2011. Sex differences in unfamiliar face identification: Evidence from matching tasks. Acta Psychologica 137, 1 (2011), 83–89.
[58]
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR) 54, 6 (2021), 1–35.
[59]
Jiani Hu Xunqiang Tao Yaohai Huang Mei Wang, Weihong Deng. [n. d.]. RFW Dataset. http://whdeng.cn/RFW/testing.html.
[60]
Christopher S Milligan. 1999. Facial recognition technology, video surveillance, and privacy. S. Cal. Interdisc. LJ 9 (1999), 295.
[61]
Vikram Mohanty, Kareem Abdol-Hamid, Courtney Ebersohl, and Kurt Luther. 2019. Second opinion: Supporting last-mile person identification with crowdsourcing and face recognition. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 86–96.
[62]
Vikram Mohanty, David Thames, Sneha Mehta, and Kurt Luther. 2019. Photo sleuth: Combining human expertise and face recognition to identify historical portraits. In Proceedings of the 24th International Conference on Intelligent User Interfaces. 547–557.
[63]
Vikram Mohanty, David Thames, Sneha Mehta, and Kurt Luther. 2020. Photo sleuth: Identifying historical portraits with face recognition and crowdsourced human expertise. ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 4 (2020), 1–36.
[64]
Ihudiya Finda Ogbonnaya-Ogburu, Angela DR Smith, Alexandra To, and Kentaro Toyama. 2020. Critical race theory for HCI. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–16.
[65]
Alice J O’Toole, Xaiobo An, Joseph Dunlop, Vaidehi Natu, and P Jonathon Phillips. 2012. Comparing face recognition algorithms to humans on challenging tasks. ACM Transactions on Applied Perception (TAP) 9, 4 (2012), 1–13.
[66]
Alice J O’Toole, P Jonathon Phillips, Fang Jiang, Janet Ayyad, Nils Penard, and Herve Abdi. 2007. Face recognition algorithms surpass humans matching faces over changes in illumination. IEEE transactions on pattern analysis and machine intelligence 29, 9 (2007), 1642–1646.
[67]
A O’Toole, P Jonathon Phillips, Fang Jiang, Janet Ayyad, Nils Penard, and Hervé Abdi. 2005. Face recognition algorithms surpass humans. IEEE Trans. PAMI (2005).
[68]
Robert Ezra Park. 1950. Race and culture. (1950).
[69]
Sidney Perkowitz. 2021. The bias in the machine: Facial recognition technology and racial disparities. MIT Case Studies in Social and Ethical Responsibilities of Computing https://doi. org/10.21428/2c646de5 62272586, 5 (2021), 15.
[70]
P Jonathon Phillips, Fang Jiang, Abhijit Narvekar, Julianne Ayyad, and Alice J O’Toole. 2011. An other-race effect for face recognition algorithms. ACM Transactions on Applied Perception (TAP) 8, 2 (2011), 1–11.
[71]
P Jonathon Phillips, Amy N Yates, Ying Hu, Carina A Hahn, Eilidh Noyes, Kelsey Jackson, Jacqueline G Cavazos, Géraldine Jeckeln, Rajeev Ranjan, Swami Sankaranarayanan, 2018. Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms. Proceedings of the National Academy of Sciences 115, 24 (2018), 6171–6176.
[72]
Julian Alberto Posada Gutierrez. 2022. The Coloniality of Data Work: Power and Inequality in Outsourced Data Production for Machine Learning. Ph. D. Dissertation.
[73]
Inioluwa Deborah Raji and Joy Buolamwini. 2019. Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 429–435.
[74]
Inioluwa Deborah Raji, Timnit Gebru, Margaret Mitchell, Joy Buolamwini, Joonseok Lee, and Emily Denton. 2020. Saving face: Investigating the ethical concerns of facial recognition auditing. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 145–151.
[75]
Elissa M Redmiles. 2021. The need for respectful technologies: Going beyond privacy., 309–313 pages.
[76]
Gillian Rhodes, Vance Locke, Louise Ewing, and Emma Evangelista. 2009. Race coding and the other-race effect in face recognition. Perception 38, 2 (2009), 232–241.
[77]
Jakob Rogstadius, Vassilis Kostakos, Aniket Kittur, Boris Smus, Jim Laredo, and Maja Vukovic. 2011. An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets. In Proceedings of the international AAAI conference on web and social media, Vol. 5. 321–328.
[78]
Drew Roselli, Jeanna Matthews, and Nisha Talagala. 2019. Managing bias in AI. In Companion Proceedings of The 2019 World Wide Web Conference. 539–544.
[79]
Sagar Samtani, Murat Kantarcioglu, and Hsinchun Chen. 2020. Trailblazing the artificial intelligence for cybersecurity discipline: a multi-disciplinary research roadmap., 19 pages.
[80]
Saiph Savage, Claudia Flores-Saviaga, Rachel Rodney, Liliana Savage, Jon Schull, and Jennifer Mankoff. 2022. The Global Care Ecosystems of 3D Printed Assistive Devices. Accessible Computing 15, 4 (2022).
[81]
Jay Scherer and Steven J Jackson. 2008. Cultural studies and the circuit of culture: Advertising, promotional culture and the New Zealand All Blacks. Cultural Studies? Critical Methodologies 8, 4 (2008), 507–526.
[82]
Morgan Klaus Scheuerman, Alex Hanna, and Emily Denton. 2021. Do datasets have politics? Disciplinary values in computer vision dataset development. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–37.
[83]
Morgan Klaus Scheuerman, Kandrea Wade, Caitlin Lustig, and Jed R Brubaker. 2020. How we’ve taught algorithms to see identity: Constructing race and gender in image databases for facial analysis. Proceedings of the ACM on Human-computer Interaction 4, CSCW1 (2020), 1–35.
[84]
Abdallah Hussein Sham, Kadir Aktas, Davit Rizhinashvili, Danila Kuklianov, Fatih Alisinanoglu, Ikechukwu Ofodile, Cagri Ozcinar, and Gholamreza Anbarjafari. 2023. Ethical AI in facial expression analysis: Racial bias. Signal, Image and Video Processing 17, 2 (2023), 399–406.
[85]
Tomáš Sixta, Julio Jacques Junior, Pau Buch-Cardona, Eduard Vazquez, and Sergio Escalera. 2020. Fairface challenge at eccv 2020: Analyzing bias in face recognition. In European conference on computer vision. Springer, 463–481.
[86]
Audrey Smedley and Brian D Smedley. 2005. Race as biology is fiction, racism as a social problem is real: Anthropological and historical perspectives on the social construction of race.American psychologist 60, 1 (2005), 16.
[87]
Angela DR Smith, Adriana Alvarado Garcia, Ian Arawjo, Audrey Bennett, Khalia Braswell, Bryan Dosono, Ron Eglash, Denae Ford, Daniel Gardner, Shamika Goddard, 2021. Keepin’it real about race in HCI. Interactions 28, 5 (2021), 28–33.
[88]
Siegfried Ludwig Sporer. 2001. Recognizing faces of other ethnic groups: An integration of theories.Psychology, Public Policy, and Law (2001).
[89]
James W Tanaka and Lara J Pierce. 2009. The neural plasticity of other-race face recognition. Cognitive, Affective, & Behavioral Neuroscience 9, 1 (2009), 122–131.
[90]
Jacob Thebault-Spieker, Sukrit Venkatagiri, David Mitchell, Chris Hurt, and Kurt Luther. 2019. Pairwise: mitigating political bias in crowdsourced content moderation. Proceedings of HCOMP (2019).
[91]
Bridget Tombleson and Katharina Wolf. 2017. Rethinking the circuit of culture: How participatory culture has transformed cross-cultural communication. Public Relations Review 43, 1 (2017), 14–25.
[92]
Mei Wang and Weihong Deng. 1804. Deep face recognition: a survey (2018). arXiv preprint arXiv:1804.06655 (1804).
[93]
Mei Wang and Weihong Deng. 2019. Mitigate bias in face recognition using skewness-aware reinforcement learning. arXiv preprint arXiv:1911.10692 (2019).
[94]
Mei Wang and Weihong Deng. 2020. Mitigating bias in face recognition using skewness-aware reinforcement learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9322–9331.
[95]
Mei Wang and Weihong Deng. 2021. Deep face recognition: A survey. Neurocomputing 429 (2021), 215–244.
[96]
Mei Wang, Weihong Deng, Jiani Hu, Xunqiang Tao, and Yaohai Huang. 2019. Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In Proceedings of the ieee/cvf international conference on computer vision. 692–702.
[97]
David White, James D Dunn, Alexandra C Schmid, and Richard I Kemp. 2015. Error rates in users of automatic face recognition software. PloS one 10, 10 (2015), e0139827.
[98]
Hoo Keat Wong, Ian D Stephen, and David RT Keeble. 2020. The own-race bias for face recognition in a multiracial society. Frontiers in psychology 11 (2020), 208.
[99]
Benjamin P Wood, Man Cao, Michael D Bond, and Dan Grossman. 2017. Instrumentation bias for dynamic data race detection. Proceedings of the ACM on Programming Languages 1, OOPSLA (2017), 1–31.
[100]
Esther Xiu Wen Wu, Bruno Laeng, and Svein Magnussen. 2012. Through the eyes of the own-race bias: Eye-tracking and pupillometry during face recognition. Social neuroscience 7, 2 (2012), 202–216.
[101]
Xingjiao Wu, Luwei Xiao, Yixuan Sun, Junhang Zhang, Tianlong Ma, and Liang He. 2022. A survey of human-in-the-loop for machine learning. Future Generation Computer Systems (2022).
[102]
Jessica L Yaros, Diana A Salama, Derek Delisle, Myra S Larson, Blake A Miranda, and Michael A Yassa. 2019. A memory computational basis for the other-race effect. Scientific reports 9, 1 (2019), 1–11.
[103]
Lixiu Yu, Paul André, Aniket Kittur, and Robert Kraut. 2014. A comparison of social, learning, and financial strategies on crowd engagement and output quality. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. 967–978.
[104]
Stefanos Zafeiriou, Cha Zhang, and Zhengyou Zhang. 2015. A survey on face detection in the wild: past, present and future. Computer Vision and Image Understanding 138 (2015), 1–24.
Index Terms
- Inclusive Portraits: Race-Aware Human-in-the-Loop Technology
Index terms have been assigned to the content through auto-classification.
Recommendations
Inclusive Manufacturing: The Impact of Disability Diversity on Productivity in a Work Integration Social Enterprise
Problem definition: This study examines the impact of disability diversity on the productivity of apparel manufacturing teams in the context of a work integration social enterprise. Two measures of disability diversity are examined: the number of ...
Returns to human capital for information technology professionals
SIGMIS CPR '03: Proceedings of the 2003 SIGMIS conference on Computer personnel research: Freedom in Philadelphia--leveraging differences and diversity in the IT workforceThis study examines compensation-tenure profiles using detailed salary data collected on IT professionals. Consistent with human capital theory, findings show that compensation increases with organizational tenure for all IT jobs. However, for a given ...
A Large Inclusive Study of Human Listening Rates
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsAs conversational agents and digital assistants become increasingly pervasive, understanding their synthetic speech becomes increasingly important. Simultaneously, speech synthesis is becoming more sophisticated and manipulable, providing the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2023
498 pages
ISBN:9798400703812
DOI:10.1145/3617694
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 October 2023
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- NSF
Conference
EAAMO '23: Equity and Access in Algorithms, Mechanisms, and Optimization
October 30 - November 1, 2023
MA, Boston, USA
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 384Total Downloads
- Downloads (Last 12 months)384
- Downloads (Last 6 weeks)56
Reflects downloads up to 02 Oct 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in