Stakeholder Involvement for Responsible AI Development: A Process Framework
Article No.: 1, Pages 1 - 14
Abstract
Stakeholder involvement (ShI) is increasingly promoted for responsible AI development; however, we lack insights into the actual process that practitioners have to complete when conducting such ShI supporting responsible AI efforts. Bridging the gap between theory and practice, this work presents a process framework of ShI for responsible AI development, formalising its stages and associated challenges. We derived an initial framework by relating literature from ShI in healthcare to responsible AI development and expanding it through AI practitioner insights obtained through semi-structured interviews (n=10). The resulting process framework enables systematic reflections about ShI for responsible AI in practice: the required stages, their order and nature, associated bottlenecks, as well as promising interventions. This is essential for informing future research and further supports practitioners by facilitating more systematic communication and ShI efforts. We recommend applications of the framework to advance (our understanding of) ShI for responsible AI development in practice.
Supplemental Material
PDF File
Semi-structured Interview Guide
- Download
- 97.62 KB
References
[1]
Christophe Abrassart, Yoshua Bengio, Guillaume Chicoisne, Nathalie de Marcellis-Warin, Marc-Antoine Dilhac, Sébastien Gambs, Vincent Gautrais, Martin Gibert, Lyse Langlois, Francois Laviolette, Pascale Lehoux, Jocelyn Maclure, Marie Martel, Joëlle Pineau, Peter Railton, Catherine Régis, Christine Tappolet, and Nathalie Voarino. 2018. Montréal Declaration for a Responsible Development of Artificial Intelligence. https://declarationmontreal-iaresponsable.com/wp-content/uploads/2023/04/UdeM_Decl_IA-Resp_LA-Declaration-ENG_WEB_09-07-19.pdf. (Accessed on 03/18/2024).
[2]
Evgeni Aizenberg and Jeroen Van Den Hoven. 2020. Designing for human rights in AI. Big Data & Society 7, 2 (2020), 2053951720949566.
[3]
AlgorithmWatch. 2020. On the European Commission’s “White Paper on Artificial Intelligence – a European approach to excellence and trust“. https://algorithmwatch.org/en/wp-content/uploads/2020/06/AlgorithmWatch_Submission_EC-White-Paper-AI_20200612.pdf. (Accessed on 03/23/2024).
[4]
SustAIn AlgorithmWatch. 2022. Artificial Intelligence: How To Make It More Sustainable. https://algorithmwatch.org/en/wp-content/uploads/2022/06/SustAIn_Magazine_2022_EN.pdf. (Accessed on 03/23/2024).
[5]
Sherry R Arnstein. 1969. A ladder of citizen participation. Journal of the American Institute of planners 35, 4 (1969), 216–224.
[6]
Matthew Barker, Emma Kallina, Dhananjay Ashok, Katherine Collins, Ashley Casovan, Adrian Weller, Ameet Talwalkar, Valerie Chen, and Umang Bhatt. 2023. FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (, Boston, MA, USA, ) (EAAMO ’23). Association for Computing Machinery, New York, NY, USA, Article 19, 15 pages. https://doi.org/10.1145/3617694.3623239
[7]
Glen Berman, Nitesh Goyal, and Michael Madaio. 2024. A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–24.
[8]
Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022. Power to the People? Opportunities and Challenges for Participatory AI. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (, Arlington, VA, USA, ) (EAAMO ’22). Association for Computing Machinery, New York, NY, USA, Article 6, 8 pages. https://doi.org/10.1145/3551624.3555290
[9]
Elizabeth Bondi, Lily Xu, Diana Acosta-Navas, and Jackson A. Killian. 2021. Envisioning Communities: A Participatory Approach Towards AI for Social Good. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (Virtual Event, USA) (AIES ’21). Association for Computing Machinery, New York, NY, USA, 425–436. https://doi.org/10.1145/3461702.3462612
[10]
Peter Bruce and Rita Shelley. 2010. Assessing stakeholder engagement. Communication Journal of New Zealand 11, 2 (2010), 30–48.
[11]
Stephanie Burkhalter, John Gastil, and Todd Kelshaw. 2002. A conceptual definition and theoretical model of public deliberation in small face—to—face groups. Communication theory 12, 4 (2002), 398–422.
[12]
Ania Calderon, Dan Taber, Hong Qu, and Jeff Wen. 2019. AI Blindspot: A discovery process for spotting unconscious biases and structural inequalities in AI systems. http://aiblindspot.media.mit.edu. (Accessed on 06/04/2024).
[13]
Shruthi Sai Chivukula, Ziqing Li, Anne C Pivonka, Jingning Chen, and Colin M Gray. 2021. Surveying the landscape of ethics-focused design methods. arXiv preprint arXiv:2102.08909 (2021).
[14]
Victoria Clarke and Virginia Braun. 2017. Thematic analysis. Vol. 12. Routledge. 297–298 pages. https://doi.org/10.1080/17439760.2016.1262613 arXiv:https://doi.org/10.1080/17439760.2016.1262613
[15]
Thomas W Concannon, Melissa Fuster, Tully Saunders, Kamal Patel, John B Wong, Laurel K Leslie, and Joseph Lau. 2014. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. Journal of general internal medicine 29 (2014), 1692–1701.
[16]
Thomas W Concannon, Paul Meissner, Jo Anne Grunbaum, Newell McElwee, Jeanne-Marie Guise, John Santa, Patrick H Conway, Denise Daudelin, Elaine H Morrato, and Laurel K Leslie. 2012. A new taxonomy for stakeholder engagement in patient-centered outcomes research. Journal of general internal medicine 27 (2012), 985–991.
[17]
Eric Corbett, Emily Denton, and Sheena Erete. 2023. Power and Public Participation in AI. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3617694.3623228
[18]
Sam Corbett-Davies, Johann D. Gaebler, Hamed Nilforoshan, Ravi Shroff, and Sharad Goel. 2024. The measure and mismeasure of fairness. J. Mach. Learn. Res. 24, 1, Article 312 (mar 2024), 117 pages.
[19]
Microsoft Corporation. 2023. Microsoft 365 - Teams. https://www.microsoft.com/en-us/microsoft-teams/log-in. (Accessed on 03/23/2024).
[20]
Ward Cunningham. 2001. Manifesto for Agile Software Development. https://agilemanifesto.org/. (Accessed on 11/26/2023).
[21]
Jiamin Dai and Karyn Moffatt. 2021. Surfacing the Voices of People with Dementia: Strategies for Effective Inclusion of Proxy Stakeholders in Qualitative Research. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (, Yokohama, Japan, ) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 374, 13 pages. https://doi.org/10.1145/3411764.3445756
[22]
Fernando Delgado, Solon Barocas, and Karen Levy. 2022. An Uncommon Task: Participatory Design in Legal AI. Proc. ACM Hum.-Comput. Interact. 6, CSCW1, Article 51 (apr 2022), 23 pages. https://doi.org/10.1145/3512898
[23]
Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (, Boston, MA, USA, ) (EAAMO ’23). Association for Computing Machinery, New York, NY, USA, Article 37, 23 pages. https://doi.org/10.1145/3617694.3623261
[24]
Wesley Hanwen Deng, Nur Yildirim, Monica Chang, Motahhare Eslami, Kenneth Holstein, and Michael Madaio. 2023. Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 705–716.
[25]
Catherine D’ignazio and Lauren F Klein. 2023. Data feminism. MIT press.
[26]
Wikipedia The Free Encyclopedia. 2023. Agile software development. https://en.wikipedia.org/wiki/Agile_software_development. (Accessed on 03/23/2024).
[27]
Laura Esmail, Emily Moore, and Alison Rein. 2015. Evaluating patient and stakeholder engagement in research: moving from theory to practice. Journal of comparative effectiveness research 4, 2 (2015), 133–145.
[28]
Michael Feffer, Michael Skirpan, Zachary Lipton, and Hoda Heidari. 2023. From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (, Montréal, QC, Canada, ) (AIES ’23). Association for Computing Machinery, New York, NY, USA, 38–48. https://doi.org/10.1145/3600211.3604661
[29]
Chief Executives Board for Coordination. 2019. A United Nations system-wide strategic approach and road map for supporting capacity development on AI. https://unsceb.org/sites/default/files/2020-09/CEB_2019_1_Add-3-EN_0.pdf. (Accessed on 03/20/2024).
[30]
CEB United Nations System Chief Executives Board for Coordination. 2022. Principles for the Ethical Use of Artificial Intelligence in the United Nations System. https://unsceb.org/principles-ethical-use-artificial-intelligence-united-nations-system. (Accessed on 01/04/2024).
[31]
Jeff Gibson. 2017. Ethical tools for designers. https://www.ethicsfordesigners.com/tools/. (Accessed on 06/04/2024).
[32]
M Maria Glymour, Mauricio Avendano, and Ichiro Kawachi. 2014. Socioeconomic status and health. Social epidemiology 2 (2014), 17–63.
[33]
ATLAS.ti Scientific Software Development GmbH. 2023. ATLAS.ti Mac (Version 23.2.1) [Computer program]. https://atlasti.com. (Accessed on 04/06/2024).
[34]
Anton Grafström and Lina Schelin. 2014. How to select representative samples. Scandinavian Journal of Statistics 41, 2 (2014), 277–290.
[35]
Michelle Greenwood. 2007. Stakeholder engagement: Beyond the myth of corporate responsibility. Journal of Business ethics 74 (2007), 315–327.
[36]
Lara Groves, Aidan Peppin, Andrew Strait, and Jenny Brennan. 2023. Going Public: The Role of Public Participation Approaches in Commercial AI Labs. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT ’23). Association for Computing Machinery, New York, NY, USA, 1162–1173. https://doi.org/10.1145/3593013.3594071
[37]
Jan Gulliksen, Bengt Göransson, Inger Boivie, Stefan Blomkvist, Jenny Persson, and Åsa Cajander. 2003. Key principles for user-centred systems design. Behaviour and Information Technology 22, 6 (2003), 397–409.
[38]
Jan Gulliksen, Bengt Göransson, Inger Boivie, Stefan Blomkvist, Jenny Persson, and Åsa Cajander. 2003. Key principles for user-centred systems design. Behaviour and Information Technology 22, 6 (2003), 397–409.
[39]
Thilo Hagendorff. 2020. The ethics of AI ethics: An evaluation of guidelines. Minds and machines 30, 1 (2020), 99–120.
[40]
Oliver L. Haimson, Kai Nham, Hibby Thach, and Aloe DeGuia. 2023. How Transgender People and Communities Were Involved in Trans Technology Design Processes. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (, Hamburg, Germany, ) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 294, 16 pages. https://doi.org/10.1145/3544548.3580972
[41]
Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudik, and Hanna Wallach. 2019. Improving fairness in machine learning systems: What do industry practitioners need?. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–16.
[42]
Tony Huzzard. 2021. Achieving impact: exploring the challenge of stakeholder engagement. European Journal of Work and Organizational Psychology 30, 3 (2021), 379–389.
[43]
Ada Lovelace Institute. 2022. Rethinking data and rebalancing digital power. https://www.adalovelaceinstitute.org/report/rethinking-data/. (Accessed on 01/04/2024).
[44]
Amnesty International. 2019. Ethical AI principles won’t solve a human rights crisis. https://www.amnesty.org/en/latest/research/2019/06/ethical-ai-principles-wont-solve-a-human-rights-crisis/. (Accessed on 04/04/2024).
[45]
Amnesty International. 2020. Comment on Proposed Artificial Intelligence Regulations. https://www.amnestyusa.org/wp-content/uploads/2020/03/Comment-on-Proposed-Artificial-Intelligence-Regulations.pdf. (Accessed on 03/23/2024).
[46]
Amnesty International. 2023. Global: Companies must act now to ensure responsible development of artificial intelligence. https://www.amnesty.org/en/latest/news/2023/06/global-companies-must-act-now-to-ensure-responsible-development-of-artificial-intelligence/. (Accessed on 03/23/2024).
[47]
ISO/IEC/IEEE. 2023. Systems and Software Engineering System Life Cycle Processes 15288:2023. https://www.iso.org/standard/81702.html. (Accessed on 01/02/2024).
[48]
Emma Kallina and Malak Sadek. 2023. Who Defines AI’s Future? The Role of Harmful AI Narratives. https://aiequalitytoolbox.com/wp-content/uploads/2023/12/Chapter-1_-Who-Defines-AIs-Future_-The-Role-of-Harmful-AI-Narratives-.docx.pdf. (Accessed on 08/04/2024).
[49]
John Karat. 1996. User centered design: quality or quackery?interactions 3, 4 (1996), 18–20.
[50]
Christopher A Le Dantec and Sarah Fox. 2015. Strangers at the gate: Gaining access, building rapport, and co-constructing community-based research. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. 1348–1358.
[51]
Q. Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: Informing Design Practices for Explainable AI User Experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3313831.3376590
[52]
Michael A Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020. Co-designing checklists to understand organizational challenges and opportunities around fairness in AI. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–14.
[53]
Donald Martin Jr, Vinodkumar Prabhakaran, Jill Kuhlberg, Andrew Smart, and William S Isaac. 2020. Participatory problem formulation for fairer machine learning through community based system dynamics. arXiv preprint arXiv:2005.07572 (2020).
[54]
Andrew McNamara, Justin Smith, and Emerson Murphy-Hill. 2018. Does ACM’s code of ethics change ethical decision making in software development?. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Lake Buena Vista, FL, USA) (ESEC/FSE 2018). Association for Computing Machinery, New York, NY, USA, 729–733. https://doi.org/10.1145/3236024.3264833
[55]
Brent Mittelstadt. 2019. Principles alone cannot guarantee ethical AI. Nature machine intelligence 1, 11 (2019), 501–507.
[56]
OECD.AI Policy Observatory. 2018. Inclusive growth, sustainable development and well-being (OECD AI Principle). https://oecd.ai/en/dashboards/ai-principles/P5. (Accessed on 03/18/2024).
[57]
Josephine Ocloo and Rachel Matthews. 2016. From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ quality & safety 25, 8 (2016), 626–632.
[58]
White House Office of Science and Technology Policy. 2022. Blueprint for an AI Bill of Rights. https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf. (Accessed on 03/20/2024).
[59]
ACM Technology Policy Office. 2022. Statement on Prinicples for Responsible Algorithmic Systems. https://www.acm.org/binaries/content/assets/public-policy/final-joint-ai-statement-update.pdf. (Accessed on 03/18/2024).
[60]
White House Office. 2023. Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/. (Accessed on 03/20/2024).
[61]
Partnership on AI. 2023. Guidelines for AI and Shared Prosperity. https://partnershiponai.org/paper/shared-prosperity/5/. (Accessed on 01/04/2024).
[62]
High-Level Expert Group on Artificial Intelligence (European Commission). 2018. Ethics Guidelines for Trustworthy AI. https://www.europarl.europa.eu/cmsdata/196377/AI%20HLEG_Ethics%20Guidelines%20for%20Trustworthy%20AI.pdf. (Accessed on 03/23/2024).
[63]
World Health Organization. 2021. Ethics and governance of artificial intelligence for health: WHO guidance. https://www.who.int/publications/i/item/9789240029200. (Accessed on 03/23/2024).
[64]
Jennifer E. O’Brien, Kathryne B. Brewer, Lisa M. Jones, Jena Corkhum, and Cynthia Fraga Rizo. 2022. Rigor and Respect: Recruitment Strategies for Engaging Vulnerable Populations in Research. Journal of Interpersonal Violence 37, 17-18 (2022), NP17052–NP17072. https://doi.org/10.1177/08862605211023497 arXiv:https://doi.org/10.1177/08862605211023497PMID: 34107811.
[65]
Aastha Pant, Rashina Hoda, Simone V. Spiegler, Chakkrit Tantithamthavorn, and Burak Turhan. 2024. Ethics in the Age of AI: An Analysis of AI Practitioners’ Awareness and Challenges. ACM Trans. Softw. Eng. Methodol. 33, 3, Article 80 (mar 2024), 35 pages. https://doi.org/10.1145/3635715
[66]
Tina Park. 2022. Making AI Inclusive: 4 Guiding Principles for Ethical Engagement. https://partnershiponai.org/wp-content/uploads/dlm_uploads/2022/07/PAI_whitepaper_making-ai-inclusive.pdf. (Accessed on 01/04/2024).
[67]
Samir Passi and Solon Barocas. 2019. Problem formulation and fairness. In Proceedings of the conference on fairness, accountability, and transparency. 39–48.
[68]
Dorian Peters and Rafael Calvo. 2023. Responsible Tech Design Library. https://www.responsibletechdesign.com. (Accessed on 06/04/2024).
[69]
Jennifer Petkovic, Alison Riddle, Elie A Akl, Joanne Khabsa, Lyubov Lytvyn, Pearl Atwere, Pauline Campbell, Kalipso Chalkidou, Stephanie M Chang, Sally Crowe, 2020. Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation. Systematic reviews 9 (2020), 1–11.
[70]
Erich Prem. 2023. From ethical AI frameworks to tools: a review of approaches. AI and Ethics 3, 3 (2023), 699–716.
[71]
Global AI Narratives Project. 2023. News, Resources & Information from the World of AI Narratives. https://www.ainarratives.com/resources1. (Accessed on 06/04/2024).
[72]
Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021. Where responsible AI meets reality: Practitioner perspectives on enablers for shifting organizational practices. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–23.
[73]
Kristin N Ray and Elizabeth Miller. 2017. Strengthening stakeholder-engaged research and research on stakeholder engagement. Journal of comparative effectiveness research 6, 4 (2017), 375–389.
[74]
Rod D Roscoe. 2021. Designing for diversity: Inclusive sampling. Ergodesign & HCI 9, 1 (2021).
[75]
Malak Sadek, Emma Kallina, Thomas Bohné, Céline Mougenot, Rafael A Calvo, and Stephen Cave. 2024. Challenges of responsible AI in practice: scoping review and recommended actions. AI & SOCIETY (2024), 1–17.
[76]
Bianca G. S. Schor, Chris Norval, Ellen Charlesworth, and Jatinder Singh. 2024. Mind The Gap: Designers and Standards on Algorithmic System Transparency for Users. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 476, 16 pages. https://doi.org/10.1145/3613904.3642531
[77]
Mona Sloane, Emanuel Moss, Olaitan Awomolo, and Laura Forlano. 2022. Participation is not a design fix for machine learning. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–6.
[78]
C Estelle Smith, Bowen Yu, Anjali Srivastava, Aaron Halfaker, Loren Terveen, and Haiyi Zhu. 2020. Keeping community in the loop: Understanding wikipedia stakeholder values for machine learning-based systems. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.
[79]
HAI Stanford. 2024. The AI Index Report 2024. https://aiindex.stanford.edu/report/. (Accessed on 15/04/2024).
[80]
Luke Stark and Anna L Hoffmann. 2019. Data Is the New What? Popular Metaphors & Professional Ethics in Emerging Data Culture. (May 2019). https://doi.org/10.22148/16.036
[81]
OECD Policy Observatory Tools. 2024. ECatalogue of Tools & Metrics for Trustworthy AI. https://oecd.ai/en/catalogue/tools. (Accessed on 06/04/2024).
[82]
Ville Vakkuri, Kai-Kristian Kemell, Joni Kultanen, Mikko Siponen, and Pekka Abrahamsson. 2019. Ethically aligned design of autonomous systems: Industry viewpoint and an empirical study. arXiv preprint arXiv:1906.07946 (2019).
[83]
Christine T Wolf, Haiyi Zhu, Julia Bullard, Min Kyung Lee, and Jed R Brubaker. 2018. The changing contours of" participation" in data-driven, algorithmic ecosystems: Challenges, tactics, and an agenda. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing. 377–384.
[84]
Richmond Y. Wong, Michael A. Madaio, and Nick Merrill. 2023. Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (April 2023), 1–27. https://doi.org/10.1145/3579621
[85]
Nur Yildirim, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, and Fernanda Viégas. 2023. Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (, Hamburg, Germany, ) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 356, 13 pages. https://doi.org/10.1145/3544548.3580900
[86]
Angie Zhang, Olympia Walker, Kaci Nguyen, Jiajun Dai, Anqing Chen, and Min Kyung Lee. 2023. Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1–32.
[87]
Haiyi Zhu, Bowen Yu, Aaron Halfaker, and Loren Terveen. 2018. Value-sensitive algorithm design: Method, case study, and lessons. Proceedings of the ACM on human-computer interaction 2, CSCW (2018), 1–23.
[88]
Douglas Zytko, Pamela J. Wisniewski, Shion Guha, Eric PS Baumer, and Min Kyung Lee. 2022. Participatory design of AI systems: Opportunities and challenges across diverse users, relationships, and application domains. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–4.
Index Terms
- Stakeholder Involvement for Responsible AI Development: A Process Framework
Recommendations
Stakeholder Involvement in Agile Software Development
NordiCHI '16: Proceedings of the 9th Nordic Conference on Human-Computer InteractionAgile software development processes (Agile), such as Scrum, DSDM, XP and Kanban, have become de facto standards for software development practice. Scrum, the most commonly used process, focuses on delivering functioning software early and continuously, ...
Developing a stakeholder-assisted agile CONOPS development process
Concepts of Operations (CONOPS) are documents describing the characteristics and intended usage of proposed and existing systems. They provide information about the requirements and future desired states the project aims to achieve. We reviewed 22 recent ...
Lay User Involvement in Developing Human-centric Responsible AI Systems: When and How?
Artificial Intelligence (AI) is increasingly used in mainstream applications to make decisions that affect a large number of people. While research has focused on involving machine learning and domain experts during the development of responsible AI ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2024
221 pages
ISBN:9798400712227
DOI:10.1145/3689904
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- EPSRC
- The Alan Turing Institute
- Studienstiftung der deutschen Wirtschaft
Conference
EAAMO '24: Equity and Access in Algorithms, Mechanisms, and Optimization
October 29 - 31, 2024
San Luis Potosi, Mexico
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 150Total Downloads
- Downloads (Last 12 months)150
- Downloads (Last 6 weeks)150
Reflects downloads up to 22 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in