Exploring Designers’ Perceptions and Practices of Collaborating with Generative AI as a Co-creative Agent in a Multi-stakeholder Design Process: Take the Domain of Avatar Design as an Example
Pages 596 - 613
Abstract
Nowadays, the traditional workflow of designers’ completing complicated design tasks has undergone a profound transformation due to the pervasive intervention of generative artificial intelligence (AI) tools, especially when multi-stakeholder participation is getting involved in the design process. Yet we know little about the designers’ perceptions and practices of collaborating with generative AI as a co-creative agent within the context of multi-stakeholder participation. To investigate these questions, we took the domain of avatar design as an example and conducted a qualitative interview study with 21 expert avatar designers who have got different levels of experience and expertise in utilizing generative AI tools in their design workflow. We found that designers not only would fall in a dilemma when deciding whether to consider AI as a co-creative agent according to different stakeholders’ interests, but they also face many challenges in effectively co-creating with the current systems, including challenges in consistently adjusting AI outputs and getting design inspiration within the iterative generation process, etc. Based on our findings, we concluded both the epistemological and creative patterns of collaborating with generative AI and highlighted several design opportunities from both technical and ethical perspectives to better support future designer-AI co-creation.
References
[1]
Hussain M. Aljaroodi, Marc T. P. Adam, Timm Teubner, and Raymond Chiong. 2023. Understanding the Importance of Cultural Appropriateness for User Interface Design: An Avatar Study. ACM Trans. Comput.-Hum. Interact. 29, 6, Article 52 (jan 2023), 27 pages. https://doi.org/10.1145/3517138
[2]
Giuseppe Amato, Malte Behrmann, Frédéric Bimbot, Baptiste Caramiaux, Fabrizio Falchi, Ander Garcia, Joost Geurts, Jaume Gibert, Guillaume Gravier, Hadmut Holken, 2019. AI in the media and creative industries. arXiv preprint arXiv:1905.04175 (2019).
[3]
Nantheera Anantrasirichai and David Bull. [n. d.]. Artificial intelligence in the creative industries: a review. 55, 1 ([n. d.]), 589–656. https://doi.org/10.1007/s10462-021-10039-7
[4]
Sofian Audry. 2021. Art in the age of machine learning. Mit Press.
[5]
Karen Barad. 2007. Meeting the universe halfway: Quantum physics and the entanglement of matter and meaning. duke university Press.
[6]
Jean-François Bélisle and H Onur Bodur. 2010. Avatars as information: Perception of consumers based on their avatars in virtual worlds. Psychology & Marketing 27, 8 (2010), 741–765.
[7]
Jeanette Blomberg, Lucy Suchman, and Randall H Trigg. 1996. Reflections on a work-oriented design project. Human–Computer Interaction 11, 3 (1996), 237–265.
[8]
Margaret A Boden. 2014. Creativity and artificial intelligence: A contradiction in terms. The philosophy of creativity: New essays (2014), 224–46.
[9]
Emanuela Bran, Cosima Rughiniş, Gheorghe Nadoleanu, and Michael G Flaherty. 2023. The Emerging Social Status of Generative AI: Vocabularies of AI Competence in Public Discourse. In 2023 24th International Conference on Control Systems and Computer Science (CSCS). IEEE, 391–398.
[10]
Nico Brand, William Odom, and Samuel Barnett. 2021. A Design Inquiry into Introspective AI: Surfacing Opportunities, Issues, and Paradoxes. In Designing Interactive Systems Conference 2021. 1603–1618.
[11]
Noel Brett. 2022. Why Do We Only Get Anime Girl Avatars? Collective White Heteronormative Avatar Design in Live Streams. Television & New Media 23, 5 (2022), 451–461.
[12]
Baptiste Caramiaux and Sarah Fdili Alaoui. [n. d.]. "Explorers of Unknown Planets": Practices and Politics of Artificial Intelligence in Visual Arts. 6 ([n. d.]), 1–24. Issue CSCW2. https://doi.org/10.1145/3555578
[13]
Eva Cetinic and James She. [n. d.]. Understanding and Creating Art with AI: Review and Outlook. 18, 2 ([n. d.]), 1–22. https://doi.org/10.1145/3475799
[14]
Eugene Ch’ng (Ed.). [n. d.]. Art by Computing Machinery: Is Machine Art Acceptable in the Artworld?15, 2 ([n. d.]), 1–17. https://doi.org/10.1145/3326338
[15]
Pei-Sze Chow. [n. d.]. Ghost in the (Hollywood) machine: Emergent applications of artificial intelligence in the film industry. 9 ([n. d.]).
[16]
Christopher Peter Clarke. 2012. Second Life in the library: an empirical study of new users’ experiences. Program 46, 2 (2012), 242–257.
[17]
Kate Crawford. 2021. The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press.
[18]
Antonio Daniele and Yi-Zhe Song. 2019. Ai+ art= human. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 155–161.
[19]
Graham Dove, Kim Halskov, Jodi Forlizzi, and John Zimmerman. 2017. UX design innovation: Challenges for working with machine learning as a design material. In Proceedings of the 2017 chi conference on human factors in computing systems. 278–288.
[20]
Marcus Peter Francis Du Sautoy. 2019. The Creativity Code: How AI is learning to write, paint and think. (No Title) (2019).
[21]
Nicolas Ducheneaut, Ming-Hui Wen, Nicholas Yee, and Greg Wadley. 2009. Body and mind: a study of avatar personalization in three virtual worlds. In Proceedings of the SIGCHI conference on human factors in computing systems. 1151–1160.
[22]
Erik H Erikson. 1968. Identity youth and crisis. Number 7. WW Norton & company.
[23]
Rebecca Fiebrink and Baptiste Caramiaux. 2016. The machine learning algorithm as creative musical tool. arXiv preprint arXiv:1611.00379 (2016).
[24]
Guo Freeman and Divine Maloney. 2021. Body, avatar, and me: The presentation and perception of self in social virtual reality. Proceedings of the ACM on human-computer interaction 4, CSCW3 (2021), 1–27.
[25]
Guo Freeman, Samaneh Zamanifard, Divine Maloney, and Alexandra Adkins. 2020. My body, my avatar: How people perceive their avatars in social virtual reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–8.
[26]
Emma Frid, Celso Gomes, and Zeyu Jin. 2020. Music creation by example. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–13.
[27]
Philip Galanter 2020. Towards ethical relationships with machines that make art. Artnodes26 (2020), 1–9.
[28]
Maia Garau, Mel Slater, David-Paul Pertaub, and Sharif Razzaque. 2005. The responses of people to virtual humans in an immersive virtual environment. Presence: Teleoperators & Virtual Environments 14, 1 (2005), 104–116.
[29]
Frederic Gmeiner, Humphrey Yang, Lining Yao, Kenneth Holstein, and Nikolas Martelaro. 2023. Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–20.
[30]
Erving Goffman. 2016. The presentation of self in everyday life. In Social Theory Re-Wired. Routledge, 482–493.
[31]
Dejan Grba. [n. d.]. Deep Else: A Critical Framework for AI Art. 2, 1 ([n. d.]), 1–32. https://doi.org/10.3390/digital2010001
[32]
Chao Guo, Yong Dou, Tianxiang Bai, Xingyuan Dai, Chunfa Wang, and Yi Wen. [n. d.]. ArtVerse: A Paradigm for Parallel Human–Machine Collaborative Painting Creation in Metaverses. 53, 4 ([n. d.]), 2200–2208. https://doi.org/10.1109/TSMC.2022.3230406
[33]
Matthew Guzdial, Nicholas Liao, and Mark Riedl. 2018. Co-creative level design via machine learning. arXiv preprint arXiv:1809.09420 (2018).
[34]
Dorothy Holland. 2001. Identity and agency in cultural worlds. Harvard University Press.
[35]
Joo-Wha Hong and Nathaniel Ming Curran. [n. d.]. Artificial Intelligence, Artists, and Art: Attitudes Toward Artwork Produced by Humans vs. Artificial Intelligence. 15, 2 ([n. d.]), 1–16. https://doi.org/10.1145/3326337
[36]
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. [n. d.]. LoRA: Low-Rank Adaptation of Large Language Models. arxiv:2106.09685 [cs]http://arxiv.org/abs/2106.09685
[37]
Yasmin B Kafai, Deborah A Fields, and Melissa S Cook. 2010. Your second selves: Player-designed avatars. Games and culture 5, 1 (2010), 23–42.
[38]
Michael Kearns and Aaron Roth. 2019. The ethical algorithm: The science of socially aware algorithm design. Oxford University Press.
[39]
Diederik P. Kingma and Max Welling. [n. d.]. An Introduction to Variational Autoencoders. 12, 4 ([n. d.]), 307–392. https://doi.org/10.1561/2200000056 arxiv:1906.02691 [cs, stat]
[40]
Thomas Kohler, Kurt Matzler, and Johann Füller. 2009. Avatar-based innovation: Using virtual worlds for real-world innovation. Technovation 29, 6-7 (2009), 395–407.
[41]
Anya Kolesnichenko, Joshua McVeigh-Schultz, and Katherine Isbister. 2019. Understanding emerging design practices for avatar systems in the commercial social vr ecology. In Proceedings of the 2019 on Designing Interactive Systems Conference. 241–252.
[42]
Beth E Kolko. 1999. Representing bodies in virtual space: The rhetoric of avatar design. The information society 15, 3 (1999), 177–186.
[43]
Andrew Kudless. 2023. Hierarchies of bias in artificial intelligence architecture: Collective, computational, and cognitive. International Journal of Architectural Computing (2023), 14780771231170272.
[44]
Octavio Kulesz. 2018. Culture, platforms and machines: the impact of artificial intelligence on the diversity of cultural expressions. Intergovernmental committee for the protection and promotion of the diversity of cultural expressions 12 (2018).
[45]
Yueen Li, Jin Gu, and Liyang Wang. [n. d.]. Research on Artificial Intelligence Ethics in the Field of Art Design. 1673, 1 ([n. d.]), 012052. https://doi.org/10.1088/1742-6596/1673/1/012052
[46]
Yi-Chi Liao, John J Dudley, George B Mo, Chun-Lien Cheng, Liwei Chan, Antti Oulasvirta, and Per Ola Kristensson. 2023. Interaction Design With Multi-objective Bayesian Optimization. IEEE Pervasive Computing 22, 1 (2023), 29–38.
[47]
Róisín Loughran and Michael O’Neill. 2017. Application Domains Considered in Computational Creativity. In ICCC. 197–204.
[48]
Zhicong Lu, Chenxinran Shen, Jiannan Li, Hong Shen, and Daniel Wigdor. [n. d.]. More Kawaii than a Real-Person Live Streamer: Understanding How the Otaku Community Engages with and Perceives Virtual YouTubers. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama Japan, 2021-05-06). ACM, 1–14. https://doi.org/10.1145/3411764.3445660
[49]
Lev Manovich. [n. d.]. Defining AI Arts: Three Proposals. ([n. d.]).
[50]
Hazel Markus and Paula Nurius. 1986. Possible selves.American psychologist 41, 9 (1986), 954.
[51]
Victoria McArthur. 2014. E is for Everyone: The Role of Stakeholders in Participatory Design and Game Studies. Journal ISSN 2368 (2014), 6103.
[52]
Jon McCormack, Toby Gifford, and Patrick Hutchings. 2019. Autonomy, authenticity, authorship and intention in computer generated art. In International conference on computational intelligence in music, sound, art and design (part of EvoStar). Springer, 35–50.
[53]
Melissa McMullen. 2016. Intercultural design competence: A guide for graphic designers working across cultural boundaries. The International Journal of Visual Design 10, 3 (2016), 19–30.
[54]
Mark Stephen Meadows. 2007. I, avatar: The culture and consequences of having a second life. New Riders.
[55]
Eitan Mendelowitz 2020. Intelligent Environments and Public Art. Artnodes26 (2020), 1–9.
[56]
Fred Miao, Irina V Kozlenkova, Haizhong Wang, Tao Xie, and Robert W Palmatier. 2022. An emerging theory of avatar marketing. Journal of Marketing 86, 1 (2022), 67–90.
[57]
Melanie Mitchell. 2019. Artificial intelligence: A guide for thinking humans. Penguin UK.
[58]
Jacquelyn Ford Morie. 2014. Avatar appearance as prima facie non-verbal communication. See Tanenbaum et al (2014), 77–102.
[59]
Michael J Muller and Sarah Kuhn. 1993. Participatory design. Commun. ACM 36, 6 (1993), 24–28.
[60]
Guglielmo Papagni, Jesse De Pagter, Setareh Zafari, Michael Filzmoser, and Sabine T. Koeszegi. [n. d.]. Artificial agents’ explainability to support trust: considerations on timing and context. 38, 2 ([n. d.]), 947–960. https://doi.org/10.1007/s00146-022-01462-7
[61]
Matteo Pasquinelli. 2020. How a Machine Learns and Fails–A Grammar of Error for Artificial Intelligence. Spheres.
[62]
Martin Ragot, Nicolas Martin, and Salomé Cojean. [n. d.]. AI-generated vs. Human Artworks. A Perception Bias Towards Artificial Intelligence?. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu HI USA, 2020-04-25). ACM, 1–10. https://doi.org/10.1145/3334480.3382892
[63]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695.
[64]
Kevin Roose. 2022. An AI-generated picture won an art prize. Artists aren’t happy. The New York Times 2, September (2022).
[65]
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. [n. d.]. DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. arxiv:2208.12242 [cs]http://arxiv.org/abs/2208.12242
[66]
Hugo Scurto, Baptiste Caramiaux, and Frédéric Bevilacqua. 2021. Prototyping machine learning through diffractive art practice. In Designing Interactive Systems Conference 2021. 2013–2025.
[67]
James She (Ed.). [n. d.]. Introduction to the Special Issue on Big Data, Machine Learning, and AI Technologies for Art and Design. 15, 2 ([n. d.]), 1–3. https://doi.org/10.1145/3338002
[68]
Jiao Sun, Q. Vera Liao, Michael Muller, Mayank Agarwal, Stephanie Houde, Kartik Talamadupula, and Justin D. Weisz. [n. d.]. Investigating Explainability of Generative AI for Code through Scenario-based Design. In 27th International Conference on Intelligent User Interfaces (Helsinki Finland, 2022-03-22). ACM, 212–228. https://doi.org/10.1145/3490099.3511119
[69]
the Precise4Q consortium, Julia Amann, Alessandro Blasimme, Effy Vayena, Dietmar Frey, and Vince I. Madai. [n. d.]. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. 20, 1 ([n. d.]), 310. https://doi.org/10.1186/s12911-020-01332-6
[70]
Thomas Waltemate, Dominik Gall, Daniel Roth, Mario Botsch, and Marc Erich Latoschik. 2018. The impact of avatar personalization and immersion on virtual body ownership, presence, and emotional response. IEEE transactions on visualization and computer graphics 24, 4 (2018), 1643–1652.
[71]
Ruoyao Wang. 2021. Computer-aided interaction of visual communication technology and art in new media scenes. Computer-Aided Design and Applications 19, S3 (2021), 75–84.
[72]
Triyono Widodo. [n. d.]. Exploration and Experimentation in Modern Art: Painting in a Personal Style. ([n. d.]). https://doi.org/10.18502/kss.v4i12.7622
[73]
Xiuxiu Wu, Kin Wai Michael Siu, Jörn Bühring, and Caterina Villani. [n. d.]. The Relationship between Creative Self-Efficacy, Achievement Motivation, and Job Burnout among Designers in China’s e-Market. 11, 11 ([n. d.]), 509. https://doi.org/10.3390/socsci11110509
[74]
Yanfang Wu, Tuenyu Lau, David Atkin, and Carolyn Lin. 2011. A comparative study of online privacy regulations in the U.S. and China. Telecommunications Policy 35 (08 2011), 603–616. https://doi.org/10.1016/j.telpol.2011.05.002
[75]
Qian Yang, Alex Scuito, John Zimmerman, Jodi Forlizzi, and Aaron Steinfeld. 2018. Investigating how experienced UX designers effectively work with machine learning. In Proceedings of the 2018 designing interactive systems conference. 585–596.
[76]
Martin Zeilinger. 2021. Tactical entanglements: AI art, creative agency, and the limits of intellectual property. meson press.
[77]
Lvmin Zhang and Maneesh Agrawala. 2023. Adding conditional control to text-to-image diffusion models. arXiv preprint arXiv:2302.05543 (2023).
Index Terms
- Exploring Designers’ Perceptions and Practices of Collaborating with Generative AI as a Co-creative Agent in a Multi-stakeholder Design Process: Take the Domain of Avatar Design as an Example
Recommendations
User Experience Design Professionals’ Perceptions of Generative Artificial Intelligence
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing SystemsAmong creative professionals, Generative Artificial Intelligence (GenAI) has sparked excitement over its capabilities and fear over unanticipated consequences. How does GenAI impact User Experience Design (UXD) practice, and are fears warranted? We ...
Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsAI-based design tools are proliferating in professional software to assist engineering and industrial designers in complex manufacturing and design tasks. These tools take on more agentic roles than traditional computer-aided design tools and are often ...
Designing Participatory AI: Creative Professionals’ Worries and Expectations about Generative AI
CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing SystemsGenerative AI, i.e., the group of technologies that automatically generate visual or written content based on text prompts, has undergone a leap in complexity and become widely available within just a few years. Such technologies potentially introduce a ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2023
634 pages
ISBN:9798400716454
DOI:10.1145/3629606
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 February 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Duke Kunshan University and Houtu Grant
Conference
CHCHI 2023
Acceptance Rates
Overall Acceptance Rate 17 of 40 submissions, 43%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 506Total Downloads
- Downloads (Last 12 months)506
- Downloads (Last 6 weeks)125
Reflects downloads up to 21 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in