research-article

DesignPrompt: Using Multimodal Interaction for Design Exploration with Generative AI

Authors:

Wendy E. MackayAuthors Info & Claims

DIS '24: Proceedings of the 2024 ACM Designing Interactive Systems Conference

Pages 804 - 818

https://doi.org/10.1145/3643834.3661588

Published: 01 July 2024 Publication History

Abstract

Visually oriented designers often struggle to create effective generative AI (GenAI) prompts. A preliminary study identified specific issues in composing and fine-tuning prompts, as well as needs in accurately translating intentions into rich input. We developed DesignPrompt, a moodboard tool that lets designers combine multiple modalities — images, color, text — into a single GenAI prompt and tweak the results. We ran a comparative structured observation study with 12 professional designers to better understand their intent expression, expectation alignment and transparency perception using DesignPrompt and text input GenAI. We found that multimodal prompt input encouraged designers to explore and express themselves more effectively. Designer’s interaction preferences change according to their overall sense of control over the GenAI and whether they are seeking inspiration or a specific image. Designers developed innovative uses of DesignPrompt, including developing elaborate multimodal prompts and creating a multimodal prompt pattern to maximize novelty while ensuring consistency.

References

[1]

Adobe. 2023. Adobe Firefly. https://firefly.adobe.com/. [Accessed 08-02-2024].

[2]

Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, and Christian Frank. 2023. MusicLM: Generating Music From Text. arxiv:2301.11325 [cs.SD]

[3]

Jim Torresen Benedikte Wallace, Kristian Nymoen and Charles Patrick Martin. 2024. Breaking from realism: exploring the potential of glitch in AI-generated dance. Digital Creativity 0, 0 (2024), 1–18. https://doi.org/10.1080/14626268.2024.2327006 arXiv:https://doi.org/10.1080/14626268.2024.2327006

[4]

Adrien Bousseau, Theophanis Tsandilas, Lora Oehlberg, and Wendy E. Mackay. 2016. How Novices Sketch and Prototype Hand-Fabricated Objects. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 397–408. https://doi.org/10.1145/2858036.2858159

Digital Library

[5]

Stephen Brade, Bryan Wang, Mauricio Sousa, Sageev Oore, and Tovi Grossman. 2023. Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Models. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology(UIST ’23). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3586183.3606725

Digital Library

[6]

Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health 11, 4 (2019), 589–597.

[7]

Andrew Brock, Jeff Donahue, and Karen Simonyan. 2019. Large Scale GAN Training for High Fidelity Natural Image Synthesis. arxiv:1809.11096 [cs.LG]

[8]

Tim Brooks, Aleksander Holynski, and Alexei Efros. 2023. InstructPix2Pix: Learning to Follow Image Editing Instructions. 18392–18402. https://doi.org/10.1109/CVPR52729.2023.01764

[9]

Daniel Buschek, Lukas Mecke, Florian Lehmann, and Hai Dang. 2021. Nine Potential Pitfalls when Designing Human-AI Co-Creative Systems. CoRR abs/2104.00358 (2021). arXiv:2104.00358https://arxiv.org/abs/2104.00358

[10]

Tracy Cassidy. 2008. Mood boards: Current practice in learning and teaching strategies and students’ understanding of the process. International Journal of Fashion Design Technology and Education (03 2008), 43–54. https://doi.org/10.1080/17543260802015154

[11]

Li-Yuan Chiou, Peng-Kai Hung, Rung-Huei Liang, and Chun-Teng Wang. 2023. Designing with AI: An Exploration of Co-Ideation with Image Generators. In Proceedings of the 2023 ACM Designing Interactive Systems Conference(DIS ’23). Association for Computing Machinery, New York, NY, USA, 1941–1954. https://doi.org/10.1145/3563657.3596001

Digital Library

[12]

John Joon Young Chung. 2022. Artistic User Expressions in AI-powered Creativity Support Tools. In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22 Adjunct). Association for Computing Machinery, New York, NY, USA, Article 95, 4 pages. https://doi.org/10.1145/3526114.3558531

Digital Library

[13]

John Joon Young Chung and Eytan Adar. 2023. PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology(UIST ’23). Association for Computing Machinery, New York, NY, USA, 1–17. https://doi.org/10.1145/3586183.3606777

Digital Library

[14]

John Joon Young Chung, Shiqing He, and Eytan Adar. 2021. The Intersection of Users, Roles, Interactions, and Technologies in Creativity Support Tools. In Proceedings of the 2021 ACM Designing Interactive Systems Conference (Virtual Event, USA) (DIS ’21). Association for Computing Machinery, New York, NY, USA, 1817–1833. https://doi.org/10.1145/3461778.3462050

Digital Library

[15]

Comfyanonymous. [n. d.]. GitHub - comfyanonymous/ComfyUI: The most powerful and modular stable diffusion GUI, api and backend with a graph/nodes interface.https://github.com/comfyanonymous/ComfyUI

[16]

Niklas Deckers, Maik Fröbe, Johannes Kiesel, Gianluca Pandolfo, Christopher Schröder, Benno Stein, and Martin Potthast. 2023. The Infinite Index: Information Retrieval on Generative Text-To-Image Models. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (Austin, TX, USA) (CHIIR ’23). Association for Computing Machinery, New York, NY, USA, 172–186. https://doi.org/10.1145/3576840.3578327

Digital Library

[17]

Prafulla Dhariwal and Alex Nichol. 2021. Diffusion Models Beat GANs on Image Synthesis. arxiv:2105.05233 [cs.LG]

[18]

Kees Dorst and Nigel Cross. 2001. Creativity in the design process: co-evolution of problem-solution. Design Studies 22, 5 (2001), 425 – 437. http://www.sciencedirect.com/science/article/pii/S0142694X01000096

[19]

Graham Dove, Kim Halskov, Jodi Forlizzi, and John Zimmerman. 2017. UX Design Innovation: Challenges for Working with Machine Learning as a Design Material. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 278–288. https://doi.org/10.1145/3025453.3025739

Digital Library

[20]

Jonas Frich, Lindsay MacDonald Vermeulen, Christian Remy, Michael Mose Biskjaer, and Peter Dalsgaard. 2019. Mapping the Landscape of Creativity Support Tools in HCI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–18. https://doi.org/10.1145/3290605.3300619

Digital Library

[21]

Jérémie Garcia, Theophanis Tsandilas, Carlos Agon, and Wendy E. Mackay. 2014. Structured observation with polyphony: a multifaceted tool for studying music composition. In Proceedings of the 2014 Conference on Designing Interactive Systems (Vancouver, BC, Canada) (DIS ’14). Association for Computing Machinery, New York, NY, USA, 199–208. https://doi.org/10.1145/2598510.2598512

Digital Library

[22]

William Gaver. 2011. Making spaces: how design workbooks work. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). Association for Computing Machinery, New York, NY, USA, 1551–1560. https://doi.org/10.1145/1978942.1979169

Digital Library

[23]

GoogleCloud. 2023. Vision AI|cloud vision API|google cloud. https://cloud.google.com/vision. [Online; accessed 3-June-2023].

[24]

Jonathan Ho, Chitwan Saharia, William Chan, David J. Fleet, Mohammad Norouzi, and Tim Salimans. 2022. Cascaded Diffusion Models for High Fidelity Image Generation. Journal of Machine Learning Research 23, 47 (2022), 1–33.

[25]

Nisha Huang, Fan Tang, Weiming Dong, and Changsheng Xu. 2022. Draw Your Art Dream: Diverse Digital Art Synthesis with Multimodal Guided Diffusion. In Proceedings of the 30th ACM International Conference on Multimedia (, Lisboa, Portugal,) (MM ’22). Association for Computing Machinery, New York, NY, USA, 1085–1094. https://doi.org/10.1145/3503161.3548282

Digital Library

[26]

Angel Hsing-Chi Hwang. 2022. Too Late to be Creative? AI-Empowered Tools in Creative Processes. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 38, 9 pages. https://doi.org/10.1145/3491101.3503549

Digital Library

[27]

Ali Jahanian, Jerry Liu, Qian Lin, Daniel Tretter, Eamonn O’Brien-Strain, Seungyon Claire Lee, Nic Lyons, and Jan Allebach. 2013. Recommendation system for automatic design of magazine covers. In Proceedings of the 2013 International Conference on Intelligent User Interfaces (Santa Monica, California, USA) (IUI ’13). Association for Computing Machinery, New York, NY, USA, 95–106. https://doi.org/10.1145/2449396.2449411

Digital Library

[28]

Youngseung Jeon, Seungwan Jin, Patrick C. Shih, and Kyungsik Han. 2021. FashionQ: An AI-Driven Creativity Support Tool for Facilitating Ideation in Fashion Design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–18. https://doi.org/10.1145/3411764.3445093

Digital Library

[29]

Youwen Kang, Zhida Sun, Sitong Wang, Zeyu Huang, Ziming Wu, and Xiaojuan Ma. 2021. MetaMap: Supporting Visual Metaphor Ideation through Multi-dimensional Example-based Exploration. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (, Yokohama, Japan,) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 427, 15 pages. https://doi.org/10.1145/3411764.3445325

Digital Library

[30]

James C Kaufman and Robert J Sternberg. 2010. The Cambridge handbook of creativity. Cambridge University Press.

[31]

Taslim Arefin Khan, Dongwook Yoon, and Joanna McGrenere. 2020. Designing an Eyes-Reduced Document Skimming App for Situational Impairments. Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376641

Digital Library

[32]

Hyung-Kwon Ko, Gwanmo Park, Hyeon Jeon, Jaemin Jo, Juho Kim, and Jinwook Seo. 2023. Large-scale Text-to-Image Generation Models for Visual Artists’ Creative Works. In Proceedings of the 28th International Conference on Intelligent User Interfaces(IUI ’23). Association for Computing Machinery, New York, NY, USA, 919–933. https://doi.org/10.1145/3581641.3584078

Digital Library

[33]

Janin Koch. 2020. Collaborative Systems for Design Inspiration. Aalto University. https://aaltodoc.aalto.fi:443/handle/123456789/46910 Accepted: 2020-10-13T09:00:05Z ISSN: 1799-4942 (electronic).

[34]

Janin Koch, Andrés Lucero, Lena Hegemann, and Antti Oulasvirta. 2019. May AI? Design Ideation with Cooperative Contextual Bandits. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300863

Digital Library

[35]

Janin Koch, Nicolas Taffin, Michel Beaudouin-Lafon, Markku Laine, Andrés Lucero, and Wendy E. Mackay. 2020. ImageSense: An Intelligent Collaborative Ideation Tool to Support Diverse Human-Computer Partnerships. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1 (May 2020), 45:1–45:27. https://doi.org/10.1145/3392850

Digital Library

[36]

Janin Koch, Nicolas Taffin, Andrés Lucero, and Wendy E. Mackay. 2020. SemanticCollage: Enriching Digital Mood Board Design with Semantic Labels. In Proceedings of the 2020 ACM Designing Interactive Systems Conference(DIS ’20). Association for Computing Machinery, New York, NY, USA, 407–418. https://doi.org/10.1145/3357236.3395494

Digital Library

[37]

Kosinkadink. [n. d.]. GitHub - Kosinkadink/ComfyUI-Advanced-ControlNet: ControlNet scheduling and masking nodes with sliding context support. https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet

[38]

Q. Vera Liao, Hariharan Subramonyam, Jennifer Wang, and Jennifer Wortman Vaughan. 2023. Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience. http://arxiv.org/abs/2302.10395 arXiv:2302.10395 [cs].

[39]

Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021. What Makes Good In-Context Examples for GPT-3?arxiv:2101.06804 [cs.CL]

[40]

Vivian Liu and Lydia B Chilton. 2022. Design Guidelines for Prompt Engineering Text-to-Image Generative Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–23. https://doi.org/10.1145/3491102.3501825

Digital Library

[41]

Vivian Liu, Jo Vermeulen, George Fitzmaurice, and Justin Matejka. 2023. 3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows. In Proceedings of the 2023 ACM Designing Interactive Systems Conference (, Pittsburgh, PA, USA,) (DIS ’23). Association for Computing Machinery, New York, NY, USA, 1955–1977. https://doi.org/10.1145/3563657.3596098

Digital Library

[42]

Lokesh Dhakar. 2023. COLOR THIEF. https://lokeshdhakar.com/projects/color-thief/. [Online; accessed 5-June-2023].

[43]

Yuwen Lu, Chengzhi Zhang, Iris Zhang, and Toby Jia-Jun Li. 2022. Bridging the Gap Between UX Practitioners’ Work Practices and AI-Enabled Design Support Tools. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems(CHI EA ’22). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3491101.3519809

Digital Library

[44]

Andrés Lucero. 2012. Framing, aligning, paradoxing, abstracting, and directing: how design mood boards work. In Proceedings of the Designing Interactive Systems Conference(DIS ’12). Association for Computing Machinery, New York, NY, USA, 438–447. https://doi.org/10.1145/2317956.2318021

Digital Library

[45]

Wendy E. Mackay and Joanna McGrenere. to appear in 2024. Comparative Structured Observation. ACM Transactions on Computer-Human Interaction (TOCHI).

[46]

Mehta, Chirag. 2007. Name that color. https://chir.ag/projects/name-that-color/. [Online; accessed 5-June-2023].

[47]

Midjourney. 2024. Midjourney. https://www.midjourney.com/. [Accessed 26-01-2024].

[48]

Emad Mostaque. 2023. Stable Diffusion Public Release — Stability AI — stability.ai. https://stability.ai/news/stable-diffusion-public-release. [Accessed 26-01-2024].

[49]

Mohammad Amin Mozaffari, Xinyuan Zhang, Jinghui Cheng, and Jin L.C. Guo. 2022. GANSpiration: Balancing Targeted and Serendipitous Inspiration in User Interface Design with Style-Based Generative Adversarial Network. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3491102.3517511

Digital Library

[50]

OpenAI. 2023. OpenAI platform. https://platform.openai.com/docs/api-reference. [Online; accessed 5-June-2023].

[51]

Openai. 2024. DALL·E 3 — openai.com. https://openai.com/dall-e-3. [Accessed 26-01-2024].

[52]

Jonas Oppenlaender. 2022. Prompt engineering for text-based generative art. arXiv preprint arXiv:2204.13988 (2022).

[53]

Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. 2022. DreamFusion: Text-to-3D using 2D Diffusion. arxiv:2209.14988 [cs.CV]

[54]

Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. 2023. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[55]

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. 2022. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. arxiv:2205.11487 [cs.CV]

[56]

Téo Sanchez. 2023. Examining the Text-to-Image Community of Practice: Why and How Do People Prompt Generative AIs?. In Proceedings of the 15th Conference on Creativity and Cognition(C&C ’23). Association for Computing Machinery, New York, NY, USA, 43–61. https://doi.org/10.1145/3591196.3593051

Digital Library

[57]

Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M. Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Stella Biderman, Leo Gao, Tali Bers, Thomas Wolf, and Alexander M. Rush. 2021. Multitask Prompted Training Enables Zero-Shot Task Generalization. CoRR abs/2110.08207 (2021). arXiv:2110.08207https://arxiv.org/abs/2110.08207

[58]

Yang Shi, Yang Wang, Ye Qi, John Chen, Xiaoyao Xu, and Kwan-Liu Ma. 2017. IdeaWall: Improving Creative Collaboration through Combinatorial Visual Stimuli. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing(CSCW ’17). Association for Computing Machinery, New York, NY, USA, 594–603. https://doi.org/10.1145/2998181.2998208

Digital Library

[59]

Patrick C. Shih, David H. Nguyen, Sen H. Hirano, David F. Redmiles, and Gillian R. Hayes. 2009. GroupMind: supporting idea generation through a collaborative mind-mapping tool. In Proceedings of the 2009 ACM International Conference on Supporting Group Work (, Sanibel Island, Florida, USA,) (GROUP ’09). Association for Computing Machinery, New York, NY, USA, 139–148. https://doi.org/10.1145/1531674.1531696

Digital Library

[60]

Hariharan Subramonyam, Roy Pea, Christopher Lawrence Pondoc, Maneesh Agrawala, and Colleen Seifert. 2024. Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfaces. arxiv:2309.14459 [cs.HC]

[61]

Lucille Alice Suchman. 2007. Human-machine reconfigurations: Plans and situated actions. Cambridge university press.

Digital Library

[62]

Jakob Tholander and Martin Jonsson. 2023. Design Ideation with AI - Sketching, Thinking and Talking with Generative Machine Learning Models. In Proceedings of the 2023 ACM Designing Interactive Systems Conference(DIS ’23). Association for Computing Machinery, New York, NY, USA, 1930–1940. https://doi.org/10.1145/3563657.3596014

Digital Library

[63]

Unsplash. [n. d.]. Unsplash Image API | Free HD Photo API — api.unsplash.com. https://api.unsplash.com/. [Accessed 29-01-2024].

[64]

Mathias Peter Verheijden and Mathias Funk. 2023. Collaborative Diffusion: Boosting Designerly Co-Creation with Generative AI. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, Article 73, 8 pages. https://doi.org/10.1145/3544549.3585680

Digital Library

[65]

Yael Vinker, Andrey Voynov, Daniel Cohen-Or, and Ariel Shamir. 2023. Concept Decomposition for Visual Exploration and Inspiration. ACM Trans. Graph. 42, 6, Article 241 (dec 2023), 13 pages. https://doi.org/10.1145/3618315

Digital Library

[66]

Hao-Chuan Wang, Dan Cosley, and Susan R. Fussell. 2010. Idea expander: supporting group brainstorming with conversationally triggered visual thinking stimuli. In Proceedings of the 2010 ACM conference on Computer supported cooperative work(CSCW ’10). Association for Computing Machinery, New York, NY, USA, 103–106. https://doi.org/10.1145/1718918.1718938

Digital Library

[67]

Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C. Schmidt. 2023. A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. arxiv:2302.11382 [cs.SE]

[68]

Qian Yang, Aaron Steinfeld, Carolyn Rosé, and John Zimmerman. 2020. Re-Examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376301

Digital Library

[69]

Nur Yildirim, Changhoon Oh, Deniz Sayar, Kayla Brand, Supritha Challa, Violet Turri, Nina Crosby Walton, Anna Elise Wong, Jodi Forlizzi, James McCann, and John Zimmerman. 2023. Creating Design Resources to Scaffold the Ideation of AI Concepts. In Proceedings of the 2023 ACM Designing Interactive Systems Conference(DIS ’23). Association for Computing Machinery, New York, NY, USA, 2326–2346. https://doi.org/10.1145/3563657.3596058

Digital Library

[70]

J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems(CHI ’23). Association for Computing Machinery, New York, NY, USA, 1–21. https://doi.org/10.1145/3544548.3581388

Digital Library

[71]

Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. 2023. Adding Conditional Control to Text-to-Image Diffusion Models. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). 3813–3824. https://doi.org/10.1109/ICCV51070.2023.00355

Cited By

Mo WSingh AHolloway C(2024)From Information Seeking to Empowerment: Using Large Language Model Chatbot in Supporting Wheelchair Life in Low Resource SettingsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675609(1-18)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675609

Index Terms

DesignPrompt: Using Multimodal Interaction for Design Exploration with Generative AI
1. Applied computing
  1. Arts and humanities
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools

Recommendations

Exploring the Design of Generative AI in Supporting Music-based Reminiscence for Older Adults
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems

Music-based reminiscence has the potential to positively impact the psychological well-being of older adults. However, the aging process and physiological changes, such as memory decline and limited verbal communication, may impede the ability of older ...
AutoSpark: Supporting Automobile Appearance Design Ideation with Kansei Engineering and Generative AI
UIST '24: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology

Rapid creation of novel product appearance designs that align with consumer emotional requirements poses a significant challenge. Text-to-image models, with their excellent image generation capabilities, have demonstrated potential in providing ...
Designing Generative AI User Interfaces for Automobiles
AutomotiveUI '24 Adjunct: Adjunct Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

As the development of Generative AI technology continues to progress, the opportunity for innovation with AI in the form of user interfaces, products and services within vehicles is expanding. Furthermore, automobiles are undergoing major transformations ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '24: Proceedings of the 2024 ACM Designing Interactive Systems Conference

July 2024

3616 pages

ISBN:9798400705830

DOI:10.1145/3643834

Editors:
Anna Vallgårda
IT University of Copenhagen
,
Li Jönsson
Malmö University
,
Jonas Fritsch
IT University of Copenhagen
,
Sarah Fdili Alaoui
LISN-Université Paris Saclay
,
Christopher A. Le Dantec
Northeastern University

Copyright © 2024 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Horizon 2020 Framework Programme

Conference

DIS '24

Sponsor:

SIGCHI

DIS '24: Designing Interactive Systems Conference

July 1 - 5, 2024

Copenhagen, Denmark

Acceptance Rates

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
769
Total Downloads

Downloads (Last 12 months)769
Downloads (Last 6 weeks)210

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mo WSingh AHolloway C(2024)From Information Seeking to Empowerment: Using Large Language Model Chatbot in Supporting Wheelchair Life in Low Resource SettingsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675609(1-18)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675609

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents