research-article

Open access

May AI?: Design Ideation with Cooperative Contextual Bandits

Authors:

Andrés Lucero,

Antti OulasvirtaAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 633, Pages 1 - 12

https://doi.org/10.1145/3290605.3300863

Published: 02 May 2019 Publication History

All formats PDF

Abstract

Design ideation is a prime creative activity in design. However, it is challenging to support computationally due to its quickly evolving and exploratory nature. The paper presents cooperative contextual bandits (CCB) as a machine-learning method for interactive ideation support. A CCB can learn to propose domain-relevant contributions and adapt their exploration/exploitation strategy. We developed a CCB for an interactive design ideation tool that 1) suggests inspirational and situationally relevant materials ("may AI?"); 2) explores and exploits inspirational materials with the designer; and 3) explains its suggestions to aid reflection. The application case of digital mood board design is presented, wherein visual inspirational materials are collected and curated in collages. In a controlled study, 14 of 16 professional designers preferred the CCB-augmented tool. The CCB approach holds promise for ideation activities wherein adaptive and steerable support is welcome but designers must retain full outcome control.

Supplementary Material

ZIP File (pn9126.zip)

1) Brief: Both design briefs used in the presented study.

Download
15.12 MB

MP4 File (paper633.mp4)

Download
254.22 MB

References

[1]

Salvatore Andolina, Khalil Klouche, Diogo Cabral, Tuukka Ruotsalo, and Giulio Jacucci. 2015. InspirationWall: Supporting Idea Generation through Automatic Information Exploration. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition. ACM, 103--106.

Digital Library

[2]

Patti Bao, Elizabeth Gerber, Darren Gergle, and David Hoffman. 2010. Momentum: Getting and Staying on Topic During a Brainstorm. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1233--1236.

Digital Library

[3]

Simone Bianco and Gianluigi Ciocca. 2015. User preferences modeling and learning for pleasing photo collage generation. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 12, 1 (2015), 6.

Digital Library

[4]

Sébastien Bubeck and Nicolo Cesa-Bianchi. 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning 5, 1 (2012), 1--122.

[5]

Tracy Cassidy. 2011. The mood board process modeled and understood as a qualitative design research tool. Fashion Practice 3, 2 (2011).

[6]

Joel Chan, Pao Siangliulue, Denisa Qori McDonald, Ruixue Liu, Reza Moradinezhad, Safa Aman, Erin T. Solovey, Krzysztof Z. Gajos, and Steven P. Dow. 2017. Semantically Far Inspirations Considered Harmful?: Accounting for Cognitive States in Collaborative Ideation. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (C&C '17). ACM, 93--105.

Digital Library

[7]

Erin Cherry and Celine Latulipe. 2014. Quantifying the creativity support of digital tools through the creativity support index. ACM Transactions on Computer--Human Interaction (TOCHI) 21, 4 (2014), 21.

Digital Library

[8]

Douglass R. Cutting, David R. Karger, Jan O. Pedersen, and John W. Tukey. 1992. Scatter/Gather: A Cluster-based Approach to Browsing Large Document Collections. In Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '92). ACM, 318--329.

Digital Library

[9]

Pedram Daee, Joel Pyykkö, Dorota Glowacka, and Samuel Kaski. 2016. Interactive Intent Modeling from Multiple Feedback Domains. In Proceedings of the 21st International Conference on Intelligent User Interfaces (IUI '16). 71--75.

Digital Library

[10]

Nicholas Davis, Chih-PIn Hsiao, Kunwar Yashraj Singh, Lisa Li, and Brian Magerko. 2016. Empirically studying participatory sense-making in abstract drawing with a co-creative cognitive agent. In Proceedings of the 21st International Conference on Intelligent User Interfaces. ACM.

Digital Library

[11]

Kees Dorst and Nigel Cross. 2001. Creativity in the design process: co-evolution of problem-solution. Design Studies 22, 5 (2001), 425--437.

[12]

James Fogarty, Jodi Forlizzi, and Scott E Hudson. 2001. Aesthetic information collages: generating decorative displays that contain information. In Proceedings of the 14th annual ACM symposium on User interface software and technology. ACM, 141--150.

Digital Library

[13]

James Fogarty, Desney Tan, Ashish Kapoor, and Simon Winder. 2008. CueFlik: Interactive Concept Learning in Image Search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, 29--38.

Digital Library

[14]

Steve Garner and Deana McDonagh-Philp. 2001. Problem interpretation and resolution via visual stimuli: the use of 'mood boards' in design education. Journal of Art & Design Education 20, 1 (2001), 57--64.

[15]

Karni Gilon, Joel Chan, Felicia Y. Ng, Hila Liifshitz-Assaf, Aniket Kittur, and Dafna Shahaf. 2018. Analogy Mining for Specific Design Needs. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, Article 121, 11 pages.

Digital Library

[16]

Milene Gonçalves, Carlos Cardoso, and Petra Badke-Schaub. 2016. Inspiration choices that matter: the selection of external stimuli during ideation. Design Science 2 (2016).

[17]

Noe Vargas Hernandez, Jami J. Shah, and Steven M. Smith. 2010. Understanding design ideation mechanisms through multilevel aligned empirical studies. Design Studies 31, 4 (2010), 382--410.

[18]

DuckDuckGo Inc. 2018. DuckDuckGo Search engine. https: //duckduckgo.com

[19]

Twinword Inc. 2018. Natural Language Processing APIs - Twinword API. https://www.twinword.com/api/word-associations.php

[20]

Mikhail Jacob, Alexander Zook, and Brian Magerko. 2013. Viewpoints AI: Procedurally Representing and Reasoning about Gestures. In DiGRA Conference.

[21]

Ben Jonson. 2005. Design ideation: the conceptual sketch in the digital age. Design Studies 26, 6 (2005), 613--624.

[22]

AI Keller, GJ Pasman, and Pieter Jan Stappers. 2006. Collections designers keep: Collecting visual material for inspiration and reference. CoDesign 2, 01 (2006), 17--33.

[23]

Yui Kita and Jun Rekimoto. 2018. V8 Storming: How Far Should Two Ideas Be?. In Proceedings of the 9th Augmented Human International Conference. ACM, 14.

Digital Library

[24]

Janin Koch, Magda Laszlo, Andrés Lucero, Antti Oulasvirta, et al. 2018. Surfing for Inspiration: Digital Inspirational Material in Design Practice. In Proceedings of DRS2018 International Conference.

[25]

Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and evaluation of a user experience questionnaire. In Symposium of the Austrian HCI and Usability Engineering Group. Springer, 63--76.

Digital Library

[26]

Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. 2010. A Contextual-bandit Approach to Personalized News Article Recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW '10). 10.

Digital Library

[27]

Andrés Lucero. 2012. Framing, aligning, paradoxing, abstracting, and directing: how design mood boards work. In Proceedings of the Designing Interactive Systems Conference. ACM.

Digital Library

[28]

Andrés Lucero. 2015. Funky-Design-Spaces: Interactive Environments for Creativity Inspired by Observing Designers Making Mood Boards. In Human--Computer Interaction. Springer, 474--492.

Digital Library

[29]

Andrés Lucero, Dzmitry Aliakseyeu, and Jean-Bernard Martens. 2007. Augmenting Mood Boards: Flexible and Intuitive Interaction in the Context of the Design Studio. In Horizontal Interactive Human-- Computer Systems 2007.

[30]

Andrés Lucero, Dzmitry Aliakseyeu, Kees Overbeeke, and JeanBernard Martens. 2009. An Interactive Support Tool to Convey the Intended Message in Asynchronous Presentations. In Proceedings of the International Conference on Advances in Computer Entertainment Technology (ACE '09). ACM, 8.

Digital Library

[31]

Nic Lupfer, Andruid Kerne, Andrew M. Webb, and Rhema Linder. 2016. Patterns of Free-form Curation: Visual Thinking with Web Content. In Proceedings of the 24th ACM International Conference on Multimedia (MM '16). ACM, 12--21.

Digital Library

[32]

Aliaksei Miniukovich and Antonella De Angeli. 2014. Quantification of Interface Visual Complexity. In Proceedings of the 2014 international working conference on advanced visual interfaces. ACM, 153--160.

Digital Library

[33]

Philippa Mothersill and V. Michael Bove. 2018. An Ontology of Computational Tools for Design Activities. In Proceedings of DRS2018 International Conference.

[34]

Changhoon Oh, Jungwoo Song, Jinhan Choi, Seonghyeon Kim, Sungwoo Lee, and Bongwon Suh. 2018. I Lead, You Help but Only with Enough Details: Understanding User Experience of Co-Creation with Artificial Intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 649.

Digital Library

[35]

OpenCV. 2018. EAST: An Efficient and Accurate Scene Text Detector (EAST text detector). https://www.pyimagesearch.com/2018/08/20/ opencv-text-detection-east-text-detector/

[36]

OpenCV. 2018. Face Detection using Haar Cascades. https://docs. opencv.org/3.4/d7/d8b/tutorial_py_face_detection.html

[37]

OpenCV. 2018. Structural Analysis and Shape Descriptors. https://docs.opencv.org/2.4/modules/imgproc/doc/structural_ analysis_and_shape_descriptors.html

[38]

Charles Sanders Peirce. 1974. Collected Papers of Charles Sanders Peirce. Vol. 5. Harvard University Press.

[39]

Carsten Rother, Lucas Bordeaux, Youssef Hamadi, and Andrew Blake. 2006. AutoCollage. In ACM SIGGRAPH 2006 Papers (SIGGRAPH '06).

Digital Library

[40]

Ben Shneiderman. 2007. Creativity support tools: Accelerating discovery and innovation. Commun. ACM 50, 12 (2007), 20--32.

Digital Library

[41]

Pao Siangliulue, Joel Chan, Krzysztof Z. Gajos, and Steven P. Dow. 2015. Providing Timely Examples Improves the Quantity and Quality of Generated Ideas. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition (C&C '15). ACM, 83--92.

Digital Library

[42]

Kyle Steinfeld. 2017. Dreams may come. In Acadia. 590--599.

[43]

Cem Tekin and Mihaela van der Schaar. 2015. Distributed Online Learning via Cooperative Contextual Bandits. IEEE Transactions on Signal Processing 63, 14 (2015), 3700--3714.

Digital Library

[44]

Belinda Thom. 2000. BoB: an Interactive Improvisational Music Companion. In Proceedings of the fourth international conference on Autonomous agents. ACM, 309--316.

Digital Library

[45]

Jun Xiao, Xuemei Zhang, Phil Cheatle, Yuli Gao, and C Brian Atkins. 2008. Mixed-Initiative Photo Collage Authoring. In Proceedings of the 16th ACM international conference on Multimedia. ACM, 509--518.

Digital Library

[46]

Georgios N Yannakakis, Antonios Liapis, and Constantine Alexopoulos. 2014. Mixed-Initiative Co-Creativity. In FDG.

Cited By

Kwon JJung EKim J(2024)Designer-Generative AI Ideation Process: Generating Images Aligned with Designer Intent in Early-Stage Concept Exploration in Product DesignArchives of Design Research10.15187/adr.2024.07.37.3.737:3(7-23)Online publication date: 31-Jul-2024
https://doi.org/10.15187/adr.2024.07.37.3.7
Jin YYoon JSelf JLee K(2024)Understanding Fashion Designers’ Behavior Using Generative AI for Early-Stage Concept Ideation and RevisionArchives of Design Research10.15187/adr.2024.07.37.3.2537:3(25-45)Online publication date: 31-Jul-2024
https://doi.org/10.15187/adr.2024.07.37.3.25
Joshi STolloczko CWenaas SHolm OLangved M(2024)Investigating How Generative AI Affects Decision-Making in Participatory Design: Shifting the space to make design choicesProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685384(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685384
Show More Cited By

Index Terms

May AI?: Design Ideation with Cooperative Contextual Bandits

Recommendations

Alchemy: experiments in interactive drawing, creativity, & serendipity
C&C '09: Proceedings of the seventh ACM conference on Creativity and cognition

This paper presents an overview of Alchemy, an experimental drawing application aimed at exploring how we can sketch, draw, and create on computers in new ways. Alchemy focuses on the absolute initial stage of the creative process, to provide an ...
Semantically Far Inspirations Considered Harmful?: Accounting for Cognitive States in Collaborative Ideation
C&C '17: Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition

Collaborative ideation systems can help people generate more creative ideas by exposing them to ideas different from their own. However, there are competing theoretical views on whether and when such exposure is helpful. Associationist theory suggests ...
The effect of group composition on divergent thinking in an interaction design activity
DIS '06: Proceedings of the 6th conference on Designing Interactive systems

Nearly 50 years of empirical research has suggested that social influences have an inhibiting effect on creativity in collaborating groups such as design teams. This suggests that design teams may not be as creative as they could be, resulting in a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Research Council (ERC)

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

74
Total Citations
View Citations
5,265
Total Downloads

Downloads (Last 12 months)1,426
Downloads (Last 6 weeks)131

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kwon JJung EKim J(2024)Designer-Generative AI Ideation Process: Generating Images Aligned with Designer Intent in Early-Stage Concept Exploration in Product DesignArchives of Design Research10.15187/adr.2024.07.37.3.737:3(7-23)Online publication date: 31-Jul-2024
https://doi.org/10.15187/adr.2024.07.37.3.7
Jin YYoon JSelf JLee K(2024)Understanding Fashion Designers’ Behavior Using Generative AI for Early-Stage Concept Ideation and RevisionArchives of Design Research10.15187/adr.2024.07.37.3.2537:3(25-45)Online publication date: 31-Jul-2024
https://doi.org/10.15187/adr.2024.07.37.3.25
Joshi STolloczko CWenaas SHolm OLangved M(2024)Investigating How Generative AI Affects Decision-Making in Participatory Design: Shifting the space to make design choicesProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685384(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685384
He JHoude SGonzalez GSilva Moran DRoss SMuller MWeisz J(2024)AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual CanvasProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663398(1-14)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3663384.3663398
Langerak TChristen SAlbaba MGebhardt CHolz CHilliges O(2024)MARLUI: Multi-Agent Reinforcement Learning for Adaptive Point-and-Click UIsProceedings of the ACM on Human-Computer Interaction10.1145/36611478:EICS(1-27)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3661147
Peng XKoch JMackay W(2024)DesignPrompt: Using Multimodal Interaction for Design Exploration with Generative AIProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661588(804-818)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661588
Lin PAndersen KSchmidt RSchoenmakers SHofmeyer HPauwels PIjsselsteijn W(2024)Text-to-Image AI as a Catalyst for Semantic Convergence in Creative CollaborationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661543(2753-2767)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661543
Wang ZShen LKuang EZhang SFan M(2024)Exploring the Impact of Artificial Intelligence-Generated Content (AIGC) Tools on Social Dynamics in UX CollaborationProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660703(1594-1606)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660703
Liu FLv JCui SLuan ZWu KZhou T(2024)Smart "Error"! Exploring Imperfect AI to Support Creative IdeationProceedings of the ACM on Human-Computer Interaction10.1145/36373988:CSCW1(1-28)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637398
Nouraei FSiu ARossi RLipka N(2024)Thinking Outside the Box: Non-Designer Perspectives and Recommendations for Template-Based Graphic Design ToolsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650967(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650967
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents