research-article

Public Access

Creative sketching partner: an analysis of human-AI co-creativity

Authors:

Safat Siddiqui,

Mary Lou Maher,

Nasrin DehbozorgiAuthors Info & Claims

IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces

Pages 221 - 230

https://doi.org/10.1145/3377325.3377522

Published: 17 March 2020 Publication History

Abstract

The creative sketching partner (CSP) is a proof of concept intelligent interface to inspire designers while sketching in response to a specified design task. With this interactive system we are studying the effect of an AI model of visual and conceptual similarity for selecting the Al's sketch response as an inspiration to the current state of the user's sketch. Specifically, we are interested in the user's behavior and response to an AI partner when engaged in a design task. By developing deep learning models of the sketches from a large-scale dataset, the user can control the amount of visual and conceptual similarity of the AI response when requesting inspiration from the CSP. We conducted a study with 50 design students to examine the participants' interaction behavior and their self reports. The participants' behavior maps into clusters that are co-related with three types of design creativity: combinatorial, exploratory, and transformational. Our findings demonstrate that the tool can facilitate ideation and overcome design fixation. In addition, analysis suggests that inspiration related to conceptual similarity is more associated with transformational creativity and inspiration related to visual similarity occurs more frequently during the detailed stages of design and is more prevalent with combinatorial creativity.

References

[1]

John A Biles et al. 1994. GenJam: A genetic algorithm for generating jazz solos. In ICMC, Vol. 94. 131--137.

[2]

Margaret A Boden. 1996. Creativity. In Artificial intelligence. Elsevier, 267--291.

[3]

Nathalie Bonnardel and Franck Zenasni. 2010. The impact of technology on creativity in design: an enhancement? Creativity and innovation management 19, 2 (2010), 180--191.

[4]

V Carbune. 2017. Recurrent neural networks for drawing classification.

[5]

Yajing Chen, Shikui Tu, Yuqi Yi, and Lei Xu. 2017. Sketch-pix2seq: a model to generate sketches of multiple categories. arXiv preprint arXiv:1709.04121 (2017).

[6]

Bo T Christensen and Christian D Schunn. 2005. Spontaneous access and analogical incubation effects. Creativity research journal 17, 2--3 (2005), 207--220.

[7]

Amitava Das and Björn Gambäck. 2014. Poetic Machine: Computational Creativity for Automatic Poetry Generation in Bengali. In ICCC. 230--238.

[8]

Nicholas Davis, Chili-Pin Hsiao, Kunwar Yashraj Singh, and Brian Magerko. 2016. Co-creative drawing agent with object recognition. In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference.

Digital Library

[9]

Kenneth D Forbus and Jeffrey Usher. 2002. Sketching for knowledge capture: A progress report. In Proceedings of the 7th international conference on Intelligent user interfaces. ACM, 71--77.

Digital Library

[10]

Ian Goodfellow. 2016. NIPS 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160 (2016).

[11]

Mark D Gross and Ellen Yi-Luen Do. 1996. The electronic cocktail napkin. Design studies 17, 1 (1996), 53--69.

[12]

David Ha and Douglas Eck. 2017. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017).

[13]

Thu-Huong Ha and Nikhil Sonnad. 2017. How do you draw a circle? https://qz.com/994486/the-way-you-draw-circles-says-a-lot-about-you/

[14]

Guy Hoffman and Gil Weinberg. 2010. Gesture-based human-robot jazz improvisation. In 2010 IEEE International Conference on Robotics and Automation. IEEE, 582--587.

[15]

Chih-Pin Hsiao. 2015. Solidsketch: Toward enactive interactions for semantic model creation. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition. ACM, 329--330.

Digital Library

[16]

Mikhail Jacob, Gaëtan Coisne, Akshay Gupta, Ivan Sysoev, Gaurav Gav Verma, and Brian Magerko. 2013. Viewpoints ai. In Ninth Artificial Intelligence and Interactive Digital Entertainment Conference.

[17]

David G Jansson and Steven M Smith. 1991. Design fixation. Design studies 12, 1 (1991), 3--11.

[18]

Gabe Johnson, Mark Gross, Ellen Yi-Luen Do, and Jason Hong. 2012. Sketch it, make it: sketching precise drawings for laser cutting. In CHI'12 Extended Abstracts on Human Factors in Computing Systems. ACM, 1079--1082.

Digital Library

[19]

Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, and Nick Fox-Gieg. 2016. The quick, draw!-ai experiment. Mount View, CA, accessed Feb 17 (2016), 2018.

[20]

Pegah Karimi, Nicholas Davis, Mary Lou Maher, Kazjon Grace, and Lina Lee. 2019. Relating Cognitive Models of Design Creativity to the Similarity of Sketches Generated by an AI Partner. In Proceedings of the 2019 on Creativity and Cognition. ACM, 259--270.

Digital Library

[21]

Pegah Karimi, Kazjon Grace, Nicholas Davis, and Mary Lou Maher. 2018. Creative sketching apprentice: Supporting conceptual shifts in sketch ideation. In International Conference on-Design Computing and Cognition. Springer, 721--738.

[22]

Pegah Karimi, Mary Lou Maher, Nicholas Davis, and Kazjon Grace. 2019. Deep Learning in a Computational Model for Conceptual Shifts in a Co-Creative Design System. arXiv preprint arXiv:1906.10188 (2019).

[23]

Antonios Liapis, Georgios N Yannakakis, and Julian Togelius. 2013. Sentient Sketchbook: Computer-aided game level authoring. In FDG. 213--220.

[24]

Julie S Linsey, Ian Tseng, Katherine Fu, Jonathan Cagan, Kristin L Wood, and C Schunn. 2010. A study of design fixation, its mitigation and perception in engineering design faculty. Journal of Mechanical Design 132, 4 (2010), 041003.

[25]

Pedro Lucas and Carlos Martinho. 2017. Stay Awhile and Listen to 3Buddy, a Co-creative Level Design Support Tool. In ICCC. 205--212.

[26]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).

[27]

Diana P Moreno, Maria C Yang, Alberto A Hernández, Julie S Linsey, and Kristin L Wood. 2015. A step beyond to overcome design fixation: a design-by-analogy approach. In Design Computing and Cognition'14. Springer, 607--624.

[28]

A Terry Purcell and John S Gero. 1996. Design and other types of fixation. Design studies 17, 4 (1996), 363--383.

[29]

Jami J Shah, Steve M Smith, and Noe Vargas-Hernandez. 2003. Metrics for measuring ideation effectiveness. Design studies 24, 2 (2003), 111--134.

[30]

Miaomiao Wen, Nancy Baym, Omer Tamuz, Jaime Teevan, Susan T Dumais, and Adam Kalai. 2015. OMG UR Funny! Computer-Aided Humor with an Application to Chat. In ICCC. 86--93.

[31]

Xinyue Yang. 2018. Scribbling Speech. http://xinyue.de/scribbling-speech.html Available at = http://xinyue.de/scribbling-speech.html.

[32]

Doris Zahner, Jeffrey V Nickerson, Barbara Tversky, James E Corter, and Jing Ma. 2010. A fix for fixation? Rerepresenting and abstracting as creative processes in the design of information systems. AIEDAM 24, 2 (2010), 231--244.

Digital Library

[33]

Matthew D Zeiler, Graham W Taylor, Rob Fergus, et al. 2011. Adaptive deconvolutional networks for mid and high level feature learning. In ICCV, Vol. 1. 6.

Cited By

Wang BZhang KChen ZShen LShen XLiu YBian JShen H(2025)An intelligent font generation system based on stroke inference, mitigating production labor and enhancing design experienceExpert Systems with Applications10.1016/j.eswa.2024.125657263(125657)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125657
Soroush MSourav MZeng Y(2025)Creativity and AIArtificial Intelligence for Design and Process Science10.1007/978-3-031-67886-8_3(29-43)Online publication date: 14-Jan-2025
https://doi.org/10.1007/978-3-031-67886-8_3
Pei YWang LXue C(2024)Human–AI Co-Drawing: Studying Creative Efficacy and Eye Tracking in Observation and CooperationApplied Sciences10.3390/app1418820314:18(8203)Online publication date: 12-Sep-2024
https://doi.org/10.3390/app14188203
Show More Cited By

Index Terms

Creative sketching partner: an analysis of human-AI co-creativity
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Collaborative interaction

Recommendations

Relating Cognitive Models of Design Creativity to the Similarity of Sketches Generated by an AI Partner
C&C '19: Proceedings of the 2019 Conference on Creativity and Cognition

This paper presents and evaluates a new method for inspiring creativity in a co-creative design system. The method uses a computational model of aconceptual shift based on clustering of deep features from a database of sketches. The co-creative ...
"Sketching" nurturing creativity: commonalities in art, design, engineering and research
CHI EA '06: CHI '06 Extended Abstracts on Human Factors in Computing Systems

The workshop seeks to bring together researchers and practitioners from diverse creative practices such as interaction design, industrial design, architectural design, media art, music, programming, writing, and scholarly work, to gain insight into the ...
Sketching for Ideation: A Structured Approach for Increasing Divergent Thinking
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Sketching is a valuable technique used across a variety of industries to aid creative thought. Although the benefits of sketching are widely known, there is limited instruction for the application of sketching in problem solving, particularly in the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces

March 2020

607 pages

ISBN:9781450371186

DOI:10.1145/3377325

General Chairs:
Fabio Paternò,
Nuria Oliver,
Program Chairs:
Cristina Conati,
Lucio Davide Spano,
Nava Tintarev

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Conference

IUI '20

Sponsor:

IUI '20: 25th International Conference on Intelligent User Interfaces

March 17 - 20, 2020

Cagliari, Italy

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

60
Total Citations
View Citations
4,531
Total Downloads

Downloads (Last 12 months)1,645
Downloads (Last 6 weeks)192

Reflects downloads up to 15 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang BZhang KChen ZShen LShen XLiu YBian JShen H(2025)An intelligent font generation system based on stroke inference, mitigating production labor and enhancing design experienceExpert Systems with Applications10.1016/j.eswa.2024.125657263(125657)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125657
Soroush MSourav MZeng Y(2025)Creativity and AIArtificial Intelligence for Design and Process Science10.1007/978-3-031-67886-8_3(29-43)Online publication date: 14-Jan-2025
https://doi.org/10.1007/978-3-031-67886-8_3
Pei YWang LXue C(2024)Human–AI Co-Drawing: Studying Creative Efficacy and Eye Tracking in Observation and CooperationApplied Sciences10.3390/app1418820314:18(8203)Online publication date: 12-Sep-2024
https://doi.org/10.3390/app14188203
Shojaei FShojaei FOsorio Torres JShih P(2024)Insights From Art Therapists on Using AI-Generated Art in Art Therapy: Mixed Methods StudyJMIR Formative Research10.2196/630388(e63038-e63038)Online publication date: 4-Dec-2024
https://doi.org/10.2196/63038
Zhang HChen PXie XJiang ZZhou ZSun L(2024)A Hybrid Prototype Method Combining Physical Models and Generative Artificial Intelligence to Support Creativity in Conceptual DesignACM Transactions on Computer-Human Interaction10.1145/368943331:5(1-34)Online publication date: 2-Sep-2024
https://dl.acm.org/doi/10.1145/3689433
Arabzadeh NDiaz FHe JSakai TIshita EOhshima HHasibi FMao JJose J(2024)Offline Evaluation of Set-Based Text-to-Image GenerationProceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region10.1145/3673791.3698424(42-53)Online publication date: 8-Dec-2024
https://dl.acm.org/doi/10.1145/3673791.3698424
Antony VHuang C(2024)ID.8: Co-Creating Visual Stories with Generative AIACM Transactions on Interactive Intelligent Systems10.1145/367227714:3(1-29)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3672277
He JHoude SGonzalez GSilva Moran DRoss SMuller MWeisz J(2024)AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual CanvasProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663398(1-14)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3663384.3663398
Muller MWeisz JHoude SRoss S(2024)Drinking Chai with Your (AI) Programming Partner: Value Tensions in the Tokenization of Future Human-AI Collaborative WorkProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663390(1-15)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3663384.3663390
Yoo DJoo J(2024)BI-CST: Behavioral Science-based Creativity Support Tool for Overcoming Design Fixation.Companion Publication of the 2024 ACM Designing Interactive Systems Conference10.1145/3656156.3663704(116-120)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3656156.3663704
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten