abstract

On How Users Edit Computer-Generated Visual Stories

Authors:

Ting-Hao (Kenneth) HuangAuthors Info & Claims

CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: LBW2711, Pages 1 - 6

https://doi.org/10.1145/3290607.3312965

Published: 02 May 2019 Publication History

Abstract

A significant body of research in Artificial Intelligence (AI) has focused on generating stories automatically, either based on prior story plots or input images. However, literature has little to say about how users would receive and use these stories. Given the quality of stories generated by modern AI algorithms, users will nearly inevitably have to edit these stories before putting them to real use. In this paper, we present the first analysis of how human users edit machine-generated stories. We obtained 962 short stories generated by one of the state-of-the-art visual storytelling models. For each story, we recruited five crowd workers from Amazon Mechanical Turk to edit it. Our analysis of these edits shows that, on average, users (i) slightly shortened machine-generated stories, (ii) increased lexical diversity in these stories, and (iii) often replaced nouns and their determiners/articles with pronouns. Our study provides a better understanding on how users receive and edit machine-generated stories, informing future researchers to create more usable and helpful story generation systems.

References

[1]

Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. O'Reilly Media, Inc.

Digital Library

[2]

Elizabeth Clark, Anne Spencer Ross, Chenhao Tan, Yangfeng Ji, and Noah A. Smith. 2018. Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories. In 23rd International Conference on Intelligent User Interfaces (IUI '18). ACM, New York, NY, USA, 329--340.

Digital Library

[3]

Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical Neural Story Generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 889--898. http://aclweb.org/anthology/P18--1082

[4]

C. Gan, Z. Gan, X. He, J. Gao, and L. Deng. 2017. StyleNet: Generating Attractive Visual Captions with Styles. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 955--964.

[5]

Andrew Hardie and Tony McEnery. 2006. Statistics. Vol. 12. Elsevier, 138--146.

[6]

Chao-Chun Hsu, Szu-Min Chen, Ming-Hsun Hsieh, and Lun-Wei Ku. 2018. Using Inter-Sentence Diverse Beam Search to Reduce Redundancy in Visual Storytelling. arXiv preprint arXiv:1805.11867 (2018).

[7]

Ting-Hao Kenneth Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, et al. 2016. Visual storytelling. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1233--1239.

[8]

Taehyeong Kim, Min-Oh Heo, Seonil Son, Kyoung-Wha Park, and Byoung-Tak Zhang. 2018. GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generation. arXiv preprint arXiv:1805.10973 (2018).

[9]

Lara J. Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O. Riedl. 2018. Event Representations for Automated Story Generation with Deep Neural Nets. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 868--875. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17046

[10]

Alexander Mathews, Lexing Xie, and Xuming He. 2018. SemStyle: Learning to Generate Stylised Image Captions Using Unaligned Text. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]

Margaret Mitchell, Francis Ferraro, Ishan Misra, et al. 2018. Proceedings of the First Workshop on Storytelling. In Proceedings of the First Workshop on Storytelling.

[12]

Melissa Roemmele. 2016. Writing stories with help from recurrent neural networks. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. AAAI Press, 4311--4312.

Digital Library

[13]

Melissa Roemmele and Andrew S. Gordon. 2015. Creative Help: A Story Writing Assistant. In Interactive Storytelling, Henrik Schoenau-Fog, Luis Emilio Bruni, Sandy Louchart, and Sarune Baceviciute (Eds.). Springer International Publishing, Cham, 81--92.

[14]

Reid Swanson and Andrew S. Gordon. 2012. Say Anything: Using Textual Case-Based Reasoning to Enable Open-Domain Interactive Storytelling. ACM Trans. Interact. Intell. Syst. 2, 3, Article 16 (Sept. 2012), 35 pages.

Digital Library

[15]

Xin Wang, Wenhu Chen, Yuan-Fang Wang, and William Yang Wang. 2018. No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. ACL, Melbourne, Victoria, Australia.

Cited By

Sevim OAslan Cobutoğlu SÖzbek ÖAkça M(2024)Yapay Zekânın Edebiyatta Kullanım SerüveniRumeliDE Dil ve Edebiyat Araştırmaları Dergisi10.29000/rumelide.1470139Online publication date: 20-Apr-2024
https://doi.org/10.29000/rumelide.1470139
Halperin BLukin S(2024)Artificial Dreams: Surreal Visual Storytelling as Inquiry Into AI 'Hallucination'Proceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660685(619-637)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660685
Lee MGero KChung JShum SRaheja VShen HVenugopalan SWambsganss TZhou DAlghamdi EAugust TBhat AChoksi MDutta SGuo JHoque MKim YKnight SNeshaei SShibani AShrivastava DShroff LSergeyuk AStark JSterman SWang SBosselut ABuschek DChang JChen SKreminski MPark JPea RRho EShen ZSiangliulue P(2024)A Design Space for Intelligent and Interactive Writing AssistantsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642697(1-35)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642697
Show More Cited By

Index Terms

On How Users Edit Computer-Generated Visual Stories
1. Computing methodologies
  1. Machine learning
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools

Recommendations

TaleBrush: Sketching Stories with Generative Pretrained Language Models
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

While advanced text generation algorithms (e.g., GPT-3) have enabled writers to co-create stories with an AI, guiding the narrative remains a challenge. Existing systems often leverage simple turn-taking between the writer and the AI in story ...
Dynamic Syuzhets: Writing and Design Methods for Playable Stories
Interactive Storytelling
Abstract
The holodeck vision of the future of Interactive Digital Storytelling (IDS) assumes a world that reacts around players as story protagonists; but, we have seen how this approach faces challenges in negotiating the delivery of narrative affect and ...
Generating Educational Interactive Stories in Computer Role-Playing Games
EC-TEL '09: Proceedings of the 4th European Conference on Technology Enhanced Learning: Learning in the Synergy of Multiple Disciplines

The aim of interactive storytelling is to tell stories with the use of computers in a new and interactive way, which immerses the reader inside the story as the protagonist and enables him to drive its course in any desired direction. Interactive ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

3673 pages

ISBN:9781450359719

DOI:10.1145/3290607

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Check for updates

Author Tags

Qualifiers

Abstract

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
243
Total Downloads

Downloads (Last 12 months)41
Downloads (Last 6 weeks)2

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sevim OAslan Cobutoğlu SÖzbek ÖAkça M(2024)Yapay Zekânın Edebiyatta Kullanım SerüveniRumeliDE Dil ve Edebiyat Araştırmaları Dergisi10.29000/rumelide.1470139Online publication date: 20-Apr-2024
https://doi.org/10.29000/rumelide.1470139
Halperin BLukin S(2024)Artificial Dreams: Surreal Visual Storytelling as Inquiry Into AI 'Hallucination'Proceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660685(619-637)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660685
Lee MGero KChung JShum SRaheja VShen HVenugopalan SWambsganss TZhou DAlghamdi EAugust TBhat AChoksi MDutta SGuo JHoque MKim YKnight SNeshaei SShibani AShrivastava DShroff LSergeyuk AStark JSterman SWang SBosselut ABuschek DChang JChen SKreminski MPark JPea RRho EShen ZSiangliulue P(2024)A Design Space for Intelligent and Interactive Writing AssistantsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642697(1-35)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642697
Fang XNg DLeung JChu S(2023)A systematic review of artificial intelligence technologies used for story writingEducation and Information Technologies10.1007/s10639-023-11741-528:11(14361-14397)Online publication date: 5-Apr-2023
https://doi.org/10.1007/s10639-023-11741-5
Chung JKim WYoo KLee HAdar EChang M(2022)TaleBrush: Sketching Stories with Generative Pretrained Language ModelsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501819(1-19)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501819
Cho HYun GLee JNam T(2022)Understanding AI-Generated Personal Narratives as Design Material for Socially Engaging Things[ ] With Design: Reinventing Design Modes10.1007/978-981-19-4472-7_75(1135-1152)Online publication date: 6-Nov-2022
https://doi.org/10.1007/978-981-19-4472-7_75
Chung JHe SAdar E(2021)The Intersection of Users, Roles, Interactions, and Technologies in Creativity Support ToolsProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462050(1817-1833)Online publication date: 28-Jun-2021
https://dl.acm.org/doi/10.1145/3461778.3462050
Strengers YQu LXu QKnibbe JBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Adhering, Steering, and Queering: Treatment of Gender in Natural Language GenerationProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376315(1-14)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376315

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents