Work in Progress

"It's Really Enjoyable to See Me Solve the Problem like a Hero": GenAI-enhanced Data Comics as a Learning Analytics Tool

Authors:

Mikaela E Milesi,

Riordan Alfredo,

Vanessa Echeverria,

Roberto Martinez-MaldonadoAuthors Info & Claims

CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

Article No.: 4, Pages 1 - 7

https://doi.org/10.1145/3613905.3651111

Published: 11 May 2024 Publication History

Abstract

Data comics are an emergent storytelling format that can enable non-experts to consume salient insights from data. Despite some research investigating the use of comic strips in education, there is limited evidence relating to how students perceive data comics about their own data as a way to reflect on their learning experience. In this paper, we summarise nursing students’ perceptions of the advantages and limitations of data comics by presenting personalised Multimodal Learning Analytics (LA) data in data comics form. We present GenAI-enhanced data comic prototypes created using a combination of the generative artificial intelligence tool, Midjourney, and graphics illustration software. Our findings indicate that while students see the potential of data comics as an engaging and enjoyable visualisation technique, concerns remain regarding the perceived lack of professionalism associated with this format.

Supplemental Material

MP4 File

Talk Video

Transcript for: Talk Video

References

[1]

Benjamin Bach, Natalie Kerracher, Kyle Wm. Hall, Sheelagh Carpendale, Jessie Kennedy, and Nathalie Henry Riche. 2016. Telling Stories about Dynamic Networks with Graph Comics. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 3670–3682. https://doi.org/10.1145/2858036.2858387

Digital Library

[2]

Benjamin Bach, Nathalie Henry Riche, Sheelagh Carpendale, and Hanspeter Pfister. 2017. The Emerging Genre of Data Comics. IEEE Computer Graphics and Applications 37, 3 (May 2017), 6–13. https://doi.org/10.1109/MCG.2017.33

Digital Library

[3]

Benjamin Bach, D. Stefaner, J. Boy, S. Drucker, L. Bartram, J. Wood, P. Ciuccarelli, Yuri Engelhardt, U. Köppen, and B. Tversky. 2018. Narrative Design Patterns for Data-Driven Storytelling. CRC Press (Taylor & Francis), New York, 107–133. https://doi.org/10.1201/9781315281575-5

[4]

Benjamin Bach, Zezhong Wang, Matteo Farinella, Dave Murray-Rust, and Nathalie Henry Riche. 2018. Design Patterns for Data Comics. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173612

Digital Library

[5]

Magdalena Boucher, Benjamin Bach, Christina Stoiber, Zezhong Wang, and Wolfgang Aigner. 2023. Educational Data Comics: What can Comics do for Education in Visualization?. In 2023 IEEE VIS Workshop on Visualization Education, Literacy, and Activities (EduVis). IEEE, Melbourne, Australia, 34–40. https://doi.org/10.1109/EduVis60792.2023.00012

[6]

Blazenka Divjak, Barbi Svetec, and Damir Horvat. 2023. Learning Analytics Dashboards: What Do Students Actually Ask For?. In LAK23: 13th International Learning Analytics and Knowledge Conference (Arlington, TX, USA) (LAK2023). Association for Computing Machinery, New York, NY, USA, 44–56. https://doi.org/10.1145/3576050.3576141

Digital Library

[7]

Vanessa Echeverria, Roberto Martinez-Maldonado, Simon Buckingham Shum, Katherine Chiluiza, Roger Granda, and Cristina Conati. 2018. Exploratory versus Explanatory Visual Learning Analytics: Driving Teachers’ Attention through Educational Data Storytelling. Journal of Learning Analytics 5, 3 (Nov. 2018), 73—97. https://doi.org/10.18608/jla.2018.53.6

[8]

Vanessa Echeverria, Lixiang Yan, Linxuan Zhao, Sophie Abel, Riordan Alfredo, Samantha Dix, Hollie Jaggard, Rosie Wotherspoon, Abra Osborne, Simon Buckingham Shum, Dragan Gasevic, and Roberto Martinez-Maldonado. 2024. TeamSlides: a Multimodal Teamwork Analytics Dashboard for Teacher-guided Reflection in a Physical Learning Space. In Proceedings of the 14th Learning Analytics and Knowledge Conference (, Kyoto, Japan, ) (LAK ’24). Association for Computing Machinery, New York, NY, USA, 112–122. https://doi.org/10.1145/3636555.3636857

Digital Library

[9]

Alejandra Gómez Ortega, Jacky Bourgeois, and Gerd Kortuem. 2024. Personal Data Comics: A Data Storytelling Approach Supporting Personal Data Literacy. In Proceedings of the XI Latin American Conference on Human Computer Interaction (Puebla, Mexico) (CLIHC ’23). Association for Computing Machinery, New York, NY, USA, Article 2, 8 pages. https://doi.org/10.1145/3630970.3630982

Digital Library

[10]

Yuxuan Huang. 2023. The Future of Generative AI: How GenAI Would Change Human-Computer Co-creation in the Next 10 to 15 Years. In Companion Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Stratford, ON, Canada) (CHI PLAY Companion ’23). Association for Computing Machinery, New York, NY, USA, 322–325. https://doi.org/10.1145/3573382.3616033

Digital Library

[11]

DaYe Kang, Tony Ho, Nicolai Marquardt, Bilge Mutlu, and Andrea Bianchi. 2021. ToonNote: Improving Communication in Computational Notebooks Using Interactive Data Comics. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 727, 14 pages. https://doi.org/10.1145/3411764.3445434

Digital Library

[12]

Riina Kleimola and Irja Leppisaari. 2022. Learning analytics to develop future competences in higher education: a case study. International Journal of Educational Technology in Higher Education 19, 1 (2022), 17. https://doi.org/10.1186/s41239-022-00318-w Export Date: 16 August 2022; Cited By: 0.

[13]

Cole Nussbaumer Knaflic. 2015. Storytelling with Data: A Data Visualization Guide for Business Professionals. John Wiley & Sons, Inc., Hoboken, New Jersey.

[14]

Roberto Martinez-Maldonado, Vanessa Echeverria, Gloria Fernandez Nieto, and Simon Buckingham Shum. 2020. From Data to Insights: A Layered Storytelling Approach for Multimodal Learning Analytics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3313831.3376148

Digital Library

[15]

Camillia Matuk, Talia Hurwich, Amy Spiegel, and Judy Diamond. 2021. How do teachers use comics to promote engagement, equity, and diversity in science classrooms?Research in Science Education 51 (2021), 685–732. https://doi.org/10.1007/s11165-018-9814-8

[16]

Scott McCloud. 2006. Making comics : storytelling secrets of comics, manga and graphic novels (1st ed. ed.). HarperPerennial, New York.

[17]

Lucas Paulsen and Euan Lindsay. 2024. Learning analytics dashboards are increasingly becoming about learning and not just analytics - A systematic review. Education and Information Technologies (2024), 30 pages. https://doi.org/10.1007/s10639-023-12401-4

Digital Library

[18]

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. arXiv preprint arXiv:2212.04356 (2022).

[19]

Edward Segel and Jeffrey Heer. 2010. Narrative Visualization: Telling Stories with Data. IEEE Transactions on Visualization and Computer Graphics 16, 6 (Nov 2010), 1139–1148. https://doi.org/10.1109/TVCG.2010.179

Digital Library

[20]

Wenxuan Wang, Haonan Bai, Jen tse Huang, Yuxuan Wan, Youliang Yuan, Haoyi Qiu, Nanyun Peng, and Michael R. Lyu. 2024. New Job, New Gender? Measuring the Social Bias in Image Generation Models. arxiv:2401.00763 [cs.SE]

[21]

Zezhong Wang, Harvey Dingwall, and Benjamin Bach. 2019. Teaching Data Visualization and Storytelling with Data Comic Workshops. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3290607.3299043

Digital Library

[22]

Zezhong Wang, Hugo Romat, Fanny Chevalier, Nathalie Henry Riche, Dave Murray-Rust, and Benjamin Bach. 2022. Interactive Data Comics. IEEE Transactions on Visualization and Computer Graphics 28, 1 (Jan 2022), 944–954. https://doi.org/10.1109/TVCG.2021.3114849

Digital Library

[23]

Zezhong Wang, Lovisa Sundin, Dave Murray-Rust, and Benjamin Bach. 2020. Cheat Sheets for Data Visualization Techniques. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376271

Digital Library

[24]

Zezhong Wang, Shunming Wang, Matteo Farinella, Dave Murray-Rust, Nathalie Henry Riche, and Benjamin Bach. 2019. Comparing Effectiveness and Engagement of Data Comics and Infographics. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300483

Digital Library

[25]

Linxuan Zhao, Vanessa Echeverria, Zachari Swiecki, Lixiang Yan, Riordan Alfredo, Xinyu Li, Dragan Gasevic, and Roberto Martinez-Maldonado. 2024. Epistemic Network Analysis for End-users: Closing the Loop in the Context of Multimodal Analytics for Collaborative Team Learning. In LAK24: 14th International Learning Analytics and Knowledge Conference (Kyoto, Japan) (LAK2024). Association for Computing Machinery, New York, NY, USA, in press. https://doi.org/10.1145/3636555.3636855

Digital Library

[26]

Zhenpeng Zhao, Rachael Marr, Jason Shaffer, and Niklas Elmqvist. 2019. Understanding Partitioning and Sequence in Data-Driven Storytelling. In Information in Contemporary Society, Natalie Greene Taylor, Caitlin Christian-Lamb, Michelle H. Martin, and Bonnie Nardi (Eds.). Springer International Publishing, Cham, 327–338.

Cited By

Yan LEcheverria VJin YFernandez‐Nieto GZhao LLi XAlfredo RSwiecki ZGašević DMartinez‐Maldonado R(2024)Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learningBritish Journal of Educational Technology10.1111/bjet.1349855:5(1900-1925)Online publication date: 22-Jun-2024
https://doi.org/10.1111/bjet.13498

Index Terms

"It's Really Enjoyable to See Me Solve the Problem like a Hero": GenAI-enhanced Data Comics as a Learning Analytics Tool
1. Human-centered computing
  1. Visualization
    1. Visualization design and evaluation methods

Recommendations

Big Data Analytics
Harnessing the power of big data analytics in the cloud to support learning analytics in mobile learning environment
Abstract
Technology enhanced learning (TEL) such as online learning environment with adaptive technologies has gained growing interest in recent past in the field of teaching and learning. In this context, mobile learning has got much momentum ...
Highlights
- Big data analytics in the cloud to support learning analytics for mobile learning.
Analytics on talent search examination data

Learning analytics and educational data mining has greatly supported the process of assessing and improving the quality of education. While learning analytics has a longer development cycle, educational data mining suffers from the inadequacy of data ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

May 2024

4761 pages

ISBN:9798400703317

DOI:10.1145/3613905

Editors:
Florian Floyd Mueller
Monash University
,
Penny Kyburz
The Australian National University
,
Julie R. Williamson
University of Glasgow
,
Corina Sas
Lancaster University

Copyright © 2024 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Check for updates

Author Tags

Qualifiers

Work in progress
Research
Refereed limited

Funding Sources

Jacobs Foundation

Conference

CHI '24

Sponsor:

CHI '24: CHI Conference on Human Factors in Computing Systems

May 11 - 16, 2024

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
773
Total Downloads

Downloads (Last 12 months)773
Downloads (Last 6 weeks)172

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yan LEcheverria VJin YFernandez‐Nieto GZhao LLi XAlfredo RSwiecki ZGašević DMartinez‐Maldonado R(2024)Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learningBritish Journal of Educational Technology10.1111/bjet.1349855:5(1900-1925)Online publication date: 22-Jun-2024
https://doi.org/10.1111/bjet.13498

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Table of Contents