Identify Design Problems Through Questioning: Exploring Role-playing Interactions with Large Language Models to Foster Design Questioning Skills
Pages 598 - 602
Abstract
Identifying design problems is a crucial step for creating plausible solutions, but it is challenging for design novices due to their limited knowledge and experience. Questioning is a promising skill that enables students to independently identify design problems without being passive or relying on instructors. This study explores role-playing interactions with Large Language Model (LLM)-powered Conversational Agents (CAs) to foster the questioning skills of novice design students. We proposed an LLM-powered CA prototype and conducted a preliminary study with 16 novice design students engaged in a real-world design class to observe the interactions between students and the LLM-powered CAs. Our findings indicate that while the CAs stimulated questioning and reduced pressure to ask questions, it also inadvertently led to over-reliance on LLM responses. We proposed design considerations and future works for LLM-powered CA to foster questioning skills.
References
[1]
Mohd. Yusof Abdullah, Noor Rahamah Abu Bakar, and Maizatul Haizan Mahbob. 2012. Students Participation in Classroom:What Motivates them to Speak up Procedia - Social and Behavioral Sciences 51 (2012), 516--522. https://doi.org/10. 1016/j.sbspro.2012.08.199 The World Conference on Design, Arts and Education (DAE-2012), May 1--3 2012, Antalya, Turkey.
[2]
Paola Andreucci-Annunziata, Andrea Riedemann, Susana Cortés, Augusto Mellado, María Teresa del Río, and Alejandro Vega-Muñoz. 2023. Conceptualizations and instructional strategies on critical thinking in higher education: A systematic review of systematic reviews. Frontiers in Education 8 (2023). https://doi.org/10.3389/feduc.2023.1141686
[3]
Aoife Ahern, Caroline Dominguez, Ciaran McNally, John J. OSullivan, and Daniela Pedrosa. 2019. A literature review of critical thinking in engineering education. Studies in Higher Education 44, 5 (2019), 816--828. https://doi.org/10. 1080/03075079.2019.1586325 arXiv:https://doi.org/10.1080/03075079.2019.1586325
[4]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77--101. https://doi.org/10.1191/1478088706qp063oa arXiv:https://www.tandfonline.com/doi/pdf/10.1191/1478088706qp063oa
[5]
Zhipeng Chen, Kun Zhou, Beichen Zhang, Zheng Gong, Wayne Xin Zhao, and Ji-Rong Wen. 2023. ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models. arXiv:2305.14323 [cs.CL] https://arxiv.org/ abs/2305.14323
[6]
Peter Dalsgaard, Christian Dindler, and Jonas Fritsch. 2013. Design argumentation in academic design education. Nordes 1, 5 (2013).
[7]
Kees Dorst and Nigel Cross. 2001. Creativity in the design process: co-evolution of problem--solution. Design studies 22, 5 (2001), 425--437.
[8]
Tonia Dousay, Jennifer LaVanchy, Trina Kilty, and Dillon Stone. 2017. Confessions of Novice Designers and Their Instructor. International Journal of Designs for Learning 8, 2 (November 2017). https://www.learntechlib.org/p/209602
[9]
Nada El Soufi and Beng Huat See. 2019. Does explicit teaching of critical thinking improve critical thinking skills of English language learners in higher education A critical review of causal evidence. Studies in Educational Evaluation 60 (2019), 140--162. https://doi.org/10.1016/j.stueduc.2018.12.006
[10]
Ozgur Eris. 2004. Effective inquiry for innovative engineering design. Vol. 10. Springer Science & Business Media.
[11]
Heidi A. Diefes-Dux Farshid Marbouti, John Mendoza-Garcia and Monica E. Cardella. 2019. Written feedback provided by first-year engineering students, undergraduate teaching assistants, and educators on design project work. European Journal of Engineering Education 44, 1--2 (2019), 179--195. https://doi.org/10.1080/ 03043797.2017.1340931 arXiv:https://doi.org/10.1080/03043797.2017.1340931
[12]
Logan Fiorella and Richard E. Mayer. 2013. The relative benefits of learning by teaching and teaching expectancy. Contemporary Educational Psychology 38, 4 (2013), 281--288. https://doi.org/10.1016/j.cedpsych.2013.06.001
[13]
Ada Hurst and Oscar G Nespoli. 2019. Comparing instructor and student verbal feedback in design reviews of a capstone design course: Differences in topic and function. International Journal of Engineering Education 35, 1 (2019), 221--231.
[14]
Hyoungwook Jin, Seonghee Lee, Hyungyu Shin, and Juho Kim. 2024. Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Education. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24). Association for Computing Machinery, New York, NY, USA, Article 652, 28 pages. https: //doi.org/10.1145/3613904.3642349
[15]
Edward Junprung. 2023. Exploring the Intersection of Large Language Models and Agent-Based Modeling via Prompt Engineering. arXiv:2308.07411 [cs.AI] https://arxiv.org/abs/2308.07411
[16]
Wei Leong Leon Loh, Hwee Mui Grace Kwek, and Wei Leong Lee. 2017. Reclarifying design problems through questions for secondary school children: An example based on design problem identification in Singapore pre-tertiary design education. (2017).
[17]
OpenAI. 2024. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https://arxiv. org/abs/2303.08774
[18]
Richard Paul and Linda Elder. 2007. Critical thinking: The art of Socratic questioning. Journal of developmental education 31, 1 (2007), 36.
[19]
Zhenhui Peng, Qiaoyi Chen, Zhiyu Shen, Xiaojuan Ma, and Antti Oulasvirta. 2024. DesignQuizzer: A Community-Powered Conversational Agent for Learning Visual Design. Proc. ACM Hum.-Comput. Interact. 8, CSCW1, Article 44 (apr 2024), 40 pages. https://doi.org/10.1145/3637321
[20]
Deepa Rao and Ieva Stupans. 2012. Exploring the potential of role play in higher education: development of a typology and teacher guidelines. Innovations in Education and Teaching International 49, 4 (2012), 427--436. https://doi.org/10. 1080/14703297.2012.728879 arXiv:https://doi.org/10.1080/14703297.2012.728879
[21]
Emma Sabzalieva and Arianna Valentini. 2023. ChatGPT and artificial intelligence in higher education: quick start guide. (2023).
[22]
Omar Shaikh, Valentino Emil Chai, Michele Gelfand, Diyi Yang, and Michael S. Bernstein. 2024. Rehearsal: Simulating Conflict to Teach Conflict Resolution. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24). Association for Computing Machinery, New York, NY, USA, Article 920, 20 pages. https://doi.org/10.1145/3613904.3642159
Index Terms
- Identify Design Problems Through Questioning: Exploring Role-playing Interactions with Large Language Models to Foster Design Questioning Skills
Recommendations
Design of agent tutee's question prompts to engage student's role-playing as tutor in a learning-by-teaching agent environment
ICLS'08: Proceedings of the 8th international conference on International conference for the learning sciences - Volume 2A learning-by-teaching environment can be used to create a context in which the student can play the role of tutor teaching the agent tutee. Without meaningful feedback from the agent, there is no reason to expect much student's engagement with the ...
An application of a virtual learning environment in support of teaching and learning for design and technology education
Audiovisual advances in Virtual Reality (VR) technology have given rise to innovative ways to teach and learn. However, so far, teaching and learning processes have been technologically driven as opposed to pedagogically led. This article identifies the ...
How learning works in design education: educating for creative awareness through formative reflexivity
DIS '12: Proceedings of the Designing Interactive Systems ConferenceThis paper reviews and extends educational principles from recent learning sciences literature to address the nuanced needs of creative design education. We have performed a variety of ethnographic and qualitative research activities, including ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2024
755 pages
ISBN:9798400711145
DOI:10.1145/3678884
- General Chairs:
- Rosta Farzan,
- Claudia López,
- Program Chairs:
- Daniel Cardoso Llach,
- Daniele Quercia,
- Maryam Mustafa,
- Shuo Niu,
- Marisol Wong-Villacrés
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 November 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
CSCW '24
Sponsor:
CSCW '24: The 27th ACM Conference on Computer-Supported Cooperative Work and Social Computing
November 9 - 13, 2024
San Jose, Costa Rica
Acceptance Rates
Overall Acceptance Rate 2,235 of 8,521 submissions, 26%
Upcoming Conference
CSCW '25
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 19Total Downloads
- Downloads (Last 12 months)19
- Downloads (Last 6 weeks)19
Reflects downloads up to 20 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in