Initial results on personalizing explanations of AI hints in an ITS
Pages 244 - 248
Abstract
Previous research on an Intelligent Tutoring System (referred to as ACSP), showed the need to personalize explanations of its AI-driven hints for users with low Need for Cognition (N4C) and low Conscientiousness (Cons.). Specifically, this work found that explanations should be provided to these users with the objective of increasing user interaction with them. In this paper, we present and evaluate design alterations to the original ACSP explanation interface aimed at achieving this objective. Our results provide initial evidence that the implemented personalization, in the form of the design alterations, had a positive impact on users with low N4C and Cons., by increasing attention to explanations and contributing to learning gains.
References
[1]
Andrea Bunt, Matthew Lount, and Catherine Lauzon. 2012. Are explanations always important? A study of deployed, low-cost intelligent interactive systems. In Proceedings of the 2012 ACM international conference on Intelligent User Interfaces. 169–178.
[2]
John T Cacioppo, Richard E Petty, and Chuan Feng Kao. 1984. The efficient assessment of need for cognition. Journal of personality assessment 48, 3 (1984), 306–307.
[3]
Cristina Conati, Oswald Barral, Vanessa Putnam, and Lea Rieger. 2021. Toward personalized XAI: A case study in intelligent tutoring systems. Artificial intelligence 298 (2021), 103503.
[4]
Kate Ehrlich, Susanna E Kirk, John Patterson, Jamie C Rasmussen, Steven I Ross, and Daniel M Gruen. 2011. Taking advice from intelligent systems: the double-edged sword of explanations. In Proceedings of the 16th international conference on Intelligent user interfaces. 125–134.
[5]
Samuel D Gosling, Peter J Rentfrow, and William B Swann Jr. 2003. A very brief measure of the Big-Five personality domains. Journal of Research in personality 37, 6 (2003), 504–528.
[6]
John Wesley Hostetter, Cristina Conati, Xi Yang, Mark Abdelshiheed, Tiffany Barnes, and Min Chi. 2023. XAI to increase the effectiveness of an Intelligent Pedagogical Agent. In Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents. 1–9.
[7]
Samad Kardan. 2017. A data mining approach for adding adaptive interventions to exploratory learning environments. Ph. D. Dissertation. University of British Columbia.
[8]
Samad Kardan and Cristina Conati. 2015. Providing adaptive support in an interactive simulation for learning: An experimental evaluation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 3671–3680.
[9]
Minjung Kim, Saebyeol Kim, Jinwoo Kim, Tae-Jin Song, and Yuyoung Kim. 2024. Do stakeholder needs differ?-Designing stakeholder-tailored Explainable Artificial Intelligence (XAI) interfaces. International Journal of Human-Computer Studies 181 (2024), 103160.
[10]
Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of the 20th international conference on intelligent user interfaces. 126–137.
[11]
Paul M Meara. 1992. EFL vocabulary tests. Citeseer.
[12]
Martijn Millecamp, Cristina Conati, and Katrien Verbert. 2022. “Knowing me, knowing you”: personalized explanations for a music recommender system. User Modeling and User-Adapted Interaction 32, 1-2 (2022), 215–252.
[13]
Martijn Millecamp, Nyi Nyi Htun, Cristina Conati, and Katrien Verbert. 2019. To explain or not to explain: the effects of personal characteristics when explaining music recommendations. In Proceedings of the 24th International Conference on Intelligent User Interfaces. 397–407.
[14]
Martijn Millecamp, Nyi Nyi Htun, Cristina Conati, and Katrien Verbert. 2020. What’s in a user? Towards personalising transparency for music recommender interfaces. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. 173–182.
[15]
Sebastiaan Rothmann and Elize P Coetzer. 2003. The big five personality dimensions and job performance. SA Journal of industrial psychology 29, 1 (2003), 68–74.
[16]
Sara Salimzadeh, Gaole He, and Ujwal Gadiraju. 2023. A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making. In Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization. 215–227.
[17]
Markel Sanz Ausin, Mehak Maniktala, Tiffany Barnes, and Min Chi. 2020. Exploring the impact of simple explanations and agency on batch deep reinforcement learning induced pedagogical policies. In Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I 21. Springer, 472–485.
[18]
Keela S Thomson and Daniel M Oppenheimer. 2016. Investigating an alternate form of the cognitive reflection test. Judgment and Decision making 11, 1 (2016), 99–113.
[19]
Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y Lim. 2019. Designing theory-driven user-centric explainable AI. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–15.
Index Terms
- Initial results on personalizing explanations of AI hints in an ITS
Recommendations
Toward personalized XAI: A case study in intelligent tutoring systems
AbstractOur research is a step toward ascertaining the need for personalization in XAI, and we do so in the context of investigating the value of explanations of AI-driven hints and feedback in Intelligent Tutoring Systems (ITS). We added an ...
Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing SystemsThe field of eXplainable artificial intelligence (XAI) has produced a plethora of methods (e.g., saliency-maps) to gain insight into artificial intelligence (AI) models, and has exploded with the rise of deep learning (DL). However, human-participant ...
A Clustering Approach for Personalizing Diversity in Collaborative Recommender Systems
UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and PersonalizationMuch of the focus of recommender systems research has been on the accurate prediction of users' ratings for unseen items. Recent work has suggested that objectives such as diversity and novelty in recommendations are also important factors in the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2024
338 pages
ISBN:9798400704338
DOI:10.1145/3627043
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 June 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
UMAP '24: 32nd ACM Conference on User Modeling, Adaptation and Personalization
July 1 - 4, 2024
Cagliari, Italy
Acceptance Rates
Overall Acceptance Rate 162 of 633 submissions, 26%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 132Total Downloads
- Downloads (Last 12 months)132
- Downloads (Last 6 weeks)9
Reflects downloads up to 31 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format