research-article

Pupillometry and Head Distance to the Screen to Predict Skill Acquisition During Information Visualization Tasks

Authors:

Sébastien Lallé,

Cristina ConatiAuthors Info & Claims

IUI '17: Proceedings of the 22nd International Conference on Intelligent User Interfaces

Pages 221 - 231

https://doi.org/10.1145/3025171.3025187

Published: 07 March 2017 Publication History

Abstract

In this paper we investigate using a variety of behavioral measures collectible with an eye tracker to predict a user's skill acquisition phase while performing various information visualization tasks with bar graphs. Our long term goal is to use this information in real-time to create user-adaptive visualizations that can provide personalized support to facilitate visualization processing based on the user's predicted skill level. We show that leveraging two additional content-independent data sources, namely information on a user's pupil dilation and head distance to the screen, yields a significant improvement for predictive accuracies of skill acquisition compared to predictions made using content-dependent information related to user eye gaze attention patterns, as was done in previous work. We show that including features from both pupil dilation and head distance to the screen improve the ability to predict users' skill acquisition state, beating both the baseline and a model using only content-dependent gaze information.

References

[1]

Akaike, H. Factor analysis and AIC. Psychometrika 52, 3 (1987), 317--332.

[2]

Amar, R., Eagan, J., and Stasko, J. Low-Level Components of Analytic Activity in Information Visualization. Proceedings of the Proceedings of the 2005 IEEE Symposium on Information Visualization, IEEE Computer Society (2005), 15--21.

Digital Library

[3]

Baker, R., Corbett, A., and Aleven, V. More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing. Proceedings of the 9th International Conference Intelligent Tutoring Systems, (2008), 406--415.

Digital Library

[4]

Baker, R.Sj., D'Mello, S.K., Rodrigo, M.M.T., and Graesser, A.C. Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive--affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies 68, 4 (2010), 223--241.

Digital Library

[5]

Barnes, T. and Stamper, J. Toward automatic hint generation for logic proof tutoring using historical student data. Proceedings of the 9th International Conference on Intelligent Tutoring Systems, Springer (2008), 373--382.

Digital Library

[6]

Beatty, J. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin 91, 2 (1982), 276--292.

[7]

Beck, J.E. and Sison, J. Using knowledge tracing in a noisy environment to measure student reading proficiencies. International Journal of Artificial Intelligence in Education 16, 2 (2006), 129--143.

Digital Library

[8]

Bednarik, R., Eivazi, S., and Vrzakova, H. A Computational Approach for Prediction of ProblemSolving Behavior Using Support Vector Machines and Eye-Tracking Data. In Y.I. Nakano, C. Conati and T. Bader, eds., Eye Gaze in Intelligent User Interfaces. Springer London, London, 2013, 111--134.

[9]

Biswas, P., Dutt, V., and Langdon, P. Comparing Ocular Parameters for Cognitive Load Measurement in EyeGaze-Controlled Interfaces for Automotive and Desktop Computing Environments. International Journal of Human-Computer Interaction 32, 1 (2016), 23--38.

[10]

Bixler, R. and D'Mello, S. Automatic Gaze-Based Detection of Mind Wandering with Metacognitive Awareness. In F. Ricci, K. Bontcheva, O. Conlan and S. Lawless, eds., User Modeling, Adaptation and Personalization. Springer, Cham, 2015, 31--43.

[11]

Bixler, R., Kopp, K., and D'Mello, S. Evaluation of a Personalized Method for Proactive Mind Wandering Reduction. Proceedings of the 4th Workshop on Personalization Approaches for Learning Environments, 22nd conference on User Modeling, Adaptation, and Personalization, Springer (2014), 33--41.

[12]

Bondareva, D., Conati, C., Feyzi-Behnagh, R., Harley, J.M., Azevedo, R., and Bouchet, F. Inferring Learning from Gaze Data during Interaction with an Environment to Support Self-Regulated Learning. In H.C. Lane, K. Yacef, J. Mostow and P. Pavlik, eds., Artificial Intelligence in Education. Springer Berlin Heidelberg, Berlin, Heidelberg, 2013, 229--238.

[13]

Boy, J., Rensink, R.A., Bertini, E., and Fekete, J.-D. A Principled Way of Assessing Visualization Literacy. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 1963--1972.

[14]

Brusilovsky, P., Ahn, J., Dumitriu, T., and Yudelson, M. Adaptive Knowledge-Based Visualization for Accessing Educational Examples. Proc. 10th International Conf. on Information Visualization, IEEE (2006), 142--150.

Digital Library

[15]

Bunt, A., Conati, C., and McGrenere, J. Supporting interface customization using a mixed-initiative approach. ACM Press (2007), 92.

Digital Library

[16]

Carenini, G., Conati, C., Hoque, E., Steichen, B., Toker, D., and Enns, J.T. Highlighting Interventions and User Differences: Informing Adaptive Information Visualization Support. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, (2014).

Digital Library

[17]

Carenini, G., Conati, C., Hoque, E., Steichen, B., Toker, D., and Enns, J.T. Highlighting Interventions and User Differences: Informing Adaptive Information Visualization Support. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2014), 1835--1844.

Digital Library

[18]

Çöltekin, A., Fabrikant, S.I., and Lacayo, M. Exploring the efficiency of users' visual analytics strategies based on sequence analysis of eye movement recordings. International Journal of Geographical Information Science 24, 10 (2010), 1559--1575.

Digital Library

[19]

Conati, C., Carenini, G., Hoque, E., Steichen, B., and Toker, D. Evaluating the impact of user characteristics and different layouts on an interactive visualization for decision making. Computer Graphics Forum, Wiley Online Library (2014), 371--380.

[20]

Conati, C., Carenini, G., Toker, D., and Lallé, S. Towards User-Adaptive Information Visualization. AAAI Conf. on Artificial Intelligence, (2015).

Digital Library

[21]

Conati, C., Hoque, E., Toker, Dereck, and Steichen, Ben. When to Adapt: Detecting User's Confusion During Visualization Processing. Proc. of the 1st International Workshop on User-Adaptive Information Visualization (WUAV), in conjunction with the 21st Conference on User Modeling, Adaptation and Personalization (UMAP), (2013).

[22]

D'Mello, S., Olney, A., Williams, C., and Hays, P. Gaze tutor: A gaze-reactive intelligent tutoring system. International Journal of Human-Computer Studies 70, 5 (2012), 377--398.

Digital Library

[23]

D'Mello, S.K., Craig, S.D., Sullins, J., and Graesser, A.C. Predicting affective states expressed through an emote-aloud procedure from AutoTutor's mixed initiative dialogue. International Journal of Artificial Intelligence in Education 16, 1 (2006), 3--28.

Digital Library

[24]

Field, A. Discovering Statistics Using IBM SPSS Statistics. Sage Publications Ltd, London, 2012.

Digital Library

[25]

Gingerich, M.J. and Conati, C. Constructing Models of User and Task Characteristics from Eye Gaze Data for User-Adaptive Information Highlighting. Twenty-Ninth AAAI Conference on Artificial Intelligence, (2015).

Digital Library

[26]

Gotz, D. and Wen, Z. Behavior-driven visualization recommendation. Proc. of the 14th international conf. on Intelligent user interfaces, ACM (2009), 315--324.

Digital Library

[27]

Green, T.M. and Fisher, B. Impact of personality factors on interface interaction and the development of user profiles: Next steps in the personal equation of interaction. Information Visualization 11, 3 (2012), 205-- 221.

Digital Library

[28]

Haas Lyons, S., Walsh, M., Aleman, E., and Robinson, J. Exploring regional futures: Lessons from Metropolitan Chicago's online MetroQuest. Technological Forecasting and Social Change 82, (2014), 23--33.

[29]

Hess, E.H. and Polt, J.M. Pupil Size in Relation to Mental Activity during Simple Problem-Solving. Science 143, 3611 (1964), 1190--1192.

[30]

Hurst, A., Hudson, S.E., and Mankoff, J. Dynamic Detection of Novice vs. Skilled Use Without a Task Model. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2007), 271--280.

Digital Library

[31]

Hyönä, J., Tommola, J., and Alaja, A.-M. Pupil dilation as a measure of processing load in simultaneous interpretation and other language tasks. The Quarterly Journal of Experimental Psychology 48, 3 (1995), 598-- 612.

[32]

Iqbal, S.T., Adamczyk, P.D., Zheng, X.S., and Bailey, B.P. Towards an index of opportunity: understanding changes in mental workload during task execution. ACM Press (2005), 311.

Digital Library

[33]

Jacobs, A.M., Fransen, B., McCurry, J.M., Heckel, F.W.P., Wagner, A.R., and Trafton, J.G. A Preliminary System for Recognizing Boredom. Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction, ACM (2009), 299--300.

Digital Library

[34]

Jaques, N., Conati, C., Harley, J.M., and Azevedo, R. Predicting Affect from Gaze Data during Interaction with an Intelligent Tutoring System. Proceedings of the 12th International Conference on Intelligent Tutoring Systems, Springer (2014), 29--38.

Digital Library

[35]

Kalyuga, S. Enhancing Instructional Efficiency of Interactive E-learning Environments: A Cognitive Load Perspective. Educational Psychology Review 19, 3 (2007), 387--399.

[36]

Kardan, S. and Conati, C. Comparing and Combining Eye Gaze and Interface Actions for Determining User Learning with an Interactive Simulation. In: Proc. of UMAP, 21st Int. Conf. on User Modeling, Adaptation and Personalization, (2013).

[37]

Kong, N., Hearst, M.A., and Agrawala, M. Extracting references between text and charts via crowdsourcing. Proceedings of the SIGCHI conference on Human Factors in Computing Systems, ACM (2014), 31--40.

Digital Library

[38]

Kuhn, M. Building predictive models in R using the caret package. Journal of Statistical Software 28, 5 (2008), 1--26.

[39]

Lallé, S., Conati, C., and Carenini, G. Predicting confusion in information visualization from eye tracking and interaction data. Proceedings on the 25th International Joint Conference on Artificial Intelligence, (2016), 2529--2535.

[40]

Lallé, S., Mostow, J., Luengo, V., and Guin, N. Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success. Proceedings of the 16th International Conference on Artificial Intelligence in Education, (2013), 161--170.

[41]

Lallé, S., Toker, D., Conati, C., and Carenini, G. Prediction of Users' Learning Curves for Adaptation while Using an Information Visualization. Proceedings of the 20th International Conference on Intelligent User Interfaces, ACM (2015), 357--368.

Digital Library

[42]

Marshall, S.P. The index of cognitive activity: Measuring cognitive workload. Proceedings of the 7th IEEE Human Factors Meeting, IEEE (2002), 5--9.

[43]

Marshall, S.P. Identifying cognitive state from eye metrics. Aviation, space, and environmental medicine 78, Supplement 1 (2007), B165--B175.

[44]

Martínez-Gómez, P. and Aizawa, A. Recognition of understanding level and language skill using measurements of reading behavior. ACM Press (2014), 95--104.

Digital Library

[45]

Mueller, F. and Lockerd, A. Cheese: tracking mouse movement activity on websites, a tool for user modeling. CHI'01 Extended Abstracts on Human Factors in Computing Systems, ACM Press (2001), 279--280.

Digital Library

[46]

Ooms, K., De Maeyer, P., and Fack, V. Study of the attentive behavior of novice and expert map users using eye tracking. Cartography and Geographic Information Science 41, 1 (2014), 37--54.

[47]

Pascual-Cid, V., Vigentini, L., and Quixal, M. Visualising Virtual Learning Environments: Case Studies of the Website Exploration Tool. IEEE (2010), 149--155.

Digital Library

[48]

Pavlik, P.I., Cen, H., and Koedinger, K.R. Performance Factors Analysis--A New Alternative to Knowledge Tracing. Proceedings of the International Conference on AI in Education, IOS Press (2009), 531--538.

Digital Library

[49]

Prendinger, H., Hyrskykari, A., Nakayama, M., Istance, H., Bee, N., and Takahasi, Y. Attentive interfaces for users with disabilities: eye gaze for intention and uncertainty estimation. Universal Access in the Information Society 8, 4 (2009), 339--354.

Digital Library

[50]

Saraiya, P., North, C., and Duca, K. An Insight-Based Methodology for Evaluating Bioinformatics Visualizations. IEEE Transactions on Visualization and Computer Graphics 11, 4 (2005), 443--456.

Digital Library

[51]

Speelman, C. and Kirsner, K. Beyond the Learning Curve. Oxford University Press, 2005.

[52]

Steichen, B., Conati, C., and Carenini, G. Inferring Visualization Task Properties, User Performance, and User Cognitive Abilities from Eye Gaze Data. ACM Transactions on Interactive Intelligent Systems, (2014).

Digital Library

[53]

Toker, D., Conati, C., Steichen, B., and Carenini, G. Individual user characteristics and information visualization: connecting the dots through eye tracking. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 295--304.

Digital Library

[54]

Toker, D. and Conati, C. Eye Tracking to Understand User Differences in Visualization Processing with Highlighting Interventions. In V. Dimitrova, T. Kuflik, D. Chin, F. Ricci, P. Dolog and G.-J. Houben, eds., User Modeling, Adaptation, and Personalization. Springer International Publishing, 2014, 219--230.

[55]

Toker, D., Steichen, B., Gingerich, M., Conati, C., and Carenini, G. Towards Facilitating User Skill Acquisition - Identifying Untrained Visualization Users through Eye Tracking. Proceedings of the 2014 international conference on Intelligent user interfaces, ACM (2014).

Digital Library

[56]

Vrochidis, S., Patras, I., and Kompatsiaris, I. An Eye tracking-based Approach to Facilitate Interactive Video Search. Proc. of the 1st ACM International Conference on Multimedia Retrieval, ACM (2011), 43:1--43:8.

Digital Library

[57]

Zhu, Y. Measuring Effective Data Visualization. Proc. of the 3rd International Symposium on Advances in Visual Computing, Springer (2007), 652--661.

Digital Library

[58]

Ziemkiewicz, C. and Kosara, R. The Shaping of Information by Visual Metaphors. IEEE Transactions on Visualization and Computer Graphics 14, 6 (2008), 1269--1276.

Digital Library

Cited By

Wenskovitch JDowling MNorth C(2023)Towards Addressing Ambiguous Interactions and Inferring User Intent with Dimension Reduction and Clustering Combinations in Visual AnalyticsACM Transactions on Interactive Intelligent Systems10.1145/3588565Online publication date: 17-Apr-2023
https://dl.acm.org/doi/10.1145/3588565
Yang KZhao XZou JDu W(2023) ATPP: A Mobile App Prediction System Based on Deep Marked Temporal Point ProcessesACM Transactions on Sensor Networks10.1145/358255519:3(1-24)Online publication date: 5-Apr-2023
https://dl.acm.org/doi/10.1145/3582555
Zhou KWang HWen JZhao W(2023)Enhancing Multi-View Smoothness for Sequential Recommendation ModelsACM Transactions on Information Systems10.1145/358249541:4(1-27)Online publication date: 8-Apr-2023
https://dl.acm.org/doi/10.1145/3582495
Show More Cited By

Index Terms

Pupillometry and Head Distance to the Screen to Predict Skill Acquisition During Information Visualization Tasks
1. Human-centered computing

Recommendations

Comparing and Combining Interaction Data and Eye-tracking Data for the Real-time Prediction of User Cognitive Abilities in Visualization Tasks

Previous work has shown that some user cognitive abilities relevant for processing information visualizations can be predicted from eye-tracking data. Performing this type of user modeling is important for devising visualizations that can detect a user'...
Beyond gaze: preliminary analysis of pupil dilation and blink rates in an fMRI study of program comprehension
EMIP '18: Proceedings of the Workshop on Eye Movements in Programming

Researchers have been employing psycho-physiological measures to better understand program comprehension, for example simultaneous fMRI and eye tracking to validate top-down comprehension models. In this paper, we argue that there is additional value in ...
Towards facilitating user skill acquisition: identifying untrained visualization users through eye tracking
IUI '14: Proceedings of the 19th international conference on Intelligent User Interfaces

A key challenge for information visualization designers lies in developing systems that best support users in terms of their individual abilities, needs, and preferences. However, most visualizations require users to first gather a certain set of skills ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '17: Proceedings of the 22nd International Conference on Intelligent User Interfaces

March 2017

654 pages

ISBN:9781450343480

DOI:10.1145/3025171

General Chairs:
George A. Papadopoulos
The University of Cyprus, Cyprus
,
Tsvi Kuflik
The University of Haifa, Israel
,
Program Chairs:
Fang Chen
Data61, CSIRO Australia
,
Carlos Duarte
The University of Lisbon, Portugal
,
Wai-Tat Fu
The University of Illinois at Urbana-Champaign, USA

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 March 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IUI'17

Sponsor:

IUI'17: 22nd International Conference on Intelligent User Interfaces

March 13 - 16, 2017

Limassol, Cyprus

Acceptance Rates

IUI '17 Paper Acceptance Rate 63 of 272 submissions, 23%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
426
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wenskovitch JDowling MNorth C(2023)Towards Addressing Ambiguous Interactions and Inferring User Intent with Dimension Reduction and Clustering Combinations in Visual AnalyticsACM Transactions on Interactive Intelligent Systems10.1145/3588565Online publication date: 17-Apr-2023
https://dl.acm.org/doi/10.1145/3588565
Yang KZhao XZou JDu W(2023) ATPP: A Mobile App Prediction System Based on Deep Marked Temporal Point ProcessesACM Transactions on Sensor Networks10.1145/358255519:3(1-24)Online publication date: 5-Apr-2023
https://dl.acm.org/doi/10.1145/3582555
Zhou KWang HWen JZhao W(2023)Enhancing Multi-View Smoothness for Sequential Recommendation ModelsACM Transactions on Information Systems10.1145/358249541:4(1-27)Online publication date: 8-Apr-2023
https://dl.acm.org/doi/10.1145/3582495
Williams AIqbal SKiseleva JWhite R(2023)Managing Tasks across the Work–Life Boundary: Opportunities, Challenges, and DirectionsACM Transactions on Computer-Human Interaction10.1145/358242930:3(1-31)Online publication date: 31-Jan-2023
https://dl.acm.org/doi/10.1145/3582429
Lin SLin J(2023)A Dense Representation Framework for Lexical and Semantic MatchingACM Transactions on Information Systems10.1145/358242641:4(1-29)Online publication date: 8-Apr-2023
https://dl.acm.org/doi/10.1145/3582426
Kosch TKarolus JZagermann JReiterer HSchmidt AWoźniak P(2023)A Survey on Measuring Cognitive Workload in Human-Computer InteractionACM Computing Surveys10.1145/358227255:13s(1-39)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3582272
Ren HFan HWang RHuo YTang RWang LBao H(2023)Data-driven Digital Lighting Design for Residential Indoor SpacesACM Transactions on Graphics10.1145/358200142:3(1-18)Online publication date: 17-Mar-2023
https://dl.acm.org/doi/10.1145/3582001
Klein CStrzodka R(2023)Tridigpu: A GPU Library for Block Tridiagonal and Banded Linear Equation SystemsACM Transactions on Parallel Computing10.1145/358037310:1(1-33)Online publication date: 29-Mar-2023
https://dl.acm.org/doi/10.1145/3580373
Yi JRen XChen Z(2023)Multi-auxiliary Augmented Collaborative Variational Auto-encoder for Tag RecommendationACM Transactions on Information Systems10.1145/357893241:4(1-25)Online publication date: 8-Apr-2023
https://dl.acm.org/doi/10.1145/3578932
Zhang LZheng DYuan MHan FWu ZLiu MLi X(2023)MultiSense: Cross-labelling and Learning Human Activities Using Multimodal Sensing DataACM Transactions on Sensor Networks10.1145/357826719:3(1-26)Online publication date: 17-Apr-2023
https://dl.acm.org/doi/10.1145/3578267
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents