Analyzing and Interpreting Eye Movements in C++: Using Holistic Models of Image Perception
Authors: 
Florian Hauser, Lisa Grabinger, Timur Ezer, Jürgen Horst Mottok, Hans GruberAuthors Info & Claims
Article No.: 72, Pages 1 - 7
Abstract
This study uses holistic models of image perception originating from radiology and psychology to analyze and interpret eye movements during code reviews in the C++ programming language. The study design is based on former experiments, but is supplemented by approaches from expertise research. The study utilizes a sample of 34 subjects whose eye movements are recorded by a Tobii Pro Spectrum 600 Hz. The results show that the holistic models of image perception are suitable for application to source code. In addition, it can be observed that the code reviews are conducted in phases, which are characterized by certain strategies (e.g. scan, error detection,...). Furthermore, experience-related differences can be detected between experts and novices, which emphasize that experts use elaborate strategies and have a comparatively better ability to collect and process information from source code.
References
[1]
Roman Bednarik. 2012. Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human Computer Studies 70, 2 (2012), 143–155. https://doi.org/10.1016/j.ijhcs.2011.09.003
[2]
Roman Bednarik and Markku Tukiainen. 2006. An eye-tracking methodology for characterizing program comprehension processes. In Eye Tracking Research and Applications (ETRA). ACM, San Diego, 125–132. https://doi.org/10.1145/1117309.1117356
[3]
Tanya Beelders and Jean-Pierre du Plessis. 2016. The Influence of Syntax Highlighting on Scanning and Reading Behaviour for Source Code. In Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists (SAICSIT), Vol. 26-28-Sept. ACM, Johannesburg, 1–10. https://doi.org/10.1145/2987491.2987536
[4]
Andrew Begel and Hana Vrzakova. 2018. Eye movements in code review. In Proceedings of the Workshop on Eye Movements in Programming (EMIP). ACM, Warsaw, 1–5. https://doi.org/10.1145/3216723.3216727
[5]
Manfred Broy. 2006. Challenges in automotive software engineering. In Proceedings of the ACM International Conference on Software Engineering (ICSE). ACM, Shanghai, 33–42.
[6]
Teresa Busjahn, Roman Bednarik, Andrew Begel, Martha Crosby, James H. Paterson, Carsten Schulte, Bonita Sharif, and Sascha Tamm. 2015. Eye Movements in Code Reading: Relaxing the Linear Order. In Proceedings of the 23rd IEEE International Conference on Program Comprehension (ICPC). IEEE, Florence, 255–265. https://doi.org/10.1109/ICPC.2015.36
[7]
Teresa Busjahn, Roman Bednarik, and Carsten Schulte. 2014. What influences dwell time during source code reading?. Analysis of element type and frequency as factors. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). ACM, New York, 335–338.
[8]
Teresa Busjahn, Carsten Schulte, and Andreas Busjahn. 2011. Analysis of code reading to gain more insight in program comprehension. In Proceedings of the 11th Koli Calling International Conference on Computing Education Research - Koli Calling ’11. ACM Press, New York, New York, USA, 1. https://doi.org/10.1145/2094131.2094133
[9]
Martha E. Crosby and Jan Stelovsky. 1989. The influence of user experience and presentation medium on strategies of viewing algorithms. In Proceedings of the 22nd Annual Hawaii International Conference on System Sciences., Vol. 13. IEEE Comput. Soc. Press, 438–446. https://doi.org/10.1109/HICSS.1989.48025
[10]
Andrew T. Duchowski. 2017. Eye Tracking Methodology (3rd ed.). Springer, Cham. https://doi.org/10.1007/978-3-319-57883-5 arxiv:arXiv:1011.1669v3
[11]
Karl Anders Ericsson, Ralf Thomas Krampe, and Clemens Tesch-Römer. 1993. The role of deliberate practice in the acquisition of expert performance. Psychological Review 100, 3 (1993), 363–406. https://graphics8.nytimes.com/images/blogs/freakonomics/pdf/DeliberatePractice(PsychologicalReview).pdf
[12]
Karl Anders Ericsson and Andreas C. Lehmann. 1996. Expert and exceptional performance: Evidence of maximal adaptation to task. Annual Review of Psychology 47 (1996), 273–305.
[13]
Karl Anders Ericsson and Tyler J. Towne. 2010. Expertise. Wiley Interdisciplinary Reviews: Cognitive Science 1, 3 (2010), 404–416. https://doi.org/10.1002/wcs.47
[14]
Andreas Gegenfurtner, Ellen Kok, Koos van Geel, Anique de Bruin, Halszka Jarodzka, Adam Szulewski, and Jeroen J.G. van Merriënboer. 2017. The challenges of studying visual expertise in medical image diagnosis. Medical Education 51, 1 (2017), 97–104. https://doi.org/10.1111/medu.13205
[15]
Andreas Gegenfurtner, Erno Lehtinen, and Roger Säljö. 2011. Expertise Differences in the Comprehension of Visualizations: A Meta-Analysis of Eye-Tracking Research in Professional Domains. Educational Psychology Review 23, 4 (2011), 523–552. https://doi.org/10.1007/s10648-011-9174-7
[16]
Andreas Gegenfurtner and Jeroen J. G. van Merriënboer. 2017. Methodologies for Studying Visual Expertise. Frontline Learning Research 5, 3 (2017), 1–13. https://doi.org/10.14786/flr.v5i3.316
[17]
Florian Hauser, Lisa Grabinger, Jürgen Mottok, and Hans Gruber. 2023. Visual Expertise in Code Reviews. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). ACM, Tübingen, 1–7. https://doi.org/10.1145/3588015.3589189
[18]
Florian Hauser, Rebecca Reuter, Ivonne Hutzler, Jürgen Mottok, and Hans Gruber. 2018. Eye Movements in Software Engineering - What Differs the Expert From the Novice?. In Proceedings of the International Conference of Education, Research and Innovation (ICERI). IATED Academy, Seville, 632–642. https://doi.org/10.21125/iceri.2018.1129
[19]
Florian Hauser, Stefan Schreistetter, Rebecca Reuter, Jürgen Mottok, Hans Gruber, Kenneth Holmqvist, and Nick Schorr. 2020. Code Reviews in C++. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). ACM, Stuttgart, 1–5. https://doi.org/10.1145/3379156.3391980
[20]
Kenneth Holmqvist, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van De Weijer. 2011. Eye tracking: A comprehensive guide to methods and measures. Oxford University Press, Oxford.
[21]
Deepika Kanojia. 2020. Saccades amplitude or saccade length calculation for each saccade in degree?https://www.researchgate.net/post/Saccades-amplitude-or-saccade-length-calculation-for-each-saccade-in-degree
[22]
Ellen M. Kok. 2016. Developing visual expertise. From Shades of Grey to Diagnostic Reasoning in Radiology. PhD-Thesis. University of Maastricht, Masstricht.
[23]
Oleksii Kononenko, Olga Baysal, and Michael W. Godfrey. 2016. Code review quality. In Proceedings of the 38th IEEE/ACM International Conference on Software Engineering (ICSE). IEEE, Austin, Texas, 1028–1038. https://doi.org/10.1145/2884781.2884840
[24]
Harold L. Kundel, Calvin F. Nodine, Emily F. Conant, and Susan P. Weinstein. 2007. Holistic Component of Image Perception in Mammogram Interpretation: Gaze-tracking Study. Radiology 242, 2 (feb 2007), 396–402. https://doi.org/10.1148/radiol.2422051997
[25]
Markus Nivala, Florian Hauser, Jurgen Mottok, and Hans Gruber. 2016. Developing visual expertise in software engineering: An eye tracking study. In Proceedings of the Global Engineering Education Conference (EDUCON). IEEE, Abu Dhabi, 613–620. https://doi.org/10.1109/EDUCON.2016.7474614
[26]
Calvin Nodine and Claudia Mello-Thoms. 2000. The Nature of Expertise in Radiology. In Handbook of Medical Imaging, Volume 1. Physics and Psychophysics. SPIE, Bellingham, WA, 859–894. https://doi.org/10.1117/3.832716.ch19
[27]
Calvin F. Nodine and Harold L. Kundel. 1987. The cognitive side of visual search in Radiology. In Eye Movements from Physiology to Cognition. Elsevier, 573–582. https://doi.org/10.1016/B978-0-444-70113-8.50081-3
[28]
Calvin F. Nodine and Claudia Mello-Thoms. 2010. The role of expertise in radiologic image interpretation. In The Handbook of Medical Image Perception and Techniques, E Samei and Elizabeth A. Krupinski (Eds.). Cambridge University Press, Cambridge, 139–156.
[29]
Unaizah Obaidellah, Mohammed Al Haek, and Peter C.-H. Cheng. 2018. A Survey on the Usage of Eye-Tracking in Computer Programming. Comput. Surveys 51, 1 (jan 2018), 1–58. https://doi.org/10.1145/3145904
[30]
Cole S. Peterson, Nahla J. Abid, Corey A. Bryant, Jonathan I. Maletic, and Bonita Sharif. 2019. Factors influencing dwell time during source code reading. In Proceedings of the 11th ACM Symposium on Eye Tracking Research and Applications - ETRA ’19. ACM Press, Denver, CO, 1–4. https://doi.org/10.1145/3314111.3319833
[31]
Eyal M. Reingold and Heather Sheridan. 2012. Eye movements and visual expertise in chess and medicine. The Oxford Handbook of Eye Movements (2012), 528–550. https://doi.org/10.1093/oxfordhb/9780199539789.013.0029
[32]
Nick Schorr. 2020. Die Rolle von Farbcodierung und Programmierexpertise auf die Fehlersuche im Quellcode Masterarbeit Universität Regensburg. Ph. D. Dissertation. Universität Regensburg.
[33]
SensoMotoric Instruments. 2017. BeGaze Manual. Teltow.
[34]
Zohreh Sharafi, Bonita Sharif, Yann Gaël Guéhéneuc, Andrew Begel, Roman Bednarik, and Martha Crosby. 2020. A practical guide on conducting eye tracking studies in software engineering. Empirical Software Engineering 25, 5 (2020), 3128–3174. https://doi.org/10.1007/s10664-020-09829-4
[35]
Zohreh Sharafi, Zéphyrin Soh, and Yann-Gaël Guéhéneuc. 2015. A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology 67, 7 (2015), 79–107. https://doi.org/10.1016/j.infsof.2015.06.008
[36]
Bonita Sharif, Michael Falcone, and Jonathan I Maletic. 2012. An eye-tracking study on the role of scan time in finding source code defects. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). ACM Press, New York, New York, 381. https://doi.org/10.1145/2168556.2168642
[37]
Heather Sheridan and Eyal M. Reingold. 2017. The holistic processing account of visual expertise in medical image perception: A review. Frontiers in Psychology 8 (2017), 1–11. https://doi.org/10.3389/fpsyg.2017.01620
[38]
Richard G. Swensson. 1980. A two-stage detection model applied to skilled visual search by radiologists. Perception and Psychophysics 27, 1 (1980), 11–16. https://doi.org/10.3758/BF03199899
[39]
Tobii Pro. 2020. Tobii Spectrum Pro. https://www.tobiipro.com/de/produkte/tobii-pro-spectrum/
[40]
Tobii Pro. 2021. Tobii Pro Lab Description. https://www.tobiipro.com/siteassets/tobii-pro/products/software/tobii-pro-lab/Tobii_Pro_Lab_Product_Description.pdf/?v=1.181
[41]
Hidetake Uwano, Masahide Nakamura, Akito Monden, and Ken-ichi Matsumoto. 2006. Analyzing Individual Performance of Source Code Review Using Reviewers’ Eye Movement. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). ACM, San Diego, CA, 133–140. https://doi.orghttp://doi.acm.org/10.1145/1117309.1117357
Index Terms
- Analyzing and Interpreting Eye Movements in C++: Using Holistic Models of Image Perception
Recommendations
Visual Expertise in Code Reviews: Using Holistic Models of Image Perception to Analyze and Interpret Eye Movements
ETRA '23: Proceedings of the 2023 Symposium on Eye Tracking Research and ApplicationsThis study uses holistic models of image perception to analyze and interpret eye movements during a code review. 23 participants (15 novices and 8 experts) take part in the experiment. The subjects’ task is to review six short code examples in C ...
Analysing EOG signal features for the discrimination of eye movements with wearable devices
PETMEI '11: Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interactionEye tracking research in human-computer interaction and experimental psychology traditionally focuses on stationary devices and a small number of common eye movements. The advent of pervasive eye tracking promises new applications, such as eye-based ...
Can the E-Z Reader Model Predict Eye Movements Over Code? Towards a Model of Eye Movements Over Source Code
ETRA '20 Short Papers: ACM Symposium on Eye Tracking Research and ApplicationsStudies of eye movements during source code reading have supported the idea that reading source code differs fundamentally from reading natural text. The paper analyzed an existing data set of natural language and source code eye movement data using the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2024
525 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- German Ministry of Education and Research
- German Ministry of Education and Research
Conference
ETRA '24: The 2024 Symposium on Eye Tracking Research and Applications
June 4 - 7, 2024
Glasgow, United Kingdom
Acceptance Rates
Overall Acceptance Rate 69 of 137 submissions, 50%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 89Total Downloads
- Downloads (Last 12 months)89
- Downloads (Last 6 weeks)32
Reflects downloads up to 20 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in