research-article

Visual Expertise in Code Reviews: Using Holistic Models of Image Perception to Analyze and Interpret Eye Movements

Authors:

Florian Hauser,

Lisa Grabinger,

Jürgen Mottok,

Hans GruberAuthors Info & Claims

ETRA '23: Proceedings of the 2023 Symposium on Eye Tracking Research and Applications

Article No.: 85, Pages 1 - 7

https://doi.org/10.1145/3588015.3589189

Published: 30 May 2023 Publication History

Abstract

This study uses holistic models of image perception to analyze and interpret eye movements during a code review. 23 participants (15 novices and 8 experts) take part in the experiment. The subjects’ task is to review six short code examples in C programming language and identify possible errors. During the experiment, their eye movements are recorded by an SMI 250 REDmobile. Additional data is collected through questionnaires and retrospective interviews. The results implicate that holistic models of image perception provide a suitable theoretical background for the analysis and interpretation of eye movements during code reviews. The assumptions of these models are particularly evident for expert programmers. Their approach can be divided into different phases with characteristic eye movement patterns. It is best described as switching between scans of the code example (global viewing) and the detailed examination of errors (focal viewing).

References

[1]

Roman Bednarik. 2012. Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human Computer Studies 70, 2 (2012), 143–155. https://doi.org/10.1016/j.ijhcs.2011.09.003

Digital Library

[2]

Roman Bednarik and Markku Tukiainen. 2006. An eye-tracking methodology for characterizing program comprehension processes. In Eye Tracking Research and Applications (ETRA). ACM, San Diego, 125–132. https://doi.org/10.1145/1117309.1117356

Digital Library

[3]

Andrew Begel and Hana Vrzakova. 2018. Eye movements in code review. In Proceedings of the Workshop on Eye Movements in Programming (EMIP). ACM, Warsaw, 1–5. https://doi.org/10.1145/3216723.3216727

Digital Library

[4]

Manfred Broy. 2006. Challenges in automotive software engineering. In Proceedings of the ACM International Conference on Software Engineering (ICSE). ACM, Shanghai, 33–42.

Digital Library

[5]

Teresa Busjahn, Roman Bednarik, Andrew Begel, Martha Crosby, James H. Paterson, Carsten Schulte, Bonita Sharif, and Sascha Tamm. 2015. Eye Movements in Code Reading: Relaxing the Linear Order. In Proceedings of the 23rd IEEE International Conference on Program Comprehension (ICPC). IEEE, Florence, 255–265. https://doi.org/10.1109/ICPC.2015.36

Digital Library

[6]

Teresa Busjahn, Roman Bednarik, and Carsten Schulte. 2014. What influences dwell time during source code reading?. Analysis of element type and frequency as factors. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). ACM, New York, 335–338.

Digital Library

[7]

Teresa Busjahn, Carsten Schulte, and Andreas Busjahn. 2011. Analysis of code reading to gain more insight in program comprehension. In Proceedings of the 11th Koli Calling International Conference on Computing Education Research - Koli Calling ’11. ACM Press, New York, New York, USA, 1. https://doi.org/10.1145/2094131.2094133

Digital Library

[8]

Martha E. Crosby and Jan Stelovsky. 1989. The influence of user experience and presentation medium on strategies of viewing algorithms. In Proceedings of the 22nd Annual Hawaii International Conference on System Sciences., Vol. 13. IEEE Comput. Soc. Press, 438–446. https://doi.org/10.1109/HICSS.1989.48025

[9]

Karl Anders Ericsson, Ralf Thomas Krampe, and Clemens Tesch-Römer. 1993. The role of deliberate practice in the acquisition of expert performance. Psychological Review 100, 3 (1993), 363–406. https://graphics8.nytimes.com/images/blogs/freakonomics/pdf/DeliberatePractice(PsychologicalReview).pdf

[10]

Karl Anders Ericsson and Tyler J. Towne. 2010. Expertise. Wiley Interdisciplinary Reviews: Cognitive Science 1, 3 (2010), 404–416. https://doi.org/10.1002/wcs.47

[11]

Andreas Gegenfurtner, Ellen Kok, Koos van Geel, Anique de Bruin, Halszka Jarodzka, Adam Szulewski, and Jeroen J.G. van Merriënboer. 2017. The challenges of studying visual expertise in medical image diagnosis. Medical Education 51, 1 (2017), 97–104. https://doi.org/10.1111/medu.13205

[12]

Andreas Gegenfurtner, Erno Lehtinen, and Roger Säljö. 2011. Expertise Differences in the Comprehension of Visualizations: A Meta-Analysis of Eye-Tracking Research in Professional Domains. Educational Psychology Review 23, 4 (2011), 523–552. https://doi.org/10.1007/s10648-011-9174-7

[13]

Andreas Gegenfurtner and Jeroen J. G. van Merriënboer. 2017. Methodologies for Studying Visual Expertise. Frontline Learning Research 5, 3 (2017), 1–13. https://doi.org/10.14786/flr.v5i3.316

[14]

Florian Hauser, Rebecca Reuter, Ivonne Hutzler, Jürgen Mottok, and Hans Gruber. 2018. Eye Movements in Software Engineering - What Differs the Expert From the Novice?. In Proceedings of the International Conference of Education, Research and Innovation (ICERI). IATED Academy, Seville, 632–642. https://doi.org/10.21125/iceri.2018.1129

[15]

Florian Hauser, Stefan Schreistetter, Rebecca Reuter, Jürgen Mottok, Hans Gruber, Kenneth Holmqvist, and Nick Schorr. 2020. Code Reviews in C++. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). ACM, Stuttgart, 1–5. https://doi.org/10.1145/3379156.3391980

Digital Library

[16]

Ellen M. Kok. 2016. Developing visual expertise. From Shades of Grey to Diagnostic Reasoning in Radiology. PhD-Thesis. University of Maastricht, Masstricht.

[17]

Oleksii Kononenko, Olga Baysal, and Michael W. Godfrey. 2016. Code review quality. In Proceedings of the 38th IEEE/ACM International Conference on Software Engineering (ICSE). IEEE, Austin, Texas, 1028–1038. https://doi.org/10.1145/2884781.2884840

Digital Library

[18]

Harold L. Kundel, Calvin F. Nodine, Emily F. Conant, and Susan P. Weinstein. 2007. Holistic Component of Image Perception in Mammogram Interpretation: Gaze-tracking Study. Radiology 242, 2 (feb 2007), 396–402. https://doi.org/10.1148/radiol.2422051997

[19]

Markus Nivala, Florian Hauser, Jurgen Mottok, and Hans Gruber. 2016. Developing visual expertise in software engineering: An eye tracking study. In Proceedings of the Global Engineering Education Conference (EDUCON). IEEE, Abu Dhabi, 613–620. https://doi.org/10.1109/EDUCON.2016.7474614

[20]

Calvin Nodine and Claudia Mello-Thoms. 2000. The Nature of Expertise in Radiology. In Handbook of Medical Imaging, Volume 1. Physics and Psychophysics. SPIE, Bellingham, WA, 859–894. https://doi.org/10.1117/3.832716.ch19

[21]

Calvin F. Nodine and Harold L. Kundel. 1987. The cognitive side of visual search in Radiology. In Eye Movements from Physiology to Cognition. Elsevier, 573–582. https://doi.org/10.1016/B978-0-444-70113-8.50081-3

[22]

Calvin F. Nodine and Claudia Mello-Thoms. 2010. The role of expertise in radiologic image interpretation. In The Handbook of Medical Image Perception and Techniques, E Samei and Elizabeth A. Krupinski (Eds.). Cambridge University Press, Cambridge, 139–156.

[23]

Unaizah Obaidellah, Mohammed Al Haek, and Peter C.-H. Cheng. 2018. A Survey on the Usage of Eye-Tracking in Computer Programming. Comput. Surveys 51, 1 (jan 2018), 1–58. https://doi.org/10.1145/3145904

Digital Library

[24]

Cole S. Peterson, Nahla J. Abid, Corey A. Bryant, Jonathan I. Maletic, and Bonita Sharif. 2019. Factors influencing dwell time during source code reading. In Proceedings of the 11th ACM Symposium on Eye Tracking Research and Applications - ETRA ’19. ACM Press, Denver, CO, 1–4. https://doi.org/10.1145/3314111.3319833

Digital Library

[25]

Eyal M. Reingold and Heather Sheridan. 2012. Eye movements and visual expertise in chess and medicine. The Oxford Handbook of Eye Movements (2012), 528–550. https://doi.org/10.1093/oxfordhb/9780199539789.013.0029

[26]

Zohreh Sharafi, Bonita Sharif, Yann Gaël Guéhéneuc, Andrew Begel, Roman Bednarik, and Martha Crosby. 2020. A practical guide on conducting eye tracking studies in software engineering. Empirical Software Engineering 25, 5 (2020), 3128–3174. https://doi.org/10.1007/s10664-020-09829-4

Digital Library

[27]

Zohreh Sharafi, Zéphyrin Soh, and Yann-Gaël Guéhéneuc. 2015. A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology 67, 7 (2015), 79–107. https://doi.org/10.1016/j.infsof.2015.06.008

Digital Library

[28]

Bonita Sharif, Michael Falcone, and Jonathan I Maletic. 2012. An eye-tracking study on the role of scan time in finding source code defects. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). ACM Press, New York, New York, 381. https://doi.org/10.1145/2168556.2168642

Digital Library

[29]

Heather Sheridan and Eyal M. Reingold. 2017. The holistic processing account of visual expertise in medical image perception: A review. Frontiers in Psychology 8 (2017), 1–11. https://doi.org/10.3389/fpsyg.2017.01620

[30]

Richard G. Swensson. 1980. A two-stage detection model applied to skilled visual search by radiologists. Perception and Psychophysics 27, 1 (1980), 11–16. https://doi.org/10.3758/BF03199899

[31]

Hidetake Uwano, Masahide Nakamura, Akito Monden, and Ken-ichi Matsumoto. 2006. Analyzing Individual Performance of Source Code Review Using Reviewers’ Eye Movement. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). ACM, San Diego, CA, 133–140. https://doi.orghttp://doi.acm.org/10.1145/1117309.1117357

Digital Library

Cited By

Hauser FGrabinger LEzer TMottok JGruber H(2024)Analyzing and Interpreting Eye Movements in C++: Using Holistic Models of Image PerceptionProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655093(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655093
Grabinger LHauser FWolff CMottok J(2024)On Eye Tracking in Software EngineeringSN Computer Science10.1007/s42979-024-03045-35:6Online publication date: 26-Jul-2024
https://dl.acm.org/doi/10.1007/s42979-024-03045-3

Index Terms

Visual Expertise in Code Reviews: Using Holistic Models of Image Perception to Analyze and Interpret Eye Movements

Recommendations

Analyzing and Interpreting Eye Movements in C++: Using Holistic Models of Image Perception
ETRA '24: Proceedings of the 2024 Symposium on Eye Tracking Research and Applications

This study uses holistic models of image perception originating from radiology and psychology to analyze and interpret eye movements during code reviews in the C++ programming language. The study design is based on former experiments, but is ...
Technology-Enhanced replays of expert gaze promote students’ visual learning in medical training
EC-TEL'12: Proceedings of the 7th European conference on Technology Enhanced Learning

Based on diagnostic performance and eye tracking data, the present study demonstrates that technology-enhanced replays of expert gaze can promote the visual learning of students in clinical visualization-based training.
Human Visual Scanpath Prediction Based on RGB-D Saliency
ICIGP '18: Proceedings of the 2018 International Conference on Image and Graphics Processing

Human visual perception is considered as a dynamic process of information acquisition, while the visual scanpath can clearly reflect the shift of our eye fixations. In the previous study of visual attention, researchers generally do the saliency ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '23: Proceedings of the 2023 Symposium on Eye Tracking Research and Applications

May 2023

441 pages

ISBN:9798400701504

DOI:10.1145/3588015

Editors:
Enkelejda Kasneci
Technical University Munich, Germany
,
Frederick Shic
University of Washington, USA
,
Mohamed Khamis
University of Glasgow, Scotland

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 May 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

German Federal Ministry of Education - Eyes on Future
German Federal Ministry of Education - HASKI
German Federal Ministry of Education - EVELIN

Conference

ETRA '23

Sponsor:

ETRA '23: 2023 Symposium on Eye Tracking Research and Applications

May 30 - June 2, 2023

Tubingen, Germany

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
106
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)1

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hauser FGrabinger LEzer TMottok JGruber H(2024)Analyzing and Interpreting Eye Movements in C++: Using Holistic Models of Image PerceptionProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655093(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655093
Grabinger LHauser FWolff CMottok J(2024)On Eye Tracking in Software EngineeringSN Computer Science10.1007/s42979-024-03045-35:6Online publication date: 26-Jul-2024
https://dl.acm.org/doi/10.1007/s42979-024-03045-3

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents