research-article

The effect of poor source code lexicon and readability on developers' cognitive load

Authors:

Sarah Fakhoury,

Venera Arnaoudova,

Olusola AdesopeAuthors Info & Claims

ICPC '18: Proceedings of the 26th Conference on Program Comprehension

Pages 286 - 296

https://doi.org/10.1145/3196321.3196347

Published: 28 May 2018 Publication History

Abstract

It has been well documented that a large portion of the cost of any software lies in the time spent by developers in understanding a program's source code before any changes can be undertaken. One of the main contributors to software comprehension, by subsequent developers or by the authors themselves, has to do with the quality of the lexicon, (i.e., the identifiers and comments) that is used by developers to embed domain concepts and to communicate with their teammates. In fact, previous research shows that there is a positive correlation between the quality of identifiers and the quality of a software project. Results suggest that poor quality lexicon impairs program comprehension and consequently increases the effort that developers must spend to maintain the software. However, we do not yet know or have any empirical evidence, of the relationship between the quality of the lexicon and the cognitive load that developers experience when trying to understand a piece of software. Given the associated costs, there is a critical need to empirically characterize the impact of the quality of the lexicon on developers' ability to comprehend a program.

In this study, we explore the effect of poor source code lexicon and readability on developers' cognitive load as measured by a cutting-edge and minimally invasive functional brain imaging technique called functional Near Infrared Spectroscopy (fNIRS). Additionally, while developers perform software comprehension tasks, we map cognitive load data to source code identifiers using an eye tracking device. Our results show that the presence of linguistic antipatterns in source code significantly increases the developers' cognitive load.

References

[1]

Surafel Lemma Abebe, Venera Arnaoudova, Paolo Tonella, Giuliano Antoniol, and Yann Gaël Guéhéneuc. 2012. Can Lexicon Bad Smells improve fault prediction?. In Proceedings of the Working Conference on Reverse Engineering (WCRE). 235--244.

Digital Library

[2]

Daniel Afergan, Evan M Peck, Erin T Solovey, Andrew Jenkins, Samuel W Hincks, Eli T Brown, Remco Chang, and Robert JK Jacob. 2014. Dynamic difficulty using brain metrics of workload. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3797--3806.

Digital Library

[3]

Venera Arnaoudova, Massimiliano Di Penta, and Giuliano Antoniol. 2016. Linguistic Antipatterns: What They are and How Developers Perceive Them. Empirical Software Engineering (EMSE) 21, 1 (2016), 104--158.

Digital Library

[4]

Venera Arnaoudova, Massimiliano Di Penta, Giuliano Antoniol, and Yann-Gaël Guéhéneuc. 2013. A New Family of Software Anti-Patterns: Linguistic Anti-Patterns. In Proceedings of the European Conference on Software Maintenance and Reengineering (CSMR). 187--196.

Digital Library

[5]

BIOPAC. 2018. BIOPAC Homepage. (March 2018). https://www.biopac.com

[6]

BIOPAC. 2018. fNIRSoft User Manual. (March 2018). https://www.biopac.com/wp-content/uploads/fnirsoft-user-manual.pdf

[7]

Raymond P.L. Buse and Westley R. Weimer. 2010. Learning a metric for code readability. IEEE Transactions on Software Engineering (TSE) 36, 4 (2010), 546--558.

Digital Library

[8]

Simon Butler, Michel Wermelinger, Yijun Yu, and Helen Sharp. 2009. Relating Identifier Naming Flaws and Code Quality: An Empirical Study. In Proceedings of the Working Conference on Reverse Engineering (WCRE). 31--35.

Digital Library

[9]

Simon Butler, Michel Wermelinger, Yijun Yu, and Helen Sharp. 2010. Exploring the Influence of Identifier Names on Code Quality: An empirical study. In Proceedings of the European Conference on Software Maintenance and Reengineering (CSMR). 156--165.

Digital Library

[10]

David T Delpy, Mark Cope, Pieter van der Zee, SR Arridge, Susan Wray, and JS Wyatt. 1988. Estimation of optical pathlength through tissue from direct time of flight measurement. Physics in Medicine & Biology 33, 12 (1988), 1433.

[11]

Eclipse. 2018. Eclipse IDE. (March 2018). https://www.eclipse.org/ide

[12]

EyeTribe. 2018. The Eye Tribe Homepage. (March 2018). https://theeyetribe.com

[13]

Sarah Fakhoury. 2018. Online Replication Package. (March 2018). https://github.com/smfakhoury/fNIRS-and-Cognitive-Load

[14]

Frank A. Fishburn, Megan E. Norr, Andrei V. Medvedev, and Chandan J. Vaidya. 2014. Sensitivity of fNIRS to cognitive state and load. Frontiers in human neuroscience 8 (2014), 76.

[15]

Benjamin Floyd, Tyler Santander, and Westley Weimer. 2017. Decoding the Representation of Code in the Brain: An fMRI Study of Code Review and Expertise. In Proceedings of the International Conference on Software Engineering (ICSE). 175--186.

Digital Library

[16]

Thomas Fritz, Andrew Begel, Sebastian C Müller, Serap Yigit-Elliott, and Manuela Züger. 2014. Using Psycho-physiological Measures to Assess Task Difficulty in Software Development. In Proceedings of the International Conference on Software Engineering (ICSE). 402--413.

Digital Library

[17]

Audrey Girouard, Erin Treacy Solovey, Leanne M. Hirshfield, Krysta Chauncey, Angelo Sassaroli, Sergio Fantini, and Robert J.K. Jacob. 2009. Distinguishing difficulty levels with non-invasive brain activity measurements. In IFIP Conference on Human-Computer Interaction. Springer, 440--452.

Digital Library

[18]

Robert J. Grissom and John J. Kim. 2005. Effect sizes for research: A broad practical approach (2nd edition ed.). Lawrence Earlbaum Associates.

[19]

Sonia Haiduc and Andrian Marcus. 2008. On the Use of Domain Terms in Source Code. In Proceedings of the International Conference on Program Comprehension (ICPC). 113--122.

Digital Library

[20]

Maurice H Halstead. 1977. Elements of software science. (1977).

[21]

Christian Herff, Dominic Heger, Ole Fortmann, Johannes Hennrich, Felix Putze, and Tanja Schultz. 2014. Mental workload during n-back task-quantified in the prefrontal cortex using fNIRS. Frontiers in Human Neuroscience 7 (2014), 935.

[22]

Yoshiharu Ikutani and Hidetake Uwano. 2014. Brain activity measurement during program comprehension with NIRS. In Proceedings of the International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). 1--6.

[23]

Seolhwa Lee, Danial Hooshyar, Hyesung Ji, Kichun Nam, and Heuiseok Lim. 2017. Mining biometric data to predict programmer expertise and task difficulty. Cluster Computing (2017), 1--11.

[24]

Andrian Marcus, Denys Poshyvanyk, and Rudolf Ferenc. 2008. Using the Conceptual Cohesion of Classes for Fault Prediction in Object-Oriented Systems. IEEE Transactions on Software Engineering (TSE) 34, 2 (2008), 287--30.

Digital Library

[25]

Thomas J. McCabe. 1976. A Complexity Measure. IEEE Transactions on Software Engineering (TSE) SE-2, 4 (1976), 308--320.

Digital Library

[26]

Sebastian C Müller and Thomas Fritz. 2016. Using (bio)metrics to predict code quality online. In Proceedings of the International Conference on Software Engineering (ICSE). 452--463.

Digital Library

[27]

Takao Nakagawa, Yasutaka Kamei, Hidetake Uwano, Akito Monden, Kenichi Matsumoto, and Daniel M. German. 2014. Quantifying programmers' mental workload during program comprehension based on cerebral blood flow measurement: a controlled experiment In Proceedings of the International Conference on Software Engineering (ICSE). 448--451.

Digital Library

[28]

Kristien Ooms, Lien Dupont, Lieselot Lapon, and Stanislav Popelka. 2015. Accuracy and precision of fixation locations recorded with the low-cost Eye Tribe tracker in different experimental setups. Journal of eye movement research 8, 1 (2015).

[29]

Denys Poshyvanyk, Yann-Gaël Guéhéneuc, Andrian Marcus, Giuliano Antoniol, and Vaclav Rajlich. 2006. Combining Probabilistic Ranking and Latent Semantic Indexing for Feature Identification. In Proceedings of the International Conference on Program Comprehension (ICPC). 137--148.

Digital Library

[30]

Daryl Posnett, Abram Hindle, and Premkumar Devanbu. 2011. A Simpler Model of Software Readability. In Proceedings of the Working Conference on Mining Software Repositories (MSR). 73--82.

Digital Library

[31]

Keith Rayner. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 124, 3 (1998), 372--422.

[32]

Iflaah Salman, Ayse Tosun Misirli, and Natalia Juristo. 2015. Are students representatives of professionals in software engineering experiments?. In Proceedings of the International Conference on Software Engineering (ICSE). 666--676.

Digital Library

[33]

Simone Scalabrino, Mario Linares-Vasquez, Denys Poshyvanyk, and Rocco Oliveto. 2016. Improving code readability models with textual features. In Proceedings of the International Conference on Program Comprehension (ICPC). 1--10.

[34]

Timothy R Shaffer, Jenna L Wise, Braden M Walters, Sebastian C Müller, Michael Falcone, and Bonita Sharif. 2015. iTrace: Enabling eye tracking on software artifacts within the IDE to support software engineering tasks. In Proceedings of the Joint Meeting on Foundations of Software Engineering (ESEC/FSE). 954--957.

Digital Library

[35]

Janet Siegmund, Christian Kästner, Sven Apel, Chris Parnin, Anja Bethmann, Thomas Leich, Gunter Saake, and André Brechmann. 2014. Understanding understanding source code with functional magnetic resonance imaging. In Proceedings of the International Conference on Software Engineering (ICSE). 378--389.

Digital Library

[36]

Janet Siegmund, Norman Peitek, Chris Parnin, Sven Apel, Johannes Hofmeister, Christian Kästner, Andrew Begel, Anja Bethmann, and André Brechmann. 2017. Measuring neural efficiency of program comprehension. In Proceedings of the Joint Meeting on Foundations of Software Engineering (ESEC/FSE). 140--150.

Digital Library

[37]

Robert R Sokal. 1958. A statistical method for evaluating systematic relationship. University of Kansas science bulletin 28 (1958), 1409--1438.

[38]

Erin Treacy Solovey, Daniel Afergan, Evan M. Peck, Samuel W. Hincks, and Robert J. K. Jacob. 2015. Designing Implicit Interfaces for Physiological Computing. ACM Transactions on Computer-Human Interaction 21, 6 (2015), 1--27.

Digital Library

[39]

Claes Wohlin, Per Runeson, Höst Martin, Magnus C. Ohlsson, Björn Regnell, and Anders Wesslén. 2000. Experimentation in Software Engineering - An Introduction. Kluwer Academic Publishers.

Digital Library

[40]

Robert K. Yin. 1994. Case Study Research: Design and Methods (2nd ed.). Sage Publications.

Cited By

Lima RSouza JFonseca BTeixeira LMello RRibeiro MGheyi RGarcia A(2024)Investigating the Social Representations of Harmful CodeJournal of Software Engineering Research and Development10.5753/jserd.2024.355412:1Online publication date: 10-Apr-2024
https://doi.org/10.5753/jserd.2024.3554
Varnavsky A(2024)Using the gaming chatbot to develop algorithmic and logical thinking skillsInformatics and education10.32517/0234-0453-2024-39-2-48-5839:2(48-58)Online publication date: 21-Jun-2024
https://doi.org/10.32517/0234-0453-2024-39-2-48-58
Lee GJu HLee SFilkov VRay BZhou M(2024)NeuroJIT: Improving Just-In-Time Defect Prediction Using Neurophysiological and Empirical Perceptions of Modern DevelopersProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695056(594-605)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695056
Show More Cited By

Index Terms

The effect of poor source code lexicon and readability on developers' cognitive load
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
2. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Software management
        Software maintenance

Recommendations

Understanding understanding source code with functional magnetic resonance imaging
ICSE 2014: Proceedings of the 36th International Conference on Software Engineering

Program comprehension is an important cognitive process that inherently eludes direct measurement. Thus, researchers are struggling with providing suitable programming languages, tools, or coding conventions to support developers in their everyday ...
Moving towards objective measures of program comprehension
ESEC/FSE 2018: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Traditionally, program comprehension research relies heavily on indirect measures of comprehension, where subjects report on their own comprehension levels or summarize part of an artifact so that researchers can instead deduce the level of ...
The Cognitive Load Affects the Interaction Pattern of Emotion and Working Memory

Emotion might selectively affect spatial and verbal cognitive activities and the selective interaction patter could be modulated by cognitive load. To test the hypotheses, the authors used event-related potentials ERPs technique to investigate the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPC '18: Proceedings of the 26th Conference on Program Comprehension

May 2018

423 pages

ISBN:9781450357142

DOI:10.1145/3196321

General Chair:
Foutse Khomh
École Polytechnique de Montréal, Canada
,
Program Chairs:
Chanchal K. Roy
University of Saskatchewan, Canada
,
Janet Siegmund
University of Passau, Germany

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 28 - 29, 2018

Gothenburg, Sweden

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

71
Total Citations
View Citations
634
Total Downloads

Downloads (Last 12 months)120
Downloads (Last 6 weeks)18

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lima RSouza JFonseca BTeixeira LMello RRibeiro MGheyi RGarcia A(2024)Investigating the Social Representations of Harmful CodeJournal of Software Engineering Research and Development10.5753/jserd.2024.355412:1Online publication date: 10-Apr-2024
https://doi.org/10.5753/jserd.2024.3554
Varnavsky A(2024)Using the gaming chatbot to develop algorithmic and logical thinking skillsInformatics and education10.32517/0234-0453-2024-39-2-48-5839:2(48-58)Online publication date: 21-Jun-2024
https://doi.org/10.32517/0234-0453-2024-39-2-48-58
Lee GJu HLee SFilkov VRay BZhou M(2024)NeuroJIT: Improving Just-In-Time Defect Prediction Using Neurophysiological and Empirical Perceptions of Modern DevelopersProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695056(594-605)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695056
Shi GKazemi FGodfrey MMcIntosh S(2024)Reevaluating the Defect Proneness of Atoms of Confusion in Java SystemsProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3686677(154-164)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3686677
Pandey MOney SBegel A(2024)Towards Inclusive Source Code Readability Based on the Preferences of Programmers with Visual ImpairmentsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642512(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642512
Ahmad HEndres MNewman KSantiesteban PShedden EWeimer WRoychoudhury APaiva AAbreu RStorey M(2024)Causal Relationships and Programming Outcomes: A Transcranial Magnetic Stimulation ExperimentProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639096(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639096
Kaur KMishra AChopra P(2024)An amalgamation of cognitive aspects in software engineeringExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122968246:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122968
Grabinger LHauser FWolff CMottok J(2024)On Eye Tracking in Software EngineeringSN Computer Science10.1007/s42979-024-03045-35:6Online publication date: 26-Jul-2024
https://dl.acm.org/doi/10.1007/s42979-024-03045-3
Bayrambas FSendurur E(2024)The comparison of two incidental learning scenarios on a digital learning platform from the cognitive load perspectiveEducation and Information Technologies10.1007/s10639-023-12241-229:9(11087-11117)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10639-023-12241-2
Li ZLuo YWang JPan YYu LLiang H(2024)Feasibility and performance enhancement of collaborative control of unmanned ground vehicles via virtual realityPersonal and Ubiquitous Computing10.1007/s00779-024-01799-428:3-4(579-595)Online publication date: 9-May-2024
https://doi.org/10.1007/s00779-024-01799-4
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents