research-article

Open access

When not to comment: questions and tradeoffs with API documentation for C++ projects

Authors:

Caitlin Sadowski,

Emerson Murphy-Hill,

Andrea KnightAuthors Info & Claims

ICSE '18: Proceedings of the 40th International Conference on Software Engineering

Pages 643 - 653

https://doi.org/10.1145/3180155.3180176

Published: 27 May 2018 Publication History

Abstract

Without usable and accurate documentation of how to use an API, developers can find themselves deterred from reusing relevant code. In C++, one place developers can find documentation is in a header file. When information is missing, they may look at the corresponding implementation code. To understand what's missing from C++ API documentation and the factors influencing whether it will be fixed, we conducted a mixed-methods study involving two experience sampling surveys with hundreds of developers at the moment they visited implementation code, interviews with 18 of those developers, and interviews with 8 API maintainers. In many cases, updating documentation may provide only limited value for developers, while requiring effort maintainers don't want to invest. We identify a set of questions maintainers and tool developers should consider when improving API-level documentation.

References

[1]

Appendix to this paper. https://goo.gl/wMg3vY.

[2]

How to Write Doc Comments for the Javadoc Tool. http://www.oracle.com/technetwork/java/javase/documentation/index-137868.html. Accessed 12 February 2018.

[3]

Stack Overflow. http://www.stackoverflow.com.

[4]

Jürgen Börstler and Barbara Paech. 2016. The Role of Method Chains and Comments in Software Readability and Comprehension---An Experiment. IEEE Transactions on Software Engineering 42, 9 (Sept. 2016), 886--898.

[5]

Joel Brandt, Philip J. Guo, Joel Lewenstein, Mira Dontcheva, and Scott R. Klemmer. 2009. Two Studies of Opportunistic Programming: Interleaving Web Foraging, Learning, and Writing Code. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1589--1598.

Digital Library

[6]

Raymond P.L. Buse and Westley Weimer. 2012. Synthesizing API Usage Examples. In Proceedings of the 34th International Conference on Software Engineering. IEEE Press, 782--792.

Digital Library

[7]

Mihaly Csikszentmihalyi and Reed Larson. 2014. Validity and Reliability of the Experience-Sampling Method. In Flow and the Foundations of Positive Psychology. Springer, 35--54.

[8]

Barthélémy Dagenais and Martin P. Robillard. 2010. Creating and Evolving Developer Documentation: Understanding the Decisions of Open Source Contributors. In Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 127--136.

Digital Library

[9]

Uri Dekel and James D. Herbsleb. 2009. Improving API Documentation Usability with Knowledge Pushing. In Proceedings of the 31st International Conference on Software Engineering. IEEE Press, 320--330.

Digital Library

[10]

Ekwa Duala-Ekoko and Martin P. Robillard. 2012. Asking and Answering Questions about Unfamiliar APIs: An Exploratory Study. In Proceedings of the 34th International Conference on Software Engineering. IEEE Press, 266--276.

Digital Library

[11]

Scott D. Fleming, Chris Scaffidi, David Piorkowski, Margaret Burnett, Rachel Bellamy, Joseph Lawrance, and Irwin Kwan. 2013. An Information Foraging Theory Perspective on Tools for Debugging, Refactoring, and Reuse Tasks. ACM Transactions on Software Engineering and Methodology 22, 2 (2013), 14:1--14:41.

Digital Library

[12]

Björn Hartmann, Mark Dhillon, and Matthew K. Chan. 2011. HyperSource: Bridging the Gap Between Source and Code-Related Web Sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2207--2210.

Digital Library

[13]

Donald E. Knuth. 1974. Structured Programming with go to Statements. Comput. Surveys 6, 4 (Dec. 1974), 261--301.

Digital Library

[14]

Andrew J. Ko, Brad A. Myers, Michael J. Coblenz, and Htet Htet Aung. 2006. An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks. IEEE Transactions on Software Engineering 32, 12 (Dec. 2006).

Digital Library

[15]

John Lakos. 1996. Large-scale C++ Software Design. Addison-Wesley.

Digital Library

[16]

Thomas D. La Toza, Gina Venolia, and Robert DeLine. 2006. Maintaining Mental Models: A Study of Developer Work Habits. In Proceedings of the 28th International Conference on Software Engineering. ACM, 492--501.

Digital Library

[17]

Timothy C. Lethbridge, Janice Singer, and Andrew Forward. 2003. How Software Engineers Use Documentation: The State of the Practice. IEEE Software 20, 6 (Nov.--Dec. 2003), 35--39.

Digital Library

[18]

Walid Maalej and Martin P. Robillard. 2013. Patterns of Knowledge in API Reference Documentation. IEEE Transactions on Software Engineering 39, 9 (Sept. 2013), 1264--1282.

Digital Library

[19]

Paul W. McBurney and Collin McMillan. 2014. Automatic Documentation Generation via Source Code Summarization of Method Context. In Proceedings of the 22nd International Conference on Program Comprehension. ACM, 279--290.

Digital Library

[20]

João Eduardo Montandon, Hudson Borges, Daniel Felix, and Marco Tulio Valente. 2013. Documenting APIs with Examples: Lessons Learned with the APIMiner Platform. In 20th Working Conference on Reverse Engineering. IEEE, 401--408.

[21]

Laura Moreno, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Andrian Marcus. 2015. How Can I Use This Method?. In Proceedings of the 37th International Conference on Software Engineering. IEEE, 880--890.

Digital Library

[22]

Yoann Padioleau, Lin Tan, and Yuanyuan Zhou. 2009. Listening to Programmers-Taxonomies and Characteristics of Comments in Operating System Code. In Proceedings of the 31st International Conference on Software Engineering. IEEE Press, 331--341.

Digital Library

[23]

Chris Parnin, Christoph Treude, and Margaret-Anne Storey. 2013. Blogging Developer Knowledge: Motivations, Challenges, and Future Directions. In Proceedings of the 2013 IEEE 21st International Conference on Program Comprehension. IEEE Press, 211--214.

[24]

Gayane Petrosyan, Martin P. Robillard, and Renato De Mori. 2015. Discovering Information Explaining API Types Using Text Classification. In Proceedings of the 37th International Conference on Software Engineering. IEEE Press, 869--879.

Digital Library

[25]

David Piorkowski, Austin Z. Henley, Tahmid Nabi, Scott D. Fleming, Christopher Scaffidi, and Margaret Burnett. 2016. Foraging and Navigations, Fundamentally: Developers' Predictions of Value and Cost. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 97--108.

Digital Library

[26]

David J. Piorkowski, Scott D. Fleming, Irwin Kwan, Margaret M. Burnett, Chris Scaffidi, Rachel K.E. Bellamy, and Joshua Jordahl. 2013. The Whats and Hows of Programmers' Foraging Diets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3063--3072.

Digital Library

[27]

Luca Ponzanelli, Alberto Bacchelli, and Michele Lanza. 2013. Seahawk: Stack Overfow in the IDE. In Proceedings of the 35th International Conference on Software Engineering. IEEE Press, 1295--1298.

Digital Library

[28]

Rachel Potvin and Josh Levenberg. 2016. Why Google Stores Billions of Lines of Code in a Single Repository. Commun. ACM 59, 7 (July 2016), 78--87.

Digital Library

[29]

Martin P. Robillard. 2009. What Makes APIs Hard to Learn? Answers from Developers. IEEE Software 26, 6 (Nov.--Dec. 2009), 27--34.

Digital Library

[30]

Martin P. Robillard and Robert DeLine. 2011. A field study of API learning obstacles. Empirical Software Engineering 16, 6 (Dec. 2011), 703--732.

Digital Library

[31]

Tobias Roehm, Rebecca Tiarks, Rainer Koschke, and Walid Maalej. 2012. How Do Professional Developers Comprehend Software?. In Proceedings of the 34th International Conference on Software Engineering. IEEE Press, 255--265.

Digital Library

[32]

Caitlin Sadowski, Kathryn T. Stolee, and Sebastian Elbaum. 2015. How Developers Search for Code: A Case Study. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 191--201.

Digital Library

[33]

Felice Salviulo and Giuseppe Scanniello. 2014. Dealing with Identifiers and Comments in Source Code Comprehension and Maintenance: Results from an Ethnographically-informed Study with Students and Professionals. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. ACM, 48:1--48:10.

Digital Library

[34]

Jonathan Sillito, Gail C. Murphy, and Kris De Volder. 2006. Questions Programmers Ask During Software Evolution Tasks. In Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 23--34.

Digital Library

[35]

Giriprasad Sridhara, Emily Hill, Divya Muppaneni, Lori Pollock, and K. Vijay-Shanker. 2010. Towards Automatically Generating Summary Comments for Java Methods. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering. ACM, 43--52.

Digital Library

[36]

Giriprasad Sridhara, Lori Pollock, and K. Vijay-Shanker. 2011. Generating Parameter Comments and Integrating with Method Summaries. In Proceedings of the 2011 IEEE 19th International Conference on Program Comprehension. IEEE, 71--80.

Digital Library

[37]

Christoph Treude and Martin P. Robillard. 2016. Augmenting API Documentation with Insights from Stack Overflow. In Proceedings of the 38th International Conference on Software Engineering. ACM, 392--403.

Digital Library

[38]

Gias Uddin and Martin P. Robillard. 2015. How API Documentation Fails. IEEE Software 32, 4 (July--Aug. 2015), 68--75.

Digital Library

[39]

Hao Zhong and Zhendong Su. 2013. Detecting API Documentation Errors. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object-Oriented Programming Systems Languages & Applications. ACM, 803--816.

Digital Library

[40]

Hao Zhong, Tao Xie, Lu Zhang, Jian Pei, and Hong Mei. 2009. MAPO: Mining and Recommending API Usage Patterns. In Proceedings of the European Conference on Object-Oriented Programming. Springer, 318--343.

Digital Library

[41]

Yu Zhou, Ruihang Gu, Taolue Chen, Zhiqiu Huang, Sebastiano Panichella, and Harald Gall. 2017. Analyzing APIs Documentation and Code to Detect Directive Defects. In Proceedings of the 39th International Conference on Software Engineering. IEEE Press, 27--37.

Digital Library

Cited By

Pan XLiu CZou YXie TXie BBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)MESIA: Understanding and Leveraging Supplementary Nature of Method-level Comments for Automatic Comment GenerationProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644401(74-86)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3644401
Subbaraman BShim SPeek N(2023)Forking a Sketch: How the OpenProcessing Community Uses Remixing to Collect, Annotate, Tune, and Extend Creative CodeProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595969(326-342)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3595969
Nassif MRobillard M(2023)A Field Study of Developer Documentation FormatExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585767(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585767
Show More Cited By

Recommendations

Taxonomy of inline code comment smells
Abstract
Code comments play a vital role in source code comprehension and software maintainability. It is common for developers to write comments to explain a code snippet, and commenting code is generally considered a good practice in software ...
A Survey on Research of Code Comment
ICMSS 2019: Proceedings of the 2019 3rd International Conference on Management Engineering, Software Engineering and Service Sciences

Code comments are one of the effective means for assisting programmers to understand the source code. High-quality code comments play an important role in areas such as software maintenance and software reuse. Good code comments can help programmers ...
Deep code comment generation
ICPC '18: Proceedings of the 26th Conference on Program Comprehension

During software maintenance, code comments help developers comprehend programs and reduce additional time spent on reading and navigating source code. Unfortunately, these comments are often mismatched, missing or outdated in the software projects. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '18: Proceedings of the 40th International Conference on Software Engineering

May 2018

1307 pages

ISBN:9781450356381

DOI:10.1145/3180155

Conference Chair:
Michel Chaudron
Chalmers University of Technology, University of Gothenburg, Sweden
,
General Chair:
Ivica Crnkovic
Chalmers University of Technology, University of Gothenburg, Sweden
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Mark Harman
Facebook and University College London, United Kingdom

Copyright © 2018 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives International 4.0 License.

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Check for updates

Qualifiers

Research-article

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 27 - June 3, 2018

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
1,053
Total Downloads

Downloads (Last 12 months)98
Downloads (Last 6 weeks)18

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pan XLiu CZou YXie TXie BBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)MESIA: Understanding and Leveraging Supplementary Nature of Method-level Comments for Automatic Comment GenerationProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644401(74-86)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3644401
Subbaraman BShim SPeek N(2023)Forking a Sketch: How the OpenProcessing Community Uses Remixing to Collect, Annotate, Tune, and Extend Creative CodeProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595969(326-342)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3595969
Nassif MRobillard M(2023)A Field Study of Developer Documentation FormatExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585767(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585767
Horvath AMacvean AMyers B(2023)Support for Long-Form Documentation Authoring and Maintenance2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00020(109-114)Online publication date: 3-Oct-2023
https://doi.org/10.1109/VL-HCC57772.2023.00020
Gu JShi JSu HLi WZang BGuan HChen H(2023)Understanding and Mitigating Twin Function Misuses in Operating System KernelIEEE Transactions on Computers10.1109/TC.2023.324036572:8(2181-2193)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1109/TC.2023.3240365
Wang XZhao LGrundy JPollock LPenta M(2023)APICad: Augmenting API Misuse Detection through Specifications from Code and DocumentsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00032(245-256)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00032
Zhu CLiu YWu XLi Y(2022)Identifying Solidity Smart Contract API Documentation ErrorsProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering10.1145/3551349.3556963(1-13)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3551349.3556963
Horvath AMyers BMacvean ARahman I(2022)Using Annotations for Sensemaking About CodeProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545667(1-16)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545667
Hu XXia XLo DWan ZChen QZimmermann TDwyer MDamian DZeller A(2022)Practitioners' expectations on automated code comment generationProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510152(1693-1705)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510152
Cummaudo AVasa RGrundy JAbdelrazek M(2022)Requirements of API Documentation: A Case Study into Computer Vision ServicesIEEE Transactions on Software Engineering10.1109/TSE.2020.304708848:6(2010-2027)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1109/TSE.2020.3047088
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents