research-article

An Empirical Study on Dynamic Typing Related Practices in Python Systems

Authors:

Wanwangying Ma,

Baowen XuAuthors Info & Claims

ICPC '20: Proceedings of the 28th International Conference on Program Comprehension

Pages 83 - 93

https://doi.org/10.1145/3387904.3389253

Published: 12 September 2020 Publication History

Abstract

The dynamic typing discipline of Python allows developers to program at a high level of abstraction. However, type related bugs are commonly encountered in Python systems due to the lack of type declaration and static type checking. Especially, the misuse of dynamic typing discipline produces underlying bugs and increases maintenance efforts. In this paper, we introduce six types of dynamic typing related practices in Python programs, which are the common but potentially risky usage of dynamic typing discipline by developers. We also implement a tool named PYDYPE to detect them. Based on this tool, we conduct an empirical study on nine real-world Python systems (with the size of more than 460KLOC) to understand dynamic typing related practices. We investigate how widespread the dynamic typing related practices are, why they are introduced into the systems, whether their usage correlates with increased likelihood of bug occurring, and how developers fix dynamic typing related bugs. The results show that: (1) dynamic typing related practices exist inconsistently in different systems and Inconsistent Variable Types is most prevalent; (2) they are introduced into systems mainly during early development phase to promote development efficiency; (3) they have a significant positive correlation with bug occurring; (4) developers tend to add type checks or exception handling to fix dynamic typing related bugs. These results benefit future research in coding convention, language design, bug detection and fixing.

References

[1]

1991. Python. https://www.python.org/

[2]

2008. django. https://github.com/django/django

[3]

2008. ipython. https://github.com/ipython/ipython

[4]

2010. scikit-learn. https://github.com/scikit-learn/scikit-learn

[5]

2010. Static typing is the root of all evil. http://www.jayway.com/2010/04/14/static-typing-is-the-root-of-all-evil/

[6]

2010. tornado. https://github.com/tornadoweb/tornado

[7]

2012. ansible. https://github.com/ansible/ansible

[8]

2012. Mypy. http://mypy-lang.org/

[9]

2014. The inconvenient truth about dynamic vs. static typing. http://blog.jooq.org/2014/12/11/the-inconvenient-truth-about-dynamic-vs-static-typing

[10]

2016. Pysonar2. https://github.com/yinwang0/pysonar2

[11]

Beatrice Åkerblom, Jonathan Stendahl, Mattias Tumlin, and Tobias Wrigstad. 2014. Tracing dynamic features in python programs. In Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 292--295.

Digital Library

[12]

Oscar Callaú, Romain Robbes, Éric Tanter, and David Röthlisberger. 2013. How (and why) developers use the dynamic features of programming languages: the case of smalltalk. Empirical Software Engineering 18, 6 (2013), 1156--1194.

Digital Library

[13]

Lin Chen, Baowen Xu, Tianlin Zhou, and Xiaoyu Zhou. 2009. A constraint based bug checking approach for Python. In 2009 33rd Annual IEEE International Computer Software and Applications Conference, Vol. 2. IEEE, 306--311.

Digital Library

[14]

Zhifei Chen, Wanwangying Ma, Wei Lin, Lin Chen, Yanhui Li, and Baowen Xu. 2018. A study on the changes of dynamic feature code when fixing bugs: towards the benefits and costs of Python dynamic features. Science China Information Sciences 61, 1 (2018), 012107.

[15]

Bruno Dufour, Christopher Goard, Laurie Hendren, Oege De Moor, Ganesh Sittampalam, and Clark Verbrugge. 2004. Measuring the dynamic behaviour of AspectJ programs. ACM SIGPLAN Notices 39, 10 (2004), 150--169.

Digital Library

[16]

Ronald Aylmer Fisher. 1992. Statistical methods for research workers. In Breakthroughs in statistics. Springer, 66--70.

[17]

Zheng Gao, Christian Bird, and Earl T Barr. 2017. To type or not to type: quantifying detectable bugs in JavaScript. In Proceedings of the 39th International Conference on Software Engineering. IEEE Press, 758--769.

Digital Library

[18]

Liang Gong, Michael Pradel, Manu Sridharan, and Koushik Sen. 2015. DLint: dynamically checking bad coding practices in JavaScript. In Proceedings of the 2015 International Symposium on Software Testing and Analysis. ACM, 94--105.

Digital Library

[19]

Alex Holkner and James Harland. 2009. Evaluating the dynamic behaviour of Python applications. In Proceedings of the Thirty-Second Australasian Conference on Computer Science-Volume 91. Australian Computer Society, Inc., 19--28.

Digital Library

[20]

Sebastian Kleinschmager, Romain Robbes, Andreas Stefik, Stefan Hanenberg, and Eric Tanter. 2012. Do static type systems improve the maintainability of software systems? An empirical study. In 2012 20th IEEE International Conference on Program Comprehension (ICPC). IEEE, 153--162.

[21]

Clemens Mayer, Stefan Hanenberg, Romain Robbes, Éric Tanter, and Andreas Stefik. 2012. An empirical study of the influence of static type systems on the usability of undocumented software. In ACM SIGPLAN Notices, Vol. 47. ACM, 683--702.

Digital Library

[22]

Leo A Meyerovich and Ariel S Rabkin. 2013. Empirical analysis of programming language adoption. In ACM SIGPLAN Notices, Vol. 48. ACM, 1--18.

Digital Library

[23]

Nevena Milojkovic, Mohammad Ghafari, and Oscar Nierstrasz. 2017. Exploiting type hints in method argument names to improve lightweight type inference. In 2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC). IEEE, 77--87.

Digital Library

[24]

Matheus Paixao, Jens Krinke, DongGyun Han, Chaiyong Ragkhitwetsagul, and Mark Harman. 2017. Are developers aware of the architectural impact of their changes?. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering. IEEE Press, 95--105.

Digital Library

[25]

Joonyoung Park, Inho Lim, and Sukyoung Ryu. 2016. Battles with false positives in static analysis of JavaScript web applications in the wild. In 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C). IEEE, 61--70.

Digital Library

[26]

Michael Pradel, Parker Schuh, and Koushik Sen. 2015. TypeDevil: Dynamic type inconsistency analysis for JavaScript. In Proceedings of the 37th International Conference on Software Engineering-Volume 1. IEEE Press, 314--324.

[27]

Michael Pradel and Koushik Sen. 2015. The good, the bad, and the ugly: An empirical study of implicit type conversions in JavaScript. In 29th European Conference on Object-Oriented Programming (ECOOP 2015). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.

[28]

Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu. 2014. A large scale study of programming languages and code quality in github. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 155--165.

Digital Library

[29]

Gregor Richards, Sylvain Lebresne, Brian Burg, and Jan Vitek. 2010. An analysis of the dynamic behavior of JavaScript programs. In ACM Sigplan Notices, Vol. 45. ACM, 1--12.

Digital Library

[30]

Jacek Śliwerski, Thomas Zimmermann, and Andreas Zeller. 2005. When do changes induce fixes?. In ACM sigsoft software engineering notes, Vol. 30. ACM, 1--5.

[31]

Andreas Stuchlik and Stefan Hanenberg. 2012. Static vs. dynamic type systems: an empirical study about the relationship between type casts and development time. ACM SIGPLAN Notices 47, 2 (2012), 97--106.

Digital Library

[32]

Michele Tufano, Fabio Palomba, Gabriele Bavota, Rocco Oliveto, Massimiliano Di Penta, Andrea De Lucia, and Denys Poshyvanyk. 2015. When and why your code starts to smell bad. In Proceedings of the 37th International Conference on Software Engineering-Volume 1. IEEE Press, 403--414.

Digital Library

[33]

Beibei Wang, Lin Chen, Wanwangying Ma, Zhifei Chen, and Baowen Xu. 2015. An empirical study on the impact of Python dynamic features on change-proneness. In SEKE. 134--139.

[34]

Zhaogui Xu, Peng Liu, Xiangyu Zhang, and Baowen Xu. 2016. Python predictive analysis for bug detection. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 121--132.

Digital Library

[35]

Zhaogui Xu, Xiangyu Zhang, Lin Chen, Kexin Pei, and Baowen Xu. 2016. Python probabilistic type inference with natural language support. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 607--618.

Digital Library

Cited By

Troppmann DFass AStaicu CFilkov VRay BZhou M(2024)Typed and Confused: Studying the Unexpected Dangers of Gradual TypingProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695549(1858-1870)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695549
Chen ZChen LYang YFeng QLi XSong W(2024)Risky Dynamic Typing-related Practices in Python: An Empirical StudyACM Transactions on Software Engineering and Methodology10.1145/364959333:6(1-35)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3649593
Peng YHu RWang RGao CLi SLyu MRoychoudhury APaiva AAbreu RStorey M(2024)Less is More? An Empirical Study on Configuration Issues in Python PyPI EcosystemProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639077(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639077
Show More Cited By

Index Terms

An Empirical Study on Dynamic Typing Related Practices in Python Systems
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools
    1. General programming languages

Index terms have been assigned to the content through auto-classification.

Recommendations

Design and evaluation of gradual typing for python
DLS '14: Proceedings of the 10th ACM Symposium on Dynamic languages

Combining static and dynamic typing within the same language offers clear benefits to programmers. It provides dynamic typing in situations that require rapid prototyping, heterogeneous data structures, and reflection, while supporting static typing ...
Risky Dynamic Typing-related Practices in Python: An Empirical Study
Python’s dynamic typing nature provides developers with powerful programming abstractions. However, many type-related bugs are accumulated in code bases of Python due to the misuse of dynamic typing. The goal of this article is to aid in the understanding ...
Typing dynamic typing

Even when programming in a statically typed language we every now and then encounter statically untypable values; such values result from interpreting values or from communicating with the outside world. To cope with this problem most languages include ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPC '20: Proceedings of the 28th International Conference on Program Comprehension

July 2020

481 pages

ISBN:9781450379588

DOI:10.1145/3387904

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICPC '20

Sponsor:

SIGSOFT

ICPC '20: 28th International Conference on Program Comprehension

July 13 - 15, 2020

Seoul, Republic of Korea

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
400
Total Downloads

Downloads (Last 12 months)88
Downloads (Last 6 weeks)12

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Troppmann DFass AStaicu CFilkov VRay BZhou M(2024)Typed and Confused: Studying the Unexpected Dangers of Gradual TypingProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695549(1858-1870)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695549
Chen ZChen LYang YFeng QLi XSong W(2024)Risky Dynamic Typing-related Practices in Python: An Empirical StudyACM Transactions on Software Engineering and Methodology10.1145/364959333:6(1-35)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3649593
Peng YHu RWang RGao CLi SLyu MRoychoudhury APaiva AAbreu RStorey M(2024)Less is More? An Empirical Study on Configuration Issues in Python PyPI EcosystemProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639077(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639077
Cao YChen LChen ZZhong JZhang XWang L(2024)Efficient Construction of Practical Python Call Graphs with Entity Knowledge BaseInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819402450010434:07(999-1024)Online publication date: 22-May-2024
https://doi.org/10.1142/S0218194024500104
Jia XZhou YHussain YYang W(2024)An Empirical Study on Python Library Dependency and Conflict Issues2024 IEEE 24th International Conference on Software Quality, Reliability and Security (QRS)10.1109/QRS62785.2024.00057(504-515)Online publication date: 1-Jul-2024
https://doi.org/10.1109/QRS62785.2024.00057
Yang YWang ZChen ZXu B(2024)COPS: An improved information retrieval-based bug localization technique using context-aware program simplificationJournal of Systems and Software10.1016/j.jss.2023.111868207(111868)Online publication date: Jan-2024
https://doi.org/10.1016/j.jss.2023.111868
Shajii ARamirez GSmajlović HRay JBerger BAmarasinghe SNumanagić IVerbrugge CLhoták OShen X(2023)Codon: A Compiler for High-Performance Pythonic Applications and DSLsProceedings of the 32nd ACM SIGPLAN International Conference on Compiler Construction10.1145/3578360.3580275(191-202)Online publication date: 17-Feb-2023
https://dl.acm.org/doi/10.1145/3578360.3580275
Cao YChen LMa WLi YZhou YWang L(2023)Towards Better Dependency Management: A First Look at Dependency Smells in Python ProjectsIEEE Transactions on Software Engineering10.1109/TSE.2022.319135349:4(1741-1765)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TSE.2022.3191353
Jin WZhong DCai YKazman RLiu T(2023)Evaluating the Impact of Possible Dependencies on Architecture-Level MaintainabilityIEEE Transactions on Software Engineering10.1109/TSE.2022.317128849:3(1064-1085)Online publication date: 1-Mar-2023
https://doi.org/10.1109/TSE.2022.3171288
Lin XHua BWang YPan Z(2023)Towards a Large-Scale Empirical Study of Python Static Type Annotations2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00046(414-425)Online publication date: Mar-2023
https://doi.org/10.1109/SANER56733.2023.00046
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents