research-article

Not All Dependencies are Equal: An Empirical Study on Production Dependencies in NPM

Authors:

Jasmine Latendresse,

Suhaib Mujahid,

Diego Elias Costa,

Emad ShihabAuthors Info & Claims

ASE '22: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering

Article No.: 73, Pages 1 - 12

https://doi.org/10.1145/3551349.3556896

Published: 05 January 2023 Publication History

Abstract

Modern software systems are often built by leveraging code written by others in the form of libraries and packages to accelerate their development. While there are many benefits to using third-party packages, software projects often become dependent on a large number of software packages. Consequently, developers are faced with the difficult challenge of maintaining their project dependencies by keeping them up-to-date and free of security vulnerabilities. However, how often are project dependencies used in production where they could pose a threat to their project’s security?

We conduct an empirical study on 100 JavaScript projects using the Node Package Manager (npm) to quantify how often project dependencies are released to production and analyze their characteristics and their impact on security. Our results indicate that less than 1% of the installed dependencies are released to production. Our analysis reveals that the functionality of a package is not enough to determine if it will be released to production or not. In fact, 59% of the installed dependencies configured as runtime dependencies are not used in production, and 28.2% of the dependencies configured as development dependencies are used in production, debunking two common assumptions of dependency management. Findings also indicate that most security alerts target dependencies not used in production, making them highly unlikely to be a risk for the security of the software. Our study unveils a more complex side of dependency management: not all dependencies are equal. Dependencies used in production are more sensitive to security exposure and should be prioritized. However, current tools lack the appropriate support in identifying production dependencies.

References

[1]

[n. d.]. Software Bill of Materials | CISA. https://www.cisa.gov/sbom

[2]

2019. 2019 State of the Software Supply Chain. https://www.sonatype.com/hubfs/SSC/2019%20SSC/SON_SSSC-Report-2019_jun16-DRAFT.pdf

[3]

2019. Eight Key Findings Illustrating How to Make Open Source Work Even Better for Developers. https://cdn2.hubspot.net/hubfs/4008838/Resources/The-2019-Tidelift-managed-open-source-survey-results.pdf

[4]

2019. webpack. https://webpack.js.org/

[5]

2020. Do ”dependencies” and ”devDependencies” matter when using Webpack?https://jsramblings.com/do-dependencies-devdependencies-matter-when-using-webpack/

[6]

2020. npm-deps-parser. https://github.com/nVisium/npm-deps-parser

[7]

2020. Securing the World’s Software. https://octoverse.github.com/static/github-octoverse-2020-security-report.pdf

[8]

2021. Create react app. https://create-react-app.dev/

[9]

2021. Help, ‘npm audit‘ says I have a vulnerability in react-scripts! · Issue #11174 · facebook/create-react-app. https://github.com/facebook/create-react-app/issues/11174

[10]

2021. rollup.js. https://rollupjs.org/guide/en/

[11]

2022. The Complete Guide to Software Composition Analysis - FOSSA. https://fossa.com/complete-guide-software-composition-analysis

[12]

2022. GitHub Advisory Database. https://github.com/advisories

[13]

2022. Snyk | Developer security | Develop fast. Stay secure.https://snyk.io/

[14]

Rabe Abdalkareem, Olivier Nourry, Sultan Wehaibi, Suhaib Mujahid, and Emad Shihab. 2017. Why do developers use trivial packages? an empirical case study on npm. Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (08 2017). https://doi.org/10.1145/3106237.3106267

Digital Library

[15]

Rabe Abdalkareem, Vinicius Oda, Suhaib Mujahid, and Emad Shihab. 2020. On the impact of using trivial packages: an empirical case study on npm and PyPI. Empirical Software Engineering 25 (01 2020), 1168–1204. https://doi.org/10.1007/s10664-019-09792-9

[16]

Mahmoud Alfadel, Diego Elias Costa, Emad Shihab, and Mouafak Mkhallalati. 2021. On the Use of Dependabot Security Pull Requests. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). 254–265. https://doi.org/10.1109/MSR52588.2021.00037

[17]

Md Atique, Reza Chowdhury, Rabe Abdalkareem, and Emad Shihab. 2019. On the Untriviality of Trivial Packages: An Empirical Study of npm JavaScript Packages. Journal of IEEE Transactions on Software Engineering 01 (2019). http://das.encs.concordia.ca/uploads/atique_tse2021.pdf

[18]

Victor R. Basili, Lionel C. Briand, and Walcélio L. Melo. 1996. How reuse influences productivity in object-oriented systems. Commun. ACM 39 (10 1996), 104–116. https://doi.org/10.1145/236156.236184

Digital Library

[19]

Chris Bogart, Christian Kästner, James Herbsleb, and Ferdian Thung. 2021. When and How to Make Breaking Changes. ACM Transactions on Software Engineering and Methodology 30 (07 2021), 1–56. https://doi.org/10.1145/3447245

Digital Library

[20]

Xiaowei Chen, Rabe Abdalkareem, Suhaib Mujahid, Emad Shihab, and Xin Xia. 2021. Helping or not Helping? Why and How Trivial Packages Impact the npm Ecosystem. Empirical Software Engineering 26 (03 2021). https://doi.org/10.1007/s10664-020-09904-w

Digital Library

[21]

Jailton Coelho, Marco Túlio Valente, Luciano Milen, and Luciana Lourdes Silva. 2020. Is this GitHub Project Maintained? Measuring the Level of Maintenance Activity of Open-Source Projects. CoRR abs/2003.04755(2020). arXiv:2003.04755https://arxiv.org/abs/2003.04755

[22]

Diego Elias Costa, Suhaib Mujahid, Rabe Abdalkareem, and Emad Shihab. 2021. Breaking Type-Safety in Go: An Empirical Study on the Usage of the unsafe Package. IEEE Transactions on Software Engineering(2021), 1–1. https://doi.org/10.1109/TSE.2021.3057720

[23]

Diego Elias Costa, Suhaib Mujahid, Rabe Abdalkareem, and Emad Shihab. 2021. Breaking Type-Safety in Go: An Empirical Study on the Usage of the unsafe Package. IEEE Transactions on Software Engineering(2021), 1–1. https://doi.org/10.1109/TSE.2021.3057720

[24]

Joel Cox, Eric Bouwers, Marko van Eekelen, and Joost Visser. 2015. Measuring Dependency Freshness in Software Systems. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 2. 109–118. https://doi.org/10.1109/ICSE.2015.140

[25]

Alexandre Decan, Tom Mens, and Philippe Grosjean. 2019. An Empirical Comparison of Dependency Network Evolution in Seven Software Packaging Ecosystems. Empirical Software Engineering 24 (02 2019). https://doi.org/10.1007/s10664-017-9589-y

Digital Library

[26]

Josh Fruhlinger. 2020. Equifax data breach FAQ: What happened, who was affected, what was the impact?https://www.csoonline.com/article/3444488/equifax-data-breach-faq-what-happened-who-was-affected-what-was-the-impact.html

[27]

Emitza Guzman, David Azócar, and Yang Li. 2014. Sentiment Analysis of Commit Comments in GitHub: An Empirical Study. In Proceedings of the 11th Working Conference on Mining Software Repositories (Hyderabad, India) (MSR 2014). Association for Computing Machinery, New York, NY, USA, 352–355. https://doi.org/10.1145/2597073.2597118

Digital Library

[28]

J. I. Hejderup. 2015. In Dependencies We Trust: How vulnerable are dependencies in software modules?repository.tudelft.nl(2015). https://repository.tudelft.nl/islandora/object/uuid:3a15293b-16f6-4e9d-b6a2-f02cd52f1a9e?collection=education

[29]

Nasif Imtiaz, Seaver Thorn, and Laurie Williams. 2021. A comparative study of vulnerability reporting by software composition analysis tools. Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) (10 2021). https://doi.org/10.1145/3475716.3475769

Digital Library

[30]

Abbas Javan Jafari, Diego Elias Costa, Rabe Abdalkareem, Emad Shihab, and Nikolaos Tsantalis. 2021. Dependency Smells in JavaScript Projects. IEEE Transactions on Software Engineering(2021), 1–1. https://doi.org/10.1109/tse.2021.3106247

Digital Library

[31]

Riivo Kikas, Georgios Gousios, Marlon Dumas, and Dietmar Pfahl. 2017. Structure and Evolution of Package Dependency Networks. In Proceedings of the 14th International Conference on Mining Software Repositories (Buenos Aires, Argentina) (MSR ’17). IEEE Press, 102–112. https://doi.org/10.1109/MSR.2017.55

Digital Library

[32]

Raula Gaikovina Kula, Daniel M. German, Ali Ouni, Takashi Ishio, and Katsuro Inoue. 2017. Do developers update their library dependencies?Empirical Software Engineering 23, 1 (may 2017), 384–417. https://doi.org/10.1007/s10664-017-9521-5

Digital Library

[33]

Tobias Lauinger, Abdelberi Chaabane, Sajjad Arshad, William Robertson, Christo Wilson, and Engin Kirda. 2017. Thou Shalt Not Depend on Me: Analysing the Use of Outdated JavaScript Libraries on the Web. In Proceedings 2017 Network and Distributed System Security Symposium. Internet Society. https://doi.org/10.14722/ndss.2017.23414

[34]

Suhaib Mujahid, Diego Elias Costa, Rabe Abdalkareem, Emad Shihab, Mohamed Aymen Saied, and Bram Adams. 2021. Toward Using Package Centrality Trend to Identify Packages in Decline. IEEE Transactions on Engineering Management(2021), 1–15. https://doi.org/10.1109/tem.2021.3122012

[35]

Emerson Murphy-Hill, Ciera Jaspan, Caitlin Sadowski, David Shepherd, Michael Phillips, Collin Winter, Andrea Knight, Edward Smith, and Matt Jorde. 2019. What Predicts Software Developers’ Productivity?IEEE Transactions on Software Engineering(2019), 1–1. https://doi.org/10.1109/tse.2019.2900308

[36]

Stack Overflow. [n. d.]. Stack Overflow Developer Survey 2021. https://insights.stackoverflow.com/survey/2021

[37]

Ivan Pashchenko, Henrik Plate, Serena Ponta, Antonino Sabetta, and Fabio Massacci. 2018. Vulnerable open source dependencies: counting those that matter. 1–10. https://doi.org/10.1145/3239235.3268920

Digital Library

[38]

Ivan Pashchenko, Henrik Plate, Serena Ponta, Antonino Sabetta, and Fabio Massacci. 2020. Vuln4Real: A Methodology for Counting Actually Vulnerable Dependencies. IEEE Transactions on Software Engineering PP (09 2020), 1–1. https://doi.org/10.1109/TSE.2020.3025443

Digital Library

[39]

Ivan Pashchenko, Duc-Ly Vu, and Fabio Massacci. 2020. A Qualitative Study of Dependency Management and Its Security Implications. Association for Computing Machinery, New York, NY, USA, 1513–1531. https://doi.org/10.1145/3372297.3417232

Digital Library

[40]

Henrik Plate, Serena Ponta, and Antonino Sabetta. 2015. Impact assessment for vulnerabilities in open-source software libraries. 411–420. https://doi.org/10.1109/ICSM.2015.7332492

Digital Library

[41]

Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu. 2014. A Large Scale Study of Programming Languages and Code Quality in Github. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering(Hong Kong, China) (FSE 2014). Association for Computing Machinery, New York, NY, USA, 155–165. https://doi.org/10.1145/2635868.2635922

Digital Library

[42]

Adriana Sejfia and Max Schäfer. 2022. Practical Automated Detection of Malicious npm Packages. arXiv preprint arXiv:2202.13953(2022).

[43]

unisil. 2021. Source Map Parser. https://github.com/unisil/source-map-parser

[44]

Haroen Viaene. 2021. feat(dependencies): update algoliasearch-helper. https://github.com/algolia/instantsearch.js/pull/4936. (Accessed on 05/04/2022).

[45]

Stefan Wagner and Emerson Murphy-Hill. 2019. Factors That Influence Productivity: A Checklist. 69–84. https://doi.org/10.1007/978-1-4842-4221-6_8

[46]

Jeff Williams and Arshan Dabirsiaghi. 2012. The unfortunate reality of insecure libraries. Asp. Secur. Inc (2012), 1–26.

[47]

Stan Zajdel, Diego Elias Costa, and Hafedh Mili. 2022. Open Source Software: An Approach to Controlling Usage and Risk in Application Ecosystems. In Proceedings of the 26TH ACM International Systems and Software Product Line Conference. arXiv. https://doi.org/10.48550/ARXIV.2206.10358

[48]

Rodrigo Zapata, Raula Kula, Bodin Chinthanet, Takashi Ishio, Kenichi Matsumoto, and Akinori Ihara. 2018. Towards Smoother Library Migrations: A Look at Vulnerable Dependency Migrations at Function Level for npm JavaScript Packages. 559–563. https://doi.org/10.1109/ICSME.2018.00067

Cited By

Weeraddana NAlfadel MMcIntosh S(2024)Dependency-Induced Waste in Continuous Integration: An Empirical Study of Unused Dependencies in the npm EcosystemProceedings of the ACM on Software Engineering10.1145/36608231:FSE(2632-2655)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660823
Ma ZMondal SChen TZhang HHassan A(2024)VulNet: Towards improving vulnerability management in the Maven ecosystemEmpirical Software Engineering10.1007/s10664-024-10448-629:4Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1007/s10664-024-10448-6
Soto-Valero CTiwari DToady TBaudry B(2023)Automatic Specialization of Third-Party Java DependenciesIEEE Transactions on Software Engineering10.1109/TSE.2023.332495049:11(5027-5045)Online publication date: 18-Oct-2023
https://dl.acm.org/doi/10.1109/TSE.2023.3324950

Index Terms

Not All Dependencies are Equal: An Empirical Study on Production Dependencies in NPM

Index terms have been assigned to the content through auto-classification.

Recommendations

Extending dependencies with conditions
VLDB '07: Proceedings of the 33rd international conference on Very large data bases

This paper introduces a class of conditional inclusion dependencies (CINDs), which extends traditional inclusion dependencies (INDs) by enforcing bindings of semantically related data values. We show that CINDs are useful not only in data cleaning, but ...
On the impact of security vulnerabilities in the npm and RubyGems dependency networks
Abstract
The increasing interest in open source software has led to the emergence of large language-specific package distributions of reusable software libraries, such as npm and RubyGems. These software packages can be subject to vulnerabilities that may ...
Gentoo package dependencies over time
MSR 2014: Proceedings of the 11th Working Conference on Mining Software Repositories

Open source distributions such as Gentoo need to accurately track dependency relations between software packages in order to install working systems. To do this, Gentoo has a carefully authored database containing those relations. In this paper, we ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ASE '22: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering

October 2022

2006 pages

ISBN:9781450394758

DOI:10.1145/3551349

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 January 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ASE '22

ASE '22: 37th IEEE/ACM International Conference on Automated Software Engineering

October 10 - 14, 2022

MI, Rochester, USA

Acceptance Rates

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
298
Total Downloads

Downloads (Last 12 months)172
Downloads (Last 6 weeks)33

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Weeraddana NAlfadel MMcIntosh S(2024)Dependency-Induced Waste in Continuous Integration: An Empirical Study of Unused Dependencies in the npm EcosystemProceedings of the ACM on Software Engineering10.1145/36608231:FSE(2632-2655)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660823
Ma ZMondal SChen TZhang HHassan A(2024)VulNet: Towards improving vulnerability management in the Maven ecosystemEmpirical Software Engineering10.1007/s10664-024-10448-629:4Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1007/s10664-024-10448-6
Soto-Valero CTiwari DToady TBaudry B(2023)Automatic Specialization of Third-Party Java DependenciesIEEE Transactions on Software Engineering10.1109/TSE.2023.332495049:11(5027-5045)Online publication date: 18-Oct-2023
https://dl.acm.org/doi/10.1109/TSE.2023.3324950
Mujahid SCosta DAbdalkareem RShihab EBissyandé TKlein JBird CSarro F(2023)Where to Go Now? Finding Alternatives for Declining Packages in the npm EcosystemProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE56229.2023.00119(1628-1639)Online publication date: 11-Nov-2023
https://dl.acm.org/doi/10.1109/ASE56229.2023.00119

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents