short-paper

A Conceptual Replication on Predicting the Severity of Software Vulnerabilities

Authors:

Sefa Eren Sahin,

Ayse TosunAuthors Info & Claims

EASE '19: Proceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering

Pages 244 - 250

https://doi.org/10.1145/3319008.3319033

Published: 15 April 2019 Publication History

Abstract

Software vulnerabilities may lead to crucial security risks in software systems. Thus, prioritization of the vulnerabilities is an important task for security teams, and assessing how severe the vulnerabilities are would help teams during fixing and maintenance activities. We replicated a prior work which aims to predict the severity of software vulnerabilities by grouping vulnerabilities into different severity levels. We follow their approach on feature extraction using word embeddings, and on prediction model using Convolutional Neural Networks (CNNs). In addition, Long Short Term Memory (LSTM) and Extreme Gradient Boosting (XGBoost) models are used. We also extend the replicated work by aiming to predict severity scores rather than levels. We carried out two experiments for predicting severity levels and severity scores of 82,974 vulnerabilities. On predicting the severity levels, our LSTM and CNN models perform similarly with an F1 score of 0.756 F1 score and 0.752, respectively. On predicting the severity scores, LSTM, CNN and XGBoost models perform 16.14%, 17.03%, 18.91% MAPE values, respectively.

References

[1]

{n. d.}. Common Vulnerability Scoring System SIG. https://www.first.org/cvss

[2]

{n. d.}. CVE - Common Vulnerabilities and Exposures (CVE). http://cve.mitre.org/

[3]

{n. d.}. CVSS v2 Complete Documentation. https://www.first.org/cvss/v2/guide

[4]

{n. d.}. Introduction to Boosted Trees ăĂŤ xgboost 0.81 documentation. https://xgboost.readthedocs.io/en/latest/tutorials/model.html

[5]

{n. d.}. NVD - Home. https://nvd.nist.gov/

[6]

Atlassian. {n. d.}. Severity Levels for Security Issues. https://www.atlassian.com/trust/security/security-severity-levels

[7]

Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '16. ACM Press, San Francisco, California, USA, 785--794.

Digital Library

[8]

Zhuobing Han, Xiaohong Li, Zhenchang Xing, Hongtao Liu, and Zhiyong Feng. 2017. Learning to Predict Severity of Software Vulnerability Using Only Vulnerability Description. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, Shanghai, 125--136.

[9]

Andrew Meneely. 2015. Chapter 8 - Analyzing Security Data. In The Art and Science of Analyzing Software Data, Christian Bird, Tim Menzies, and Thomas Zimmermann (Eds.). Morgan Kaufmann, Boston, 215--229.

[10]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. arXiv.1301.37S1 {cs} (Jan. 2013). http://arxiv.org/abs/1301.3781 arXiv: 1301.3781.

[11]

Stephan Neuhaus, Thomas Zimmermann, Christian Holler, and Andreas Zeller. 2007. Predicting Vulnerable Software Components. In Proceedings of the 14th ACM Conference on Computer and Communications Security (CCS '07). ACM, New York, NY, USA, 529--540.

Digital Library

[12]

Viet Hung Nguyen and Le Minh Sang Tran. 2010. Predicting Vulnerable Software Components with Dependency Graphs. In Proceedings of the 6th International Workshop on Security Measurements and Metrics (MetriSec '10). ACM, New York, NY, USA, 3:1--3:8.

Digital Library

[13]

Sefa Eren Sahin. 2019. Predict severity of software vulnerabilities using Deep Neural Networks and Gradient Boosting (XGBoost): erensahin/vulnerability-severity-predictor. https://github.com/erensahin/vulnerability-severity-predictor

[14]

R. Scandariato, J. Walden, A. Hovsepyan, and W. Joosen. 2014. Predicting Vulnerable Software Components via Text Mining. IEEE Transactions on Software Engineering 40, 10 (Oct. 2014), 993--1006.

[15]

Yonghee Shin, Andrew Meneely, Laurie Williams, and Jason A. Osborne. 2011. Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities. IEEE Transactions on Software Engineering 37, 6 (Nov. 2011), 772--787.

Digital Library

[16]

Georgios Spanos, Lefteris Angelis, and Dimitrios Toloudis. 2017. Assessment of Vulnerability Severity using Text Mining. ACM, 49.

Digital Library

Cited By

Freitas TNovo CSoares JDutra ICorreia MShariati BMartins R(2024)HAL 9000: a Risk Manager for ITSs2024 IEEE 6th International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications (TPS-ISA)10.1109/TPS-ISA62245.2024.00044(322-331)Online publication date: 28-Oct-2024
https://doi.org/10.1109/TPS-ISA62245.2024.00044
Mirtaheri SPugliese AMovahedkor NMajd A(2024)Advanced Automated Vulnerability Scoring: Improving Performance with a Fine-Tuned BERT-CNN Model2024 11th International Symposium on Telecommunications (IST)10.1109/IST64061.2024.10843410(109-113)Online publication date: 9-Oct-2024
https://doi.org/10.1109/IST64061.2024.10843410
Mirtaheri SPugliese A(2024)Leveraging Generative AI to Enhance Automated Vulnerability Scoring2024 IEEE Conference on Dependable, Autonomic and Secure Computing (DASC)10.1109/DASC64200.2024.00014(57-64)Online publication date: 5-Nov-2024
https://doi.org/10.1109/DASC64200.2024.00014
Show More Cited By

Index Terms

A Conceptual Replication on Predicting the Severity of Software Vulnerabilities
1. Security and privacy
  1. Software and application security
2. Software and its engineering
  1. Software creation and management
    1. Software post-development issues
      1. Software evolution

Recommendations

A vulnerability severity prediction method based on bimodal data and multi-task learning
Highlights
- A new vulnerability severity prediction method is proposed to improve the F1 score.
- The GraphCodeBert is used to provide comprehensive information for prediction.
- Multi-task learning is used to enhance the generalization ability of ...
Abstract
Facing the increasing number of software vulnerabilities, the automatic analysis of vulnerabilities has become an important task in the field of software security. However, the existing severity prediction methods are mainly based on ...
CVE Severity Prediction From Vulnerability Description - A Deep Learning Approach
Abstract
The Common Vulnerabilities and Exposures (CVE) system is a widely used standard for identifying and tracking known vulnerabilities in software systems. The severity of these vulnerabilities must be determined in order to prioritize mitigation ...
Predicting Severity of Software Vulnerability Based on Grey System Theory
Proceedings of the ICA3PP International Workshops and Symposiums on Algorithms and Architectures for Parallel Processing - Volume 9532

Vulnerabilities usually represents the risk level of software, therefore, it is of high value to predict vulnerabilities so as to evaluate the security level of software. Current researches mainly focus on predicting the number of vulnerabilities or the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

EASE '19: Proceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering

April 2019

345 pages

ISBN:9781450371452

DOI:10.1145/3319008

Program Chairs:
Shaukat Ali,
Vahid Garousi

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

IT University of Copenhagen

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 April 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

EASE '19

EASE '19: Evaluation and Assessment in Software Engineering

April 15 - 17, 2019

Copenhagen, Denmark

Acceptance Rates

EASE '19 Paper Acceptance Rate 20 of 73 submissions, 27%;

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
289
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Freitas TNovo CSoares JDutra ICorreia MShariati BMartins R(2024)HAL 9000: a Risk Manager for ITSs2024 IEEE 6th International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications (TPS-ISA)10.1109/TPS-ISA62245.2024.00044(322-331)Online publication date: 28-Oct-2024
https://doi.org/10.1109/TPS-ISA62245.2024.00044
Mirtaheri SPugliese AMovahedkor NMajd A(2024)Advanced Automated Vulnerability Scoring: Improving Performance with a Fine-Tuned BERT-CNN Model2024 11th International Symposium on Telecommunications (IST)10.1109/IST64061.2024.10843410(109-113)Online publication date: 9-Oct-2024
https://doi.org/10.1109/IST64061.2024.10843410
Mirtaheri SPugliese A(2024)Leveraging Generative AI to Enhance Automated Vulnerability Scoring2024 IEEE Conference on Dependable, Autonomic and Secure Computing (DASC)10.1109/DASC64200.2024.00014(57-64)Online publication date: 5-Nov-2024
https://doi.org/10.1109/DASC64200.2024.00014
Dhawan NSharma RChattopadhyay SChoudhary AJain V(2024)Enhancing Green Bean Anthracnose Severity Detection via Integrated CNN-LSTM Models2023 4th International Conference on Intelligent Technologies (CONIT)10.1109/CONIT61985.2024.10626282(1-4)Online publication date: 21-Jun-2024
https://doi.org/10.1109/CONIT61985.2024.10626282
A MKota KBabu AS S(2024)CVE Severity Prediction From Vulnerability Description - A Deep Learning ApproachProcedia Computer Science10.1016/j.procs.2024.04.294235(3105-3117)Online publication date: 2024
https://doi.org/10.1016/j.procs.2024.04.294
Malhotra RVidushi (2024)Text mining based an automatic model for software vulnerability severity predictionInternational Journal of System Assurance Engineering and Management10.1007/s13198-024-02371-215:8(3706-3724)Online publication date: 31-May-2024
https://doi.org/10.1007/s13198-024-02371-2
Chen BZou WCai BMeng QLiu WLi PChen L(2024)An empirical study on the potential of word embedding techniques in bug report management tasksEmpirical Software Engineering10.1007/s10664-024-10510-329:5Online publication date: 25-Jul-2024
https://doi.org/10.1007/s10664-024-10510-3
Wang RXu SJi XTian YGong LWang K(2024)An extensive study of the effects of different deep learning models on code vulnerability detection in Python codeAutomated Software Engineering10.1007/s10515-024-00413-431:1Online publication date: 31-Jan-2024
https://doi.org/10.1007/s10515-024-00413-4
Malhotra RVidushi V(2023)Impact of Word Embedding Methods on Software Vulnerability Severity Prediction Models2023 13th International Conference on Cloud Computing, Data Science & Engineering (Confluence)10.1109/Confluence56041.2023.10048868(293-297)Online publication date: 19-Jan-2023
https://doi.org/10.1109/Confluence56041.2023.10048868
Bassi DSingh H(2023)A Systematic Literature Review on Software Vulnerability Prediction ModelsIEEE Access10.1109/ACCESS.2023.331261311(110289-110311)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3312613
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten