Article

Toward a text classification system for the quality assessment of software requirements written in natural language

Authors:

Olga Ormandjieva,

Ishrar Hussain,

Leila KosseimAuthors Info & Claims

SOQUA '07: Fourth international workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting

Pages 39 - 45

https://doi.org/10.1145/1295074.1295082

Published: 03 September 2007 Publication History

Abstract

Requirements Engineering (RE) is concerned with the gathering, analyzing, specifying and validating of user requirements that are documented mostly in natural language. The artifact produced by the RE process is the software requirements specification (SRS) document. The success of a software project largely depends on the quality of SRS documentation, which serves as an input to the design, coding and testing phases. This paper approaches the problem of the automatic quality assessment of textual requirements from an innovative point of view, namely the use of the Natural Language Processing (NLP) text classification technique. The paper proposes a quality model for the requirements text and a text classification system to automate the quality assessment process. A large study evaluating the discriminatory power of the quality characteristics and the feasibility of a tool for the automatic detection of ambiguities in requirements documentation is presented. The study also provides a benchmark for such an evaluation and an upper bound on what we can expect automatic requirements quality assessment tools to achieve. The reported research is part of a larger project on the applicability of NLP techniques to assess the quality of artifacts produced in RE.

References

[1]

Ambriola, V., Gervasi, V., "Processing natural language requirements," In proceedings of Automated Software Engineering (ASE'97): 12th IEEE International Conference, November 1--5, 1997, pp. 36--45, 1997.

Digital Library

[2]

Carletta, J., "Assessing agreement on classification tasks: The kappa statistic," Computational Linguistics, 22(2), 1996, pp. 249--254.

Digital Library

[3]

Cohen, J., "A coefficient of agreement for nominal scales," Educational and Psychological Measurement, 20, 1960, pp. 37--46.

[4]

Cyre, W. R., "A Requirements Sublanguage for Automated Analysis," International Journal of Intelligent Systems, 10 (7), pp. 665--689, July 1995.

[5]

Denger, C., Berry, D., Kamsties, E., "Higher Quality Requirements Specifications through Natural Language Patterns," SWSTE, p. 80, IEEE International Conference on Software-Science, Technology & Engineering, 2003.

Digital Library

[6]

Fabbrini, F., Fusani, M., Gnesi, S., and Lami, G., "An Automatic Quality Evaluation for Natural Language Requirements," Proceedings of the Seventh International Workshop on Requirements Engineering: Foundation for Software Quality REFSQ'01, Interlaken, Switzerland, June 4--5, 2001.

[7]

Fantechi, A., Gnesi, S., Ristori, G., Carenini, M., Vanocchi, M., and Moreschini, P., "Assisting requirement formalization by means of natural language translation," Formal Methods in System Design, vol. 4, pp. 243--263, 1994.

Digital Library

[8]

Kamsties, E., Berry, D. M., and Paech, B., "Detecting Ambiguities in Requirements Documents Using Inspections," p. 68--80 in Proceedings of the First Workshop on Inspection in Software Engineering (WISE'01), Paris, France, July 23, 2001.

[9]

Klein, D. and Manning, C. D., "Accurate Unlexicalized Parsing", Proceedings of the 41st Meeting of the Association for Computational Linguistics, 2003.

Digital Library

[10]

Lami, G., Gnesi, S., Fabbrini, F., Fusani, M., and Trentanni, G., "An Automatic Tool for the Analysis of Natural Language Requirements," published as Technical Report 2004--TR-40, Consiglio Nazionale delle Ricerche, Instituto di Scienza e Tecnologie dell'Informazione 'A. Faedo', 2004.

[11]

Landis, J. R. and Koch, G. G., "The measurement of observer agreement for categorical data," Biometrics, 33, 1977, pp. 159--174.

[12]

Letier, E., Kramer, J., Magee, J. and Uchitel, S., "Monitoring and Control in Scenario-Based Requirements Analysis," Proceedings ICSE 2005 - 27th International Conference on Software Engineering, ACM Press, St. Louis, Missouri, USA, May 2005.

Digital Library

[13]

Lin, D., "Dependency-based Evaluation of MINIPAR," In Workshop on the Evaluation of Parsing Systems, Granada, Spain, May, 1998.

[14]

Lu, R., Jin, Z., and Wan, R., "Requirement Specification in Pseudo-Natural Language in PROMIS," In proceedings of 19th International Computer Software and Applications Conference (COMPSAC'95), pp. 96--101, 1995.

Digital Library

[15]

Meyer, B., "On Formalism in Specifications," IEEE Software, 2(1): pp. 6--26, January 1985.

Digital Library

[16]

Osborne, M. and MacNish, C. K., "Processing natural language software requirement specifications," In Proceedings of ICRE'96: 2nd IEEE International Conference on Requirements Engineering, pp. 229--236. IEEE Press, 1996.

Digital Library

[17]

Rolland, C. and Proix, C., "A Natural Language Approach For Requirements Engineering," Proceedings of the Fourth International Conference CAiSE'92 on Advanced Information Systems Engineering, vol. 593 of Lecture Notes in Computer Science, pp. 257--277, Manchester, United Kingdom, 1992.

[18]

Wilson, W., "Writing Effective Requirements Specifications," USAF Software Technology Conference, Utah, 1997.

[19]

Wilson, W., Rosenberg, L. and Hyatt, L., "Automated Quality Analysis of Natural Language Requirement Specifications," 14th Annual Pacific Northwest Software Quality Conference, Portland, 1996.Bowman, B., Debray, S. K., and Peterson, L. L. Reasoning about naming systems. ACM Trans. Program. Lang. Syst., 15, 5 (Nov. 1993), 795--825.

Cited By

Balwani STiwari SDasgupta SSharma A(2024)An Approach for Providing Recommendation for Requirements Non-Conformant with Requirement Templates (RTs)Proceedings of the 17th Innovations in Software Engineering Conference10.1145/3641399.3641412(1-11)Online publication date: 22-Feb-2024
https://dl.acm.org/doi/10.1145/3641399.3641412
Saqib MMustaqeem MJawed MAbdulaziz AKhan AKhan J(2024)Deep-transfer learning inspired natural language processing system for software requirements classificationKnowledge and Information Systems10.1007/s10115-024-02248-767:1(839-861)Online publication date: 5-Oct-2024
https://doi.org/10.1007/s10115-024-02248-7
Noor MSithungu SLebea K(2024)A Framework for Cyberwarfare Attribution TechniquesProceedings of Ninth International Congress on Information and Communication Technology10.1007/978-981-97-3302-6_33(407-417)Online publication date: 25-Jul-2024
https://doi.org/10.1007/978-981-97-3302-6_33
Show More Cited By

Index Terms

Toward a text classification system for the quality assessment of software requirements written in natural language
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Project and people management
2. Software and its engineering
  1. Software creation and management
    1. Software development process management
  2. Software notations and tools
    1. Software configuration management and version control systems

Recommendations

Resolving Ambiguities in Natural Language Software Requirements: A Comprehensive Survey

Requirements Engineering is one of the most vital activities in the entire Software Development Life Cycle. The success of the software is largely dependent on how well the users' requirements have been understood and converted into appropriate ...
Framework for quantitatively evaluating the quality requirements of software system
HCI International'13: Proceedings of the 15th international conference on Human Interface and the Management of Information: information and interaction design - Volume Part I

Quality requirements (QR) are a description which indicates how well the software's behavior is to be executed. It is widely recognized that quality requirements are vital for the success of software systems. Therefore, to define the quality ...
A methodology for the classification of quality of requirements using machine learning techniques

ContextOne of the most important factors in the development of a software project is the quality of their requirements. Erroneous requirements, if not detected early, may cause many serious problems, such as substantial additional costs, failure to meet ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SOQUA '07: Fourth international workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting

September 2007

120 pages

ISBN:9781595937247

DOI:10.1145/1295074

General Chair:
Mauro Pezzè
University of Lugano, Switzerland, and University of Milano-Bicocca, Italy

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery
SIGSOFT: ACM Special Interest Group on Software Engineering
CEPIS: The Council of European Professional Informatics Societies

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 September 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ESEC/FSE07

Sponsor:

ACM
SIGSOFT
CEPIS

ESEC/FSE07: Joint 11th European Software Engineering Conference 2007

September 3 - 4, 2007

Dubrovnik, Croatia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
1,216
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)3

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Balwani STiwari SDasgupta SSharma A(2024)An Approach for Providing Recommendation for Requirements Non-Conformant with Requirement Templates (RTs)Proceedings of the 17th Innovations in Software Engineering Conference10.1145/3641399.3641412(1-11)Online publication date: 22-Feb-2024
https://dl.acm.org/doi/10.1145/3641399.3641412
Saqib MMustaqeem MJawed MAbdulaziz AKhan AKhan J(2024)Deep-transfer learning inspired natural language processing system for software requirements classificationKnowledge and Information Systems10.1007/s10115-024-02248-767:1(839-861)Online publication date: 5-Oct-2024
https://doi.org/10.1007/s10115-024-02248-7
Noor MSithungu SLebea K(2024)A Framework for Cyberwarfare Attribution TechniquesProceedings of Ninth International Congress on Information and Communication Technology10.1007/978-981-97-3302-6_33(407-417)Online publication date: 25-Jul-2024
https://doi.org/10.1007/978-981-97-3302-6_33
Limaylla-Lunarejo MCondori-Fernandez NLuaces MHong JLanperne MPark JCerny TShahriar H(2023)Towards a FAIR Dataset for non-functional requirementsProceedings of the 38th ACM/SIGAPP Symposium on Applied Computing10.1145/3555776.3578611(1414-1421)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3555776.3578611
Kadebu PSikka STyagi RChiurunge P(2023)A classification approach for software requirements towards maintainable securityScientific African10.1016/j.sciaf.2022.e0149619(e01496)Online publication date: Mar-2023
https://doi.org/10.1016/j.sciaf.2022.e01496
Cheligeer CHuang JWu GBhuiyan NXu YZeng Y(2022)Machine learning in requirements elicitation: a literature reviewArtificial Intelligence for Engineering Design, Analysis and Manufacturing10.1017/S089006042200016636Online publication date: 26-Oct-2022
https://doi.org/10.1017/S0890060422000166
Buchmann EHauser S(2021)Text Mining for Standardized Quality Criteria of Natural-Language IT-Requirements2021 IEEE 29th International Requirements Engineering Conference (RE)10.1109/RE51729.2021.00043(390-395)Online publication date: Sep-2021
https://doi.org/10.1109/RE51729.2021.00043
Atapour-Abarghouei ABonner SMcGough A(2021)Rank over Class: The Untapped Potential of Ranking in Natural Language Processing2021 IEEE International Conference on Big Data (Big Data)10.1109/BigData52589.2021.9671386(3950-3959)Online publication date: 15-Dec-2021
https://doi.org/10.1109/BigData52589.2021.9671386
Dawei WAlfred RObit JOn C(2021)A Literature Review on Text Classification and Sentiment Analysis ApproachesComputational Science and Technology10.1007/978-981-33-4069-5_26(305-323)Online publication date: 16-Mar-2021
https://doi.org/10.1007/978-981-33-4069-5_26
Huang MLai PLin PYou YTsai RHsu W(2020)Biomedical named entity recognition and linking datasets: survey and our recent developmentBriefings in Bioinformatics10.1093/bib/bbaa05421:6(2219-2238)Online publication date: 30-Jun-2020
https://doi.org/10.1093/bib/bbaa054
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten