research-article

On the effect of using SysML requirement diagrams to comprehend requirements: results from two controlled experiments

Authors:

Giuseppe Scanniello,

Miroslaw Staron,

Rogardt HeldalAuthors Info & Claims

EASE '14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering

Article No.: 49, Pages 1 - 10

https://doi.org/10.1145/2601248.2601259

Published: 13 May 2014 Publication History

Abstract

We carried out a controlled experiment and an external replication to investigate whether the use of requirement diagrams of the System Modeling Language (SysML) helps in the comprehensibility of requirements. The original experiment was conducted at the University of Basilicata in Italy with Bachelor students, while its replication was executed at the University of Gothenburg in Sweden with Bachelor and Master students. A total of 87 participants took part in the experiment and its replication. The achieved results indicated that the comprehension of requirements is statistically significant when requirements specification documents include requirement diagrams without any impact on the time to accomplish comprehension tasks. On the basis of our results, we also present and discuss possible implications from the practitioner and researcher perspectives.

References

[1]

S. M. Abrahão, C. Gravino, E. I. Pelozo, G. Scanniello, and G. Tortora. Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: Results from a family of five experiments. IEEE Trans. on Soft. Eng., 39(3), 2013.

Digital Library

[2]

J. Aranda, N. Ernst, J. Horkoff, and S. Easterbrook. A framework for empirical evaluation of model comprehensibility. In Proceedings of the International Workshop on Modeling in Software Engineering, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[3]

E. Arisholm, L. C. Briand, S. E. Hove, and Y. Labiche. The impact of UML documentation on software maintenance: An experimental evaluation. IEEE Trans. Softw. Eng., 32(6):365--381, 2006.

Digital Library

[4]

R. Baker. Modern permutation test software. In E. Edgington, editor, Randomization Tests, Marcel Decker, 1995.

[5]

V. R. Basili, S. Green, O. Laitenberger, F. Lanubile, F. Shull, L. S. Sørumgård, and M. V. Zelkowitz. The empirical investigation of perspective-based reading. Empirical Software Engineering, 1(2):133--164, 1996.

Digital Library

[6]

V. R. Basili and H. D. Rombach. The TAME project: Towards improvement-oriented software environments. IEEE Trans. Software Eng., 14(6):758--773, 1988.

Digital Library

[7]

L. C. Briand, D. Falessi, S. Nejati, M. Sabetzadeh, and T. Yue. Traceability and sysml design slices to support safety inspections: A controlled experiment. ACM Trans. Softw. Eng. Methodol., 23(1):9, 2014.

Digital Library

[8]

B. Bruegge and A. H. Dutoit. Object-Oriented Software Engineering: Using UML, Patterns and Java, 2nd edition. Prentice-Hall, 2003.

Digital Library

[9]

D. Budgen, A. J. Burn, O. P. Brereton, B. A. Kitchenham, and R. Pretorius. Empirical evidence about the UML: a systematic literature review. Software: Practice and Experience, 41(4):363--392, 2011.

Digital Library

[10]

J. Carver, L. Jaccheri, S. Morasca, and F. Shull. Issues in using students in empirical studies in software engineering education. In Proc. of the International Symposium on Software Metrics, pages 239--. IEEE Computer Society, 2003.

Digital Library

[11]

J. Cohen. Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Earlbaum Associates, Hillsdale, NJ, 1988.

[12]

W. J. Conover. Practical Nonparametric Statistics. Wiley, 3rd Edition, 1998.

[13]

J. L. Devore and N. Farnum. Applied Statistics for Engineers and Scientists. Duxbury, 1999.

[14]

B. Dobing and J. Parsons. How UML is used. Communications of the ACM, 49(5):109--113, 2006.

Digital Library

[15]

P. Ellis. The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results. Cambridge University Press, 2010.

[16]

D. Falessi, S. Nejati, M. Sabetzadeh, L. Briand, and A. Messina. SafeSlice: a model slicing and design safety inspection tool for SysML. In Proceedings of European conference on Foundations of Software Engineering, pages 460--463, New York, NY, USA, 2011. ACM.

Digital Library

[17]

S. Friedenthal, A. Moore, and R. Steiner. A Practical Guide to SysML: Systems Modeling Language. The MK/OMG Press. Elsevier Science, 2008.

Digital Library

[18]

O. M. Group. Omg certified systems modeling professional (ocsmp).

[19]

O. M. Group. SysML v1.3.

[20]

M. Höst, B. Regnell, and C. Wohlin. Using students as subjects: comparative study of students and professionals in lead-time impact assessment. Empirical Softw. Engg., 5(3):201--214, Nov. 2000.

Digital Library

[21]

A. Jedlitschka, M. Ciolkowski, and D. Pfahl. Reporting experiments in software engineering. In F. Shull, J. Singer, and D. I. K. Sjøberg, editors, Guide to Advanced Empirical Software Engineering, pages 201--228. Springer London, 2008.

[22]

N. Juristo and A. Moreno. Basics of Software Engineering Experimentation. Kluwer Academic Publishers, Englewood Cliffs, NJ, 2001.

Digital Library

[23]

V. Kampenes, T. Dyba, J. Hannay, and I. Sjoberg. A systematic review of effect size in software engineering experiments. Information & Software Technology, 49(11-12):1073--1086, 2006.

Digital Library

[24]

E. Kamsties, A. von Knethen, and R. Reussner. A controlled experiment to evaluate how styles affect the understandability of requirements specifications. Information & Software Technology, 45(14):955--965, 2003.

[25]

B. Kitchenham, H. Al-Khilidar, M. Babar, M. Berry, K. Cox, J. Keung, F. Kurniawati, M. Staples, H. Zhang, and L. Zhu. Evaluating guidelines for reporting empirical software engineering studies. Empirical Software Engineering, 13:97--121, 2008.

Digital Library

[26]

B. Kitchenham, S. Pfleeger, L. Pickard, P. Jones, D. Hoaglin, K. El Emam, and J. Rosenberg. Preliminary guidelines for empirical research in software engineering. IEEE Trans. on Soft. Eng., 28(8):721--734, 2002.

Digital Library

[27]

H. Levene. Robust tests for equality of variances. In I. Olkin, editor, Contributions to probability and statistics. Stanford Univ. Press., Palo Alto, CA, 1960.

[28]

S. Nejati, M. Sabetzadeh, D. Falessi, L. C. Briand, and T. Coq. A SysML-based approach to traceability management and design slicing in support of safety certification: Framework, tool support, and case studies. Information & Software Technology, 54(6):569--590, 2012.

Digital Library

[29]

OMG. Unified Modeling Language (UML) specification, version 2.0. Technical report, Object Management Group, July 2005.

[30]

S. L. Pfleeger and W. Menezes. Marketing technology to software practitioners. IEEE Software, 17(1):27--33, 2000.

Digital Library

[31]

F. Ricca, M. D. Penta, M. Torchiano, P. Tonella, and M. Ceccato. How developers' experience and ability influence web application comprehension tasks supported by uml stereotypes: A series of four experiments. IEEE Trans. Software Eng., 36(1):96--118, 2010.

Digital Library

[32]

M. Scaife and Y. Rogers. External cognition: how do graphical representations work? International Journal of Human-Computer Studies, 45(2):185--213, 1996.

Digital Library

[33]

G. Scanniello, C. Gravino, M. Genero, J. A. Cruz-Lemus, and G. Tortora. On the impact of UML analysis models on source code comprehensibility and modifiability. ACM Trans. on Soft. Eng. and Meth., 23(2), 2014.

Digital Library

[34]

G. Scanniello, C. Gravino, and G. Tortora. Investigating the role of UML in the software modeling and maintenance - a preliminary industrial survey. In Proc. of the International Conference on Enterprise Information Systems, pages 141--148. SciTePress, 2010.

[35]

S. Shapiro and M. Wilk. An analysis of variance test for normality. Biometrika, 52(3-4):591--611, 1965.

[36]

F. Shull, M. G. Mendoncça, V. Basili, J. Carver, J. C. Maldonado, S. Fabbri, G. H. Travassos, and M. C. Ferreira. Knowledge-sharing issues in experimental software engineering. Empirical Software Engineering, 9(1-2):111--137, March 2004.

Digital Library

[37]

D. I. K. Sjøberg, J. E. Hannay, O. Hansen, V. B. Kampenes, A. Karahasanovic, N. Liborg, and A. C. Rekdal. A survey of controlled experiments in software engineering. IEEE Trans. on Soft. Eng., 31(9):733--753, 2005.

Digital Library

[38]

S. Vegas, N. Juristo, A. Moreno, M. Solari, and P. Letelier. Analysis of the influence of communication between researchers on experiment replication. In Proc. of the International Symposium on Empirical Software Engineering, pages 28--37, New York, NY, USA, 2006. ACM.

Digital Library

[39]

J. Verelst. The influence of the level of abstraction on the evolvability of conceptual models of information systems. In Proc. of the International Symposium on Empirical Software Engineering, pages 17--26, Washington, DC, USA, 2004. IEEE Computer Society.

Digital Library

[40]

C. Wohlin, P. Runeson, M. Höst, M. Ohlsson, B. Regnell, and A. Wesslén. Experimentation in Software Engineering. Springer, 2012.

Digital Library

[41]

R. Young. Effective Requirements Practice. Addison-Wesley, Boston, MA, 2001.

Digital Library

Cited By

Kitchenham BMadeyski LScanniello GGravino C(2022)The Importance of the Correlation in Crossover ExperimentsIEEE Transactions on Software Engineering10.1109/TSE.2021.307048048:8(2802-2813)Online publication date: 1-Aug-2022
https://doi.org/10.1109/TSE.2021.3070480
Cruz MBernardez BDuran AGalindo JRuiz-Cortes A(2020)Replication of Studies in Empirical Software Engineering: A Systematic Mapping Study, From 2013 to 2018IEEE Access10.1109/ACCESS.2019.29521918(26773-26791)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2019.2952191
Staron MStaron M(2020)Software Architectures—Views and DocumentationAutomotive Software Architectures10.1007/978-3-030-65939-4_2(19-53)Online publication date: 17-Dec-2020
https://doi.org/10.1007/978-3-030-65939-4_2
Show More Cited By

Index Terms

On the effect of using SysML requirement diagrams to comprehend requirements: results from two controlled experiments
1. Software and its engineering

Recommendations

Integrated product-service analysis using SysML requirement diagrams

During the design of integrated products and services, such as in a product-service system PSS, an analysis of specific customer needs and the requirements to fulfil them is an important activity needed to determine value propositions based on the ...
A proposed extension to the SysML requirements diagram
SE '08: Proceedings of the IASTED International Conference on Software Engineering

There are several approaches to document requirements, from totally informal and document-oriented to formal languages/methods that may represent requirements graphically. The SysML Requirements Diagram has the advantage of being created specifically to ...
Investigating the Role of Use Cases in the Construction of Class Diagrams

Several approaches have been proposed for the transition from functional requirements to object-oriented design. In a use case-driven development process, the use cases are important input for the identification of classes and their methods. There is, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

EASE '14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering

May 2014

486 pages

ISBN:9781450324762

DOI:10.1145/2601248

General Chair:
Martin Shepperd
Brunel University, UK
,
Program Chairs:
Tracy Hall
Brunel University
,
Ingunn Myrtveit
The Norwegian Business School

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Brunel University: Brunel University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

EASE '14

Sponsor:

Brunel University

EASE '14: 18th International Conference on Evaluation and Assessment in Software Engineering

May 13 - 14, 2014

England, London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
186
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kitchenham BMadeyski LScanniello GGravino C(2022)The Importance of the Correlation in Crossover ExperimentsIEEE Transactions on Software Engineering10.1109/TSE.2021.307048048:8(2802-2813)Online publication date: 1-Aug-2022
https://doi.org/10.1109/TSE.2021.3070480
Cruz MBernardez BDuran AGalindo JRuiz-Cortes A(2020)Replication of Studies in Empirical Software Engineering: A Systematic Mapping Study, From 2013 to 2018IEEE Access10.1109/ACCESS.2019.29521918(26773-26791)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2019.2952191
Staron MStaron M(2020)Software Architectures—Views and DocumentationAutomotive Software Architectures10.1007/978-3-030-65939-4_2(19-53)Online publication date: 17-Dec-2020
https://doi.org/10.1007/978-3-030-65939-4_2
Labunets KMassaci FTedeschi ABener ATurhan BBiffl S(2017)Graphical vs. tabular notations for risk modelsProceedings of the 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1109/ESEM.2017.40(267-276)Online publication date: 9-Nov-2017
https://dl.acm.org/doi/10.1109/ESEM.2017.40
Labunets KMassacci FPaci FMarczak SOliveira F(2017)Model comprehension for security risk assessmentEmpirical Software Engineering10.1007/s10664-017-9502-822:6(3017-3056)Online publication date: 1-Dec-2017
https://dl.acm.org/doi/10.1007/s10664-017-9502-8
Staron MStaron M(2017)Software Architectures: Views and DocumentationAutomotive Software Architectures10.1007/978-3-319-58610-6_2(19-50)Online publication date: 27-Jun-2017
https://doi.org/10.1007/978-3-319-58610-6_2
de la Vara JMarín BGiachetti GAyora CGenero MJedlitschka AJørgensen MScanniello GSampath SCaivano DPort D(2016)Do Models Improve the Understanding of Safety Compliance Needs?Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2961111.2962621(1-6)Online publication date: 8-Sep-2016
https://dl.acm.org/doi/10.1145/2961111.2962621
Amyot DAnda ABaslyman MLessard LBruel J(2016)Towards Improved Requirements Engineering with SysML and the User Requirements Notation2016 IEEE 24th International Requirements Engineering Conference (RE)10.1109/RE.2016.58(329-334)Online publication date: Sep-2016
https://doi.org/10.1109/RE.2016.58
Del Carpio AAngarita L(2015)Towards the Development of a Framework for Encouraging the Learning of SPICE Model by Using Knowledge GraphsSoftware Process Improvement and Capability Determination10.1007/978-3-319-19860-6_16(203-216)Online publication date: 3-Jun-2015
https://doi.org/10.1007/978-3-319-19860-6_16

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents