Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3377811.3380332acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

A novel approach to tracing safety requirements and state-based design models

Published: 01 October 2020 Publication History

Abstract

Traceability plays an essential role in assuring that software and systems are safe to use. Automated requirements traceability faces the low precision challenge due to a large number of false positives being returned and mingled with the true links. To overcome this challenge, we present a mutation-driven method built on the novel idea of proactively creating many seemingly correct tracing targets (i.e., mutants of a state machine diagram), and then exploiting model checking within process mining to automatically verify whether the safety requirement's properties hold in the mutants. A mutant is killed if its model checking fails; otherwise, it is survived. We leverage the underlying killed-survived distinction, and develop a correlation analysis procedure to identify the traceability links. Experimental evaluation results on two automotive systems with 27 safety requirements show considerable precision improvements compared with the state-of-the-art.

References

[1]
Bernhard K. Aichernig, Jakob Auer, Elisabeth Jöbstl, Robert Korosec, Willibald Krenn, Rupert Schlick, and Birgit Vera Schmidt. 2014. Model-Based Mutation Testing of an Industrial Measurement Device. In International Conference on Tests and Proofs (TAP). York, UK, 1--19.
[2]
Bernhard K. Aichernig, Harald Brandl, Elisabeth Jöbstl, Willibald Krenn, Rupert Schlick, and Stefan Tiran. 2015. Killing Strategies for Model-Based Mutation Testing. Software Testing, Verification & Reliability 25, 8 (December 2015), 716--748.
[3]
Mounifah Alenazi, Nan Niu, and Juha Savolainen. 2019. A Process Mining Based Approach to Improving Defect Detection of SysML Models. In Late Breaking Results Track, International Conference on Automated Software Engineering (ASE). Last accessed: February 2020.
[4]
Mounifah Alenazi, Nan Niu, and Juha Savolainen. 2019. SysML Modeling Mistakes and Their Impacts on Requirements. In International Model-Driven Requirements Engineering Workshop (MoDRE). Jeju Island, South Korea, 14--23.
[5]
Mounifah Alenazi, Nan Niu, Wentao Wang, and Juha Savolainen. 2018. Using Obstacle Analysis to Support SysML-Based Model Testing for Cyber Physical Systems. In International Model-Driven Requirements Engineering Workshop (MoDRE). Banff, Canada, 46--55.
[6]
Mounifah Alenazi, Deepak Reddy, and Nan Niu. 2018. Assuring Virtual PLC in the Context of SysML Models. In International Conference on Software Reuse (ICSR). Madrid, Spain, 121--136.
[7]
Shaukat Ali, Tao Yue, and Lionel C. Briand. 2014. Does Aspect-Oriented Modeling Help Improve the Readability of UML State Machines? Software and System Modeling 13, 3 (July 2014), 1189--1221.
[8]
Paul E. Ammann, Paul E. Black, and William Majurski. 1998. Using Model Checking to Generate Tests from Specifications. In International Conference on Formal Engineering Methods (ICFEM). Brisbane, Australia, 46--55.
[9]
Kelly Androutsopoulos, David Clark, Mark Harman, Jens Krinke, and Laurence Tratt. 2013. State-Based Model Slicing: A Survey. Comput. Surveys 45, 4 (August 2013), 53:1--53:36.
[10]
Markus Borg, Per Runeson, and Anders Ardö. 2014. Recovering from a Decade: A Systematic Mapping of Information Retrieval Approaches to Software Trace-ability. Empirical Software Engineering 19, 6 (December 2014), 1565--1616.
[11]
Lionel C. Briand, Davide Falessi, Shiva Nejati, Mehrdad Sabetzadeh, and Tao Yue. 2014. Traceability and SysML Design Slices to Support Safety Inspections: A Controlled Experiment. ACM Transactions on Software Engineering and Methodology 23, 1 (February 2014), 9:1--9:43.
[12]
Darius Cepulis and Nan Niu. 2018. Creating Socio-Technical Patches for Information Foraging: A Requirements Traceability Case Study. In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Lisbon, Portugal, 17--21.
[13]
Jinghui Chen, Micayla Goodrum, Ronald A. Metoyer, and Jane Cleland-Huang. 2018. How Do Practitioners Perceive Assurance Cases in Safety-Critical Software Systems?. In International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE). Gothenburg, Sweden, 57--60.
[14]
Christine Choppy and Gianna Reggio. 2009. A Method for Developing UML State Machines. In ACM Symposium on Applied Computing (SAC). Honolulu, HI, USA, 382--388.
[15]
Edmund M. Clarke, Orna Grumberg, and Doron A. Peled. 2001. Model Checking. MIT Press.
[16]
Jane Cleland-Huang. 2017. Safety Stories in Agile Development. IEEE Software 34, 4 (July/August 2017), 16--19.
[17]
Jane Cleland-Huang, Adam Czauderna, Marek Gibiec, and John Emenecker. 2010. A Machine Learning Approach for Tracing Regulatory Codes to Product Specific Requirements. In International Conference on Software Engineering (ICSE). Cape Town, South Africa, 155--164.
[18]
Jane Cleland-Huang, Orlena Gotel, Jane Huffman Hayes, Patrick Mäder, and Andrea Zisman. 2014. Software Traceability: Trends and Future Directions. In Future of Software Engineering (FOSE). Hyderabad, India, 55--69.
[19]
Jane Cleland-Huang, Mats Heimdahl, Jane Hayes, Robyn R. Lutz, and Patrick Mäder. 2012. Trace Queries for Safety Requirements in High Assurance Systems. In International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ). Essen, Germany, 179--193.
[20]
Jane Cleland-Huang and Michael Vierhauser. 2018. Discovering, Analyzing, and Managing Safety Stories in Agile Projects. In International Requirements Engineering Conference (RE). Banff, Canada, 262--273.
[21]
Daniela Colangelo, Daniele Compare, Paola Inverardi, and Patrizio Pelliccione. 2006. Reducing Software Architecture Models Complexity: A Slicing and Abstraction Approach. In International Conference on Formal Techniques for Networked and Distributed Systems (FORTE). Paris, France, 243--258.
[22]
David Cuddeback, Alex Dekhtyar, and Jane Huffman Hayes. 2010. Automated Requirements Traceability: the Study of Human Analysts. In International Requirements Engineering Conference (RE). Sydney, Australia, 231--240.
[23]
Richard A. DeMillo, Richard J. Lipton, and Frederick G. Sayward. 1978. Hints on Test Data Selection: Help for the Practicing Programmer. IEEE Computer 11, 4 (April 1978), 34--41.
[24]
Sanford Friedenthal, Alan Moore, and Rick Steiner. 2012. Water Distiller Example Using Functional Analysis. In A Practical Guide to SysML (Second Edition), Sanford Friedenthal, Alan Moore, and Rick Steiner (Eds.). The MK/OMG Press, 393--429.
[25]
Antonio Frisoli, Luigi Borelli, Alberto Montagner, Simone Marcheschi, Caterina Procopio, Fabio Salsedo, Massimo Bergamasco, Maria C. Carboncini, Martina Tolaini, and Bruno Rossi. 2007. Arm Rehabilitation with a Robotic Exoskeleleton in Virtual Reality. In International Conference on Rehabilitation Robotics (ICORR). Noordwijk, The Netherlands, 631--642.
[26]
Ross B. Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, OH, USA, 580--587.
[27]
Micayla Goodrum, Jane Cleland-Huang, Robyn R. Lutz, Jinghui Cheng, and Ronald A. Metoyer. 2017. What Requirements Knowledge Do Developers Need to Manage Change in Safety-Critical Systems?. In International Requirements Engineering Conference (RE). Lisbon, Portugal, 90--99.
[28]
Maria Fernanda Granda, Nelly Condori-Fernández, Tanja E. J. Vos, and Oscar Pastor. 2016. Using Model Checking to Generate Tests from Specifications. In International Conference on Information Systems Development (ISD). Katowice, Poland, 17--37.
[29]
Jin Guo, Jinghui Cheng, and Jane Cleland-Huang. 2017. Semantically Enhanced Software Traceability Using Deep Learning Techniques. In International Conference on Software Engineering (ICSE). Buenos Aires, Argentina, 3--14.
[30]
Jane Hayes, Alex Dekhtyar, and Senthil Karthikeyan Sundaram. 2006. Advancing Candidate Link Generation for Requirements Tracing: the Study of Methods. IEEE Transactions on Software Engineering 32, 1 (January 2006), 4--19.
[31]
International Electrotechnical Commission. 2010. Functional Safety of Electrical / Electronic / Programmable Electronic Safety-Related System (IEC 61508). https://www.iec.ch/functionalsafety/. Last accessed: February 2020.
[32]
Yue Jia and Mark Harman. 2011. An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37, 5 (September/October 2011), 649--678.
[33]
Lin Jiang, Hui Liu, and He Jiang. 2019. Machine Learning Based Recommendation of Method Names: How Far are We. In International Conference on Automated Software Engineering (ASE). San Diego, CA, USA, 602--614.
[34]
K. N. King and Jeff Offutt. 1991. A Fortran Language System for Mutation-Based Software Testing. Software: Practice and Experience 21, 7 (July 1991), 685--718.
[35]
Bogdan Korel, Inderdeep Singh, Luay Ho Tahat, and Boris Vaysburg. 2003. Slicing of State-Based Models. In International Conference on Software Maintenance (ICSM). Amsterdam, The Netherlands, 34--43.
[36]
Nancy G. Leveson, Mats Heimdahl, Holly Hildreth, and Jon Damon Reese. 1994. Requirements Specification for Process-Control Systems. IEEE Transactions on Software Engineering 20, 9 (September 1994), 684--707.
[37]
Hui Liu, Zhifeng Xu, and Yanzhen Zou. 2018. Deep Learning Based Feature Envy Detection. In International Conference on Automated Software Engineering (ASE). Montpellier, France, 385--396.
[38]
Patrick Mäder and Jane Cleland-Huang. 2013. A Visual Language for Modeling and Executing Traceability Queries. Software and System Modeling 12, 3 (July 2013), 537--553.
[39]
Patrick Mäder, Orlena Gotel, and Ilka Philippow. 2009. Getting Back to Basics: Promoting the Use of a Traceability Information Model in Practice. In International Workshop on Traceability in Emerging Forms of Software Engineering (TEFSE). Vancouver, Canada, 21--25.
[40]
Patrick Mäder, Paul L. Jones, Yi Zhang, and Jane Cleland-Huang. 2013. Strategic Traceability for Safety-Critical Projects. IEEE Software 30, 3 (May/June 2013), 58--66.
[41]
Salome Maro, Jan-Philipp Steghöfer, Jane Hayes, Jane Cleland-Huang, and Miroslaw Staron. 2018. Vetting Automatically Generated Trace Links: What Information is Useful to Human Analysts?. In International Requirements Engineering Conference (RE). Banff, Canada, 52--63.
[42]
Lei Mi and Kerong Ben. 2011. A Method of Software Specification Mutation Testing Based on UML State Diagram for Consistency Checking. Procedia Engineering 15 (2011), 110--114.
[43]
Valéry M. Monthe, Laurent Nana, Georges E. Kouamou, and Claude Tangha. 2016. A Decision Support Framework for the Choice of Languages and Methods for the Design of Real Time Embedded Systems. Journal of Software Engineering and Applications 9 (2016), 353--397.
[44]
Shiva Nejati, Mehrdad Sabetzadeh, Chetan Arora, Lionel C. Briand, and Felix Mandoux. 2016. Automated Change Impact Analysis between SysML Models of Requirements and Design. In International Symposium on Foundations of Software Engineering (FSE). Seattle, WA, USA, 242--253.
[45]
Shiva Nejati, Mehrdad Sabetzadeh, Marsha Chechik, Steve Easterbrook, and Pamela Zave. 2007. Matching and Merging of Statecharts Specifications. In International Conference on Software Engineering (ICSE). Minneapolis, MN, USA, 54--64.
[46]
Shiva Nejati, Mehrdad Sabetzadeh, Davide Falessi, Lionel C. Briand, and Thierry Coq. 2012. A SysML-Based Approach to Traceability Management and Design Slicing in Support of Safety Certification: Framework, Tool Support, and Case Studies. Information & Software Technology 54, 6 (June 2012), 569--590.
[47]
Nan Niu, Anas Mahmoud, Zhangji Chen, and Gary Bradshaw. 2013. Departures from Optimality: Understanding Human Analyst's Information Foraging in Assisted Requirements Tracing. In International Conference on Software Engineering (ICSE). San Francisco, CA, USA, 572--581.
[48]
Nan Niu, Sandeep Reddivari, and Zhangji Chen. 2013. Keeping Requirements on Track via Visual Analytics. In International Requirements Engineering Conference (RE). Rio de Janeiro, Brazil, 205--214.
[49]
Nan Niu, Wentao Wang, and Arushi Gupta. 2016. Gray Links in the Use of Requirements Traceability. In International Symposium on Foundations of Software Engineering (FSE). Seattle, WA, USA, 384--395.
[50]
Nan Niu, Wentao Wang, Arushi Gupta, Mona Assarandarban, Li Da Xu, Juha Savolainen, and Jing-Ru C. Cheng. 2018. Requirements Socio-Technical Graphs for Managing Practitioners' Traceability Questions. IEEE Transactions on Computational Social Systems 5, 4 (December 2018), 1152--1162.
[51]
No Magic, Inc. 2020. MagicDraw. https://www.nomagic.com/products/magicdraw. Last accessed: February 2020.
[52]
Bashar Nuseibeh and Steve Easterbrook. 2000. Requirements Engineering: A Roadmap. In Future of Software Engineering (FOSE). Limerick, Ireland, 35--46.
[53]
Object Management Group. 2020. Systems Modeling Language (SysML). http://www.omgsysml.org. Last accessed: February 2020.
[54]
Vesa Ojala. 2006. A Slicer for UML State Machines. Technical Report HUT-TCS-25. Laboratory for Theoretical Computer Science, Helsinki University of Technology, Espoo, Finland.
[55]
Rajwinder Kaur Panesar-Walawege, Mehrdad Sabetzadeh, and Lionel C. Briand. 2011. Using Model-Driven Engineering for Managing Safety Evidence: Challenges, Vision and Experience. In International Workshop on Software Certification (WoSoCER). Hiroshima, Japan, 7--12.
[56]
Process Mining Group, Eindhoven University of Technology. 2016. ProM Tools. http://www.promtools.org. Last accessed: February 2020.
[57]
Patrick Rempel, Patrick Mäder, Tobias Kuschke, and Jane Cleland-Huang. 2014. Mind the Gap: Assessing the Conformance of Software Traceability to Relevant Guidelines. In International Conference on Software Engineering (ICSE). Hyderabad, India, 943--954.
[58]
Enno Ruijters and Mariëlle Stoelinga. 2015. Fault Tree Analysis: A Survey of the State-of-the-Art in Modeling, Analysis and Tools. Computer Science Review 15--16, 3 (February-May 2015), 29--62.
[59]
Mehrdad Sabetzadeh, Shiva Nejati, Lionel C. Briand, and Anne-Heidi Evensen Mills. 2011. Using SysML for Modeling of Safety-Critical Software-Hardware Interfaces: Guidelines and Industry Experience. In International Symposium on High-Assurance Systems Engineering (HASE). Boca Raton, FL, USA, 193--201.
[60]
Wilhelm Schäfer and Heike Wehrheim. 2007. The Challenges of Building Advanced Mechatronic Systems. In Future of Software Engineering (FOSE). Minneapolis, MN, USA, 72--84.
[61]
Zilvinas Strolia and Saulius Pavalkis. 2017. Building Executable SysML Model. https://blog.nomagic.com/building-executable-sysml-model-automatic-transmission-system-part-1/. Last accessed: February 2020.
[62]
Haiying Sun, Mingsong Chen, Min Zhang, Jing Liu, and Ying Zhang. 2014. Improving Defect Detection Ability of Derived Test Cases Based on Mutated UML Activity Diagrams. In IEEE Annual Computer Software and Applications Conference (COMPSAC). Atlanta, GA, USA, 275--280.
[63]
Frank Tip. 1995. A Survey of Program Slicing Techniques. Journal of Programming Languages 3, 3 (1995), 121--189.
[64]
U.S. Department of Defense. 1980. Procedures for Performing a Failure Mode Effect and Criticality Analysis (MIL-STD-1629A). http://www.fmea-fmeca.com/milstd1629.pdf. Last accessed: February 2020.
[65]
Wil M.P. van der Aalst and Kees van Hee. 2004. Workflow Management: Models, Methods, and Systems. MIT Press.
[66]
Wil M. P. van der Aalst, H. T. de Beer, and Boudewijn F. van Dongen. 2005. Process Mining and Verification of Properties: An Approach Based on Temporal Logic. In International Conferences "On the Move to Meaningful Internet Systems" (OTM). Agia Napa, Cyprus, 130--147.
[67]
Wentao Wang, Arushi Gupta, Nan Niu, Li Da Xu, Jing-Ru C.Cheng, and Zhendong Niu. 2018. Automatically Tracing Dependability Requirements via Term-Based Relevance Feedback. IEEE Transactions on Industrial Informatics 14, 1 (January 2018), 342--349.
[68]
Wentao Wang, Nan Niu, Mounifah Alenazi, and Li Da Xu. 2019. In-Place Trace-ability for Automated Production Systems: A Survey of PLC and SysML Tools. IEEE Transactions on Industrial Informatics 15, 6 (June 2019), 3155--3162.
[69]
Wentao Wang, Nan Niu, Hui Liu, and Zhendong Niu. 2018. Enhancing Automated Requirements Traceability by Resolving Polysemy. In International Requirements Engineering Conference (RE). Banff, Canada, 40--51.
[70]
Mark Weiser. 1979. Program Slices: Formal, Psychological, and Practical Investigations of an Automatic Program Abstraction Method. Ph.D. Dissertation. University of Michigan, Ann Arbor, MI, USA.

Cited By

View all
  • (2024)Requirements-Driven Slicing of Simulink Models using LLMs2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW)10.1109/REW61692.2024.00014(72-82)Online publication date: 24-Jun-2024
  • (2023)AMULET: a Mutation Language Enabling Automatic Enrichment of SysML ModelsACM Transactions on Embedded Computing Systems10.1145/3624583Online publication date: 16-Sep-2023
  • (2022)STPChain: a Crowdsourced Software Engineering Method for Software Traceability and Fine-grained Privacy Based on Blockchain2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC54236.2022.00135(849-859)Online publication date: Jun-2022
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering
June 2020
1640 pages
ISBN:9781450371216
DOI:10.1145/3377811
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

  • KIISE: Korean Institute of Information Scientists and Engineers
  • IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. mutation analysis
  2. process mining
  3. requirements engineering
  4. systems modeling language (SysML)
  5. traceability

Qualifiers

  • Research-article

Funding Sources

Conference

ICSE '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)41
  • Downloads (Last 6 weeks)5
Reflects downloads up to 24 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Requirements-Driven Slicing of Simulink Models using LLMs2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW)10.1109/REW61692.2024.00014(72-82)Online publication date: 24-Jun-2024
  • (2023)AMULET: a Mutation Language Enabling Automatic Enrichment of SysML ModelsACM Transactions on Embedded Computing Systems10.1145/3624583Online publication date: 16-Sep-2023
  • (2022)STPChain: a Crowdsourced Software Engineering Method for Software Traceability and Fine-grained Privacy Based on Blockchain2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC54236.2022.00135(849-859)Online publication date: Jun-2022
  • (2022)Testing software’s changing features with environment-driven abstraction identificationRequirements Engineering10.1007/s00766-022-00390-827:4(405-427)Online publication date: 1-Dec-2022
  • (2021)An RE’21 Workshop on Environment-Driven Requirements Engineering (EnviRE’21)2021 IEEE 29th International Requirements Engineering Conference Workshops (REW)10.1109/REW53955.2021.00075(421-421)Online publication date: Sep-2021
  • (2021)Towards Norm Classification: An Initial Analysis of HIPAA Breaches2021 IEEE 29th International Requirements Engineering Conference Workshops (REW)10.1109/REW53955.2021.00074(415-420)Online publication date: Sep-2021
  • (2021)Towards Technology Acceptance: a Bayesian Network of soft requirements, the case of the NHS COVID-19 Test and Trace App2021 IEEE 29th International Requirements Engineering Conference Workshops (REW)10.1109/REW53955.2021.00026(146-152)Online publication date: Sep-2021
  • (2021)Environment-Driven Abstraction Identification for Requirements-Based Testing2021 IEEE 29th International Requirements Engineering Conference (RE)10.1109/RE51729.2021.00029(245-256)Online publication date: Sep-2021
  • (2021)Reliability of Convolutional Neural Networks: Failure Metrics with Metamorphic Test Cases2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI)10.1109/IRI51335.2021.00017(75-82)Online publication date: 10-Aug-2021
  • (2020)Faulty Requirements Made Valuable: On the Role of Data Quality in Deep Learning2020 IEEE Seventh International Workshop on Artificial Intelligence for Requirements Engineering (AIRE)10.1109/AIRE51212.2020.00016(61-69)Online publication date: Sep-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media