research-article

Public Access

A combinatorial approach for exposing off-nominal behaviors

Authors:

Kaushik Madala,

Daniel AceitunaAuthors Info & Claims

ICSE '18: Proceedings of the 40th International Conference on Software Engineering

Pages 910 - 920

https://doi.org/10.1145/3180155.3180204

Published: 27 May 2018 Publication History

Abstract

Off-nominal behaviors (ONBs) have been a major concern in the areas of embedded systems and safety-critical systems. To address ONB problems, some researchers have proposed model-based approaches that can expose ONBs by analyzing natural language requirements documents. While these approaches produced promising results, they require a lot of human effort and time. In this paper, to reduce human effort and time, we propose a combinatorial-based approach, Combinatorial Causal Component Model (Combi-CCM), which uses structured requirements patterns and combinations generated using the IPOG algorithm. We conducted an empirical study using several requirements documents to evaluate our approach, and our results indicate that the proposed approach can reduce human effort and time while maintaining the same ONB exposure ability obtained by the control techniques.

References

[1]

D. Aceituna and H. Do. 2015. Exposing the susceptibility of off-nominal behaviors in reactive system requirements. In IEEE 23rd International Requirements Engineering Conference (RE). 136--145.

[2]

C. M. Belcastro. 2012. Validation and Verification (V&V) of Safety-Critical Systems Operating under Off-Nominal Conditions. Springer Berlin Heidelberg, Berlin, Heidelberg, 399--419.

[3]

D. M. Berry. 2007. Ambiguity in natural language requirements documents. In Monterey Workshop. Springer, 1--7.

[4]

Boston Scientific. 2007. PACEMAKER system specification. Technical Report. Boston Scientific.

[5]

Marco Bozzano, Alessandro Cimatti, Joost-Pieter Katoen, Viet Yen Nguyen, Thomas Noll, and Marco Roveri. 2009. The COMPASS approach: Correctness, modelling and performability of aerospace systems. In International Conference on Computer Safety, Reliability, and Security. 173--186.

Digital Library

[6]

Reliability Analysis Center. 1993. Failure Mode, Effects and Criticality Analysis (FMECA). (1993). http://www.dtic.mil/dtic/tr/fulltext/u2/a278508.pdf

[7]

J. Day, K. Donahue, M. D. Ingham, A. Kadesch, A. Kennedy, and E. Post. 2012. Modeling Off-Nominal Behavior in SysML. In AIAA Infotech. 19--21.

[8]

X. Devroey, M. Cordy, P. Schobbens, A. Legay, and P. Heymans. 2015. State machine flattening, a mapping study and tools assessment. In IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 1--8.

[9]

V. Estivill-Castro, R. Hexel, and D. A. Rosenblueth. 2012. Failure mode and effects analysis (FMEA) and model-checking of software for embedded systems by sequential scheduling of vectors of logic-labelled finite-state machines. In 7th IET International Conference on System Safety, incorporating the Cyber Security Conference. 1--6.

[10]

N. Fenton and M. Neil. 2014. Decision Support Software for Probabilistic Risk Assessment Using Bayesian Networks. IEEE Software 31, 2 (Mar 2014), 21--26.

[11]

E. Feuvrier-Danziger, C. Dunkers, M. Kosowski, and D. Marschner. 2015. Oddbotics. (2015). https://sites.google.com/site/mrsdproject201415teamd/documents/presentations

[12]

D. Firesmith. 2012. The Need to Specify Requirements for Off-Nominal Behaviors. (2012). https://insights.sei.cmu.edu/sei

[13]

D. C. Foyle and B. L. Hooey. 2003. Improving evaluation and system design through the use of off-nominal testing: A methodology for scenario development. In Wright State University. 397--402.

[14]

G. C. Fraccone, V. Volovoi, A. E. Colón, and M. Blake. 2011. Novel air traffic procedures: investigation of off-nominal scenarios and potential hazards. Journal of Aircraft 48, 1 (2011), 127--140.

[15]

A. O. Gomes and M. V. M. Oliveira. 2011. Formal Development of a Cardiac Pacemaker: From Specification to Code. Springer Berlin Heidelberg, Berlin, Heidelberg, 210--225.

Digital Library

[16]

D. L. Iverson. 2004. Inductive system health monitoring. In In Proceedings of The 2004 International Conference on Artificial Intelligence (IC-AI04), Las Vegas.

[17]

M. Jackson. 2010. DigitalHome Software Requirements Specification. (2010). http://fmt.isti.cnr.it/nlreqdataset/

[18]

E. Jee, I. Lee, and O. Sokolsky. 2010. Assurance Cases in Model-Driven Development of the Pacemaker Software. Springer Berlin Heidelberg, Berlin, Heidelberg, 343--356.

Digital Library

[19]

D. C. Jensen and I. Y. Tumer. 2013. Modeling and Analysis of Safety in Early Design. Procedia Computer Science 16 (2013), 824--833. 2013 Conference on Systems Engineering Research.

[20]

Z. Jiang, M. Pajic, and R. Mangharam. 2011. Model-Based Closed-Loop Testing of Implantable Pacemakers. In IEEE/ACM Second International Conference on Cyber-Physical Systems. 131--140.

Digital Library

[21]

Y. Jou, K. Yang, M. Liao, and C. Liaw. 2016. Multi-criteria failure mode effects and criticality analysis method: a comparative case study on aircraft braking system. International Journal of Reliability and Safety 10, 1 (2016), 1--21.

[22]

H. Kim, D. Bae, V. Debroy, and W. E. Wong. 2011. Deriving Data Dependence from/for UML State Machine Diagrams. In 5th International Conference on Secure Software Integration and Reliability Improvement (SSIRI). 118--126.

Digital Library

[23]

J. Kloos, T. Hussain, and R. Eschbach. 2011. Risk-Based Testing of Safety-Critical Embedded Systems Driven by Fault Tree Analysis. In IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops. 26--33.

Digital Library

[24]

D. R. Kuhn, R. N. Kacker, and Y. Lei. 2016. Estimating t-Way Fault Profile Evolution During Testing. In IEEE 40th Annual Computer Software and Applications Conference (COMPSAC), Vol. 2. 596--597.

[25]

T. Kurtoglu and I. Y. Tumer. 2008. A graph-based fault identification and propagation framework for functional design of complex systems. Journal of Mechanical Design 130, 5 (2008), 051401.

[26]

T. Kurtoglu, I. Y. Tumer, and D. C. Jensen. 2010. A functional failure reasoning methodology for evaluation of conceptual system architectures. Research in Engineering Design 21, 4 (01 Oct 2010), 209--234.

[27]

K. Lano. 2009. Slicing of UML state machines. In Proceedings of the 9th WSEAS international conference on Applied informatics and communications. World Scientific and Engineering Academy and Society (WSEAS), 63--69.

Digital Library

[28]

Y. Lei, R. Kacker, D. R. Kuhn, V. Okun, and J. Lawrence. 2008. IPOG/IPOG-D: efficient test generation for multi-way combinatorial testing. Software Testing, Verification and Reliability 18, 3 (2008), 125--148.

Digital Library

[29]

N. G. Leveson. 2004. Role of software in spacecraft accidents. Journal of spacecraft and Rockets 41, 4 (2004), 564--575.

[30]

L. Liu, B. Pan, T. Wang, Q. Li, M. Aktas, and M. Gamell. 2012. Automatic Delivery System. (2012). http://eceweb1.rutgers.edu/~marsic/books/SE/projects/OTHER/2012-g4-report3.pdf

[31]

H. K. Lo, C. Y. Huang, Y. R. Chang, W. C. Huang, and J. R. Chang. 2005. Reliability and Sensitivity Analysis of Embedded Systems with Modular Dynamic Fault Trees. In TENCON - IEEE Region 10 Conference. 1--6.

[32]

M. Luisa, F. Mariangela, and N. I. Pierluigi. 2004. Market Research for Requirements Analysis Using Linguistic Tools. Requirements Engineering 9, 1 (2004), 40--56.

Digital Library

[33]

K. Madala, H. Do, and D. Aceituna. 2017. Hierarchical Model Exploration for Exposing Off-Nominal Behaviors. In 14th Workshop on Model-Driven Engineering, Verification and Validation (MoDeVVa) co-located with ACM/IEEE 20th International Conference on Model Driven Engineering Languages and Systems (MODELS 2017).

[34]

A. Mavi, P. Wilkinson, A. Harwood, and M. Novak. 2009. Easy approach to requirements syntax (EARS). In 17th IEEE International Requirements Engineering Conference (RE). 317--322.

Digital Library

[35]

A. Mavin. 2012. Listen, Then Use EARS. IEEE Software 29, 2 (March 2012), 17--18.

Digital Library

[36]

A. Mavin and P. Wilkinson. 2010. Big Ears (The Return of "Easy Approach to Requirements Engineering"). In 18th IEEE International Requirements Engineering Conference (RE). 277--282.

Digital Library

[37]

McMaster University. 2007. Pacemaker Formal Methods Challenge. (April 2007). http://sqrl.mcmaster.ca/pacemaker.htm

[38]

M. Melin. 2010. Requirements Specification AB Mail Robot. (2010). http://www.isy.liu.se/edu/projekt/tsrt10/2010/postrobot-2010/images/general_doc.pdf

[39]

E. So, J. Ajtum, Y. Moy, and Y. L. Quach. 2005. Requirements Specification AB Mail Robot. (2005). http://www.ecs.umass.edu/ece/sdp/sdp05/preston/sdp_data/Requirement%20Specification.doc

[40]

S. Teng and S. Ho. 1996. Failure mode and effects analysis: an integrated approach for product design and process control. International journal of quality & reliability management 13, 5 (1996), 8--26.

[41]

L. A. Tuan, M. C. Zheng, and Q. T. Tho. 2010. Modeling and Verification of Safety Critical Systems: A Case Study on Pacemaker. In 4th International Conference on Secure Software Integration and Reliability Improvement. 23--32.

Digital Library

[42]

A. Valmari. 1998. The state explosion problem. Springer Berlin Heidelberg, Berlin, Heidelberg, 429--528.

Digital Library

[43]

A. T. Vemuri, M. M. Polycarpou, and S. A. Diakourtis. 1998. Neural network based fault detection in robotic manipulators. IEEE Transactions on Robotics and Automation 14, 2 (Apr 1998), 342--348.

[44]

S. Verma, S. Lozito, K. Thomas, and D. Ballinger. 2008. Procedures for Off-Nominal Cases: Very Closely Spaced Parallel Runway Operations. In IEEE/AIAA 27th Digital Avionics Systems Conference (DASC). 2.C.4--1âĂŞ2.C.4--11.

[45]

J. Wang, W. Dong, and Z. Qi. 2002. Slicing Hierarchical Automata for Model Checking UML Statecharts. Lecture Notes in Computer Science 2495 (2002), 435--446.

Digital Library

[46]

N. Yatapanage, K. Winter, and S. Zafar. 2010. Slicing behavior tree models for verification. Theoretical Computer Science (2010), 125--139.

[47]

L. Yu, Y. Lei, R. N. Kacker, and D. R. Kuhn. 2013. Acts: A combinatorial test generation tool. In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation (ICST). 370--375.

Digital Library

Cited By

Madala KDo HTenbergen B(2023)ADSA – Association-Driven Safety Analysis to Expose Unknown Safety IssuesIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.324860621:1(216-228)Online publication date: 24-Feb-2023
https://dl.acm.org/doi/10.1109/TDSC.2023.3248606
Madala KDo HTenbergen B(2023)Identifying safety issues from energy conservation requirementsJournal of Software: Evolution and Process10.1002/smr.251535:7Online publication date: 2-Jul-2023
https://dl.acm.org/doi/10.1002/smr.2515
Madala KDo H(2022)A Hierarchical Dependency-Driven Scenario-Based Testing for Autonomous VehiclesSmart Cities, Green Technologies, and Intelligent Transport Systems10.1007/978-3-031-17098-0_15(297-312)Online publication date: 28-Sep-2022
https://doi.org/10.1007/978-3-031-17098-0_15
Show More Cited By

Index Terms

A combinatorial approach for exposing off-nominal behaviors
1. Computer systems organization
  1. Embedded and cyber-physical systems
2. Software and its engineering
  1. Software creation and management
    1. Designing software
      1. Requirements analysis

Recommendations

A systematic approach to transforming system requirements into model checking specifications
ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software Engineering

We propose a method that addresses the following dilemma: model checking can formally expose off-nominal behaviors and unintended scenarios in the requirements of concurrent reactive systems. Requirements engineers and non-technical stakeholders who ...
An artificial intelligence-based model-driven approach for exposing off-nominal behaviors
ICSE '19: Proceedings of the 41st International Conference on Software Engineering: Companion Proceedings

With an increase in the automation of cyber-physical systems (e.g., automated vehicles and robots), quality problems such as off-nominal behaviors (ONBs) have also increased. While there are techniques that can find ONBs at the requirements engineering ...
Real-Time Estimation of Lane-to-Lane Turning Flows at Isolated Signalized Junctions
In this paper, we develop rule- and model-based approaches for the real-time estimation of lane-to-lane turning flows. Our aim is to determine the turning proportions of vehicles based on detector information at isolated signalized junctions and thereby ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '18: Proceedings of the 40th International Conference on Software Engineering

May 2018

1307 pages

ISBN:9781450356381

DOI:10.1145/3180155

Conference Chair:
Michel Chaudron
Chalmers University of Technology, University of Gothenburg, Sweden
,
General Chair:
Ivica Crnkovic
Chalmers University of Technology, University of Gothenburg, Sweden
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Mark Harman
Facebook and University College London, United Kingdom

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 27 - June 3, 2018

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
391
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)8

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Madala KDo HTenbergen B(2023)ADSA – Association-Driven Safety Analysis to Expose Unknown Safety IssuesIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.324860621:1(216-228)Online publication date: 24-Feb-2023
https://dl.acm.org/doi/10.1109/TDSC.2023.3248606
Madala KDo HTenbergen B(2023)Identifying safety issues from energy conservation requirementsJournal of Software: Evolution and Process10.1002/smr.251535:7Online publication date: 2-Jul-2023
https://dl.acm.org/doi/10.1002/smr.2515
Madala KDo H(2022)A Hierarchical Dependency-Driven Scenario-Based Testing for Autonomous VehiclesSmart Cities, Green Technologies, and Intelligent Transport Systems10.1007/978-3-031-17098-0_15(297-312)Online publication date: 28-Sep-2022
https://doi.org/10.1007/978-3-031-17098-0_15
Madala KPiparia SBlanco EDo HBryce R(2021)Model elements identification using neural networks: a comprehensive studyRequirements Engineering10.1007/s00766-020-00332-226:1(67-96)Online publication date: 1-Mar-2021
https://dl.acm.org/doi/10.1007/s00766-020-00332-2
Madala KHang KDo HTenbergen B(2020)SACC - A property driven approach to expose undesired behaviors among system’s components2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE5003.2020.00043(380-390)Online publication date: Oct-2020
https://doi.org/10.1109/ISSRE5003.2020.00043
Madala KDo HAceituna D(2019)Exposing off-nominal behaviors in multi-robot coordinationProceedings of the 2nd International Workshop on Robotics Software Engineering10.1109/RoSE.2019.00006(17-24)Online publication date: 27-May-2019
https://dl.acm.org/doi/10.1109/RoSE.2019.00006
Madala KMussbacher GAtlee JBultan T(2019)An artificial intelligence-based model-driven approach for exposing off-nominal behaviorsProceedings of the 41st International Conference on Software Engineering: Companion Proceedings10.1109/ICSE-Companion.2019.00085(214-217)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICSE-Companion.2019.00085
Madala KDo HAceituna D(2018)A Knowledge Acquisition Approach for Off-Nominal Behaviors2018 4th International Workshop on Requirements Engineering for Self-Adaptive, Collaborative, and Cyber Physical Systems (RESACS)10.1109/RESACS.2018.00012(36-43)Online publication date: Aug-2018
https://doi.org/10.1109/RESACS.2018.00012
Madala KPiparia SDo HBryce R(2018)Finding Component State Transition Model Elements Using Neural Networks: An Empirical Study2018 5th International Workshop on Artificial Intelligence for Requirements Engineering (AIRE)10.1109/AIRE.2018.00014(54-61)Online publication date: Aug-2018
https://doi.org/10.1109/AIRE.2018.00014

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents