Cause–effect graphing is a well-known requirement-based and systematic testing method with a heur... more Cause–effect graphing is a well-known requirement-based and systematic testing method with a heuristic approach. Since it was introduced by Myers in 1979, there have not been any sufficiently comprehensive studies to generate test inputs from these graphs. However, there exist several methods for test input generation from Boolean expressions. Cause–effect graphs can be more convenient for a wide variety of users compared to Boolean expressions. Moreover, they can be used to enforce common constraints and rules on the system variables of different expressions of the system. This study proposes a new mutant-based test input generation method, Spectral Testing for Boolean specification models based on spectral analysis of Boolean expressions using mutations of the original expression. Unlike Myers’ method, Spectral Testing is an algorithmic and deterministic method, in which we model the possible faults systematically. Furthermore, the conversion of cause–effect graphs between Boolean expressions is explored so that the existing test input generation methods for Boolean expressions can be exploited for cause–effect graphing. A software is developed as an open-source extendable tool for generating test inputs from cause–effect graphs by using different methods and performing mutation analysis for quantitative evaluation on these methods for further analysis and comparison. Selected methods, MI, MAX-A, MUTP, MNFP, CUTPNFP, MUMCUT, Unique MC/DC, and Masking MC/DC are implemented together with Myers’ technique and the proposed Spectral Testing in the developed tool. For mutation testing, 9 common fault types of Boolean expressions are modeled, implemented, and generated in the tool. An XML-based standard on top of GraphML representing a cause–effect graph is proposed and is used as the input type to the approach. An empirical study is performed by a case study on 5 different systems with various requirements, including the benchmark set from the TCAS-II system. Our results show that the proposed XML-based cause–effect graph model can be used to represent system requirements. The developed tool can be used for test input generation from proposed cause–effect graph models and can perform mutation analysis to distinguish between the methods with respect to the effectiveness of test inputs and their mutant kill scores. The proposed Spectral Testing method outperforms the state-of-the-art methods in the context of critical systems, regarding both the effectiveness and mutant kill scores of the generated test inputs, and increasing the chances of revealing faults in the system and reducing the cost of testing. Moreover, the proposed method can be used as a separate or complementary method to other well-performing test input generation methods for covering specific fault types.
International Journal of Software Engineering and Knowledge Engineering
Boolean expressions are highly involved in control flows of programs and software specifications.... more Boolean expressions are highly involved in control flows of programs and software specifications. Coverage criteria for Boolean expressions aim at producing minimal test suites to detect software faults. There exist various testing criteria, efficiency of which is usually evaluated through mutation analysis. This paper proposes an integer programming-based minimal test suite generation technique relying on mutation analysis. The proposed technique also takes into account the cost of fault detection. The technique is optimal such that the resulting test suite guarantees to detect all the mutants under given fault assumptions, while maximizing the average percentage of fault detection of a test suite. Therefore, the approach presented can also be considered as a reference method to check the efficiency of any common technique. The method is evaluated using four well-known real benchmark sets of Boolean expressions and is also exemplary compared with MCDC criterion. The results show th...
2020 Turkish National Software Engineering Symposium (UYMS), 2020
Model-based GUI testing is an important concept in Software GUI testing. Manual testing is a time... more Model-based GUI testing is an important concept in Software GUI testing. Manual testing is a time-consuming labor and heavily error-prone. It has several well-accepted models that Software Testing community has been working and contributing to them for many years. This paper reviews different models used in model-based GUI testing and presents a case study with a proposed approach for how to convert several well-accepted models to ESG (Event Sequence Graphs) to generate test cases and execute them with an aim to consolidate past and future works in a single model.
This paper proposes a framework for testing behavioral model of sequential circuits implemented i... more This paper proposes a framework for testing behavioral model of sequential circuits implemented in Hardware Description Language (HDL). The concept of Ideal Testing is applied for achieving reliability and validity of both positive and negative testing. The HDL program is first modeled by a Finite State Machine (FSM) which is then converted to a Regular Expression (RE). This RE is used to construct test sequences. For positive testing, the original (fault-free) FSM model is used, while for negative testing its mutant model(s) are used to define requirements of ideal testing in conjunction with model-based and code-based mutation testing. A demonstrating example based on a real-life-like Traffic Light Controller (TLC) validates the proposed approach and analyzes its characteristic features.
2015 IEEE 39th Annual Computer Software and Applications Conference, 2015
With the network’s application in various areas of Human life, there are many security threats by... more With the network’s application in various areas of Human life, there are many security threats by various attacks such as viruses and information interception. The network applications have attracted continuous attentions from both academia and industry. Especially, the large scale networks (e. g. social networks and wireless sensor network) aggravate the problem of cyber-attack even further. To address these issues of large scale networks, there has been increased interest to propose and design new algorithms and methodologies for network security and applications.
Cause–effect graphing is a well-known requirement-based and systematic testing method with a heur... more Cause–effect graphing is a well-known requirement-based and systematic testing method with a heuristic approach. Since it was introduced by Myers in 1979, there have not been any sufficiently comprehensive studies to generate test inputs from these graphs. However, there exist several methods for test input generation from Boolean expressions. Cause–effect graphs can be more convenient for a wide variety of users compared to Boolean expressions. Moreover, they can be used to enforce common constraints and rules on the system variables of different expressions of the system. This study proposes a new mutant-based test input generation method, Spectral Testing for Boolean specification models based on spectral analysis of Boolean expressions using mutations of the original expression. Unlike Myers’ method, Spectral Testing is an algorithmic and deterministic method, in which we model the possible faults systematically. Furthermore, the conversion of cause–effect graphs between Boolean expressions is explored so that the existing test input generation methods for Boolean expressions can be exploited for cause–effect graphing. A software is developed as an open-source extendable tool for generating test inputs from cause–effect graphs by using different methods and performing mutation analysis for quantitative evaluation on these methods for further analysis and comparison. Selected methods, MI, MAX-A, MUTP, MNFP, CUTPNFP, MUMCUT, Unique MC/DC, and Masking MC/DC are implemented together with Myers’ technique and the proposed Spectral Testing in the developed tool. For mutation testing, 9 common fault types of Boolean expressions are modeled, implemented, and generated in the tool. An XML-based standard on top of GraphML representing a cause–effect graph is proposed and is used as the input type to the approach. An empirical study is performed by a case study on 5 different systems with various requirements, including the benchmark set from the TCAS-II system. Our results show that the proposed XML-based cause–effect graph model can be used to represent system requirements. The developed tool can be used for test input generation from proposed cause–effect graph models and can perform mutation analysis to distinguish between the methods with respect to the effectiveness of test inputs and their mutant kill scores. The proposed Spectral Testing method outperforms the state-of-the-art methods in the context of critical systems, regarding both the effectiveness and mutant kill scores of the generated test inputs, and increasing the chances of revealing faults in the system and reducing the cost of testing. Moreover, the proposed method can be used as a separate or complementary method to other well-performing test input generation methods for covering specific fault types.
International Journal of Software Engineering and Knowledge Engineering
Boolean expressions are highly involved in control flows of programs and software specifications.... more Boolean expressions are highly involved in control flows of programs and software specifications. Coverage criteria for Boolean expressions aim at producing minimal test suites to detect software faults. There exist various testing criteria, efficiency of which is usually evaluated through mutation analysis. This paper proposes an integer programming-based minimal test suite generation technique relying on mutation analysis. The proposed technique also takes into account the cost of fault detection. The technique is optimal such that the resulting test suite guarantees to detect all the mutants under given fault assumptions, while maximizing the average percentage of fault detection of a test suite. Therefore, the approach presented can also be considered as a reference method to check the efficiency of any common technique. The method is evaluated using four well-known real benchmark sets of Boolean expressions and is also exemplary compared with MCDC criterion. The results show th...
2020 Turkish National Software Engineering Symposium (UYMS), 2020
Model-based GUI testing is an important concept in Software GUI testing. Manual testing is a time... more Model-based GUI testing is an important concept in Software GUI testing. Manual testing is a time-consuming labor and heavily error-prone. It has several well-accepted models that Software Testing community has been working and contributing to them for many years. This paper reviews different models used in model-based GUI testing and presents a case study with a proposed approach for how to convert several well-accepted models to ESG (Event Sequence Graphs) to generate test cases and execute them with an aim to consolidate past and future works in a single model.
This paper proposes a framework for testing behavioral model of sequential circuits implemented i... more This paper proposes a framework for testing behavioral model of sequential circuits implemented in Hardware Description Language (HDL). The concept of Ideal Testing is applied for achieving reliability and validity of both positive and negative testing. The HDL program is first modeled by a Finite State Machine (FSM) which is then converted to a Regular Expression (RE). This RE is used to construct test sequences. For positive testing, the original (fault-free) FSM model is used, while for negative testing its mutant model(s) are used to define requirements of ideal testing in conjunction with model-based and code-based mutation testing. A demonstrating example based on a real-life-like Traffic Light Controller (TLC) validates the proposed approach and analyzes its characteristic features.
2015 IEEE 39th Annual Computer Software and Applications Conference, 2015
With the network’s application in various areas of Human life, there are many security threats by... more With the network’s application in various areas of Human life, there are many security threats by various attacks such as viruses and information interception. The network applications have attracted continuous attentions from both academia and industry. Especially, the large scale networks (e. g. social networks and wireless sensor network) aggravate the problem of cyber-attack even further. To address these issues of large scale networks, there has been increased interest to propose and design new algorithms and methodologies for network security and applications.
Uploads
Papers by Fevzi Belli