Nothing Special   »   [go: up one dir, main page]

skip to main content
review-article

Model-based security testing in IoT systems: : A Rapid Review

Published: 01 December 2023 Publication History

Abstract

Context:

Security testing is a challenging and effort-demanding task in IoT scenarios. The heterogeneous devices expose different vulnerabilities that can influence the methods and cost of security testing. Model-based security testing techniques support the systematic generation of test cases for the assessment of security requirements by leveraging the specifications of the IoT system model and of the attack templates.

Objective:

This paper aims to review the adoption of model-based security testing in the context of IoT, and then provides the first systematic and up-to-date comprehensive classification and analysis of research studies in this topic.

Method:

We conducted a systematic literature review analyzing 803 publications and finally selecting 17 primary studies, which satisfied our inclusion criteria and were classified according to a set of relevant analysis dimensions.

Results:

We report the state-of-the-art about the used formalisms, the test techniques, the objectives, the target applications and domains; we also identify the targeted security attacks, and discuss the challenges, gaps and future research directions.

Conclusion:

Our review represents the first attempt to systematically analyze and classify existing studies on model-based security testing for IoT. According to the results, model-based security testing has been applied in core IoT domains. Models complexity and the need of modeling evolving scenarios that include heterogeneous open software and hardware components remain the most important shortcomings. Our study shows that model-based security testing of IoT applications is a promising research direction. The principal future research directions deal with: extending the existing modeling formalisms in order to capture all peculiarities and constraints of complex and large scale IoT networks; the definition of context-aware and dynamic evolution modeling approaches of IoT entities; and the combination of model-based testing techniques with other security test strategies such as penetration testing or learning techniques for model inference.

References

[1]
Anand P., Singh Y., Selwal A., Alazab M., Tanwar S., Kumar N., IoT vulnerability assessment for sustainable computing: threats, current solutions, and open challenges, IEEE Access 8 (2020) 168825–168853,.
[2]
Nadir I., Ahmad Z., Mahmood H., Shah G.A., Shahzad F., Umair M., Khan H., Gulzar U., An auditing framework for vulnerability analysis of IoT system, in: Proceedings of the IEEE European Symposium on Security and Privacy Workshops, EuroS&PW, IEEE, 2019, pp. 39–47,.
[3]
Atwady Y., Hammoudeh M., A survey on authentication techniques for the internet of things, in: Proceedings of the International Conference on Future Networks and Distributed Systems, 2017,.
[4]
Johari R., Kaur I., Tripathi R., Gupta K., Penetration testing in IoT network, in: Proceedings of 5th International Conference on Computing, Communication and Security, ICCCS, IEEE, 2020, pp. 1–7,.
[5]
Siboni S., Sachidananda V., Meidan Y., Bohadana M., Mathov Y., Bhairav S., Shabtai A., Elovici Y., Security testbed for Internet-of-Things devices, IEEE Trans. Reliab. 68 (1) (2019) 23–44,.
[6]
Utting M., Pretschner A., Legeard B., A taxonomy of model-based testing approaches, Softw. Test. Verif. Reliab. 22 (5) (2012) 297–312,.
[7]
Garousi V., Keleş A.B., Balaman Y., Güler Z.Ö., Arcuri A., Model-based testing in practice: An experience report from the web applications domain, J. Syst. Softw. 180 (2021),.
[8]
Peleska J., Brauer J., Huang W.-l., Model-based testing for avionic systems proven benefits and further challenges, in: Proceedings of International Symposium on Leveraging Applications of Formal Methods, Springer, 2018, pp. 82–103,.
[9]
Morin B., Harrand N., Fleurey F., Model-based software engineering to tame the IoT jungle, IEEE Softw. 34 (1) (2017) 30–36,.
[10]
Siegel J.E., Kumar S., Sarma S.E., The future internet of things: Secure, efficient, and model-based, IEEE Internet Things J. 5 (4) (2017) 2386–2398,.
[11]
Kirchhof J.C., Rumpe B., Schmalzing D., Wortmann A., MontiThings: Model-driven development and deployment of reliable IoT applications, J. Syst. Softw. 183 (2022),.
[12]
Berrouyne I., Adda M., Mottu J.-M., Tisi M., A model-driven methodology to accelerate software engineering in the Internet of Things, IEEE Internet Things J. (2022),.
[13]
Fortino G., Gravina R., Russo W., Savaglio C., Modeling and simulating Internet-of-Things systems: A hybrid agent-oriented approach, Comput. Sci. Eng. 19 (5) (2017) 68–76,.
[14]
Ahmad A., Bouquet F., Fourneret E., Legeard B., Model-based testing for internet of things systems, in: Advances in Computers, Vol. 108, Elsevier, 2018, pp. 1–58,.
[15]
Felderer M., Zech P., Breu R., Büchler M., Pretschner A., Model-based security testing: a taxonomy and systematic classification, Softw. Test. Verif. Reliab. 26 (2) (2016) 119–148,.
[16]
Peroli M., De Meo F., Viganò L., Guardini D., MobSTer: A model-based security testing framework for web applications, Softw. Test. Verif. Reliab. 28 (8) (2018),.
[17]
Mahmood S., Nguyen H.N., Shaikh S.A., Systematic threat assessment and security testing of automotive over-the-air (OTA) updates, Veh. Commun. 35 (2022),.
[18]
Robles-Ramirez D.A., Escamilla-Ambrosio P.J., Tryfonas T., IoTsec: UML extension for internet of things systems security modelling, in: Proceedings of International Conference on Mechatronics, Electronics and Automotive Engineering, ICMEAE, IEEE, 2017, pp. 151–156,.
[19]
Bodei C., Degano P., Ferrari G.-L., Galletta L., Modelling and analysing IoT systems, J. Parallel Distrib. Comput. 157 (2021) 233–242,.
[20]
Anwer F., Nazir M., Mustafa K., Security testing, in: H. Mohanty J.M., Balakrishnan A. (Eds.), Trends in Software Testing, Springer, 2017, pp. 35–66,.
[21]
Murad G., Badarneh A., Qusef A., Almasalha F., Software testing techniques in IoT, in: Proceedings of 8th International Conference on Computer Science and Information Technology, CSIT, IEEE, 2018, pp. 17–21,.
[22]
Dias J.P., Couto F., Paiva A.C., Ferreira H.S., A brief overview of existing tools for testing the internet-of-things, in: Proceedings of International Conference on Software Testing, Verification and Validation Workshops, ICSTW, IEEE, 2018, pp. 104–109,.
[23]
Cortés M., Saraiva R., Souza M., Mello P., Soares P., Adoption of software testing in internet of things: A systematic literature mapping, in: Proceedings of the IV Brazilian Symposium on Systematic and Automated Software Testing, 2019, pp. 3–11,.
[24]
Kh T., Hamarash I., Model-Based Quality Assessment of Internet of Things Software Applications: A Systematic Mapping Study, Int. J. Interact. Mob. Technol. (iJIM) 14 (2020) 128–152,.
[25]
Bures M., Klima M., Rechtberger V., Bellekens X., Tachtatzis C., Atkinson R., Ahmed B.S., Interoperability and integration testing methods for IoT systems: A systematic mapping study, in: Proceedings of International Conference on Software Engineering and Formal Methods, Springer, 2020, pp. 93–112,.
[26]
Mahmood S., Nguyen H.N., Shaikh S.A., Automotive cybersecurity testing: Survey of testbeds and methods, in: Digital Transformation, Cyber Security and Resilience of Modern Societies, Springer, 2021, pp. 219–243,.
[27]
Tauqeer O.B., Jan S., Khadidos A.O., Khadidos A.O., Khan F.Q., Khattak S., Analysis of security testing techniques, Intell. Autom. Soft Comput. 29 (1) (2021) 291–306,.
[28]
Zhu S., Yang S., Gou X., Xu Y., Zhang T., Wan Y., Survey of testing methods and testbed development concerning Internet of Things, Wirel. Pers. Commun. 123 (1) (2022) 165–194,.
[29]
Heiding F., Katsikeas S., Lagerström R., Research communities in cyber security vulnerability assessments: A comprehensive literature review, Comp. Sci. Rev. 48 (2023),.
[30]
Ahmad E., Model-based system engineering of the Internet of Things: A bibliometric literature analysis, IEEE Access (2023),.
[31]
Sommer F., Kriesten R., Kargl F., Survey of model-based security testing approaches in the automotive domain, IEEE Access (2023),.
[32]
Cartaxo B., Pinto G., Soares S., Rapid reviews in software engineering, in: Contemporary Empirical Methods in Software Engineering, Springer, 2020, pp. 357–384,.
[33]
Hamel C., Michaud A., Thuku M., Skidmore B., Stevens A., Nussbaumer-Streit B., Garritty C., Defining rapid reviews: a systematic scoping review and thematic analysis of definitions and defining characteristics of rapid reviews, J. Clin. Epidemiol. 129 (2021) 74–85,.
[34]
Thelwall M., Sud P., Scopus 1900–2020: Growth in articles, abstracts, countries, fields, and journals, Quant. Sci. Stud. 3 (1) (2022) 37–50,.
[35]
Kitchenham B., Procedures for Performing Systematic Reviews, Vol. 33, No. 2004, Keele University, Keele, UK, 2004, pp. 1–26.
[36]
Medvidovic N., Rosenblum D.S., Redmiles D.F., Robbins J.E., Modeling software architectures in the unified modeling language, ACM Trans. Softw. Eng. Methodol. 11 (1) (2002) 2–57,.
[37]
Matheu-García S.N., Hernández-Ramos J.L., Skarmeta A.F., Baldini G., Risk-based automated assessment and testing for the cybersecurity certification and labelling of IoT devices, Comput. Stand. Interfaces 62 (2019) 64–83,.
[38]
Cabot J., Gogolla M., Object constraint language (OCL): a definitive guide, in: International School on Formal Methods for the Design of Computer, Communication and Software Systems, Springer, 2012, pp. 58–90,.
[39]
Ahmad A., Bouquet F., Fourneret E., Le Gall F., Legeard B., Model-based testing as a service for IoT platforms, in: Proceedings of International Symposium on Leveraging Applications of Formal Methods, Springer, 2016, pp. 727–742,.
[40]
Jürjens J., UMLsec: Extending UML for secure systems development, in: Proceedings of International Conference on the Unified Modeling Language, Springer, 2002, pp. 412–425,.
[41]
Lodderstedt T., Basin D., Doser J., Secureuml: A UML-based modeling language for model-driven security, in: Proceedings of International Conference on the Unified Modeling Language, Springer, 2002, pp. 426–441,.
[42]
Mahmoodi Y., Reiter S., Viehl A., Bringmann O., Rosenstiel W., Model-guided security analysis of interconnected embedded systems, in: Proceedings of International Conference on Model-Based Software and Systems Engineering, 2018, pp. 602–609,.
[43]
Mahmoodi Y., Reiter S., Viehl A., Bringmann O., Rosenstiel W., Attack surface modeling and assessment for penetration testing of IoT system designs, in: Proceedings of 21st Euromicro Conference on Digital System Design, DSD, IEEE, 2018, pp. 177–181,.
[44]
Arcile J., André É., Timed automata as a formalism for expressing security: A survey on theory and practice, ACM Comput. Surv. (2022),.
[45]
Larsen K.G., Lorber F., Nielsen B., 20 years of UPPAAL enabled industrial model-based validation and beyond, in: Proceedings of International Symposium on Leveraging Applications of Formal Methods, Springer, 2018, pp. 212–229,.
[46]
Lallie H.S., Debattista K., Bal J., A review of attack graph and attack tree visual syntax in cyber security, Comput. Sci. Rev. 35 (2020) 100219,.
[47]
Krichen M., Alroobaea R., A new model-based framework for testing security of IoT systems in smart cities using attack trees and price timed automata, in: Proceedings of 14th International Conference on Evaluation of Novel Approaches to Software Engineering, SCITEPRESS-Science and Technology Publications, 2019, pp. 570–577,.
[48]
Gehlot V., From Petri NETS to colored Petri NETS: A tutorial introduction to nets based formalism for modeling and simulation, in: Proceedings of Winter Simulation Conference, WSC, 2019, pp. 1519–1533,.
[49]
Valadares D.C.G., de Carvalho César Sobrinho Á.A., Perkusich A., Gorgonio K.C., Formal verification of a trusted execution environment-based architecture for IoT applications, IEEE Internet Things J. 8 (23) (2021) 17199–17210,.
[50]
Khedker U., Sanyal A., Sathe B., Data Flow Analysis: Theory and Practice, CRC Press, Taylor & Francis Group, 2009,.
[51]
Wirtz R., Heisel M., A systematic method to describe and identify security threats based on functional requirements, in: Proceedings of International Conference on Risks and Security of Internet and Systems, Springer, 2019, pp. 205–221,.
[52]
Marksteiner S., Priller P., A model-driven methodology for automotive cybersecurity test case generation, in: Proceedings of IEEE European Symposium on Security and Privacy Workshops, EuroS&PW, IEEE, 2021, pp. 129–135,.
[53]
Rak M., Salzillo G., Granata D., ESSecA: An automated expert system for threat modelling and penetration testing for IoT ecosystems, Comput. Electr. Eng. 99 (2022),.
[54]
Allweyer T., BPMN 2.0: Introduction to the Standard for Business Process Modeling, BoD–Books on Demand, ISBN 978-3-8370-9331-5, 2016.
[55]
Daoudagh S., Lonetti F., Marchetti E., XACMET: XACML testing & modeling: An automated model-based testing solution for access control systems, Softw. Qual. J. 28 (1) (2020) 249–282,.
[56]
Lunkeit A., Schieferdecker I., Model-based security testing-deriving test models from artefacts of security engineering, in: Proceedings of International Conference on Software Testing, Verification and Validation Workshops, ICSTW, IEEE, 2018, pp. 244–251,.
[57]
Miller A., Maple S., Powell R., Danen V., Papadopoulos E., State of open source security report, Snyk, London, Tel Aviv, Boston, 2020, Retrieved on July 29th, 2023 from https://snyk.io/series/open-source-security/report-2020/.
[58]
Shah Y., Sengupta S., A survey on classification of cyber-attacks on IoT and IIoT devices, in: Proceedings of the 11th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference, UEMCON, IEEE, 2020, pp. 406–413,.
[59]
Woolf N., DDoS attack that disrupted internet was largest of its kind in history, experts say, Guardian 26 (2016) Retrieved on July 29th, 2023 from https://www.theguardian.com/technology/2016/oct/26/ddos-attack-dyn-mirai-botnet.
[60]
Greenberg A., A hacker tried to poison a Florida city’s water supply, officials say, 2021, Wired magazine. Retrieved on July 29th, 2023 from https://www.wired.com/story/oldsmar-florida-water-utility-hack.
[61]
Lounis K., Zulkernine M., Attacks and defenses in short-range wireless technologies for IoT, IEEE Access 8 (2020) 88892–88932,.
[62]
Khanam S., Ahmedy I.B., Idris M.Y.I., Jaward M.H., Sabri A.Q.B.M., A survey of security challenges, attacks taxonomy and advanced countermeasures in the Internet of Things, IEEE Access 8 (2020) 219709–219743,.
[63]
Sengupta J., Ruj S., Bit S.D., A comprehensive survey on attacks, security issues and blockchain solutions for IoT and IIoT, J. Netw. Comput. Appl. 149 (2020),.
[64]
Xiao L., Wan X., Lu X., Zhang Y., Wu D., IoT security techniques based on machine learning: How do IoT devices use AI to enhance security?, IEEE Signal Process. Mag. 35 (5) (2018) 41–49,.
[65]
Chandan A.R., Khairnar V.D., Security testing methodology of IoT, in: Proceedings of International Conference on Inventive Research in Computing Applications, ICIRCA, IEEE, 2018, pp. 1431–1435,.
[66]
Gui Z., Shu H., Kang F., Xiong X., Firmcorn: Vulnerability-oriented fuzzing of IoT firmware via optimized virtual execution, IEEE Access 8 (2020) 29826–29841,.
[67]
Xie W., Jiang Y., Tang Y., Ding N., Gao Y., Vulnerability detection in IoT firmware: A survey, in: Proceedings of 23rd International Conference on Parallel and Distributed Systems, ICPADS, IEEE, 2017, pp. 769–772,.
[68]
Chen C.-K., Zhang Z.-K., Lee S.-H., Shieh S., Penetration testing in the IoT age, Computer 51 (4) (2018) 82–85,.
[69]
Cartaxo B., Pinto G., Soares S., The role of rapid reviews in supporting decision-making in software engineering practice, in: Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering, 2018, pp. 24–34,.
[70]
Cartaxo B., Pinto G., Fonseca B., Ribeiro M., Pinheiro P., Baldassarre M.T., Soares S., Software engineering research community viewpoints on rapid reviews, in: Proceedings of ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM, IEEE, 2019, pp. 1–12,.
[71]
Reynen E., Robson R., Ivory J., Hwee J., Straus S.E., Tricco A.C., et al., A retrospective comparison of systematic reviews with same-topic rapid reviews, J. Clin. Epidemiol. 96 (2018) 23–34,.
[72]
Wohlin C., Guidelines for snowballing in systematic literature studies and a replication in software engineering, in: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, 2014, pp. 1–10,. Art. no. 38.
[73]
Jabbar R., Krichen M., Kharbeche M., Fetais N., Barkaoui K., A formal model-based testing framework for validating an IoT solution for blockchain-based vehicles communication, in: Proceedings of 15th International Conference on Evaluation of Novel Approaches to Software Engineering. SCITEPRESS-Science and Technology Publications, 2020, pp. 595–602,.
[74]
Krichen M., Mechti S., Alroobaea R., Said E., Singh P., Khalaf O.I., Masud M., A formal testing model for operating room control system using internet of things, Comput. Mater. Continua 66 (3) (2021) 2997–3011,.
[75]
Matheu S.N., Hernández-Ramos J.L., Pérez S., Skarmeta A.F., Extending MUD profiles through an automated IoT security testing methodology, IEEE Access 7 (2019) 149444–149463,.
[76]
Neisse R., Baldini G., Steri G., Ahmad A., Fourneret E., Legeard B., Improving internet of things device certification with policy-based management, in: Proceedings of Global Internet of Things Summit, GIoTS, IEEE, 2017, pp. 1–6,.
[77]
Marksteiner S., Ramler R., Sochor H., Integrating threat modeling and automated test case generation into industrialized software security testing, in: Proceedings of the Third Central European Cybersecurity Conference, 2019, pp. 1–3,.
[78]
Ahmad A., Baldini G., Cousin P., Matheu S.N., Skarmeta A., Fourneret E., Legeard B., Large scale IoT security testing, benchmarking and certification, in: Cognitive Hyperconnected Digital Transformation: Internet of Things Intelligence Evolution, ISBN 9781003337584, 2017, pp. 189–220.
[79]
Aichernig B.K., Muškardin E., Pferscher A., Learning-based fuzzing of IoT message brokers, in: Proceedings of 14th Conference on Software Testing, Verification and Validation, ICST, IEEE, 2021, pp. 47–58,.
[80]
Matheu S.N., Pérez S., Ramos J.L.H., Skarmeta A., On the automation of security testing for IoT constrained scenarios, in: Proceedings of International Conference on Information Security Applications, Springer, 2019, pp. 286–298,.
[81]
Krichen M., Cheikhrouhou O., Lahami M., Alroobaea R., Maâlej A.J., Towards a model-based testing framework for the security of internet of things for smart city applications, in: Proceedings of International Conference on Smart Cities, Infrastructure, Technologies and Applications, Springer, 2017, pp. 360–365,.
[82]
Krichen M., Lahami M., Cheikhrouhou O., Alroobaea R., Maâlej A.J., Security testing of internet of things for smart city applications: A formal approach, in: Smart Infrastructure and Applications, Springer, 2020, pp. 629–653,.
[83]
Wang R., Kristensen L.M., Meling H., Stolz V., Automated test case generation for the Paxos single-decree protocol using a Coloured Petri Net model, J. Log. Algebraic Methods Program. 104 (2019) 254–273,.
[84]
Tretmans J., On the existence of practical testers, in: ModelEd, TestEd, TrustEd, Springer, 2017, pp. 87–106,.
[85]
Legeard B., Bouzy A., Smartesting certifyIt: Model-based testing for enterprise IT, in: Proceedings of IEEE Sixth International Conference on Software Testing, Verification and Validation, ICST, IEEE, 2013, pp. 391–397,.
[86]
Willcock C., Deiß T., Tobies S., Keil S., Engler F., Schulz S., An introduction to TTCN-3, John Wiley & Sons, 2011,.
[87]
Marijan D., Liaaen M., Gotlieb A., Sen S., Ieva C., Titan: Test suite optimization for highly configurable software, in: Proceedings of the IEEE International Conference on Software Testing, Verification and Validation, ICST, IEEE, 2017, pp. 524–531,.
[88]
Lohiya R., Thakkar A., Application domains, evaluation data sets, and research challenges of IoT: A systematic review, IEEE Internet Things J. 8 (11) (2020) 8774–8798,.
[89]
Rudin C., Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nat. Mach. Intell. 1 (5) (2019) 206–215,.
[90]
Carlini N., Poisoning the unlabeled dataset of Semi-Supervised learning, in: Proceedings of 30th USENIX Security Symposium, USENIX Security 21, USENIX Association, ISBN 978-1-939133-24-3, 2021, pp. 1577–1592. Retrieved on July 29th, 2023 from https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-poisoning.
[91]
Sabbagh M., Fei Y., Kaeli D., Secure speculative execution via RISC-V open hardware design, in: Proceedings of Fifth Workshop on Computer Architecture Research with RISC-V, CARRV 2021, 2021, pp. 1–7,.

Cited By

View all
  • (2025)Using rapid reviews to support software engineering practice: a systematic review and a replication studyEmpirical Software Engineering10.1007/s10664-024-10545-630:1Online publication date: 1-Feb-2025
  • (2024)Towards an MDRE Approach to Verify Security and safety of Heterogeneous IoT AppsProceedings of the ACM/IEEE 6th International Workshop on Software Engineering Research & Practices for the Internet of Things10.1145/3643794.3648289(65-72)Online publication date: 20-Apr-2024

Index Terms

  1. Model-based security testing in IoT systems: A Rapid Review
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Information and Software Technology
    Information and Software Technology  Volume 164, Issue C
    Dec 2023
    366 pages

    Publisher

    Butterworth-Heinemann

    United States

    Publication History

    Published: 01 December 2023

    Author Tags

    1. Internet of Things
    2. Model-based testing
    3. Security testing

    Qualifiers

    • Review-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 22 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Using rapid reviews to support software engineering practice: a systematic review and a replication studyEmpirical Software Engineering10.1007/s10664-024-10545-630:1Online publication date: 1-Feb-2025
    • (2024)Towards an MDRE Approach to Verify Security and safety of Heterogeneous IoT AppsProceedings of the ACM/IEEE 6th International Workshop on Software Engineering Research & Practices for the Internet of Things10.1145/3643794.3648289(65-72)Online publication date: 20-Apr-2024

    View Options

    View options

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media