Article

The Road Towards Autonomous Cybersecurity Agents: Remedies for Simulation Environments

Authors:

Martin Drašar,

Shanchieh Jay YangAuthors Info & Claims

Computer Security. ESORICS 2023 International Workshops: CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, The Hague, The Netherlands, September 25–29, 2023, Revised Selected Papers, Part II

Pages 738 - 749

https://doi.org/10.1007/978-3-031-54129-2_43

Published: 25 September 2023 Publication History

Abstract

One of the fundamental challenges in developing autonomous cybersecurity agents (AICA) is providing them with appropriate training environments for skills acquisition and evaluation. Current reinforcement learning (RL) algorithms rely on myriads of training runs to instill proper behavior, and this is reasonably achievable only within a simulated environment. In this paper, we explore the topic of simulation models and environments for RL and present an assessment framework to compare simulation models designed for simulating cyberattack scenarios. We examine four existing simulation tools, including a new one by the authors of the paper, and discuss their properties, particularly in terms of deployability, to support RL-based AICA. In the example of complex scenarios, we compare the two most sophisticated simulation tools and discuss their strengths.

References

[1]

Andrew, A., Spillard, S., Collyer, J., Dhir, N.: Developing optimal causal cyber-defence agents via cyber security simulation (2022).

[2]

Bodeau, D., Graubart, R., Heinbockel, W.: Mapping the cyber terrain: Enabling cyber defensibility claims and hypotheses to be stated and evaluated with greater rigor and utility. Tech. rep., The MITRE Corporation., Bedford, MA, USA (2013). https://www.mitre.org/sites/default/files/publications/mapping-cyber-terrain-13-4175.pdf

[3]

Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym (2016).

[4]

Drasar, M.: Cyst api documentation (2023), https://muni.cz/go/cyst/

[5]

Li, L., Fayad, R., Taylor, A.: Cygil: A cyber gym for training autonomous agents over emulated network systems (2021).

[6]

Microsoft: Cyberbattlesim (2021). https://github.com/microsoft/cyberbattlesim, created by Christian Seifert, Michael Betser, William Blum, James Bono, Kate Farris, Emily Goren, Justin Grana, Kristian Holsheimer, Brandon Marken, Joshua Neil, Nicole Nichols, Jugal Parikh, Haoran Wei

[7]

MITRE: Caldera: A scalable, adversary emulation platform (2022). https://caldera.mitre.org

[8]

Molina-Markham, A., Miniter, C., Powell, B., Ridley, A.: Network environment design for autonomous cyberdefense (2021).

[9]

Moskal, S., Yang, S.J.: Cyberattack action-intent-framework for mapping intrusion observables. CoRR abs/2002.07838 (2020). https://arxiv.org/abs/2002.07838

[10]

Moskal S, Yang SJ, and Kuhl ME Cyber threat assessment via attack scenario simulation using an integrated adversary and network modeling approach J. Defense Model. Simul. 2018 15 1 13-29

[11]

Raymond, D., Cross, T., Conti, G., Nowatkowski, M.: Key terrain in cyberspace: Seeking the high ground. In: 2014 6th International Conference On Cyber Conflict (CyCon 2014), pp. 287–300. IEEE, Tallinn, Estonia (2014).

[12]

Rush, G., Tauritz, D.R., Kent, A.D.: Coevolutionary agent-based network defense lightweight event system (candles). In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation. p. 859–866. GECCO Companion ’15, Association for Computing Machinery, New York, NY, USA (2015).

Digital Library

[13]

Schoonover, K., et al.: Galaxy: A network emulation framework for cybersecurity. In: 11th USENIX Workshop on Cyber Security Experimentation and Test (CSET 18). USENIX Association, Baltimore, MD (Aug 2018). https://www.usenix.org/conference/cset18/presentation/schoonover

[14]

Standen, M., Lucas, M., Bowman, D., Richer, T.J., Kim, J., Marriott, D.: Cyborg: a gym for the development of autonomous cyber agents (2021). https://doi.org/10.48550/ARXIV.2108.09118

[15]

The Technical Cooperation Program: TTCP CAGE Challenge 2 (2022). https://github.com/cage-challenge/cage-challenge-2

Cited By

Ruman ÁDrašar MSadlek LYang SCeleda P(2024)Adversary Tactic Driven Scenario and Terrain Generation with Partial Infrastructure SpecificationProceedings of the 19th International Conference on Availability, Reliability and Security10.1145/3664476.3664523(1-11)Online publication date: 30-Jul-2024
https://dl.acm.org/doi/10.1145/3664476.3664523

Recommendations

Using cognitive agents in social simulations

Multi-Agent-Based Social Simulation (MABS) is a paradigm devoted to using agents as the modelling metaphor to simulate autonomous entities in a social world composed of a number of independent and interacting entities. Such models try to reproduce real ...
Co-development of real-time systems and their simulation environments
APSEC '00: Proceedings of the Seventh Asia-Pacific Software Engineering Conference

A real-time system consists of control software, the hardware controlled by the software, and the environment that physically interacts with the controlled hardware. Simulating control software with simplistic assumptions of the environment ...
Cybersecurity Simulator for Connected and Autonomous Vehicles
MobiHoc '23: Proceedings of the Twenty-fourth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing

In recent years, we have witnessed a significant rise in both the popularity and capability of Connected and Autonomous Vehicles (CAVs). This progress has been facilitated, in part, by advancements in CAV simulators that enable researchers to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

Computer Security. ESORICS 2023 International Workshops: CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, The Hague, The Netherlands, September 25–29, 2023, Revised Selected Papers, Part II

Sep 2023

784 pages

ISBN:978-3-031-54128-5

DOI:10.1007/978-3-031-54129-2

Editors:
Sokratis Katsikas
https://ror.org/05xg72x27Norwegian University of Science and Technology, Gjøvik, Norway
,
Habtamu Abie
https://ror.org/02gm7te43Norwegian Computing Center, Oslo, Norway
,
Silvio Ranise
https://ror.org/05trd4x28University of Trento, Trento, Italy
,
Luca Verderame
https://ror.org/0107c5v14University of Genoa, Genoa, Italy
,
Enrico Cambiaso
https://ror.org/04zaypm56Consiglio Nazionale delle Ricerche (CNR), Genoa, Italy
,
Rita Ugarelli
SINTEF A.S., Oslo, Norway
,
Isabel Praça
Instituto Superior de Engenharia do Porto, Porto, Portugal
,
Wenjuan Li
https://ror.org/0030zas98Hong Kong Polytechnic University, Hong Kong, China
,
Weizhi Meng
https://ror.org/04qtj9h94Technical University of Denmark, Kongens Lyngby, Denmark
,
Steven Furnell
https://ror.org/01ee9ar58University of Nottingham, Nottingham, UK
,
Basel Katt
https://ror.org/05xg72x27Norwegian University of Science and Technology, Gjøvik, Norway
,
Sandeep Pirbhulal
https://ror.org/02gm7te43Norwegian Computing Center, Oslo, Norway
,
Ankur Shukla
https://ror.org/02jqtg033Institute for Energy Technology (IFE), Halden, Norway
,
Michele Ianni
https://ror.org/02rc97e94University of Calabria, Rende, Italy
,
Mila Dalla Preda
https://ror.org/039bp8j42University of Verona, Verona, Italy
,
Kim-Kwang Raymond Choo
https://ror.org/01kd65564The University of Texas at San Antonio, San Antonio, TX, USA
,
Miguel Pupo Correia
https://ror.org/01c27hj86University of Lisbon, Lisbon, Portugal
,
Abhishta Abhishta
https://ror.org/006hf6230University of Twente, Enschede, The Netherlands
,
Giovanni Sileno
https://ror.org/04dkp9463University of Amsterdam, Amsterdam, The Netherlands
,
Mina Alishahi
https://ror.org/018dfmf50Open University in the Netherlands, Heerlen, The Netherlands
,
Harsha Kalutarage
https://ror.org/04f0qj703Robert Gordon University, Aberdeen, UK
,
Naoto Yanai
https://ror.org/035t8zc32Osaka University, Osaka, Japan

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 25 September 2023

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ruman ÁDrašar MSadlek LYang SCeleda P(2024)Adversary Tactic Driven Scenario and Terrain Generation with Partial Infrastructure SpecificationProceedings of the 19th International Conference on Availability, Reliability and Security10.1145/3664476.3664523(1-11)Online publication date: 30-Jul-2024
https://dl.acm.org/doi/10.1145/3664476.3664523

View Options

View options

Figures

Tables

Media

View Table of Conten