Abstract
Conventional safety engineering is not sufficient to deal with Artificial Intelligence (AI) and Autonomous Systems (AS). Some authors propose dynamic safety approaches to deal with the challenges related to AI and AS. These approaches are referred to as dynamic risk management, dynamic safety management, dynamic assurance, or runtime certification [4]. These dynamic safety approaches are related to each other and the research in this field is increasing. In this paper, we structure the research challenges and solution approaches in order to explain why dynamic risk management is needed for dependability of autonomous systems. We will present 5 research areas in this large research field and name for each research area some concrete approaches or standardization activities. We hope the problem decomposition helps to foster effective research collaboration and enables researchers to better navigate the challenges surrounding dynamic risk management.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kurd, Z., Kelly, T., McDermid, J., Calinescu, R., Kwiatkowska, M.: Establishing a framework for dynamic risk management in ‘intelligent’ aero-engine control. In: Buth, B., Rabe, G., Seyfarth, T. (eds.) SAFECOMP 2009. LNCS, vol. 5775, pp. 326–341. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04468-7_26
Trapp, M., et al.: Towards safety-awareness and dynamic safety management. In: 2018 14th European Dependable Computing Conference (EDCC), Iasi, Romania (2018)
Asaadi, E., Denney, E., Menzies, J., Pai, G.J., Petroff, D.: Dynamic assurance cases: a pathway to trusted autonomy. Computer 53(12), 35–46 (2020)
Rushby, J.: Runtime certification. In: Leucker, M. (ed.) RV 2008. LNCS, vol. 5289, pp. 21–35. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89247-2_2
Kagermann, H., et al.: Das fachforum autonome systeme im hightech-forum der bundesregierung –chancen und risiken für wirtschaft, wissenschaft und gesellschaft. Final report, Berlin (2017)
Saidi, S., Ziegenbein, D., Deshmukh, J.V., Ernst, R.: Autonomous systems design: charting a new discipline. IEEE Design Test. 39(1), 8–23 (2022)
Feth, P., Schneider, D., Adler, R.: A conceptual safety supervisor definition and evaluation framework for autonomous systems. In: Tonetta, S., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2017. LNCS, vol. 10488, pp. 135–148. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66266-4_9
Shalev-Shwartz, S., Shammah, S., Shashua, A.: On a formal model of safe and scalable self-driving cars. ArXiv abs/1708.06374 (2017)
Reich, J., Wellstein, M., Sorokos, I., Oboril, F., Scholl, K.U.: Towards a software component to perform situation-aware dynamic risk assessment for autonomous vehicles. In: European Dependable Computing Conference - EDCC 2021 Workshops (2021)
Huang, H.-M., et al.: Autonomy levels for unmanned systems (ALFUS) framework: an update. In: Proceedings of the 2005 SPIE Defense and Security Symposium, Orlando, Florida (2005)
Adler, R.: A Model-based approach for exploring the space of adaptation behaviors of safety-related embedded systems. Dissertation. Fraunhofer Verlag (2013)
Trapp, M., Adler, R., Forster, M., Junger, J.: Runtime adaptation in safety-critical automotive systems (2007)
Henne, M., et al.: Benchmarking uncertainty estimation methods for deep learning with safety-related metrics. In: SafeAI Workshop@AAAI Conference (2020)
Kläs, M., et al.: Handling uncertainties of data-driven models in compliance with safety constraints for autonomous behavior. In: 2021 17th European Dependable Computing Conference (EDCC), Munich, Germany, pp. 95–102 (2021)
Groß, J., et al.: Architectural patterns for handling runtime uncertainty of data-driven models in safety-critical perception. In: Trapp, Saglietti, Spisländer, Bitsch (eds.) SAFECOMP 2022. LNCS, vol. 13414, pp. 284–297. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-14835-4_19
Schneider, D., Trapp, M.:. Conditional safety certification of open adaptive systems. ACM Trans. Auton. Adapt. Syst. 8(2), 1–20 (2013). Article 8
Zeller, M., et al.: Open dependability exchange metamodel: a format to exchange safety information. In: Annual Reliability and Maintainability Symposium (RAMS), USA (2023)
Rushby, J.: The interpretation and evaluation of assurance cases (2015). http://www.csl.sri.com/users/rushby/papers/sri-csl-15-1-assurance-cases.pdf30T
Bishop, P., Povyakalo, A., Strigini, L.: Bootstrapping confidence in future safety from past safe operation. In: 2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE), Charlotte, NC, USA, pp. 97–108 (2022)
Hawkins, R., Conmy, P.: Identifying runtime monitoring requirements for autonomous systems through the analysis of safety arguments. In: Proceedings of 42nd International Conference on Computer Safety, Reliability and Security (Safecomp 2023) (2023, to appear)
Denney, E., Pai, G., Habli, I.: Dynamic safety cases for through-life safety assurance. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Florence, Italy, pp. 587–590 (2015). https://doi.org/10.1109/ICSE.2015.199
Schneider, D., Trapp, M.: B-space: dynamic management and assurance of open systems of systems. J. Internet Serv. Appl. 9, 15 (2018). https://doi.org/10.1186/s13174-018-0084-5
Schleiss, P., Carella, F., Kurzidem, I.: Towards continuous safety assurance for autonomous systems. In: 2022 6th International Conference on System Reliability and Safety (ICSRS), Venice, Italy, pp. 457–462 (2022). https://doi.org/10.1109/ICSRS56243.2022.10067323
DIN Spec homepage, DIN SPEC 92005 Artificial Intelligence - Uncertainty quantification in machine learning. https://www.din.de/de/forschung-und-innovation/din-spec/alle-geschaeftsplaene/wdc-beuth:din21:360097552. Accessed June 2023
Press Release “LOPAAS project”. https://www.iese.fraunhofer.de/en/media/press/pm-2021-10-18-paradigmenwechsel-se.html. Accessed June 2023
DREAMS workshop homepage. https://www.iese.fraunhofer.de/en/seminare_training/edcc-workshop.html, visited 06.2023
ISO/IEC FDIS 23894: Information technology—Artificial intelligence—Guidance on risk management (2022)
Acknowledgments
This work has been funded by the project “LOPAAS” (Layers of Protection Architecture for Autonomous Systems) as part of the internal funding program “ICON” of the Fraunhofer-Gesellschaft.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Adler, R., Reich, J., Hawkins, R. (2023). Structuring Research Related to Dynamic Risk Management for Autonomous Systems. In: Guiochet, J., Tonetta, S., Schoitsch, E., Roy, M., Bitsch, F. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2023 Workshops. SAFECOMP 2023. Lecture Notes in Computer Science, vol 14182. Springer, Cham. https://doi.org/10.1007/978-3-031-40953-0_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-40953-0_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40952-3
Online ISBN: 978-3-031-40953-0
eBook Packages: Computer ScienceComputer Science (R0)