research-article

Open access

From Explainable AI to Explainable Simulation: Using Machine Learning and XAI to understand System Robustness

Authors:

Niclas Feldkamp,

Steffen StrassburgerAuthors Info & Claims

SIGSIM-PADS '23: Proceedings of the 2023 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

Pages 96 - 106

https://doi.org/10.1145/3573900.3591114

Published: 21 June 2023 Publication History

All formats PDF

Abstract

Evaluating robustness is an important goal in simulation-based analysis. Robustness is achieved when the controllable factors of a system are adjusted in such a way that any possible variance in uncontrollable factors (noise) has minimal impact on the variance of the desired output. The optimization of system robustness using simulation is a dedicated and well-established research direction. However, once a simulation model is available, there is a lot of potential to learn more about the inherent relationships in the system, especially regarding its robustness. Data farming offers the possibility to explore large design spaces using smart experiment design, high performance computing, automated analysis, and interactive visualization. Sophisticated machine learning methods excel at recognizing and modelling the relation between large amounts of simulation input and output data. However, investigating and analyzing this modelled relationship can be very difficult, since most modern machine learning methods like neural networks or random forests are opaque black boxes. Explainable Artificial Intelligence (XAI) can help to peak into this black box, helping us to explore and learn about relations between simulation input and output. In this paper, we introduce a concept for using Data Farming, machine learning and XAI to investigate and understand system robustness of a given simulation model.

References

[1]

Alejandro Barredo-Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58, 82–115.

Digital Library

[2]

Vaishak Belle and Ioannis Papantonis. 2021. Principles and Practice of Explainable Machine Learning. Front. Big Data 4, 688969.

[3]

Filip K. Dosilovic, Mario Brcic, and Nikica Hlupic. 2018. Explainable Artificial Intelligence: A Survey. In 41st International Convention on Information and Communication Technology, Electronics and Microelectronics. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey, 210–215.

[4]

Niclas Feldkamp. 2021. Data Farming Output Analysis Using Explainable AI. In Proceedings of the 2021 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey.

[5]

Niclas Feldkamp, Soeren Bergmann, Florian Conrad, and Steffen Strassburger. 2022. A Method Using Generative Adversarial Networks for Robustness Optimization. ACM Trans. Model. Comput. Simul. 32, 2, 1–22.

Digital Library

[6]

Niclas Feldkamp, Soeren Bergmann, and Steffen Strassburger. 2015. Knowledge Discovery in Manufacturing Simulations. In Proceedings of the 3rd ACM SIGSIM Conference on Principles of Advanced Discrete Simulation. SIGSIM PADS ’15. ACM Press, New York, New York, 3–12.

Digital Library

[7]

Niclas Feldkamp, Soeren Bergmann, and Steffen Strassburger. 2020. Knowledge Discovery in Simulation Data. ACM Trans. Model. Comput. Simul. 30, 4, 1–25.

Digital Library

[8]

Niclas Feldkamp, Soeren Bergmann, Steffen Strassburger, and Thomas Schulze. 2017. Knowledge Discovery and Robustness Analysis in Manufacturing Simulations. In Proceedings of the 2017 Winter Simulation Conference. IEEE Inc.

Digital Library

[9]

Niclas Feldkamp, Jonas Genath, and Steffen Strassburger. 2022. Explainable AI For Data Farming Output Analysis: A Use Case for Knowledge Generation Through Black-Box Classifiers. In Proceedings of the 2022 Winter Simulation Conference. IEEE, 1152–1163.

[10]

Jonas Genath. 2021. Automation within the Process of Knowledge Discovery in Simulation Data [Poster]. In Proceedings of the 2021 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey.

[11]

Jonas Genath, Sören Bergmann, Niclas Feldkamp, Sven Spieckermann, and Stephan Stauber. 2022. Development of an Integrated Solution for Data Farming and Knowledge Discovery in Simulation Data. SNE 32, 3, 121–126.

[12]

Bryce Goodman and Seth Flaxman. 2017. European Union Regulations on Algorithmic Decision-Making and a “Right to Explanation”. AIMag 38, 3, 50–57.

Digital Library

[13]

Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2019. A Survey of Methods for Explaining Black Box Models. ACM Comput. Surv. 51, 5, 1–42.

Digital Library

[14]

G. E. Horne and T. Meyer. 2010. Data farming and defense applications. In MODSIM World Conference and Expo. Langley Research Center, Hampton, VA, 74–82.

[15]

Gary E. Horne and Ted E. Meyer. 2005. Data Farming: Discovering Surprise. In Proceedings of the 2005 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey, 1082–1087.

[16]

Gary E. Horne and Klaus-Peter Schwierz. 2008. Data Farming Around The World Overview. In Proceedings of the 2008 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey, 1442–1447.

[17]

Joachim Hunker, Anne Antonia Scheidler, Markus Rabe, and Hendrik van der Valk. 2021. A New Data Farming Procedure Model for a Farming for Mining Method in Logistics Networks. In Proceedings of the 2021 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey.

[18]

Florian Klug. 2013. The internal bullwhip effect in car manufacturing. International Journal of Production Research 51, 1, 303–322.

[19]

Tobias Lechler, Martin Sjarov, and Jörg Franke. 2021. Data Farming in Production Systems - A Review on Potentials, Challenges and Exemplary Applications. Procedia CIRP 96, 230–235.

[20]

Yi-Shan Lin, Wen-Chuan Lee, and Z. B. Celik. 2020. What Do You See? Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors, arxiv.org Preprint. http://arxiv.org/pdf/2009.10639v1.

[21]

Thomas W. Lucas, W. D. Kelton, Paul J. Sánchez, Susan M. Sanchez, and Ben L. Anderson. 2015. Changing the paradigm: Simulation, now a method of first resort. Naval Research Logistics 62, 4, 293–303.

[22]

Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan and R. Garnett, Eds. Curran Associates, Inc, 4765–4774.

[23]

Maximilian Alber, Sebastian Lapuschkin, Philipp Seegerer, Miriam Hägele, Kristof T. Schütt, Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller, Sven Dähne, and Pieter-Jan Kindermans. 2019. iNNvestigate Neural Networks! Journal of Machine Learning Research 20, 93, 1–8.

[24]

Christoph Molnar. 2019. Interpretable Machine Learning. A Guide for Making Black Box Models Explainable (1st edition). Lulu Com, Zürich.

[25]

Christoph Molnar, Gunnar König, Julia Herbinger, Timo Freiesleben, Susanne Dandl, Christian A. Scholbeck, Giuseppe Casalicchio, Moritz Grosse-Wentrup, and Bernd Bischl. 2022. General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models. In xxAI - Beyond Explainable AI, Andreas Holzinger, Randy Goebel, Ruth Fong, Taesup Moon, Klaus-Robert Müller and Wojciech Samek, Eds. Lecture Notes in Computer Science. Springer International Publishing, Cham, 39–68.

[26]

Manuel E. Morocho-Cayamcela, Haeyoung Lee, and Wansu Lim. 2019. Machine Learning for 5G/B5G Mobile and Wireless Communications: Potential, Limitations, and Future Directions. IEEE Access 7, 137184–137206.

[27]

Vijayan N. Nair, Bovas Abraham, Jock MacKay, John A. Nelder, George Box, Madhav S. Phadke, Raghu N. Kacker, Jerome Sacks, William J. Welch, Thomas J. Lorenzen, Anne C. Shoemaker, Kwok L. Tsui, James M. Lucas, Shin Taguchi, Raymond H. Myers, G. G. Vining, and C. F. J. Wu. 1992. Taguchi's Parameter Design. A Panel Discussion. Technometrics 34, 2, 127.

Digital Library

[28]

Michael K. Painter, Madhav Erraguntla, Gary L. Hogg, and Brian Beachkofski. 2006. Using Simulation, Data Mining, and Knowledge Discovery Techniques for Optimized Aircraft Engine Fleet Management. In Proceedings of the 2006 Winter Simulation Conference. Monterey, CA, U.S.A., Dec 3-6, 2006. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey.

[29]

Gyung-Jin Park, Tae-Hee Lee, Kwon H. Lee, and Kwang-Hyeon Hwang. 2006. Robust Design. An Overview. AIAA Journal 44, 1, 181–191.

[30]

Amir Parnianifard, A. S. Azfanizam, M.K.A. Ariffin, and M.I.S. Ismail. 2018. An overview on robust design hybrid metamodeling: Advanced methodology in process optimization under uncertainty. 10.5267/j.ijiec 9, 1–32.

[31]

Madhav S. Phadke. 1989. Quality engineering using robust design. Prentice Hall, Englewood Cliffs, N.J.

[32]

Gabriëlle Ras, Marcel van Gerven, and Pim Haselager. 2018. Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges. In Explainable and Interpretable Models in Computer Vision and Machine Learning, Hugo J. Escalante, Sergio Escalera, Isabelle Guyon, Xavier Baró, Yağmur Güçlütürk, Umut Güçlü and Marcel van Gerven, Eds. The Springer Series on Challenges in Machine Learning. Springer International Publishing, Cham, 19–36.

[33]

Marco T. Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why Should I Trust You?”. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA, 1135–1144.

Digital Library

[34]

Marco T. Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-Precision Model-Agnostic Explanations. Proceedings of the AAAI Conference on Artificial Intelligence 32, 1, 1527–1535.

[35]

S. M. Sanchez. 2000. Robust Design: Seeking the Best of all Possible Worlds. In Proceedings of the 2000 Winter Simulation Conference. IEEE Inc, Piscataway, N.J., 69–76.

[36]

Susan M. Sanchez. 1994. A Robust Design Tutorial. In Proceedings of the 1994 Winter Simulation Conference, 106–113.

[37]

Susan M. Sanchez. 2014. Simulation Experiments: Better Data, Not Just Big Data. In Proceedings of the 2014 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey, 805–816.

[38]

Susan M. Sanchez. 2020. Data Farming: Methods for the Present, Opportunities for the Future. ACM Trans. Model. Comput. Simul. 30, 4, 1–30.

Digital Library

[39]

Susan M. Sanchez and Paul J. Sanchez. 2020. Robustness Revisited: Simulation Optimization Viewed Through A Different Lens. In Proceedings of the 2020 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey, 60–74.

[40]

Andrew D. Selbst and Julia Powles. 2017. Meaningful information and the right to explanation. International Data Privacy Law 7, 4, 233–242.

[41]

Lynne Serre and Maude Amyot-Bourgeois. 2022. An Application of Automated Machine Learning Within a Data Farming Process. In Proceedings of the 2022 Winter Simulation Conference. IEEE, 2013–2024.

[42]

Lynne Serre, Maude Amyot-Bourgeois, and Brittany Astles. 2021. Use of Shapley Additive Explanations in Interpreting Agent-Based Simulations of Military Operational Scenarios. In 2021 Annual Modeling and Simulation Conference (ANNSIM). IEEE, 1–12.

[43]

Amitojdeep Singh, Sourya Sengupta, and Vasudevan Lakshminarayanan. 2020. Explainable Deep Learning Models in Medical Image Analysis. Journal of imaging 6, 6, 1–19.

[44]

Steffen Strassburger, Sören Bergmann, Niclas Feldkamp, Kristina Sokoll, and Matthias Clausing. 2018. Data Farming Research Project with Audi and VW. In 2018 Plant Simulation Worldwide User Conference.

[45]

Genichi Taguchi. 1995. Quality engineering (Taguchi methods) for the development of electronic circuit technology. IEEE Trans. Rel. 44, 2, 225–229.

[46]

Erico Tjoa and Cuntai Guan. 2020. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI. IEEE Transactions on Neural Networks and Learning Systems, 1–21.

[47]

H. Vieira, Susan M. Sanchez, K. H. Kienitz, and M. C. N. Belderrain. 2013. Efficient, nearly orthogonal-and-balanced, mixed designs. An effective way to conduct trade-off analyses via simulation. Journal of Simulation 7, S4, 264–275.

[48]

Hélcio Vieira. 2012. NOB_Mixed_512DP_v2.xls design spreadsheet (2012). Retrieved February 1, 2023 from http://harvest.nps.edu/.

Cited By

Chen ZLiang NLi HZhang HLi HYan LHu ZChen YZhang YWang YKe DShi N(2024)Exploring explainable AI features in the vocal biomarkers of lung diseaseComputers in Biology and Medicine10.1016/j.compbiomed.2024.108844179:COnline publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2024.108844
Elsayed AEl-Mihoub TManss CMiedtank ANolle LStahl F(2024)Interactive Simulator Framework for XAI Applications in Aquatic EnvironmentsArtificial Intelligence XLI10.1007/978-3-031-77915-2_11(144-157)Online publication date: 29-Nov-2024
https://doi.org/10.1007/978-3-031-77915-2_11

Index Terms

From Explainable AI to Explainable Simulation: Using Machine Learning and XAI to understand System Robustness
1. Computing methodologies
  1. Machine learning
  2. Modeling and simulation

Recommendations

Explainable artificial intelligence (XAI): Precepts, models, and opportunities for research in construction
Abstract
Machine learning (ML) and deep learning (DL) are both branches of AI. As a form of AI, ML automatically adapts to changing datasets with minimal human interference. Deep learning is a subset of ML that uses artificial neural networks to imitate ...
Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
Abstract
The past decade has seen significant progress in artificial intelligence (AI), which has resulted in algorithms being adopted for resolving a variety of problems. However, this success has been met by increasing model complexity and ...
Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
Highlights
- We review concepts related to the explainability of AI methods (XAI).
- We comprehensive analyze the XAI literature organized in two taxonomies.
- We identify future research directions of the XAI field.
- We discuss potential ...
Abstract
In the last few years, Artificial Intelligence (AI) has achieved a notable momentum that, if harnessed appropriately, may deliver the best of expectations over many application sectors across the field. For this to occur shortly in Machine ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSIM-PADS '23: Proceedings of the 2023 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

June 2023

173 pages

ISBN:9798400700309

DOI:10.1145/3573900

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGSIM: ACM Special Interest Group on Simulation and Modeling

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2023

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SIGSIM-PADS '23

Sponsor:

SIGSIM

SIGSIM-PADS '23: SIGSIM Conference on Principles of Advanced Discrete Simulation

June 21 - 23, 2023

FL, Orlando, USA

Acceptance Rates

Overall Acceptance Rate 398 of 779 submissions, 51%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
976
Total Downloads

Downloads (Last 12 months)577
Downloads (Last 6 weeks)85

Reflects downloads up to 14 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen ZLiang NLi HZhang HLi HYan LHu ZChen YZhang YWang YKe DShi N(2024)Exploring explainable AI features in the vocal biomarkers of lung diseaseComputers in Biology and Medicine10.1016/j.compbiomed.2024.108844179:COnline publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2024.108844
Elsayed AEl-Mihoub TManss CMiedtank ANolle LStahl F(2024)Interactive Simulator Framework for XAI Applications in Aquatic EnvironmentsArtificial Intelligence XLI10.1007/978-3-031-77915-2_11(144-157)Online publication date: 29-Nov-2024
https://doi.org/10.1007/978-3-031-77915-2_11

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten