Abstract
The recent advancements of Artificial Intelligence (AI) have generated a lot of interest in the robotics community. Indeed, AI can find application in a wide variety of problems. Among these, social navigation of mobile robots is a big challenge, where ensuring non-harmful behaviors of the robotic system is fundamental.
In this paper, we consider a simulated navigation problem that involves a fleet of mobile agents moving in a cross scenario, governed by a human-like behavior. With the purpose of avoiding collisions among them, we show how safe and explainable AI (XAI) methods can constitute useful tools to tailor the parameters of the behavior towards a safe, collision-free, navigation. We first explore how global native rule-based classification provides interpretable characterizations of the agents’ behavior. Afterwards, we derive safety regions, \(\mathcal {S}_{\varepsilon }\), denoting the zones in the parameters space where collisions are avoided, with a maximum error given by \(\varepsilon \). The design of the regions is based on scalable classifiers, a technique to tune the decision function of a machine learning (ML) classifier so to bound its error on a desired class to a predefined level, combined with either probabilistic scaling (probabilistic safety regions, PSR), or with conformal prediction theory (conformal safety regions, CSR). Finally, we investigate how explainability can be provided to these regions by extracting local rules from their boundaries.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
Navigation Playground, see https://idsia-robotics.github.io/navground/.
- 4.
- 5.
- 6.
We remind that the error of a rule corresponds to its false positive rate, where the positive class is intended as the one predicted by the rule (see Eq. 13). Hence, in our case, the error refers to the percentage of collisions wrongly classified as non-collisions.
References
Angelopoulos, A.N., Bates, S.: A gentle introduction to conformal prediction and distribution-free uncertainty quantification (2021). arXiv preprint arXiv:2107.07511
Brunke, L., et al.: Safe learning in robotics: from learning-based control to safe reinforcement learning. Annu. Rev. Control Robot. Auton. Syst. 5, 411–444 (2022)
Carlevaro, A., Alamo, T., Dabbene, F., Mongelli, M.: Probabilistic safety regions via finite families of scalable classifiers (2023). arXiv preprint arXiv:2309.04627
Carlevaro, A., Cantarero, T.A., Dabbene, F., Mongelli, M.: Conformal predictions for probabilistically robust scalable machine learning classification (2024). arXiv preprint arXiv:2403.10368
Cebollada, S., Payá, L., Flores, M., Peidró, A., Reinoso, O.: A state-of-the-art review on mobile robotics tasks using artificial intelligence and visual data. Expert Syst. Appl. 167, 114195 (2021)
Choi, H., et al.: On the use of simulation in robotics: opportunities, challenges, and suggestions for moving forward. Proc. Natl. Acad. Sci. 118(1), e1907856118 (2021)
D, F.L., E, G.G.: Trustworthy autonomous vehicles. Scientific analysis or review, Anticipation and foresight, Technical guidance KJ-NA-30942-EN-N (online), Luxembourg (Luxembourg) (2021). https://doi.org/10.2760/120385(online)
Dwivedi, R., et al.: Explainable AI (XAI): core ideas, techniques, and solutions. ACM Comput. Surv. 55(9), 1–33 (2023)
Emaminejad, N., Akhavian, R.: Trustworthy AI and robotics: implications for the AEC industry. Autom. Constr. 139, 104298 (2022)
Everett, M., Chen, Y.F., How, J.P.: Collision avoidance in pedestrian-rich environments with deep reinforcement learning. IEEE Access 9, 10357–10377 (2021). https://doi.org/10.1109/ACCESS.2021.3050338
Fan, T., Long, P., Liu, W., Pan, J.: Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios. Int. J. Robot. Res. 39(7), 856–892 (2020)
Ferrari, E., Verda, D., Pinna, N., Muselli, M.: A novel rule-based modeling and control approach for the optimization of complex water distribution networks. In: Advances in System-Integrated Intelligence: Proceedings of the 6th International Conference on System-Integrated Intelligence (SysInt 2022), September 7-9, 2022, Genova, Italy. pp. 33–42. Springer (2022). https://doi.org/10.1007/978-3-031-16281-7_4
Fontana, M., Zeni, G., Vantini, S.: Conformal prediction: a unified review of theory and new challenges. Bernoulli 29(1), 1–23 (2023)
Friedman, J.H., Popescu, B.E.: Predictive learning via rule ensembles (2008)
Fu, Y., Li, C., Yu, F.R., Luan, T.H., Zhang, Y.: A survey of driving safety with sensing, vehicular communications, and artificial intelligence-based collision avoidance. IEEE Trans. Intell. Transp. Syst. 23(7), 6142–6163 (2021)
Guzzi, J., Giusti, A., Gambardella, L.M., Theraulaz, G., Di Caro, G.A.: Human-friendly robot navigation in dynamic environments. In: 2013 IEEE international conference on robotics and automation, pp. 423–430. IEEE (2013)
Hassija, V., et al.: Interpreting black-box models: a review on explainable artificial intelligence. Cogn. Comput. 16(1), 45–74 (2024)
High-Level Expert Group on AI: Ethics guidelines for trustworthy AI. Report, European Commission, Brussels (2019). https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai
Hu, Y., Wang, Y., Hu, K., Li, W.: Adaptive obstacle avoidance in path planning of collaborative robots for dynamic manufacturing. J. Intell. Manuf. 34(2), 789–807 (2023)
https://2018.ds3-datascience-polytechnique.fr/wp-content/uploads/2018/06/DS3-309.pdf
Kahn, G., Villaflor, A., Pong, V., Abbeel, P., Levine, S.: Uncertainty-aware reinforcement learning for collision avoidance (2017). arXiv preprint arXiv:1702.01182
Kim, Youngmin, Allmendinger, Richard, López-Ibáñez, Manuel: Safe learning and optimization techniques: towards a survey of the state of the art. In: Heintz, Fredrik, Milano, Michela, O’Sullivan, Barry (eds.) TAILOR 2020. LNCS (LNAI), vol. 12641, pp. 123–139. Springer, Cham (2021).https://doi.org/10.1007/978-3-030-73959-1_12
Liu, Y., et al.: Trustworthy LLMs: a survey and guideline for evaluating large language models’ alignment (2023)
Longo, L., et al.: Explainable artificial intelligence (XAI) 2.0: a manifesto of open challenges and interdisciplinary research directions. Inf. Fusion 102301 (2024)
Luo, W., Sun, W., Kapoor, A.: Multi-robot collision avoidance under uncertainty with probabilistic safety barrier certificates. Adv. Neural. Inf. Process. Syst. 33, 372–383 (2020)
Lütjens, B., Everett, M., How, J.P.: Safe reinforcement learning with model uncertainty estimates. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 8662–8668. IEEE (2019)
Mackowiak, R., Ardizzone, L., Kothe, U., Rother, C.: Generative classifiers as a basis for trustworthy image classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2971–2981 (2021)
Mahdi, H., Akgun, S.A., Saleh, S., Dautenhahn, K.: A survey on the design and evolution of social robots-past, present and future. Robot. Auton. Syst. 156, 104193 (2022)
Mammarella, M., Mirasierra, V., Lorenzen, M., Alamo, T., Dabbene, F.: Chance-constrained sets approximation: a probabilistic scaling approach. Automatica 137, 110108 (2022)
Moldovan, T.M., Abbeel, P.: Safe exploration in markov decision processes (2012). arXiv preprint arXiv:1205.4810
Molnar, C.: Interpretable machine learning. Lulu. com (2020)
Moussaïd, M., Helbing, D., Theraulaz, G.: How simple rules determine pedestrian behavior and crowd disasters. Proc. Natl. Acad. Sci. 108(17), 6884–6888 (2011)
Muselli, M.: Switching neural networks: a new connectionist model for classification (2005). https://doi.org/10.1007/11731177_4
Narteni, S., Orani, V., Cambiaso, E., Rucco, M., Mongelli, M.: On the intersection of explainable and reliable AI for physical fatigue prediction. IEEE Access 10, 76243–76260 (2022). https://doi.org/10.1109/ACCESS.2022.3191907
Narteni, S., Orani, V., Vaccari, I., Cambiaso, E., Mongelli, M.: Sensitivity of logic learning machine for reliability in safety-critical systems. IEEE Intell. Syst. 37(5), 66–74 (2022)
Ravankar, A., Ravankar, A.A., Rawankar, A., Hoshino, Y.: Autonomous and safe navigation of mobile robots in vineyard with smooth collision avoidance. Agriculture 11(10), 954 (2021)
Ribeiro, M.T., Singh, S., Guestrin, C.: Anchors: High-precision model-agnostic explanations. In: AAAI Conference on Artificial Intelligence (AAAI) (2018)
Rubio, F., Valero, F., Llopis-Albert, C.: A review of mobile robots: concepts, methods, theoretical framework, and applications. Int. J. Adv. Rob. Syst. 16(2), 1729881419839596 (2019)
Santos, N.B., Bavaresco, R.S., Tavares, J.E., Ramos, G.d.O., Barbosa, J.L.: A systematic mapping study of robotics in human care. Robot. Auton. Syst. 144, 103833 (2021)
Soori, M., Arezoo, B., Dastres, R.: Artificial intelligence, machine learning and deep learning in advanced robotics, a review. Cognitive Robotics (2023)
Sulaiman, A., et al.: Artificial intelligence-based secured power grid protocol for smart city. Sensors 23(19), 8016 (2023). https://doi.org/10.3390/s23198016
Terra, A., Riaz, H., Raizer, K., Hata, A., Inam, R.: Safety vs. efficiency: Ai-based risk mitigation in collaborative robotics. In: 2020 6th International Conference on Control, Automation and Robotics (ICCAR), pp. 151–160 (2020). https://doi.org/10.1109/ICCAR49639.2020.9108037
Theile, M., Bayerlein, H., Caccamo, M., Sangiovanni-Vincentelli, A.L.: Learning to recharge: UAV coverage path planning through deep reinforcement learning (2023)
Von Rueden, L., et al.: Informed machine learning-a taxonomy and survey of integrating prior knowledge into learning systems. IEEE Trans. Knowl. Data Eng. 35(1), 614–633 (2021)
Xiao, X., Liu, B., Warnell, G., Stone, P.: Motion planning and control for mobile robot navigation using machine learning: a survey. Auton. Robot. 46(5), 569–597 (2022)
Zhang, J., Cheung, B., Finn, C., Levine, S., Jayaraman, D.: Cautious adaptation for reinforcement learning in safety-critical settings. In: International Conference on Machine Learning, pp. 11055–11065. PMLR (2020)
Zhang, J., Zhang, Z.M.: Ethics and governance of trustworthy medical artificial intelligence. BMC Med. Informatics Decis. Making 23(1), 7 (2023)
Zhu, K., Zhang, T.: Deep reinforcement learning based mobile robot navigation: a review. Tsinghua Sci. Technol. 26(5), 674–691 (2021)
Zhu, Y., Wang, Z., Chen, C., Dong, D.: Rule-based reinforcement learning for efficient robot navigation with space reduction. IEEE/ASME Trans. Mechatron. 27(2), 846–857 (2022). https://doi.org/10.1109/TMECH.2021.3072675
Acknowledgements
This work was partially supported by REXASI-PRO H-EU project, call HORIZON-CL4-2021-HUMAN-01-01, Grant agreement ID: 101070028.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Narteni, S., Carlevaro, A., Guzzi, J., Mongelli, M. (2024). Ensuring Safe Social Navigation via Explainable Probabilistic and Conformal Safety Regions. In: Longo, L., Lapuschkin, S., Seifert, C. (eds) Explainable Artificial Intelligence. xAI 2024. Communications in Computer and Information Science, vol 2156. Springer, Cham. https://doi.org/10.1007/978-3-031-63803-9_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-63803-9_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-63802-2
Online ISBN: 978-3-031-63803-9
eBook Packages: Computer ScienceComputer Science (R0)