Reactive and Safe Road User Simulations using Neural Barrier Certificates
Authors: 
Yue Meng, Zengyi Qin, Chuchu FanAuthors Info & Claims
Pages 6299 - 6306
Abstract
Reactive and safe agent modellings are important for nowadays traffic simulator designs and safe planning applications. In this work, we proposed a reactive agent model which can ensure safety without comprising the original purposes, by learning only high-level decisions from expert data and a low level decentralized controller guided by the jointly learned decentralized barrier certificates. Empirical results show that our learned road user simulation models can achieve a significant improvement in safety comparing to state-of-the-art imitation learning and pure control-based methods, while being similar to human agents by having smaller error to the expert data. Moreover, our learned reactive agents are shown to generalize better to unseen traffic conditions, and react better to other road users and therefore can help understand challenging planning problems pragmatically.
References
[1]
A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “Carla: An open urban driving simulator,” in Conference on robot learning. PMLR, 2017, pp. 1–16.
[2]
B. Wymann, E. Espie, C. Guionneau, C. Dimitrakakis, R. Coulom, and A. Sumner, “Torcs, the open racing car simulator,” Software available at http://torcs.sourceforge.net, vol. 4, no. 6, p. 2, 2000.
[3]
D. Krajzewicz, “Traffic simulation with sumo–simulation of urban mobility,” in Fundamentals of traffic simulation. Springer, 2010, pp. 269–293.
[4]
A. Best, S. Narang, D. Barber, and D. Manocha, “Autonovi: Autonomous vehicle planning with dynamic maneuvers and traffic constraints,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 2629–2636.
[5]
M. Bain and C. Sammut, “A framework for behavioural cloning.” in Machine Intelligence 15, 1995, pp. 103–129.
[6]
S. Russell, “Learning agents for uncertain environments,” in Proceedings of the eleventh annual conference on Computational learning theory, 1998, pp. 101–103.
[7]
A. Y. Ng, S. J. Russell, et al., “Algorithms for inverse reinforcement learning.” in Icml, vol. 1, 2000, p. 2.
[8]
R. P. Bhattacharyya, D. J. Phillips, C. Liu, J. K. Gupta, K. Driggs-Campbell, and M. J. Kochenderfer, “Simulating emergent properties of human driving behavior using multi-agent reward augmented imitation learning,” in 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 789–795.
[9]
D. Hadfield-Menell, S. Milli, P. Abbeel, S. Russell, and A. Dragan, “Inverse reward design,” arXiv preprint arXiv:1711.02827, 2017.
[10]
A. D. Ames, J. W. Grizzle, and P. Tabuada, “Control barrier function based quadratic programs with application to adaptive cruise control,” in 53rd IEEE Conference on Decision and Control. IEEE, 2014, pp. 6271–6278.
[11]
A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada, “Control barrier functions: Theory and applications,” in 2019 18th European Control Conference (ECC). IEEE, 2019, pp. 3420–3431. [Online]. Available: http://ames.caltech.edu/ames2019control.pdf
[12]
Z. Qin, K. Zhang, Y. Chen, J. Chen, and C. Fan, “Learning safe multi-agent control with decentralized neural barrier certificates,” arXiv preprint arXiv:2101.05436, 2021.
[13]
“Ngsim. next generation simulation,” http://ngsim.fhwa.dot.gov/, 2006.
[14]
R. Krajewski, J. Bock, L. Kloeker, and L. Eckstein, “The highd dataset: A drone dataset of naturalistic vehicle trajectories on german highways for validation of highly automated driving systems,” in 2018 21st International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2018, pp. 2118–2125.
[15]
R. Krajewski, T. Moers, J. Bock, L. Vater, and L. Eckstein, “The round dataset: A drone dataset of road user trajectories at roundabouts in germany,” in 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2020, pp. 1–6.
[16]
A. Robicquet, A. Sadeghian, A. Alahi, and S. Savarese, “Learning social etiquette: Human trajectory prediction in crowded scenes,” in European Conference on Computer Vision (ECCV), 2020.
[17]
D. Yang, L. Li, K. Redmill, and Ü. Özgüner, “Top-view trajectories: A pedestrian dataset of vehicle-crowd interaction from controlled experiments and crowded campus,” in 2019 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2019, pp. 899–904.
[18]
J. Ho and S. Ermon, “Generative adversarial imitation learning,” arXiv preprint arXiv:1606.03476, 2016.
[19]
R. P. Bhattacharyya, D. J. Phillips, B. Wulfe, J. Morton, A. Kuefler, and M. J. Kochenderfer, “Multi-agent imitation learning for driving simulation,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 1534–1539.
[20]
A. Liu, G. Shi, S.-J. Chung, A. Anandkumar, and Y. Yue, “Robust regression for safe exploration in control,” in Learning for Dynamics and Control. PMLR, 2020, pp. 608–619.
[21]
F. Berkenkamp, M. Turchetta, A. P. Schoellig, and A. Krause, “Safe model-based reinforcement learning with stability guarantees,” arXiv preprint arXiv:1705.08551, 2017.
[22]
R. Cheng, G. Orosz, R. M. Murray, and J. W. Burdick, “End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 3387–3395.
[23]
X. Li and C. Belta, “Temporal logic guided safe reinforcement learning using control barrier functions,” arXiv preprint arXiv:1903.09885, 2019.
[24]
J. Clark and D. Amodei, “Faulty reward functions in the wild,” Internet: https://blog.openai.com/faulty-reward-functions, 2016.
[25]
S. Prajna and A. Jadbabaie, “Safety verification of hybrid systems using barrier certificates,” in International Workshop on Hybrid Systems: Computation and Control. Springer, 2004, pp. 477–492.
[26]
P. Wieland and F. Allgöwer, “Constructive safety using control barrier functions,” IFAC Proceedings Volumes, vol. 40, no. 12, pp. 462–467, 2007.
[27]
Y. Chen, H. Peng, and J. Grizzle, “Obstacle avoidance for low-speed autonomous vehicles with barrier function,” IEEE Transactions on Control Systems Technology, vol. 26, no. 1, pp. 194–206, 2017.
[28]
J. Ferlez, M. Elnaggar, Y. Shoukry, and C. Fleming, “Shieldnn: A provably safe nn filter for unsafe nn controllers,” arXiv preprint arXiv:2006.09564, 2020.
[29]
L. Wang, D. Han, and M. Egerstedt, “Permissive barrier certificates for safe stabilization using sum-of-squares,” in 2018 Annual American Control Conference (ACC). IEEE, 2018, pp. 585–590.
[30]
M. Srinivasan, A. Dabholkar, S. Coogan, and P. Vela, “Synthesis of control barrier functions using a supervised machine learning approach,” arXiv preprint arXiv:2003.04950, 2020.
[31]
K. Long, C. Qian, J. Cortes, and N. Atanasov, “Learning barrier´ functions with memory for robust safe navigation,” arXiv preprint arXiv:2011.01899, 2020.
[32]
U. Borrmann, L. Wang, A. D. Ames, and M. Egerstedt, “Control barrier certificates for safe swarm behavior,” IFAC-PapersOnLine, vol. 48, no. 27, pp. 68–73, 2015.
[33]
L. Wang, A. D. Ames, and M. Egerstedt, “Safety barrier certificates for collisions-free multirobot systems,” IEEE Transactions on Robotics, vol. 33, no. 3, pp. 661–674, 2017.
[34]
Y. Chen, A. Singletary, and A. D. Ames, “Guaranteed obstacle avoidance for multi-robot operations with limited actuation: a control barrier function approach,” IEEE Control Systems Letters, vol. 5, no. 1, pp. 127–132, 2020.
[35]
H. K. Khalil and J. W. Grizzle, Nonlinear systems. Prentice hallUpper Saddle River, NJ, 2002, vol. 3.
[36]
V. Lakshmikantham and S. Leela, Differential and Integral Inequalities: Theory and Applications: Volume I: Ordinary Differential Equations. Academic press, 1969.
[37]
P. Glotfelter, J. Cortes, and M. Egerstedt, “Nonsmooth barrier func-´ tions with applications to multi-robot systems,” IEEE control systems letters, vol. 1, no. 2, pp. 310–315, 2017.
[38]
C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652–660.
[39]
C. E. Garcia, D. M. Prett, and M. Morari, “Model predictive control: Theory and practice—a survey,” Automatica, vol. 25, no. 3, pp. 335–348, 1989.
[40]
J. A. E. Andersson, J. Gillis, G. Horn, J. B. Rawlings, and M. Diehl, “CasADi – A software framework for nonlinear optimization and optimal control,” Mathematical Programming Computation, vol. 11, no. 1, pp. 1–36, 2019.
[41]
R. P. Bhattacharyya, R. Senanayake, K. Brown, and M. J. Kochenderfer, “Online parameter estimation for human driver behavior prediction,” in 2020 American Control Conference (ACC). IEEE, 2020, pp. 301–306.
[42]
M. Treiber and A. Kesting, “The intelligent driver model with stochasticity-new insights into traffic flow oscillations,” Transportation research procedia, vol. 23, pp. 174–187, 2017.
[43]
S. E. Lee, E. C. Olsen, W. W. Wierwille, et al., “A comprehensive examination of naturalistic lane-changes,” United States. National Highway Traffic Safety Administration, Tech. Rep., 2004.
Index Terms
- Reactive and Safe Road User Simulations using Neural Barrier Certificates
Index terms have been assigned to the content through auto-classification.
Recommendations
Safe Learning of Quadrotor Dynamics Using Barrier Certificates
2018 IEEE International Conference on Robotics and Automation (ICRA)To effectively control complex dynamical systems, accurate nonlinear models are typically needed. However, these models are not always known. In this paper, we present a data-driven approach based on Gaussian processes that learns models of quadrotors ...
Safe Drive Map Concept for Road Curve Monitoring
DSD '15: Proceedings of the 2015 Euromicro Conference on Digital System DesignWe present a technique for dangerous curve monitoring relying on the innovative concept of Safe Driving Map (SDM), a geo-referenced database with data about safe vehicle behavior in the monitored area, also considering different weather conditions. A ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sep 2021
7915 pages
Copyright © 2021.
Publisher
IEEE Press
Publication History
Published: 27 September 2021
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in