research-article

Learning from experience for rapid generation of local car maneuvers

Authors:

Krzysztof Ćwian,

Piotr SkrzypczyńskiAuthors Info & Claims

Volume 105, Issue C

https://doi.org/10.1016/j.engappai.2021.104399

Published: 01 October 2021 Publication History

Abstract

Being able to rapidly respond to the changing scenes and traffic situations by generating feasible local paths is of pivotal importance for car autonomy. We propose to train a deep neural network (DNN) to plan feasible and nearly-optimal paths for kinematically constrained vehicles in a small constant time. Our DNN model is trained using a novel weakly supervised approach and a gradient-based policy search. On real and simulated scenes and a large set of local planning problems, we demonstrate that our approach outperforms the existing planners with respect to the number of successfully completed tasks. While the path generation time is about 40 ms, the generated paths are smooth and comparable to those obtained from conventional path planners.

Highlights

•

Reinforcement learning based path planning for car-like vehicles.

•

Fast generation of local maneuvers for cars using machine learning.

•

A novel differentiable loss function for training path planning neural networks.

•

A dataset of real-world planning problems for comparing path planners.

•

Real-time path planning for autonomous vehicles in the CARLA simulator.

References

[1]

Arab, A., Yu, K., Yi, J., Song, D., 2016. Motion planning for aggressive autonomous vehicle maneuvers. In: IEEE International Conference on Automation Science and Engineering. pp. 221–226.

[2]

Belter, D., Labȩcki, P., Skrzypczyński, P., 2012. Estimating terrain elevation maps from sparse and uncertain multi-sensor data. In: IEEE International Conference on Robotics and Biomimetics (ROBIO). pp. 715–722.

[3]

Bency M.J., Qureshi A.H., Yip M.C., Neural path planning: Fixed time, near-optimal path generation via oracle imitation, 2019.

[4]

Berenson, D., Abbeel, P., Goldberg, K., 2012. A robot path planning framework that learns from experience. In: IEEE International Conference on Robotics and Automation. Saint Paul, pp. 3671–3678.

[5]

Bhardwaj, M., Choudhury, S., Boots, B., Srinivasa, S.S., 2019. Leveraging experience in lazy search. In: Robotics: Science and Systems. Freiburg im Breisgau.

[6]

Chen, B., Dai, B., Lin, Q., Ye, G., Liu, H., Song, L., 2020. Learning to plan in high dimensions via neural exploration-exploitation trees. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020.

[7]

Chen, J., Yuan, B., Tomizuka, M., 2019a. Deep imitation learning for autonomous driving in generic urban scenarios with enhanced safety. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 2884–2890.

[8]

Chen J., Zhan W., Tomizuka M., Autonomous driving motion planning with constrained iterative LQR, IEEE Trans. Intell. Veh. 4 (2) (2019) 244–254.

[9]

Craw S., Manhattan distance, in: Sammut C., Webb G.I. (Eds.), Encyclopedia of Machine Learning and Data Mining, Springer, Boston, 2017, pp. 790–791.

[10]

Cybenko G., Approximation by superpositions of a sigmoidal function, Math. Control. Signals Syst. 2 (4) (1989) 303–314.

[11]

Deisenroth M.P., Neumann G., Peters J., A survey on policy search for robotics, Found. Trends Robotics 2 (1–2) (2013) 1–141.

[12]

Dolgov D., Thrun S., Montemerlo M., Diebel J., Path planning for autonomous driving in unknown environments, in: Experimental Robotics, in: STAR, 54, Springer, 2009, pp. 55–64.

[13]

Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V., 2017. CARLA: An open urban driving simulator. In: Proceedings of the 1st Annual Conference on Robot Learning. pp. 1–16.

[14]

Duan Y., Chen X., Houthooft R., Schulman J., Abbeel P., Benchmarking deep reinforcement learning for continuous control, in: Balcan M.F., Weinberger K.Q. (Eds.), Proceedings of the 33rd International Conference on Machine Learning, in: Proceedings of Machine Learning Research, 48, PMLR, New York, USA, 2016, pp. 1329–1338.

[15]

Gammell J.D., Barfoot T.D., Srinivasa S.S., Informed sampling for asymptotically optimal path planning, IEEE Trans. Robot. 34 (4) (2018) 966–984.

[16]

Gammell J.D., Barfoot T.D., Srinivasa S.S., Batch informed trees (BIT*): Informed asymptotically optimal anytime search, Int. J. Robot. Res. 39 (5) (2020) 543–567.

[17]

Gammell, J.D., Srinivasa, S.S., Barfoot, T.D., 2015. Batch informed trees (BIT∗): Sampling-based optimal planning via the heuristically guided search of implicit random geometric graphs. In: IEEE International Conference on Robotics and Automation. pp. 3067–3074.

[18]

Gawron T., Michałek M.M., A G3-continuous extend procedure for path planning of mobile robots with limited motion curvature and state constraints, Appl. Sci. 8 (11) (2018).

[19]

Geiger A., Lenz P., Stiller C., Urtasun R., Vision meets robotics: The KITTI dataset, Int. J. Robot. Res. (2013).

Digital Library

[20]

González D., Pírez J., Milanís V., Nashashibi F., A review of motion planning techniques for automated vehicles, IEEE Trans. Intell. Transp. Syst. 17 (4) (2016) 1135–1145.

[21]

Groeger J.A., Understanding Driving: Applying Cognitive Psychology To a Complex Everyday Task, Psychology Press, East Sussex, 2000.

[22]

Hornik K., Approximation capabilities of multilayer feedforward networks, Neural Netw. 4 (2) (1991) 251–257.

Digital Library

[23]

Hussein A., Gaber M.M., Elyan E., Jayne C., Imitation learning: A survey of learning methods, ACM Comput. Surv. 50 (2) (2017).

[24]

Jetchev N., Toussaint M., Fast motion planning from experience: Trajectory prediction for speeding up movement generation, Auton. Robots 34 (1) (2013) 1573–7527.

[25]

Karaman S., Frazzoli E., Sampling-based algorithms for optimal motion planning, Int. J. Robot. Res. 30 (7) (2011) 846–894.

Digital Library

[26]

Karaman, S., Frazzoli, E., 2013. Sampling-based optimal motion planning for non-holonomic dynamical systems. In: IEEE International Conference on Robotics and Automation. Karlsruhe, pp. 5041–5047.

[27]

Kingma, D.P., Ba, J., 2015. Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.

[28]

LaValle S.M., Planning Algorithms, Cambridge University Press, 2006.

Digital Library

[29]

Li Y., Littlefield Z., Bekris K.E., Asymptotically optimal sampling-based kinodynamic planning, Int. J. Robot. Res. 35 (5) (2016) 528–564.

[30]

Manessi F., Rozza A., Learning combinations of activation functions, in: 2018 24th International Conference on Pattern Recognition (ICPR), 2018, pp. 61–66,.

[31]

Michałek M.M., Gawron T., VFO path following control with guarantees of positionally constrained transients for unicycle-like robots with constrained control input, J. Intell. Robot. Syst. 89 (1) (2018) 191–210.

[32]

Mishkin D., Sergievskiy N., Matas J., Systematic evaluation of convolution neural network advances on the imagenet, Comput. Vis. Image Underst. 161 (2017) 11–19,. https://www.sciencedirect.com/science/article/pii/S1077314217300814.

Digital Library

[33]

Phillips, M., Cohen, B., Chitta, S., Likhachev, M., 2012. E-Graphs: Bootstrapping planning with experience graphs. In: Robotics, Science and Systems Conference.

[34]

Pivtoraiko M., Knepper R.A., Kelly A., Differentially constrained mobile robot motion planning in state lattices, J. Field Robot. 26 (3) (2009) 308–333.

[35]

Puterman M.L., Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, 2014.

[36]

Qureshi, A.H., Simeonov, A., Bency, M.J., Yip, M.C., 2019. Motion planning networks. In: IEEE International Conference on Robotics and Automation. pp. 2118–2124.

[37]

Sadat, A., Ren, M., Pokrovsky, A., Lin, Y., Yumer, E., Urtasun, R., 2019. Jointly learnable behavior and trajectory planning for self-driving vehicles. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 3949–3956.

[38]

Strub M.P., Gammell J.D., Adaptively informed trees (AIT∗): Fast asymptotically optimal path planning through adaptive heuristics, 2020.

[39]

Strub, M.P., Gammell, J.D., 2020b. Advanced BIT* (ABIT∗): Sampling-based planning with advanced graph-search techniques. In: IEEE International Conference on Robotics and Automation. Paris, pp. 3671–3678.

[40]

Sucan I.A., Moll M., Kavraki L.E., The open motion planning library, IEEE Robot. Autom. Mag. 19 (4) (2012) 72–82.

[41]

Sünderhauf N., Brock O., Scheirer W., Hadsell R., Fox D., Leitner J., Upcroft B., Abbeel P., Burgard W., Milford M., Corke P., The limits and potentials of deep learning for robotics, Int. J. Robot. Res. 37 (4–5) (2018) 405–420.

[42]

Sutton R.S., Barto A.G., Introduction To Reinforcement Learning, MIT Press, Cambridge, 1998.

Digital Library

[43]

Yoon S., Lee D., Jung J., Shim D., Spline-based RRT* using piecewise continuous collision-checking algorithm for car-like vehicles, J. Intell. Robot. Syst. 90 (2017) 1–13.

[44]

Zhang J., Singh S., Low-drift and real-time lidar odometry and mapping, Auton. Robots 41 (2017) 401–416.

Cited By

Hou XGan MZhang JZhao SJi Y(2023)Vehicle ride comfort optimization in the post-braking phase using residual reinforcement learningAdvanced Engineering Informatics10.1016/j.aei.2023.10219858:COnline publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.aei.2023.102198
Kicki PSkrzypczyński P(2022)Speeding up deep neural network-based planning of local car maneuvers via efficient B-spline path construction2022 International Conference on Robotics and Automation (ICRA)10.1109/ICRA46639.2022.9812313(4422-4428)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1109/ICRA46639.2022.9812313

Index Terms

Learning from experience for rapid generation of local car maneuvers
1. Computer systems organization
2. Computing methodologies
  1. Artificial intelligence
    1. Planning and scheduling
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Merging planning in dense traffic scenarios using interactive safe reinforcement learning
Highlights
- A controller to prevent secondary crashes after an initial rear-end collision is proposed.
- Rule-based switching control is embedded into reinforcement learning.
- Pre-collision control and post-collision control are combined.
- ...
Abstract
Autonomous navigation in dense traffic scenarios, such as on-ramp forced merging, still poses significant challenges for autonomous vehicles to prevent accidents and alleviate traffic congestion. This paper introduces a novel motion planning ...
Nash double Q-based multi-agent deep reinforcement learning for interactive merging strategy in mixed traffic
Abstract
The interaction between ramp and mainline vehicles plays a crucial role in merging areas, especially in the mixed-traffic environment. The driving behaviours of human drivers are uncertain and diverse, and the uncertainty makes it more complex ...
Comparison of Deep Reinforcement Learning Path-Following System Based on Road Geometry and an Adaptive Cruise Control for Autonomous Vehicles
Hybrid Artificial Intelligent Systems
Abstract
This paper presents an intelligent path-following system implemented using deep reinforcement learning based road curvature influence. The vehicle dynamics features are obtained with a 3-DOF vehicle model. The intelligent system consists of a deep ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Engineering Applications of Artificial Intelligence

Engineering Applications of Artificial Intelligence Volume 105, Issue C

Oct 2021

522 pages

ISSN:0952-1976

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 October 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hou XGan MZhang JZhao SJi Y(2023)Vehicle ride comfort optimization in the post-braking phase using residual reinforcement learningAdvanced Engineering Informatics10.1016/j.aei.2023.10219858:COnline publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.aei.2023.102198
Kicki PSkrzypczyński P(2022)Speeding up deep neural network-based planning of local car maneuvers via efficient B-spline path construction2022 International Conference on Robotics and Automation (ICRA)10.1109/ICRA46639.2022.9812313(4422-4428)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1109/ICRA46639.2022.9812313

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents