Abstract
Planning and decision-making of autonomous driving is an active and challenging topic currently. Deep reinforcement learning-based approaches seek to solve the problem in an end-to-end manner, but generally require a large amount of sample data and confronted with high dimensionality of input data and complex models, which lead to slow convergence and cannot learn effectively with noisy data. Most of deep reinforcement learning-based approaches use a sample reward function. Due to the complicated and volatile traffic scenarios, these approaches cannot satisfy the driving policy requirement. To address the issues, a multi-sensing and multi-constraint reward function (MSMC-SAC) based deep reinforcement learning method is proposed. The inputs of the proposed method include front-view image, point cloud from LiDAR, as well as the bird's-eye view generated from the perception results. The multi-sensing input is first passed to an encoding network to obtain the representation in latent space and then forward to a SAC-based learning module. A multiple rewards function considering various constraints, such as the error of transverse-longitudinal distance and heading angle, smoothness, velocity, and the possibility of collision, is designed. The performance of the proposed method in different typical traffic scenarios is validated with CARLA [1]. The effects of multiple reward functions are compared. The simulation results show that the presented approach can learn the driving policies in many complex scenarios, such as straight ahead, passing the intersections, and making turning, and outperforms against the existing typical deep reinforcement learning methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Dosovitskiy, A., Ros, G., Codevilla, F., et al.: CARLA: an open urban driving simulator. arXiv preprint arXiv:1711.03938 (2017)
Silver, D., Bagnell, J.A., Stentz, A.: Learning from demonstration for autonomous navigation in complex unstructured terrain. Int. J. Robot. Res. 29(12), 1565–1592 (2010)
Paden, B., Čáp, M., Yong, S.Z., et al.: A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Trans. Intell. Vehicles 1(1), 33–55 (2016)
Ziegler, J., Bender, P., Schreiber, M., et al.: Making bertha drive-an autonomous journey on a historic route. IEEE Intell. Transp. Syst. Mag. 6(2), 8–20 (2014)
Mnih, V., Kavukcuoglu, K., Silver, D., et al.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)
Lange, S., Riedmiller, M., Voigtländer, A.: Autonomous reinforcement learning on raw visual input data in a real world application. In: The 2012 international joint conference on neural networks (IJCNN), pp. 1–8. IEEE (2012)
Yu, A., Palefsky-Smith, R., Bedi, R.: Deep reinforcement learning for simulated autonomous vehicle control. In: Course Project Reports: Winter, pp. 1–7 (2016)
Sallab, A.E., Abdou, M., Perot, E., et al.: End-to-end deep reinforcement learning for lane keeping assist. arXiv preprint arXiv:1612.04340 (2016)
Kendall, A., Hawke, J., Janz, D., et al.: Learning to drive in a day. In: International Conference on Robotics and Automation (ICRA), pp. 8248–8254. IEEE (2019)
Haarnoja, T., Zhou, A., Abbeel, P., et al.: Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International Conference on Machine Learning, pp. 1861–1870. PMLR (2018)
Bansal, M., Krizhevsky, A., Ogale, A.: Chauffeurnet: learning to drive by imitating the best and synthesizing the worst. arXiv preprint arXiv:1812.03079 (2018)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Woo, J., Yu, C., Kim, N.: Deep reinforcement learning-based controller for path following of an unmanned surface vehicle. Ocean Eng. 183, 155–166 (2019)
Nelson, D.R., Barber, D.B., McLain, T.W., et al.: Vector field path following for miniature air vehicles. IEEE Trans. Rob. 23(3), 519–529 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Z., Wang, H., Cui, X., Zheng, C. (2021). A Multi-sensing Input and Multi-constraint Reward Mechanism Based Deep Reinforcement Learning Method for Self-driving Policy Learning. In: Liu, XJ., Nie, Z., Yu, J., Xie, F., Song, R. (eds) Intelligent Robotics and Applications. ICIRA 2021. Lecture Notes in Computer Science(), vol 13016. Springer, Cham. https://doi.org/10.1007/978-3-030-89092-6_63
Download citation
DOI: https://doi.org/10.1007/978-3-030-89092-6_63
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89091-9
Online ISBN: 978-3-030-89092-6
eBook Packages: Computer ScienceComputer Science (R0)