research-article

Robust tracking control of uncertain nonlinear systems with adaptive dynamic programming

Authors:

Guanbin GaoAuthors Info & Claims

Volume 471, Issue C

Pages 21 - 30

https://doi.org/10.1016/j.neucom.2021.10.081

Published: 30 January 2022 Publication History

Abstract

Although robust regulation problem has been well studied, solving robust tracking control via online learning has not been fully solved, in particular for nonlinear systems. This paper develops an online adaptive learning technique to complete the robust tracking control design for nonlinear uncertain systems, which uses the ideas of adaptive dynamic programming (ADP) proposed for optimal control. An augmented system is first constructed using the tracking error and reference trajectory, so as to reformulate the tracking control into a modified robust regulation problem. Then, an equivalence between the robust control and the optimal control is established by using a constructive discounted cost function, which allows to design the robust control by tackling the optimal control of its nominal system. Then, the derived Hamilton–Jacobi-Bellman (HJB) equation is solved by training a critic neural network (NN). Finally, an adaptive learning algorithm is adopted to online directly update the unknown NN weights, where the convergence can be guaranteed. The closed-loop system stability is rigorously proved and extensive simulation results are given to show the effectiveness of the developed learning algorithm.

References

[1]

T. Başar, P. Bernhard, H-infinity optimal control and related minimax design problems: a dynamic game approach, Springer Science & Business Media, Berlin, Germany, 2008.

[2]

F. Lin, R.D. Brandt, J. Sun, Robust control of nonlinear systems: compensating for uncertainty, Int. J. Control 56 (6) (1992) 1453–1459.

[3]

K. Zhou, J. Doyle, Essentials of robust control, Prentice hall NJ, USA, 1997.

[4]

F. Lin, An optimal control approach to robust control design, Int. J. Control 73 (3) (2000) 177–186.

[5]

Q. Chen, S. Xie, M. Sun, X. He, Adaptive nonsingular fixed-time attitude stabilization of uncertain spacecraft, IEEE Trans. Aerosp. Electron. Syst. 54 (6) (2018) 2937–2950.

[6]

S. Wang, H. Yu, J. Yu, J. Na, X. Ren, Neural-network-based adaptive funnel control for servo mechanisms with unknown dead-zone, IEEE Trans. Cybern. 50 (4) (2019) 1383–1394.

[7]

M. Chen, S.S. Ge, Adaptive neural output feedback control of uncertain nonlinear systems with unknown hysteresis using disturbance observer, IEEE Trans. Industr. Electron. 62 (12) (2015) 7706–7716.

[8]

P. Gahinet, A. Nemirovskii, A.J. Laub, M. Chilali, redThe LMI control toolbox, in: Proceedings of 1994 33rd IEEE Conference on Decision and Control, 3, 1994, pp. 2038–2041.

[9]

F. Lin, Robust control design: an optimal control approach, John Wiley & Sons Ltd, Chichester, England, 2007.

[10]

F.L. Lewis, D. Vrabie, Reinforcement learning and adaptive dynamic programming for feedback control, IEEE Circuits Syst. Mag. 9 (3) (2009) 32–50.

Digital Library

[11]

P.J. Werbos, Approximate dynamic programming for real-time control and neural modeling, Handbook of Intelligent Control Neural Fuzzy & Adaptive Approaches, D.A.White and D.A. Sofge, Eds (1992) 493–525.

[12]

D. Liu, Q. Wei, D. Wang, X. Yang, H. Li, Adaptive Dynamic Programming With Applications in Optimal Control, Cham, Springer, Switzerland, 2017.

[13]

J. Na, H. Guido, Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous-time unknown nonlinear systems, IEEE/CAA J. Automatica Sinica 1 (4) (2014) 412–422.

[14]

H. Dong, X. Zhao, B. Luo, redOptimal tracking control for uncertain nonlinear systems with prescribed performance via critic-only ADP, IEEE Transactions on Systems, Man, and Cybernetics: Systems (2020(In press)) 1–13.

[15]

Y. Lv, X. Ren, J. Na, Adaptive optimal tracking controls of unknown multi-input systems based on nonzero-sum game theory, J. Franklin Inst. 356 (15) (2019) 8255–8277.

[16]

Y. Lv, X. Ren, Approximate nash solutions for multiplayer mixed-zero-sum game with reinforcement learning, IEEE Trans. Syst. Man Cybern. Syst. 48 (12) (2018) 2739–2750.

[17]

J. Na, B. Wang, G. Li, S. Zhan, W. He, Nonlinear constrained optimal control of wave energy converters with adaptive dynamic programming, IEEE Trans. Industr. Electron. 66 (10) (2019) 7904–7915.

[18]

H. Ren, Y. Wen, C. Liu, Integral reinforcement learning off-policy method for solving nonlinear multi-player nonzero-sum games with saturated actuator, Neurocomputing 335 (2019) 96–104.

[19]

Y. Jiang, Z.-P. Jiang, Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics, Automatica 48 (10) (2012) 2699–2704.

Digital Library

[20]

Y. Jiang, Z.-P. Jiang, Robust adaptive dynamic programming and feedback stabilization of nonlinear systems, IEEE Trans. Neural Networks Learning Systems 25 (5) (2014) 882–893.

[21]

C. Wei, J. Luo, H. Dai, G. Duan, redLearning-based adaptive attitude control of spacecraft formation With guaranteed prescribed performance, IEEE Trans. Cybern. 49 (11) (2019) 4004–4016.

[22]

X. Yang, H. He, Adaptive critic designs for event-triggered robust control of nonlinear systems with unknown dynamics, IEEE Trans. Cybern. 49 (6) (2019) 2255–2267.

[23]

D. Wang, D. Liu, C. Mu, Y. Zhang, Neural network learning and robust stabilization of nonlinear systems with dynamic uncertainties, IEEE Trans. Neural Networks Learn. Syst. 29 (4) (2018) 1342–1351.

[24]

D. Liu, D. Wang, F.Y. Wang, H. Li, X. Yang, Neural-network-based online hjb solution for optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems, IEEE Trans. Cybern. 44 (12) (2014) 2834–2847.

[25]

J. Zhao, J. Na, G. Gao, Adaptive dynamic programming based robust control of nonlinear systems with unmatched uncertainties, Neurocomputing 395 (2020) 56–65.

[26]

H. Zhang, L. Cui, X. Zhang, Y. Luo, Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method, IEEE Trans. Neural Networks 22 (12) (2011) 2226–2236.

[27]

H. Modares, F.L. Lewis, Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning, Automatica 50 (7) (2014) 1780–1792.

Digital Library

[28]

A. Heydari, S. Balakrishnan, Fixed-final-time optimal tracking control of input-affine nonlinear systems, Neurocomputing 129 (2014) 528–539.

[29]

D. Wang, D. Liu, Y. Zhang, H. Li, Neural network robust tracking control with adaptive critic framework for uncertain nonlinear systems, Neural Networks 97 (2018) 11–18.

[30]

D. Wang, C. Mu, Adaptive-critic-based robust trajectory tracking of uncertain dynamics and its application to a spring-mass-damper system, IEEE Trans. Industr. Electron. 65 (1) (2018) 654–663.

[31]

D. Wang, L. Cheng, J. Yan, Self-learning robust control synthesis and trajectory tracking of uncertain dynamics, IEEE Trans. Cybern. (2020) 1–9.

[32]

Y. Zhu, D. Zhao, X. Li, Using reinforcement learning techniques to solve continuous-time non-linear optimal tracking problem without system dynamics, IET Control Theory Appl. 10 (12) (2016) 1339–1347.

[33]

F. Luan, J. Na, Y. Huang, G. Gao, Adaptive neural network control for robotic manipulators with guaranteed finite-time convergence, Neurocomputing 337 (2019) 153–164.

[34]

M. Abu-Khalaf, F.L. Lewis, Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network hjb approach, Automatica 41 (5) (2005) 779–791.

Digital Library

[35]

K.G. Vamvoudakis, F.L. Lewis, Online actor-critic Algorithm to solve the continuous-time infinite horizon optimal control problem, Automatica 46 (5) (2010) 878–888.

Digital Library

[36]

F. Ding, X. Liu, J. Chu, redGradient-based and least-squares-based iterative algorithms for Hammerstein systems using the hierarchical identification principle, IET Control Theory Appl. 7 (2) (2013) 176–184.

[37]

J. Na, S. Wang, Y. Liu, Y. Huang, X. Ren, Finite-time convergence adaptive neural network control for nonlinear servo systems, IEEE Trans. Cybern. 50 (6) (2019) 2568–2579.

Cited By

Qin CZhu TJiang KZhang J(2024)Dynamic event-triggered robust safety control for multiplayer fully cooperative games with mismatched uncertainties and asymmetric input constraintsApplied Intelligence10.1007/s10489-023-05233-954:1(749-766)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-023-05233-9
Zhang DWang YJiang KLiang L(2024)Safe optimal robust control of nonlinear systems with asymmetric input constraints using reinforcement learningApplied Intelligence10.1007/s10489-023-05184-154:1(1-13)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-023-05184-1
Peng BCui XZhou K(2024)Event-triggered self-learning-based tracking control for nonlinear constrained-input systems with uncertain disturbancesNeural Computing and Applications10.1007/s00521-024-09442-236:13(7007-7023)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1007/s00521-024-09442-2
Show More Cited By

Index Terms

Robust tracking control of uncertain nonlinear systems with adaptive dynamic programming

Index terms have been assigned to the content through auto-classification.

Recommendations

Neural network robust tracking control with adaptive critic framework for uncertain nonlinear systems

In this paper, we aim to tackle the neural robust tracking control problem for a class of nonlinear systems using the adaptive critic technique. The main contribution is that a neural-network-based robust tracking control scheme is established for ...
Robust Tracking and Model Following Control with Zero Tracking Error for Uncertain Dynamical Systems

The problem of robust tracking and model following for a class of linear dynamical systems with time-varying uncertain parameters and disturbances is considered. A class of continuous (nonlinear) state feedback controllers is proposed for robust ...
Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming

This paper presents an adaptive dynamic programming-based guaranteed cost neural tracking control algorithm for a class of continuous-time matched uncertain nonlinear systems. By introducing an augmented system and employing a modified cost function with ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 471, Issue C

Jan 2022

378 pages

ISSN:0925-2312

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 30 January 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Qin CZhu TJiang KZhang J(2024)Dynamic event-triggered robust safety control for multiplayer fully cooperative games with mismatched uncertainties and asymmetric input constraintsApplied Intelligence10.1007/s10489-023-05233-954:1(749-766)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-023-05233-9
Zhang DWang YJiang KLiang L(2024)Safe optimal robust control of nonlinear systems with asymmetric input constraints using reinforcement learningApplied Intelligence10.1007/s10489-023-05184-154:1(1-13)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-023-05184-1
Peng BCui XZhou K(2024)Event-triggered self-learning-based tracking control for nonlinear constrained-input systems with uncertain disturbancesNeural Computing and Applications10.1007/s00521-024-09442-236:13(7007-7023)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1007/s00521-024-09442-2
Che GYu Z(2023)ADP based output-feedback fault-tolerant tracking control for underactuated AUV with actuators faultsJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-22397645:4(5871-5883)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/JIFS-223976
Zhu QSun HZhao ZLiu YZhao J(2023)Adaptive Optimal Control of Hybrid Electric Vehicle Power Battery via Policy LearningComputational Intelligence and Neuroscience10.1155/2023/82885272023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/8288527
Kozlov OKondratenko YSkakodub O(2022)Information Technology for Parametric Optimization of Fuzzy Systems Based on Hybrid Grey Wolf AlgorithmsSN Computer Science10.1007/s42979-022-01333-43:6Online publication date: 3-Sep-2022
https://dl.acm.org/doi/10.1007/s42979-022-01333-4

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents