Q-learning based tracking control with novel finite-horizon performance index
Authors: 
Wei Wang, Ke Wang, Zixin Huang, Chaoxu Mu, Haoxian ShiAuthors Info & Claims
References
[1]
Z. Huang, W. Bai, T. Li, Y. Long, C.P. Chen, H. Liang, H. Yang, Adaptive reinforcement learning optimal tracking control for strict-feedback nonlinear systems with prescribed performance, Inf. Sci. 621 (2023) 407–423.
[2]
C. Mu, Y. Zhang, Z. Gao, C. Sun, ADP-based robust tracking control for a class of nonlinear systems with unmatched uncertainties, IEEE Trans. Syst. Man Cybern. Syst. 50 (11) (2020) 4056–4067.
[3]
Y. Pan, S. Fu, J. Wang, W. Zhang, Optimal output tracking of Boolean control networks, Inf. Sci. 626 (2023) 524–536.
[4]
R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction, MIT Press, 2018.
[5]
C.J. Watkins, P. Dayan, Q-learning, Mach. Learn. 8 (3–4) (1992) 279–292.
[6]
M. Li, J. Sun, H. Zhang, Z. Ming, Based on Q-learning optimal tracking control schemes for linear Itô stochastic systems with Markovian jumps, IEEE Trans. Circuits Syst. II, Express Briefs 70 (3) (2023) 1094–1098.
[7]
B. Kiumarsi, F.L. Lewis, H. Modares, A. Karimpour, M.-B. Naghibi-Sistani, Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics, Automatica 50 (4) (2014) 1167–1175.
[8]
C. Mu, Q. Zhao, C. Sun, Z. Gao, An ADDHP-based Q-learning algorithm for optimal tracking control of linear discrete-time systems with unknown dynamics, Appl. Soft Comput. 82 (2019).
[9]
X. Wen, H. Shi, C. Su, X. Jiang, P. Li, J. Yu, Novel data-driven two-dimensional Q-learning for optimal tracking control of batch process with unknown dynamics, ISA Trans. 125 (2022) 10–21.
[10]
S.A.A. Rizvi, A.J. Pertzborn, Z. Lin, Reinforcement learning based optimal tracking control under unmeasurable disturbances with application to HVAC systems, IEEE Trans. Neural Netw. Learn. Syst. 33 (12) (2022) 7523–7533.
[11]
J. Zhao, C. Yang, W. Gao, L. Zhou, Reinforcement learning and optimal setpoint tracking control of linear systems with external disturbances, IEEE Trans. Ind. Inform. 18 (11) (2022) 7770–7779.
[12]
H. Shi, C. Yang, X. Jiang, C. Su, P. Li, Novel two-dimensional off-policy Q-learning method for output feedback optimal tracking control of batch process with unknown dynamics, J. Process Control 113 (2022) 29–41.
[13]
H. Shi, X. Wen, X. Jiang, C. Su, Two-dimensional model-free optimal tracking control for batch processes with packet loss, IEEE Trans. Control Netw. Syst. 10 (2) (2023) 1032–1045.
[14]
D. Wang, J. Ren, M. Ha, Discounted linear Q-learning control with novel tracking cost and its stability, Inf. Sci. 626 (2023) 339–353.
[15]
B. Luo, D. Liu, T. Huang, D. Wang, Model-free optimal tracking control via critic-only Q-learning, IEEE Trans. Neural Netw. Learn. Syst. 27 (10) (2016) 2134–2144.
[16]
S. Song, M. Zhu, X. Dai, D. Gong, Model-free optimal tracking control of nonlinear input-affine discrete-time systems via an iterative deterministic Q-learning algorithm, IEEE Trans. Neural Netw. Learn. Syst. 35 (1) (2024) 999–1012.
[17]
J. Li, Z. Xiao, P. Li, J. Cao, Robust optimal tracking control for multiplayer systems by off-policy Q-learning approach, Int. J. Robust Nonlinear 31 (1) (2021) 87–106.
[18]
Y. Peng, Q. Chen, W. Sun, Reinforcement Q-learning algorithm for H ∞ tracking control of unknown discrete-time linear systems, IEEE Trans. Syst. Man Cybern. Syst. 50 (11) (2020) 4109–4122.
[19]
Q. Wei, R. Song, B. Li, X. Lin, Self-Learning Optimal Control of Nonlinear Systems, Springer, 2018.
[20]
D. Wang, D. Liu, Q. Wei, Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach, Neurocomputing 78 (1) (2012) 14–22.
[21]
A. Heydari, S. Balakrishnan, Fixed-final-time optimal tracking control of input-affine nonlinear systems, Neurocomputing 129 (2014) 528–539.
[22]
C. Li, D. Liu, H. Li, Finite horizon optimal tracking control of partially unknown linear continuous-time systems using policy iteration, IET Control Theory Appl. 9 (12) (2015) 1791–1801.
[23]
R. Song, Y. Xie, Z. Zhang, Data-driven finite-horizon optimal tracking control scheme for completely unknown discrete-time nonlinear systems, Neurocomputing 356 (2019) 206–216.
[24]
H. Zhang, X. Cui, Y. Luo, H. Jiang, Finite-horizon H ∞ tracking control for unknown nonlinear systems with saturating actuators, IEEE Trans. Neural Netw. Learn. Syst. 29 (4) (2018) 1200–1212.
[25]
C. Possieri, G.P. Incremona, G.C. Calafiore, A. Ferrara, An iterative data-driven linear quadratic method to solve nonlinear discrete-time tracking problems, IEEE Trans. Autom. Control 66 (11) (2021) 5514–5521.
[26]
W. Wang, X. Xie, C. Feng, Model-free finite-horizon optimal tracking control of discrete-time linear systems, Appl. Math. Comput. 433 (2022).
[27]
C. Li, J. Ding, F.L. Lewis, T. Chai, A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systems, Automatica 129 (2021).
[28]
D. Simon, Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches, John Wiley & Sons, 2006.
Index Terms
- Q-learning based tracking control with novel finite-horizon performance index
Index terms have been assigned to the content through auto-classification.
Recommendations
Discounted linear Q-learning control with novel tracking cost and its stability
AbstractIn this article, in order to achieve optimal tracking control of unknown linear discrete systems, a model-free scheme based on Q-learning is established online. First, we introduce an innovative performance index function, so as to eliminate the ...
Computing a Classic Index for Finite-Horizon Bandits
This paper considers the efficient exact computation of the counterpart of the Gittins index for a finite-horizon discrete-state bandit, which measures for each initial state the average productivity, given by the maximum ratio of expected total ...
Model-free finite-horizon optimal tracking control of discrete-time linear systems
Highlights- In this paper, the finite horizon linear quadratic tracking problem transforms to the finite-horizon linear quadratic regulator problem with the help of the ...
AbstractConventionally, the finite-horizon linear quadratic tracking (FHLQT) problem relies on solving the time-varying Riccati equations and the time-varying non-causal difference equations as the system dynamics is known. In this paper, with ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Elsevier Inc.
Publisher
Elsevier Science Inc.
United States
Publication History
Published: 18 October 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in