Q-learning based tracking control with novel finite-horizon performance index
Authors: 
Wei Wang, Ke Wang, Zixin Huang, Chaoxu Mu, Haoxian ShiAuthors Info & Claims
References
[1]
Z. Huang, W. Bai, T. Li, Y. Long, C.P. Chen, H. Liang, H. Yang, Adaptive reinforcement learning optimal tracking control for strict-feedback nonlinear systems with prescribed performance, Inf. Sci. 621 (2023) 407–423.
[2]
C. Mu, Y. Zhang, Z. Gao, C. Sun, ADP-based robust tracking control for a class of nonlinear systems with unmatched uncertainties, IEEE Trans. Syst. Man Cybern. Syst. 50 (11) (2020) 4056–4067.
[3]
Y. Pan, S. Fu, J. Wang, W. Zhang, Optimal output tracking of Boolean control networks, Inf. Sci. 626 (2023) 524–536.
[4]
R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction, MIT Press, 2018.
[5]
C.J. Watkins, P. Dayan, Q-learning, Mach. Learn. 8 (3–4) (1992) 279–292.
[6]
M. Li, J. Sun, H. Zhang, Z. Ming, Based on Q-learning optimal tracking control schemes for linear Itô stochastic systems with Markovian jumps, IEEE Trans. Circuits Syst. II, Express Briefs 70 (3) (2023) 1094–1098.
[7]
B. Kiumarsi, F.L. Lewis, H. Modares, A. Karimpour, M.-B. Naghibi-Sistani, Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics, Automatica 50 (4) (2014) 1167–1175.
[8]
C. Mu, Q. Zhao, C. Sun, Z. Gao, An ADDHP-based Q-learning algorithm for optimal tracking control of linear discrete-time systems with unknown dynamics, Appl. Soft Comput. 82 (2019).
[9]
X. Wen, H. Shi, C. Su, X. Jiang, P. Li, J. Yu, Novel data-driven two-dimensional Q-learning for optimal tracking control of batch process with unknown dynamics, ISA Trans. 125 (2022) 10–21.
[10]
S.A.A. Rizvi, A.J. Pertzborn, Z. Lin, Reinforcement learning based optimal tracking control under unmeasurable disturbances with application to HVAC systems, IEEE Trans. Neural Netw. Learn. Syst. 33 (12) (2022) 7523–7533.
[11]
J. Zhao, C. Yang, W. Gao, L. Zhou, Reinforcement learning and optimal setpoint tracking control of linear systems with external disturbances, IEEE Trans. Ind. Inform. 18 (11) (2022) 7770–7779.
[12]
H. Shi, C. Yang, X. Jiang, C. Su, P. Li, Novel two-dimensional off-policy Q-learning method for output feedback optimal tracking control of batch process with unknown dynamics, J. Process Control 113 (2022) 29–41.
[13]
H. Shi, X. Wen, X. Jiang, C. Su, Two-dimensional model-free optimal tracking control for batch processes with packet loss, IEEE Trans. Control Netw. Syst. 10 (2) (2023) 1032–1045.
[14]
D. Wang, J. Ren, M. Ha, Discounted linear Q-learning control with novel tracking cost and its stability, Inf. Sci. 626 (2023) 339–353.
[15]
B. Luo, D. Liu, T. Huang, D. Wang, Model-free optimal tracking control via critic-only Q-learning, IEEE Trans. Neural Netw. Learn. Syst. 27 (10) (2016) 2134–2144.
[16]
S. Song, M. Zhu, X. Dai, D. Gong, Model-free optimal tracking control of nonlinear input-affine discrete-time systems via an iterative deterministic Q-learning algorithm, IEEE Trans. Neural Netw. Learn. Syst. 35 (1) (2024) 999–1012.
[17]
J. Li, Z. Xiao, P. Li, J. Cao, Robust optimal tracking control for multiplayer systems by off-policy Q-learning approach, Int. J. Robust Nonlinear 31 (1) (2021) 87–106.
[18]
Y. Peng, Q. Chen, W. Sun, Reinforcement Q-learning algorithm for H ∞ tracking control of unknown discrete-time linear systems, IEEE Trans. Syst. Man Cybern. Syst. 50 (11) (2020) 4109–4122.
[19]
Q. Wei, R. Song, B. Li, X. Lin, Self-Learning Optimal Control of Nonlinear Systems, Springer, 2018.
[20]
D. Wang, D. Liu, Q. Wei, Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach, Neurocomputing 78 (1) (2012) 14–22.
[21]
A. Heydari, S. Balakrishnan, Fixed-final-time optimal tracking control of input-affine nonlinear systems, Neurocomputing 129 (2014) 528–539.
[22]
C. Li, D. Liu, H. Li, Finite horizon optimal tracking control of partially unknown linear continuous-time systems using policy iteration, IET Control Theory Appl. 9 (12) (2015) 1791–1801.
[23]
R. Song, Y. Xie, Z. Zhang, Data-driven finite-horizon optimal tracking control scheme for completely unknown discrete-time nonlinear systems, Neurocomputing 356 (2019) 206–216.
[24]
H. Zhang, X. Cui, Y. Luo, H. Jiang, Finite-horizon H ∞ tracking control for unknown nonlinear systems with saturating actuators, IEEE Trans. Neural Netw. Learn. Syst. 29 (4) (2018) 1200–1212.
[25]
C. Possieri, G.P. Incremona, G.C. Calafiore, A. Ferrara, An iterative data-driven linear quadratic method to solve nonlinear discrete-time tracking problems, IEEE Trans. Autom. Control 66 (11) (2021) 5514–5521.
[26]
W. Wang, X. Xie, C. Feng, Model-free finite-horizon optimal tracking control of discrete-time linear systems, Appl. Math. Comput. 433 (2022).
[27]
C. Li, J. Ding, F.L. Lewis, T. Chai, A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systems, Automatica 129 (2021).
[28]
D. Simon, Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches, John Wiley & Sons, 2006.
Index Terms
- Q-learning based tracking control with novel finite-horizon performance index
Index terms have been assigned to the content through auto-classification.
Recommendations
Discounted linear Q-learning control with novel tracking cost and its stability
AbstractIn this article, in order to achieve optimal tracking control of unknown linear discrete systems, a model-free scheme based on Q-learning is established online. First, we introduce an innovative performance index function, so as to eliminate the ...
Computing a Classic Index for Finite-Horizon Bandits
This paper considers the efficient exact computation of the counterpart of the Gittins index for a finite-horizon discrete-state bandit, which measures for each initial state the average productivity, given by the maximum ratio of expected total ...
Model-free finite-horizon optimal tracking control of discrete-time linear systems
Highlights- In this paper, the finite horizon linear quadratic tracking problem transforms to the finite-horizon linear quadratic regulator problem with the help of the ...
AbstractConventionally, the finite-horizon linear quadratic tracking (FHLQT) problem relies on solving the time-varying Riccati equations and the time-varying non-causal difference equations as the system dynamics is known. In this paper, with ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Elsevier Inc.
Publisher
Elsevier Science Inc.
United States
Publication History
Published: 01 October 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025