Abstract
This article presents an algorithm that combines a FAST-based algorithm (Flexible Adaptable-Size Topology), called ARM, and Q-learning algorithm. The ARM is a self organizing architecture. Dynamically adjusting the size of sensitivity regions of each neuron and adaptively pruning one of the redundant neurons, the ARM can preserve resources (available neurons) to accommodate more categories. The Q-learning is a dynamic programming-based reinforcement learning method, in which the learned action-value function, Q, directly approximates Q*, the optimal action-value function, independent of the policy being followed. In the proposed method, the ARM acts as a cluster to categorize input vectors from the outside world. Clustered results are then sent to the Q-learning architecture in order that it learns to present the best actions to the outside world. The effect of the algorithm is shown through computer simulations of the well-known control of balancing an inverted pendulum on a cart.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Watkins, C.J.C.H., Dayan, P.: Technical Note: Q-Learning, Machine Learning, vol. 8(3-4), pp. 279–292 (1992)
Claude, F., Touzet,: Q-Learning for Robot. In: Arbib, M.A. (edr.), Handbook of Brain Theory and Neural Networks, pp. 934–937(2003)
Parker, L.E., Touzet, C., Fernandez, F.: Techniques for Learning in Multi-robot Teams. In: Balch, T., Parker, L.E. (eds.) Robot Teams: From Diversity to Polymorphism, Peters, A.K, Natick, MA (2001)
Hwang, K.S., Tan, S.W., Chen, C.C.: Cooperative Strategy Based on Adaptive Q-learning for Robot Soccer Systems, IEEE Transactions on Fuzzy Systems, Vol. 12, Issue: 4, (2004) 569-576
Carpenter,G.A., Grossberg ,S.:.The ART of Adaptive Pattern Recognition by A Self-organizing Neural Network, IEEE Computer, 21(3), (1988)77-88
Albus ,J. S.: A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC), Trans. ASME, J. Dynamic Syst. Meas., Contr., Vol. 97, (1975) 220-227
Pérez-Uribe,A.:Structure-adaptable Digital Neural Networks, PhD Thesis 2052, Swiss Federal Institute of Technology-Lausanne, Lausanne(1999)
Perez,A., Sanchez ,E.:The FAST Architecture: A Neural Network with Flexible Adaptable-size Topology, Proceedings of Fifth International Conference on Microelectronics for Neural Networks, 12-14, (1996) 337-340
Alpaydin,A.E.:Neural Models of Incremental Supervised and Unsupervised Learning. PhD thesis, Swiss Federal Institute of Technology-Lausanne, Lausanne, DPFL, Thesis 863(1990).
Barto,Andrew.G.,Sutton,Richard.S.,Anderson,Charles.W.:Neuronlike Adaptive Elements that Can Solve Difficult Learning Control Problems,IEEE Transactions on System, Man, and Cybernetics SMC-13 (1983)834-846
Peng, J., Wiliams, R.J.: Incremental Multi-Step Q Learning, Machine Learning, 22, (1996) 283-290
Sutton, R.S., Barto, A.G.: Reinforcement Learning An Introduction, Cambridge, Mass., MIT Press(1998)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hsu, YP., Hwang, KS., Lin, HY. (2007). An ARM-Based Q-Learning Algorithm. In: Huang, DS., Heutte, L., Loog, M. (eds) Advanced Intelligent Computing Theories and Applications. With Aspects of Contemporary Intelligent Computing Techniques. ICIC 2007. Communications in Computer and Information Science, vol 2. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74282-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-74282-1_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74281-4
Online ISBN: 978-3-540-74282-1
eBook Packages: Computer ScienceComputer Science (R0)