Abstract
In this paper, we propose Q-learning with adaptive state space construction. This provides an efficient method to construct the state space suitable for Q-learning to accomplish the task in continuous sensor space. In the proposed algorithm, a robot starts with single state covering whole sensor space. A new state is generated incrementally by segmenting a sub-region of the sensor space or combining the existing states. The criterion for incremental segmentation and combination is derived from Q-learning algorithm. Simulation results show that the proposed algortithm is able to construct the sensor space effectively to accomplish the task. The resulting state space reveals the sensor space in a Voronoi tessellation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Whitehead, S. D., Ballard, D.H.: Learning to perceive and act by trial and error. ML 7 (1991) 45–83
Chrisman, L.: Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach. Proc. of AAAI-92 (1992) 183–188
Tan, M.: Cost-sensitive reinforcement learning for adaptive classification and control. Proc. of AAAI-91 (1991)
Krose, B.J.A., van Dam, J.W.M.: Adaptive state space quantisation for reinforcement learning of collision-free navigation. Proc. of IROS 2 (1992) 1327–1331
Dubrawski, A., Reignier, P.: Learning to Categorize Perceptual Space of a Mobile Robot Using Fuzzy-ART Neural Network. Proc. of IROS 2 (1994) 1272–1277
Takahashi, Y., Asada, M., Hosoda, K.: Reasonable Performance in Less Learning Time by Real Robot Based on Incremental State Space Segmentation. Proc. of IROS 3 (1996) 1518–1524
Ishiguro, H., Sato, R., Ishida, T.: Robot Oriented State Space Construction. Proc. of IROS 3 (1996) 1496–1501.
Chapman, D., Kaebling, L.P.: Input Generalisation in Delayed Reinforcement Learning: an Algorithm and Performance Comparisons. Proc. of IJCAI-91 (1991) 726–731
Munos, R., Patinel, J.: Reinforcement learning with dynamic covering of state-action space: Partitioning Q-learning. Proc. of SAB (1994) 354–363
Murao, H., Kitamura, S.: Q-Learning with Adaptive State Segmentation (QLASS). Proc. of CIRA (1997) 179–184
Watkins, C: Learning from Delayed Rewards. Ph.D. Dissertation of Cambridge University (1989)
Naito, T., Odagiri, R., Matsunaga, Y., Tanifuji, M., Murase, K.: Genetic evolution of a logic circuit which controls an autonomous mobile robot. Proc. of ICES96 (1996)
MaCallum, R.A.: Overcoming Incomplete Perception with Utile Distinction Memory. Proc. of ICML (1993) 190–196
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Murao, H., Kitamura, S. (1998). Q-Learning with Adaptive State Space Construction. In: Birk, A., Demiris, J. (eds) Learning Robots. EWLR 1997. Lecture Notes in Computer Science(), vol 1545. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49240-2_2
Download citation
DOI: https://doi.org/10.1007/3-540-49240-2_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65480-3
Online ISBN: 978-3-540-49240-5
eBook Packages: Springer Book Archive