Abstract
The paper presents a two-layered system for (1) learning and encoding a periodic signal without any knowledge on its frequency and waveform, and (2) modulating the learned periodic trajectory in response to external events. The system is used to learn periodic tasks on a humanoid HOAP-2 robot. The first layer of the system is a dynamical system responsible for extracting the fundamental frequency of the input signal, based on adaptive frequency oscillators. The second layer is a dynamical system responsible for learning of the waveform based on a built-in learning algorithm. By combining the two dynamical systems into one system we can rapidly teach new trajectories to robots without any knowledge of the frequency of the demonstration signal. The system extracts and learns only one period of the demonstration signal. Furthermore, the trajectories are robust to perturbations and can be modulated to cope with a dynamic environment. The system is computationally inexpensive, works on-line for any periodic signal, requires no additional signal processing to determine the frequency of the input signal and can be applied in parallel to multiple dimensions. Additionally, it can adapt to changes in frequency and shape, e.g. to non-stationary signals, such as hand-generated signals and human demonstrations.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Buchli, J., & Ijspeert, A. (2004). A simple, adaptive locomotion toy-system. In S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam, & J. Meyer (Eds.), From animals to animats 8. Proceedings of the eighth international conference on the simulation of adaptive behavior (SAB’04) (pp. 153–162). Cambridge: MIT.
Buchli, J., Righetti, L., & Ijspeert, A. (2006). Engineering entrainment and adaptation in limit cycle systems—from biological inspiration to applications in robotics. Biological Cybernetics, 95(6), 645–664.
Bullock, D., & Grossberg, S. (1989). VITE and FLETE: neural modules for trajectory formation and postural control. In W. Hersberger (Ed.), Volitional control (pp. 253–297). Amsterdam: Elsevier.
Calinon, S., Guenter, F., & Billard, A. (2007). On learning, representing and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics, Part B, Special issue on robot learning by observation, demonstration and imitation, 37(2), 286–298.
Crespi, A., Badertscher, A., & Guignard, A. (2005). AmphiBot I: An amphibious snake-like robot. Robotics and Autonomous Systems, 50(4), 163–175.
Drumwright, E., Jenkins, O. C., & Mataric, M. (2004). Exemplar-based primitives for humanoid movement classification and control. In Proceedings of the 2004 IEEE international conference on robotics and automation (ICRA 2004), pp. 140–145.
Fod, A., Mataric, M., & Jenkins, O. C. (2002). Automated derivation of primitives for movement classification. Autonomous Robots, 12(1), 39–54.
Gams, A., Zlajpah, L., & Lenarcic, J. (2007). Imitating human acceleration of a gyroscopic device. Robotica, 25(4), 501–509.
Grimes, D., Rashid, D., & Rao, R. (2006). Learning nonparametric models for probabilistic imitation. In NIPS, pp. 521–528.
Guenter, F., Hersch, M., Calinon, S., & Billard, A. (2007). Reinforcement learning for imitating constrained reaching movements. RSJ Advanced Robotics, Special Issue on Imitative Robots, 21(13), 1521–1544.
Harris, C. M., & Wolpert, D. M. (1998). Signal-dependent noise determines motor planning. Nature, 394(6695), 780–784. doi:10.1038/29528.
Hersch, M., Guenter, F., Calinon, S., & Billard, A. (2008). Dynamical system modulation for robot learning via kinesthetic demonstrations. IEEE Transactions on Robotics.
Ijspeert, A., Hallam, J., & Willshaw, D. (1999). Evolving swimming controllers for a simulated lamprey with inspiration from neurobiology. Adaptive Behavior, 7(2), 151–172.
Ijspeert, A., Nakanishi, J. & Schaal, S. (2002a). Learning attractor landscapes for learning motor primitives. In Advances in neural information processing systems 15 (NIPS2002).
Ijspeert, A., Nakanishi, J., & Schaal, S. (2002b). Learning rhythmic movements by demonstration using nonlinear oscillators. In Proceedings of the IEEE/RSJ Int. Conference on Intelligent Robots and Systems (IROS2002) (pp. 958–963).
Ijspeert, A., Nakanishi, J., & Schaal, S. (2002c). Movement imitation with nonlinear dynamical systems in humanoid robots. In IEEE International Conference on Robotics and Automation (ICRA2002) (pp. 1398–1403).
Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: a review. Neural Networks, 21(4), 642–653.
Inamura, T., Toshima, I., Tanie, H., & Nakamura, Y. (2004). Embodied symbol emergence based on mimesis theory. International Journal of Robotic Research, 23(4–5), 363–377.
Kawamura, S., & Fukao, N. (1994). Interpolation for input torque patterns obtained through learning control. In Proceedings of the third international conference on automation, robotics and computer vision (ICARCV’94).
Kawato, M. (1996). Trajectory formation in arm movements: minimization principles and procedures. In H. Zelaznik (Ed.), Advance in motor learning and control (pp. 225–259). Champaign: Human Kinetics Publisher.
Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. International Journal of Robotic Research, 5(1), 90–98.
Li, P., & Horowitz, R. (1999). Passive velocity field control of mechanical manipulators. IEEE Transactions on Robotics and Automation, 15(4), 751–763.
Ljung, L., & Söderström, T. (1986). Theory and Practice of Recursive Identification. Cambridge: MIT.
Maass, W., Natschläger, T., & Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531–2560.
Mataric, M. (1998). Behavior-based robotics as a tool for synthesis of artificial behavior and analysis of natural behavior. Trends in Cognitive Science, 2(3), 82–87.
Mezger, J. W. I., & Giese, M. (2005). Trajectory synthesis by hierarchical spatio-temporal correspondence: comparison of different methods. In APGV ’05: Proceedings of the 2nd symposium on Applied perception in graphics and visualization (pp. 25–32). New York: ACM.
Miyamoto, H., Schaal, S., Gandolfo, F., Koike, Y., Osu, R., Nakano, E., Wada, Y., & Kawato, M. (1996). A Kendama learning robot based on bi-directional theory. Neural Networks, 9, 1281–1302.
Mussa-Ivaldi, F. (1997). Nonlinear force fields: a distributed system of control primitives for representing and learning movements. In IEEE International Symposium on Computational Intelligence in Robotics and Automation (pp. 84–90). Los Alamitos: IEEE, Computer Society.
Nishii, J. (1998). A learning model for oscillatory networks. Neural Networks, 11(2), 249–257.
Okada, M., Tatani, K., & Nakamura, Y. (2002). Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion. In Proceedings of ICRA 2002 (Vol. 2, pp. 1410–1415). IEEE.
Paine, R. W., & Tani, J. (2004). Motor primitive and sequence self-organization in a hierarchical recurrent neural network. Neural Networks, 17, 1291–1309.
Righetti, L., & Ijspeert, A. (2006). Programmable central pattern generators: an application to biped locomotion control. In Proceedings of the 2006 IEEE international conference on robotics and automation (pp. 1585–1590).
Righetti, L., Buchli, J., & Ijspeert, A. (2006). Dynamic Hebbian learning in adaptive frequency oscillators. Physica D, 216(2), 269–281.
Rohrer, B., & Hogan, N. (2003). Avoiding spurious submovement decompositions: a globally optimal algorithm. Biological Cybernetics, 89(3), 190–199.
Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3, 233–242.
Schaal, S., & Atkeson, C. (1998). Constructive incremental learning from only local information. Neural Computation, 10, 2047–2084.
Schaal, S., Mohajerian, P., & Ijspeert, A. (2007). Dynamics systems vs. optimal control a unifying view. Progress in Brain Research, 165, 425–445.
Simard, P., & LeCun, Y. (1991). Reverse TDNN: an architecture for trajectory generation. In NIPS (pp. 579–588).
Tani, J., & Ito, M. (2003). Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment. IEEE Transactions on Systems, Man, and Cybernetics, 33(4), 481–488.
Tsuji, T., Tanaka, Y., Morasso, P., Sanguineti, V., & Kaneko, M. (2002). Bio-mimetic trajectory generation of robots via artificial potential field with time base generator. Systems, Man and Cybernetics, Part C, IEEE Transactions on, 32(4), 426–439.
Ude, A., Riley, M., Nemec, B., Kos, A., Asfour, T., & Cheng, G. (2007). Synthesizing goal-directed actions from a library of example movements. In IEEE-RAS/RSJ international conference on humanoid robots (Humanoids 2007).
Zegers, P., & Sundareshan, M. (2003). Trajectory generation and modulation using dynamic neural networks. IEEE Transactions on Neural Networks, 14(3), 520–533.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gams, A., Ijspeert, A.J., Schaal, S. et al. On-line learning and modulation of periodic movements with nonlinear dynamical systems. Auton Robot 27, 3–23 (2009). https://doi.org/10.1007/s10514-009-9118-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10514-009-9118-y