Nothing Special   »   [go: up one dir, main page]

skip to main content
article

Dynamical movement primitives: Learning attractor models for motor behaviors

Published: 01 February 2013 Publication History

Abstract

Nonlinear dynamical systems have been used in many disciplines to model complex behaviors, including biological motor control, robotics, perception, economics, traffic prediction, and neuroscience. While often the unexpected emergent behavior of nonlinear systems is the focus of investigations, it is of equal importance to create goal-directed behavior e.g., stable locomotion from a system of coupled oscillators under perceptual guidance. Modeling goal-directed behavior with nonlinear systems is, however, rather difficult due to the parameter sensitivity of these systems, their complex phase transitions in response to subtle parameter changes, and the difficulty of analyzing and predicting their long-term behavior; intuition and time-consuming parameter tuning play a major role. This letter presents and reviews dynamical movement primitives, a line of research for modeling attractor behaviors of autonomous nonlinear dynamical systems with the help of statistical learning techniques. The essence of our approach is to start with a simple dynamical system, such as a set of linear differential equations, and transform those into a weakly nonlinear system with prescribed attractor dynamics by means of a learnable autonomous forcing term. Both point attractors and limit cycle attractors of almost arbitrary complexity can be generated. We explain the design principle of our approach and evaluate its properties in several example applications in motor control and robotics.

References

[1]
Atkeson, C. G., Hale, J., Kawato, M., Kotosaka, S., Pollick, F., Riley, M., et al. (2000). Using humanoid robots to study human behaviour. IEEE Intelligent Systems, 15, 46-56.
[2]
Bernstein, N. A. (1967). The control and regulation of movements. London: Pergamon Press.
[3]
Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008). Robot programming by demonstration. In B. Siciliano & O. Khatib (Eds.), Handbook of robotics. Cambridge, MA: MIT Press.
[4]
Billard, A., & Mataric, M. (2001). Learning human arm movements by imitation: Evaluation of a biologically-inspired architecture. Robotics and Autonomous Systems, 941, 1-16.
[5]
Bishop, C. M. (2006). Pattern recognition and machine learning. New York: Springer.
[6]
Buchli, J., Righetti, L., & Ijspeert, A. J. (2006). Engineering entrainment and adaptation in limit cycle systems--from biological inspiration to applications in robotics. Biological Cybernetics, 95(6), 645-664.
[7]
Bühler, M., & Koditschek, D. E. (1990). From stable to chaotic juggling: Theory, simulation, and experiments. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 845-865). Piscataway, NJ: IEEE.
[8]
Bullock, D., & Grossberg, S. (1989). VITE and FLETE: Neural modules for trajectory formation and postural control. In W. A. Hersberger (Ed.), Volitional control (pp. 253-297). New York: Elsevier.
[9]
Burridge, R. R., Rizzi, A. A., & Koditschek, D. E. (1999). Sequential composition of dynamically dexterous robot behaviors. International Journal of Robotics Research, 18(6), 534-555.
[10]
Chevallereau, C., Westervelt, E. R., & Grizzle, J. W. (2005). Asymptotically stable running for a five-link, four-actuator, planar, bipedal robot. International Journal of Robotics Research, 24(6), 431-464.
[11]
Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience. Cambridge, MA: MIT Press.
[12]
Dijkstra, T. M., Schoner, G., Giese, M. A., & Gielen, C. C. (1994). Frequency dependence of the action-perception cycle for postural control in a moving visual environment: Relative phase dynamics. Biol Cybern, 71(6), 489-501.
[13]
Fajen, B. R., & Warren, W. H. (2003). Behavioral dynamics of steering, obstacle avoidance, and route selection. J. Exp. Psychol. Hum. Percept. Perform., 29(2), 343- 362.
[14]
Flash, T., & Hogan, N. (1985). The coordination of arm movements: An experimentally confirmed mathematical model. Journal of Neuroscience, 5(7), 1688- 1703.
[15]
Flash, T., & Sejnowski, T. (2001).Computational approaches to motor control. Current Opinion in Neurobiology, 11, 655-662.
[16]
Friedland, B. (1986). Control system design: An introduction to state-space methods. New York: McGraw-Hill.
[17]
Gams, A., Ijspeert, A., Schaal, S., & Lenarcic, J. (2009). On-line learning and modulation of periodic movements with nonlinear dynamical systems. Autonomous Robots, 27(1), 3-23.
[18]
Getting, P.A. (1988). Comparative analysis of invertebrate central pattern generators. In A. H. Cohen, S. Rossignol, & S. Grillner (Eds.), Neural control of rhythmic movements in vertebrates (pp. 101-127). New York: Wiley.
[19]
Giszter, S. F., Mussa-Ivaldi, F. A., & Bizzi, E. (1993). Convergent force fields organized in the frog's spinal cord. Journal of Neuroscience, 13, 467-491.
[20]
Gomi, H., & Kawato, M. (1996). Equilibrium-point control hypothesis examined by measured arm stiffness during multijoint movement. Science, 272, 117-220.
[21]
Gomi, H., & Kawato, M. (1997). Human arm stiffness and equilibrium-point trajectory during multi-joint movement. Biol. Cybern., 76(3), 163-171.
[22]
Gribovskaya, E., Khansari-Zadeh, M., & Billard, A. (2010). Learning nonlinear multivariate dynamics of motion in robotic manipulators. International Journal of Robotics Research, 30(1), 80-117.
[23]
Grillner, S. (1981). Control of locomotion in bipeds, tetrapods and fish. In V. B. Brooks (Ed.), Handbook of physiology: The nervous system, vol. 2: Motor control (pp. 1179-1236). Bethesda, MD: American Physiology Society.
[24]
Guckenheimer, J., & Holmes, P. (1983). Nonlinear oscillators, dynamical systems, and bifurcations of vector fields. New York: Springer.
[25]
Haken, H., Kelso, J. A. S., Fuchs, A., & Pandya, A. S. (1990). Dynamic pattern recognition of coordinated biological movement. Neural Networks, 3, 395-401.
[26]
Hatsopoulos, N. G., & Warren, W. H. J. (1996). Resonance tuning in rhythmic arm movements. Journal of Motor Behavior, 28(1), 3-14.
[27]
Hoffmann, H., Pastor, P., Park, D.-H., & Schaal, S. (2009). Biologically-inspired dynamical systems for movement generation: Automatic real-time goal adaptation and obstacle avoidance. In International Conference on Robotics and Automation (pp. 2587-2592). Piscataway, NJ: IEEE.
[28]
Hollerbach, J. M. (1984). Dynamic scaling of manipulator trajectories. Transactions of the ASME, 106, 139-156.
[29]
Ijspeert, A. J. (2001). A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander. Biological Cybernetics, 84(5), 331-348.
[30]
Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: A review. Neural Netw., 21(4), 642-653.
[31]
Ijspeert, A. J., Hallam, J., & Willshaw, D. (1999). Evolving swimming controllers for a simulated lamprey with inspiration from neurobiology. Adaptive Behavior, 7(2), 151-172.
[32]
Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002a). Learning rhythmic movements by demonstration using nonlinear oscillators. In Proceedings of the IEEE/RSJ Int. Conference on Intelligent Robots and Systems (pp. 958-963). Piscataway, NJ: IEEE.
[33]
Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002b). Movement imitation with nonlinear dynamical systems in humanoid robots. In IEEE International Conference on Robotics and Automation (pp. 1398-1403). Piscataway, NJ: IEEE.
[34]
Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2003). Learning control policies for movement imitation and movement recognition. In S. T. S. Becker & K. Obermayer (Eds.), Advances in neural information processing systems, 15 (pp. 1547-1554). Cambridge, MA: MIT Press.
[35]
Jackson, E. A. (1989). Perspectives of nonlinear dynamics. Cambridge: Cambridge University Press.
[36]
Jaeger, H., & Haas, H. (2004). Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science, 304(5667), 78-80.
[37]
Joshi, P., & Maass, W. (2005). Movement generation with circuits of spiking neurons. Neural Computation, 17(8), 1715-1738.
[38]
Kawato, M. (1996). Trajectory formation in armmovements:Minimization principles and procedures. In H. N. Zelaznik (Ed.), Advances in motor learning and control (pp. 225-259). Champaign, IL: Human Kinetics.
[39]
Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behavior. Cambridge, MA: MIT Press.
[40]
Kelso, J. A. S., Scholtz, J. P., & Schoner, G. (1988).Dynamics governs switching among patterns of coordination in biological movement. Physics Letters A, 134(1), 8-12.
[41]
Khansari-Zadeh, S.M., & Billard, A. (2010). Imitation learning of globally stable nonlinear point-to-point robot motions using nonlinear programming. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 2676-2683). Piscataway, NJ: IEEE.
[42]
Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. International Journal of Robotics Research, 5(1), 90-98.
[43]
Klavins, E., & Koditschek, D. (2001). Stability of coupled hybrid oscillators. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 4200-4207). Piscataway, NJ: IEEE.
[44]
Kober, J., & Peters, J. (2009). Learning motor primitives for robotics. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 2112-2118). Piscataway, NJ: IEEE.
[45]
Koditschek, D. E. (1987). Exact robot navigation by means of potential functions: Some topological considerations. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 211-223). Piscataway, NJ: IEEE.
[46]
Kugler, P. N., & Turvey, M. T. (1987). Information, natural law, and the self-assembly of rhythmic movement. Hillsdale, NJ: Erlbaum.
[47]
Kulvicius, T., Ning, K., Tamosiunaite, M., & Worgötter, F. (2012). Joining movement sequences: Modified dynamic movement primitives for robotics applications exemplified on handwriting. IEEE Transactions on Robotics, 28(1), 145-157.
[48]
Latash, M. L. (1993). Control of human movement. Champaign, IL: Human Kinetics.
[49]
Li, P., & Horowitz, R. (1999). Passive velocity field control of mechanical manipulators. IEEE Transactions on Robotics and Automation, 15(4), 751-763.
[50]
Lohmiller, W., & Slotine, J. J. (1998). On contraction analysis for nonlinear systems. Automatica, 34(6), 683-696.
[51]
Maass, W., Natschläger, T., & Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531-2560.
[52]
Matthews, P. C., Mirollo, R. E., & Strogatz, S. H. (1991). Dynamics of a large system of coupled nonlinear oscillators. Physica D, 52, 293-331.
[53]
McCrea, D. A., & Rybak, I. A. (2008). Organization ofmammalian locomotor rhythm and pattern generation. Brain Research Reviews, 57, 134-146.
[54]
Miyamoto, H., Schaal, S., Gandolfo, F., Koike, Y., Osu, R., Nakano, E., et al. (1996). A kendama learning robot based on bi-directional theory. Neural Networks, 9, 1281-1302.
[55]
Mussa-Ivaldi, F. A. (1997). Nonlinear force fields: a distributed system of control primitives for representing and learning movements. In Proceedings of the IEEE International Symposium on Computational Intelligence in Robotics and Automation (pp. 84-90). San Mateo, CA: IEEE Computer Society.
[56]
Mussa-Ivaldi, F. A. (1999). Modular features of motor control and learning. Current Opinion in Neurobiology, 9(6), 713-717.
[57]
Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., & Kawato, M. (2004). Learning from demonstration and adaptation of biped locomotion. Robotics and Autonomous Systems, 47, 79-91.
[58]
Okada, M., Tatani, K., & Nakamura, Y. (2002). Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 1410-1415). Piscataway, NJ: IEEE.
[59]
Paine, R. W., & Tani, J. (2004). Motor primitive and sequence self-organization in a hierarchical recurrent neural network. Neural Networks, 17(8-9), 1291-1309.
[60]
Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009). Learning and generalization of motor skills by learning from demonstration. In International Conference on Robotics and Automation (pp. 763-768). Piscataway, NJ: IEEE.
[61]
Perk, B. E., & Slotine, J. J. E. (2006). Motion primitives for robotic flight control. arXiv:cs/0609140v2 {cs.RO}.
[62]
Peters, J., & Schaal, S. (2008). Reinforcement learning of motor skills with policy gradients. Neural Networks, 21(4), 682-697.
[63]
Pongas, D., Billard, A., & Schaal, S. (2005). Rapid synchronization and accurate phase-locking of rhythmic motor primitives. In Proceedings of the IEEEInternational Conference on Intelligent Robots and Systems (pp. 2911-2916). Piscataway, NJ: IEEE.
[64]
Righetti, L., Buchli, J., & Ijspeert, A. J. (2006). Dynamic Hebbian learning in adaptive frequency oscillators. Physica D, 216(2), 269-281.
[65]
Rimon, E., & Koditschek, D. (1992). Exact robot navigation using artificial potential functions. IEEE Transactions on Robotics and Automation, 8(5), 501-518.
[66]
Rizzi, A. A., & Koditschek, D. E. (1994). Further progress in robot juggling: Solvable mirror laws. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 2935-2940). Piscataway, NJ: IEEE.
[67]
Rizzolatti, G., & Arbib, M. A. (1998). Language within our grasp. Trends Neurosci., 21(5), 188-194.
[68]
Sakoe, H., & Chiba, S. (1987). Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech and Signal Processing, 26(1), 43-49.
[69]
Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6), 233-242.
[70]
Schaal, S., & Atkeson, C. G. (1994). Assessing the quality of learned local models. In J. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 160-167). San Mateo, CA: Morgan Kaufmann.
[71]
Schaal, S., & Atkeson, C. G. (1998). Constructive incremental learning from only local information. Neural Computation, 10(8), 2047-2084.
[72]
Schaal, S., Ijspeert, A., & Billard, A. (2003). Computational approaches to motor learning by imitation. Philosophical Transactions of the Royal Society of London: Series B, Biological Sciences, 358(1431), 537-547.
[73]
Schaal, S., Mohajerian, P., & Ijspeert, A. (2007). Dynamics systems vs. optimal contro--a unifying view. Prog. Brain Res., 165, 425-445.
[74]
Schaal, S., & Sternad, D. (1998). Programmable pattern generators. In Proceedings of the International Conference onComputational Intelligence in Neuroscience (pp. 48-51). Piscataway, NJ: IEEE.
[75]
Schaal, S., Sternad, D., Osu, R., & Kawato, M. (2004). Rhythmic movement is not discrete. Nature Neuroscience, 7(10), 1137-1144.
[76]
Schöner, G. (1990). A dynamic theory of coordination of discrete movement. Biological Cybernetics, 63, 257-270.
[77]
Schöner, G., & Kelso, J. A. S. (1988). Dynamic pattern generation in behavioral and neural systems. Science, 239, 1513-1520.
[78]
Schöner, G., & Santos, C. (2001). Control of movement time and sequential action through attractor dynamics: A simulation study demonstrating object interception and coordination. In Proceedings of the 9th International Symposium on Intelligent Robotic Systems. http://spiderman-2.laas.fr/sirs2001/proceedings/
[79]
Sciavicco, L., & Siciliano, B. (2000). Modelling and control of robot manipulators. New York: Springer.
[80]
Scott, A. (2005). Encyclopedia of nonlinear science. New York: Routledge.
[81]
Slotine, J. J. E., & Li, W. (1991). Applied nonlinear control. Upper Saddle River, NJ: Prentice Hall.
[82]
Sternad, D., Amazeen, E., & Turvey, M. (1996). Diffusive, synaptic, and synergetic coupling: An evaluation through inphase and antiphase rhythmic movements. Journal of Motor Behavior, 28, 255-269.
[83]
Strogatz, S.H. (1994). Nonlinear dynamics and chaos:With applications to physics, biology, chemistry, and engineering. Reading, MA: Addison-Wesley.
[84]
Swinnen, S. P., Li, Y., Dounskaia, N., Byblow, W., Stinear, C., & Wagemans, J. (2004). Perception-action coupling during bimanual coordination: The role of visual perception in the coalition of constraints that govern bimanual action. J. Mot. Behav., 36(4), 394-398, 402-407 (discussion 408-417).
[85]
Taga, G., Yamaguchi, Y., & Shimizu, H. (1991). Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment. Biological Cybernetics, 65, 147-159.
[86]
Thelen, E., & Smith, L. B. (1994). A dynamical systems approach to the development of cognition and action. Cambridge, MA: MIT Press.
[87]
Theodorou, E., Buchli, J., & Schaal, S. (2010). Reinforcement learning in high dimensional state spaces: A path integral approach. Journal of Machine Learning Research, 2010(11), 3137-3181.
[88]
Todorov, E. (2004). Optimality principles in sensorimotor control. Nature Neuroscience, 7, 907-915.
[89]
Tsuji, T., Tanaka, Y.,Morasso, P. G., Sanguineti, V., & Kaneko, M. (2002). Biomimetic trajectory generation of robots via artificial potential field with time base generator. IEEE Transactions on Systems, Man, and Cybernetics--Part C, 32(4), 426- 439.
[90]
Turvey, M. T. (1990). Coordination. Am. Psychol., 45(8), 938-953.
[91]
Ude, A., Gams, A., Asfour, T., & Morimoto, J. (2010). Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Transactions on Robotics, 26(5), 800-815.
[92]
Wada, Y., & Kawato, M. (2004). A via-point time optimization algorithm for complex sequential trajectory formation. Neural Networks, 17(3), 353-364.
[93]
Wolpert, D. M. (1997). Computational approaches to motor control. Trends Cogn. Sci., 1(6), 209-216.
[94]
Wyffels, F., & Schrauwen, B. (2009). Design of a central pattern generator using reservoir computing for learning human motion. In ATEQUAL 2009: 2009 EC-SIS Symposium on Advanced Technologies for Enhanced Quality of Life (LABRS and ARTIPED 2009): Proceedings (pp. 118-122). Los Alamitos, CA: IEEE Computer Society.

Cited By

View all
  • (2024)Robot control based on motor primitivesInternational Journal of Robotics Research10.1177/0278364924125878243:12(1959-1991)Online publication date: 1-Oct-2024
  • (2024)A structured prediction approach for robot imitation learningInternational Journal of Robotics Research10.1177/0278364923120465643:2(113-133)Online publication date: 1-Feb-2024
  • (2024)Learning to Assist Different Wearers in Multitasks: Efficient and Individualized Human-in-the-Loop Adaptation Framework for Lower-Limb ExoskeletonIEEE Transactions on Robotics10.1109/TRO.2024.346876840(4699-4718)Online publication date: 1-Jan-2024
  • Show More Cited By
  1. Dynamical movement primitives: Learning attractor models for motor behaviors

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Neural Computation
    Neural Computation  Volume 25, Issue 2
    February 2013
    278 pages

    Publisher

    MIT Press

    Cambridge, MA, United States

    Publication History

    Published: 01 February 2013

    Qualifiers

    • Article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 25 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Robot control based on motor primitivesInternational Journal of Robotics Research10.1177/0278364924125878243:12(1959-1991)Online publication date: 1-Oct-2024
    • (2024)A structured prediction approach for robot imitation learningInternational Journal of Robotics Research10.1177/0278364923120465643:2(113-133)Online publication date: 1-Feb-2024
    • (2024)Learning to Assist Different Wearers in Multitasks: Efficient and Individualized Human-in-the-Loop Adaptation Framework for Lower-Limb ExoskeletonIEEE Transactions on Robotics10.1109/TRO.2024.346876840(4699-4718)Online publication date: 1-Jan-2024
    • (2024)MMP++: Motion Manifold Primitives With Parametric Curve ModelsIEEE Transactions on Robotics10.1109/TRO.2024.344406840(3950-3963)Online publication date: 1-Jan-2024
    • (2024)PRIMP: PRobabilistically-Informed Motion Primitives for Efficient Affordance Learning From DemonstrationIEEE Transactions on Robotics10.1109/TRO.2024.339005240(2868-2887)Online publication date: 16-Apr-2024
    • (2024)ViolinBot: A Framework for Imitation Learning of Violin Bowing Using Fuzzy Logic and PCAIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.340914632:9(5005-5017)Online publication date: 1-Sep-2024
    • (2024)A Framework of Robot Manipulability Learning and Control and Its Application in TeleroboticsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2023.329766532:1(266-280)Online publication date: 1-Jan-2024
    • (2024)Learning periodic skills for robotic manipulationRobotics and Autonomous Systems10.1016/j.robot.2024.104763180:COnline publication date: 1-Oct-2024
    • (2024)Dynamic Global/Local multi-layer motion planner architecture for autonomous Cognitive Surgical RobotsRobotics and Autonomous Systems10.1016/j.robot.2024.104758180:COnline publication date: 1-Oct-2024
    • (2024)MT-RSLRobotics and Computer-Integrated Manufacturing10.1016/j.rcim.2024.10281790:COnline publication date: 1-Dec-2024
    • Show More Cited By

    View Options

    View options

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media