research-article

Open access

Learning Visuomotor Policies with Deep Movement Primitives

Authors:

Michail Theofanidis,

Asil Bozcuoglu,

Maria KyrariniAuthors Info & Claims

PETRA '21: Proceedings of the 14th PErvasive Technologies Related to Assistive Environments Conference

Pages 140 - 146

https://doi.org/10.1145/3453892.3453899

Published: 29 June 2021 Publication History

All formats PDF

Abstract

In this paper, we present a novel method to learn end-to-end visuomotor policies for robotic manipulators. The method computes state-action mappings in a supervised learning manner from video demonstrations and robot trajectories. We show that the robot learns to perform different tasks by associating image features with the corresponding movement primitives of different grasp poses. To evaluate the effectiveness of the proposed learning method, we conduct experiments with a PR2 robot in a simulation environment. The purpose of these experiments is to evaluate the system’s ability to perform manipulation tasks.

References

[1]

Brenna D Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. 2009. A survey of robot learning from demonstration. Robotics and autonomous systems 57, 5 (2009), 469–483.

[2]

Aude G Billard, Sylvain Calinon, and Rüdiger Dillmann. 2016. Learning from humans. In Springer handbook of robotics. Springer, 1995–2014.

[3]

Sylvain Calinon, Florent Guenter, and Aude Billard. 2007. On learning, representing, and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 37, 2(2007), 286–298.

Digital Library

[4]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 1126–1135.

Digital Library

[5]

Chelsea Finn and Sergey Levine. 2017. Deep visual foresight for planning robot motion. In 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2786–2793.

Digital Library

[6]

Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, and Sergey Levine. 2017. One-shot visual imitation learning via meta-learning. arXiv preprint arXiv:1709.04905(2017).

[7]

Andrej Gams, Aleš Ude, Jun Morimoto, 2018. Deep encoder-decoder networks for mapping raw images to dynamic movement primitives. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 1–6.

[8]

Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. science 313, 5786 (2006), 504–507.

[9]

Auke Jan Ijspeert, Jun Nakanishi, Heiko Hoffmann, Peter Pastor, and Stefan Schaal. 2013. Dynamical movement primitives: learning attractor models for motor behaviors. Neural computation 25, 2 (2013), 328–373.

[10]

Jens Kober, Katharina Mülling, Oliver Krömer, Christoph H Lampert, Bernhard Schölkopf, and Jan Peters. 2010. Movement templates for learning of hitting and batting. In 2010 IEEE International Conference on Robotics and Automation. IEEE, 853–858.

[11]

Jens Kober, Andreas Wilhelm, Erhan Oztop, and Jan Peters. 2012. Reinforcement learning to adjust parametrized motor primitives to new situations. Autonomous Robots 33, 4 (2012), 361–379.

[12]

Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. 2016. End-to-end training of deep visuomotor policies. The Journal of Machine Learning Research 17, 1 (2016), 1334–1373.

Digital Library

[13]

Sergey Levine, Peter Pastor, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen. 2018. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. The International Journal of Robotics Research 37, 4-5(2018), 421–436.

[14]

Katharina Muelling, Jens Kober, and Jan Peters. 2010. Learning table tennis with a mixture of motor primitives. In Humanoid Robots (Humanoids), 2010 10th IEEE-RAS International Conference on. IEEE, 411–416.

[15]

Katharina Mülling, Jens Kober, Oliver Kroemer, and Jan Peters. 2013. Learning to select and generalize striking movements in robot table tennis. The International Journal of Robotics Research 32, 3 (2013), 263–279.

Digital Library

[16]

Peter Pastor, Heiko Hoffmann, Tamim Asfour, and Stefan Schaal. 2009. Learning and generalization of motor skills by learning from demonstration. In Robotics and Automation, 2009. ICRA’09. IEEE International Conference on. IEEE, 763–768.

Digital Library

[17]

Affan Pervez, Yuecheng Mao, and Dongheui Lee. 2017. Learning deep movement primitives using convolutional neural networks. In 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids). IEEE, 191–197.

Digital Library

[18]

Deirdre Quillen, Eric Jang, Ofir Nachum, Chelsea Finn, Julian Ibarz, and Sergey Levine. 2018. Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 6284–6291.

Digital Library

[19]

Stefan Schaal. 1999. Is imitation learning the route to humanoid robots?Trends in cognitive sciences 3, 6 (1999), 233–242.

[20]

C Sylvain. 2009. Robot programming by demonstration: A probabilistic approach.

[21]

Tianhe Yu, Chelsea Finn, Annie Xie, Sudeep Dasari, Tianhao Zhang, Pieter Abbeel, and Sergey Levine. 2018. One-shot imitation from observing humans via domain-adaptive meta-learning. arXiv preprint arXiv:1802.01557(2018).

Cited By

Mavsar MRidge BPahič RMorimoto JUde A(2024)Simulation-Aided Handover Prediction From Video Using Recurrent Image-to-Motion NetworksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.317572035:1(494-506)Online publication date: Jan-2024
https://doi.org/10.1109/TNNLS.2022.3175720
Hu YAbu-Dakka FChen FLuo XLi ZKnoll ADing W(2024)Fusion dynamical systems with machine learning in imitation learning: A comprehensive overviewInformation Fusion10.1016/j.inffus.2024.102379108(102379)Online publication date: Aug-2024
https://doi.org/10.1016/j.inffus.2024.102379

Recommendations

Dynamic Movement Primitives: Volumetric Obstacle Avoidance Using Dynamic Potential Functions
Abstract
Obstacle avoidance for Dynamic Movement Primitives (DMPs) is still a challenging problem. In our previous work, we proposed a framework for obstacle avoidance based on superquadric potential functions to represent volumes. In this work, we extend ...
Probabilistic movement primitives based multi-task learning framework
Abstract
With the increasing complexity of industrial production and manufacturing tasks, industrial robots are expected to learn intricate operations from simple actions easily and quickly with adaption to dynamic environment. In this paper, a task-...
Highlights
- A task-parameterized skill learning framework for robots’ Learning from Demonstration (LfD) is established.
- The extrapolation issue in LfD is addressed where the actions to be learned are not shown in demonstration region.
- ...
Gesture learning and execution in a humanoid robot via dynamic movement primitives

A system for learning and executing gestures in a humanoid robot has been developed.Gestures are represented via the use of DMPs on the robotic platform REEM.Agnostic knowledge is considered, so our approach can be easily extended.Gestures can be ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

PETRA '21: Proceedings of the 14th PErvasive Technologies Related to Assistive Environments Conference

June 2021

593 pages

ISBN:9781450387927

DOI:10.1145/3453892

Conference Chair:
Fillia Makedon

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

PETRA '21

PETRA '21: The 14th PErvasive Technologies Related to Assistive Environments Conference

June 29 - July 2, 2021

Corfu, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
483
Total Downloads

Downloads (Last 12 months)109
Downloads (Last 6 weeks)13

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mavsar MRidge BPahič RMorimoto JUde A(2024)Simulation-Aided Handover Prediction From Video Using Recurrent Image-to-Motion NetworksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.317572035:1(494-506)Online publication date: Jan-2024
https://doi.org/10.1109/TNNLS.2022.3175720
Hu YAbu-Dakka FChen FLuo XLi ZKnoll ADing W(2024)Fusion dynamical systems with machine learning in imitation learning: A comprehensive overviewInformation Fusion10.1016/j.inffus.2024.102379108(102379)Online publication date: Aug-2024
https://doi.org/10.1016/j.inffus.2024.102379

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents