article

Tactile Guidance for Policy Adaptation

Authors:

Brenna D. Argall,

Eric L. Sauser,

Aude G. BillardAuthors Info & Claims

Foundations and Trends in Robotics, Volume 1, Issue 2

Pages 79 - 133

https://doi.org/10.1561/2300000012

Published: 01 February 2011 Publication History

Abstract

Demonstration learning is a powerful and practical technique to develop robot behaviors. Even so, development remains a challenge and possible demonstration limitations, for example correspondence issues between the robot and demonstrator, can degrade policy performance. This work presents an approach for policy improvement through a tactile interface located on the body of the robot. We introduce the Tactile Policy Correction (TPC) algorithm, that employs tactile feedback for the refinement of a demonstrated policy, as well as its reuse for the development of other policies. The TPC algorithm is validated on humanoid robot performing grasp positioning tasks. The performance of the demonstrated policy is found to improve with tactile corrections. Tactile guidance also is shown to enable the development of policies able to successfully execute novel, undemonstrated, tasks. We further show that different modalities, namely teleoperation and tactile control, provide information about allowable variability in the target behavior in different areas of the state space.

References

[1]

P. Abbeel and A. Y. Ng, "Exploration and apprenticeship learning in reinforcement learning," in Proceedings of the 22nd International Conference on Machine Learning (ICML'05), Bonn, Germany, 2005.

[2]

B. Argall, S. Chernova, B. Browning, and M. Veloso, "A survey of robot learning from demonstration," Robotics and Autonomous Systems, vol. 57, no. 5, pp. 469-483, 2009.

Digital Library

[3]

B. D. Argall, "Learning Mobile Robot Motion Control from Demonstration and Corrective Feedback," PhD thesis, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, March 2009.

[4]

B. D. Argall, E. L. Sauser, and A. G. Billard, "Tactile guidance for policy refinement and reuse," in 9th IEEE International Conference on Development and Learning (ICDL '10), Ann Arbor, Michigan, USA, 2010.

[5]

P. Baerlocher and R. Boulic, "An inverse kinematics architecture enforcing an arbitrary number of strict priority levels," International Journal of Computer Graphics, vol. 20, 2004.

[6]

J. A. Bagnell and J. G. Schneider, "Autonomous helicopter control using reinforcement learning policy search methods," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'01), Seoul, Korea, 2001.

[7]

D. C. Bentivegna, "Learning from Observation Using Primitives," PhD thesis, College of Computing, Georgia Institute of Technology, Atlanta, GA, July 2004.

[8]

A. Billard, S. Callinon, R. Dillmann, and S. Schaal, "Robot programming by demonstration," in Handbook of Robotics, (B. Siciliano and O. Khatib, eds.), New York, NY, USA: Chapter 59, Springer, 2008.

[9]

S. Calinon and A. Billard, "Incremental learning of gestures by imitation in a humanoid robot," in Proceedings of the 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI'07), Arlington, Virginia, USA, 2007.

[10]

S. Calinon, F. D'halluin, D. G. Caldwell, and A. Billard, "Handling of multiple constraints and motion alternatives in a robot programming by demonstration framework," in Proceedings of the IEEE-RAS International Conference on Humanoids Robots, Paris, France, 2009.

[11]

S. Chernova and M. Veloso, "Learning equivalent action choices from demonstration," in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'08), Nice, France, 2008.

[12]

D. Cohn, Z. Ghahramani, and M. Jordan, "Active learning with statistical models," Artificial Intelligence Research, vol. 4, pp. 129-145, 1996.

Digital Library

[13]

D. H. Grollman and O. C. Jenkins, "Dogged learning for robots," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '07), Rome, Italy, 2007.

[14]

G. Grunwald, G. Schreiber, A. Albu-Chaffer, and G. Hirzinger, "Programming by touch: The different way of human-robot interaction," IEEE Transactions on Industrial Electronics, vol. 50, no. 4, 2003.

[15]

F. Guenter, M. Hersch, S. Calinon, and A. Billard, "Reinforcement learning for imitating constrained reaching movements," RSJ Advanced Robotics, Special Issue on Imitative Robots, vol. 21, pp. 1521-1544, 2007.

[16]

R. Jakel, S. R. Schmidt-Rohr, M. Losch, and R. Dillmann, "Representation and constrained planning of manipulation strategies in the context of programming by demonstration," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '10), Anchorage, Alaska, USA, 2010.

[17]

M. Kaiser, H. Friedrich, and R. Dillmann, "Obtaining good performance from a bad teacher," in Programming by Demonstration vs. Learning from Examples Workshop at ML'95, Tahoe City, California, USA, 1995.

[18]

S. M. Khansari-Zadeh and A. Billard, "BM: An iterative algorithm to learn stable non-linear dynamical systems with gaussian mixture models," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '10), Anchorage, Alaska, USA, 2010.

[19]

J. Kober and J. Peters, "Learning motor primitives for robotics," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '09), Kobe, Japan, 2009.

[20]

T. Minato, Y. Yoshikawa, T. Noda, S. Ikemoto, H. Ishiguro, and M. Asada, "CB2: A child robot with biomimetic body for cognitive developmental robotics," in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '07), San Diego, California, USA, 2007.

[21]

C. L. Nehaniv and K. Dautenhahn, "The correspondence problem," in Imitation in Animals and Artifacts, (K. Dautenhahn and C. L. Nehaniv, eds.), Cambridge, MA, USA: Chapter 2, MIT Press, 2002.

[22]

M. N. Nicolescu and M. J. Mataric, "Methods for robot task learning: Demonstrations, generalization and practice," in Proceedings of the Second International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS'03), Melbourne, Victoria, Australia, 2003.

[23]

P. K. Pook and D. H. Ballard, "Recognizing teleoperated manipulations," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '93), Atlanta, Georgia, USA, 1993.

[24]

J. Saunders, C. L. Nehaniv, and K. Dautenhahn, "Teaching robots by moulding behavior and scaffolding the environment," in First Annual Conference on Human-Robot Interactions (HRI '06), Salt Lake City, Utah, USA, 2006.

[25]

M. Stolle and C. G. Atkeson, "Knowledge transfer using local features," in Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL'07), USA: Honolulu, Hawaii, 2007.

[26]

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA, London, England: The MIT Press, 1998.

[27]

J. D. Sweeney and R. A. Grupen, "A model of shared grasp affordances from demonstration," in Proceedings of the IEEE-RAS International Conference on Humanoids Robots (Humanoids'07), Japan: Tokyo, 2007.

[28]

N. Tsagarakis, G. Metta, G. Sandini, D. Vernon, R. Beira, F. Becchi, L. Righetti, J. Santos-Victor, A. Ijspeert, M. Carrozza, and D. Cald-well, "iCub: The design and realization of an open humanoid platform for cognitive and neuroscience research," Advanced Robotics, vol. 21, 2007.

[29]

K. Wada and T. Shibata, "Social effects of robot therapy in a care house -- change of social network of the residents for two months," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '07), Italy: Rome, 2007.

Cited By

Candon KZhou HGillet SVázquez MCastellano GRiek LCakmak MLeite I(2023)Verbally Soliciting Human Feedback in Continuous Human-Robot CollaborationProceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3568162.3576980(290-300)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3568162.3576980
Jackson ANorthcutt BSukthankar GKim JTapus ASirkin DJung MKwak S(2019)The benefits of immersive demonstrations for teaching robotsProceedings of the 14th ACM/IEEE International Conference on Human-Robot Interaction10.5555/3378680.3378748(326-334)Online publication date: 11-Mar-2019
https://dl.acm.org/doi/10.5555/3378680.3378748
Kaya OOztop E(2018)Effective Robot Skill Synthesis via Divided Control2018 IEEE International Conference on Robotics and Biomimetics (ROBIO)10.1109/ROBIO.2018.8664825(766-771)Online publication date: 12-Dec-2018
https://dl.acm.org/doi/10.1109/ROBIO.2018.8664825

Recommendations

Policy adaptation with tactile feedback
HRI '11: Proceedings of the 6th international conference on Human-robot interaction

Behavior adaptation with execution experience is a practical feature for any policy learning system. Our work provides performance feedback to a robot learner in the form of tactile corrections from a human teacher, for the purpose of policy refinement ...
Body-Penetrating Tactile Phantom Sensations
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

In tactile interaction, a phantom sensation refers to an illusion felt on the skin between two distant points at which vibrations are applied. It can improve the perceptual spatial resolution of tactile stimulation with a few tactors. All phantom ...
Equiangular navigation guidance of a wheeled mobile robot with local obstacle avoidance
ROBIO '09: Proceedings of the 2008 IEEE International Conference on Robotics and Biomimetics

We consider the problem of navigation of wheeled mobile robot (WMR) towards an unknown target in a cluttered environment. The biologically inspired navigation algorithm is the Equiangular Navigation Guidance (ENG) combined with a local obstacle ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Foundations and Trends in Robotics

Foundations and Trends in Robotics Volume 1, Issue 2

February 2011

57 pages

ISSN:1935-8253

EISSN:1935-8261

Issue’s Table of Contents

Publisher

Now Publishers Inc.

Hanover, MA, United States

Publication History

Published: 01 February 2011

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Candon KZhou HGillet SVázquez MCastellano GRiek LCakmak MLeite I(2023)Verbally Soliciting Human Feedback in Continuous Human-Robot CollaborationProceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3568162.3576980(290-300)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3568162.3576980
Jackson ANorthcutt BSukthankar GKim JTapus ASirkin DJung MKwak S(2019)The benefits of immersive demonstrations for teaching robotsProceedings of the 14th ACM/IEEE International Conference on Human-Robot Interaction10.5555/3378680.3378748(326-334)Online publication date: 11-Mar-2019
https://dl.acm.org/doi/10.5555/3378680.3378748
Kaya OOztop E(2018)Effective Robot Skill Synthesis via Divided Control2018 IEEE International Conference on Robotics and Biomimetics (ROBIO)10.1109/ROBIO.2018.8664825(766-771)Online publication date: 12-Dec-2018
https://dl.acm.org/doi/10.1109/ROBIO.2018.8664825

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents