Sim-to-Real Robot Learning from Pixels with Progressive Nets

Andrei A. Rusu, Matej Večerík, Thomas Rothörl, Nicolas Heess, Razvan Pascanu, Raia Hadsell
Proceedings of the 1st Annual Conference on Robot Learning, PMLR 78:262-270, 2017.

Abstract

Applying end-to-end learning to solve complex, interactive, pixel-driven control tasks on a robot is an unsolved problem. Deep Reinforcement Learning algorithms are too slow to achieve performance on a real robot, but their potential has been demonstrated in simulated environments. We propose using \emphprogressive networks to bridge the reality gap and transfer learned policies from simulation to the real world. The progressive net approach is a general framework that enables reuse of everything from low-level visual features to high-level policies for transfer to new tasks, enabling a compositional, yet simple, approach to building complex skills. We present an early demonstration of this approach with a number of experiments in the domain of robot manipulation that focus on bridging the reality gap. Unlike other proposed approaches, our real-world experiments demonstrate successful task learning from raw visual input on a fully actuated robot manipulator. Moreover, rather than relying on model-based trajectory optimisation, the task learning is accomplished using only deep reinforcement learning and sparse rewards.

Cite this Paper

BibTeX
@InProceedings{pmlr-v78-rusu17a, title = {Sim-to-Real Robot Learning from Pixels with Progressive Nets}, author = {Rusu, Andrei A. and Večerík, Matej and Rothörl, Thomas and Heess, Nicolas and Pascanu, Razvan and Hadsell, Raia}, booktitle = {Proceedings of the 1st Annual Conference on Robot Learning}, pages = {262--270}, year = {2017}, editor = {Levine, Sergey and Vanhoucke, Vincent and Goldberg, Ken}, volume = {78}, series = {Proceedings of Machine Learning Research}, month = {13--15 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v78/rusu17a/rusu17a.pdf}, url = {https://proceedings.mlr.press/v78/rusu17a.html}, abstract = {Applying end-to-end learning to solve complex, interactive, pixel-driven control tasks on a robot is an unsolved problem. Deep Reinforcement Learning algorithms are too slow to achieve performance on a real robot, but their potential has been demonstrated in simulated environments. We propose using \emphprogressive networks to bridge the reality gap and transfer learned policies from simulation to the real world. The progressive net approach is a general framework that enables reuse of everything from low-level visual features to high-level policies for transfer to new tasks, enabling a compositional, yet simple, approach to building complex skills. We present an early demonstration of this approach with a number of experiments in the domain of robot manipulation that focus on bridging the reality gap. Unlike other proposed approaches, our real-world experiments demonstrate successful task learning from raw visual input on a fully actuated robot manipulator. Moreover, rather than relying on model-based trajectory optimisation, the task learning is accomplished using only deep reinforcement learning and sparse rewards.} }
Endnote
%0 Conference Paper %T Sim-to-Real Robot Learning from Pixels with Progressive Nets %A Andrei A. Rusu %A Matej Večerík %A Thomas Rothörl %A Nicolas Heess %A Razvan Pascanu %A Raia Hadsell %B Proceedings of the 1st Annual Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2017 %E Sergey Levine %E Vincent Vanhoucke %E Ken Goldberg %F pmlr-v78-rusu17a %I PMLR %P 262--270 %U https://proceedings.mlr.press/v78/rusu17a.html %V 78 %X Applying end-to-end learning to solve complex, interactive, pixel-driven control tasks on a robot is an unsolved problem. Deep Reinforcement Learning algorithms are too slow to achieve performance on a real robot, but their potential has been demonstrated in simulated environments. We propose using \emphprogressive networks to bridge the reality gap and transfer learned policies from simulation to the real world. The progressive net approach is a general framework that enables reuse of everything from low-level visual features to high-level policies for transfer to new tasks, enabling a compositional, yet simple, approach to building complex skills. We present an early demonstration of this approach with a number of experiments in the domain of robot manipulation that focus on bridging the reality gap. Unlike other proposed approaches, our real-world experiments demonstrate successful task learning from raw visual input on a fully actuated robot manipulator. Moreover, rather than relying on model-based trajectory optimisation, the task learning is accomplished using only deep reinforcement learning and sparse rewards.
APA
Rusu, A.A., Večerík, M., Rothörl, T., Heess, N., Pascanu, R. & Hadsell, R.. (2017). Sim-to-Real Robot Learning from Pixels with Progressive Nets. Proceedings of the 1st Annual Conference on Robot Learning, in Proceedings of Machine Learning Research 78:262-270 Available from https://proceedings.mlr.press/v78/rusu17a.html.

Related Material