Abstract
In this paper, we investigate the challenge of Pre-trained Language Models (PLMs) for continual task planning. PLM-based planner is difficult to incorporate incremental experience without risking catastrophic forgetting or overwhelming the model parameters. Inspired by human cognition, we propose the Experience Adapter, a novel method that avoids the need for model re-training or fine-tuning. The adapter continually collects experiences externally, including observation memory and human feedback, represented in memory graph and rules. Using these, the adapter directs task planning and corrects behavior not aligning with human expectations. Our method, not relying on the planner’s inherent structure, pairs easily with various foundational planning methods. In experiments on everyday tasks within the VirtualHome environment, we show that our approach significantly improves task success rate from 47% to 64%. This non-invasive method fits seamlessly within existing model-serving pipelines without altering the model training.
J. Zhang and J. Liao—Contribute equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brohan, A., et al.: Do as i can, not as i say: grounding language in robotic affordances. In: Conference on Robot Learning, pp. 287–318 (2023)
Chen, M., et al.: Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)
Chen, Z., et al.: Vision transformer adapter for dense predictions. arXiv preprint arXiv:2205.08534 (2022)
De Lange, M., et al.: A continual learning survey: defying forgetting in classification tasks. IEEE Trans. Pattern Anal. Mach. Intell. 44(7), 3366–3385 (2021)
Driess, D., et al.: PaLM-E: an embodied multimodal language model. arXiv preprint arXiv:2303.03378 (2023)
Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., Parikh, D.: Making the V in VQA matter: elevating the role of image understanding in visual question answering. In: Conference on Computer Vision and Pattern Recognition, pp. 6325–6334 (2017)
Guhur, P.L., Chen, S., Pinel, R.G., Tapaswi, M., Laptev, I., Schmid, C.: Instruction-driven history-aware policies for robotic manipulations. In: Conference on Robot Learning, pp. 175–187. PMLR (2023)
Houlsby, N., et al.: Parameter-efficient transfer learning for NLP. In: International Conference on Machine Learning, pp. 2790–2799 (2019)
Huang, C., Mees, O., Zeng, A., Burgard, W.: Visual language maps for robot navigation. arXiv preprint arXiv:2210.05714 (2022)
Huang, W., Abbeel, P., Pathak, D., Mordatch, I.: Language models as zero-shot planners: extracting actionable knowledge for embodied agents. In: International Conference on Machine Learning, pp. 9118–9147 (2022)
Huang, W., et al.: Inner monologue: embodied reasoning through planning with language models (2022)
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., Díaz-Rodríguez, N.: Continual learning for robotics: definition, framework, learning strategies, opportunities and challenges. Inf. Fusion 58, 52–68 (2020)
Lewkowycz, A., et al.: Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206.14858 (2022)
Li, S., et al.: Pre-trained language models for interactive decision-making. Adv. Neural Inf. Process. Syst. 35, 31199–31212 (2022)
Li, Y., Mao, H., Girshick, R., He, K.: Exploring plain vision transformer backbones for object detection. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13669, pp. 280–296. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20077-9_17
McFarlane, R.: A survey of exploration strategies in reinforcement learning. McGill University (2018)
Puig, X., et al.: Virtualhome: simulating household activities via programs. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8494–8502 (2018)
Stickland, A.C., Murray, I.: Bert and pals: projected attention layers for efficient adaptation in multi-task learning. In: International Conference on Machine Learning, pp. 5986–5995 (2019)
Wang, Z., Cai, S., Liu, A., Ma, X., Liang, Y.: Describe, explain, plan and select: interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560 (2023)
Zhao, W.X., et al.: A survey of large language models. arXiv preprint arXiv:2303.18223 (2023)
Acknowledgement
This research was supported by Key Research Project of Zhejiang Lab (Grant No. G2021NB0AL03) and National Natural Science Foundation of China (Grant No. U21A20488).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, J. et al. (2023). Experience Adapter: Adapting Pre-trained Language Models for Continual Task Planning. In: Yang, H., et al. Intelligent Robotics and Applications. ICIRA 2023. Lecture Notes in Computer Science(), vol 14271. Springer, Singapore. https://doi.org/10.1007/978-981-99-6495-6_33
Download citation
DOI: https://doi.org/10.1007/978-981-99-6495-6_33
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-6494-9
Online ISBN: 978-981-99-6495-6
eBook Packages: Computer ScienceComputer Science (R0)