Nothing Special   »   [go: up one dir, main page]

skip to main content
10.5555/3306127.3332008acmconferencesArticle/Chapter ViewAbstractPublication PagesaamasConference Proceedingsconference-collections
research-article

Memory based Multiagent One Shot Learning

Published: 08 May 2019 Publication History

Abstract

One shot learning is particularly difficult in multiagent systems where the relevant information is distributed across agents, and inter-agent interactions shape global emergent behavior. This paper introduces a distributed learning framework called Distributed Modular Memory Unit (DMMU) that creates a shared external memory to enable one shot adaptive learning in multiagent systems. In DMMU, a shared external memory is selectively accessed by agents acting asynchronously and in parallel. Each agent processes its own stream of sequential information independently while interacting with the shared external memory to identify, retain, and propagate salient information. This enables DMMU to rapidly assimilate task features from a group of distributed agents, consolidate it into a reconfigurable external memory, and use it for one shot multiagent learning. We compare the performance of the DMMU framework on a simulated cybersecurity task with traditional feedforward ensembles, LSTM based agents, and a centralized framework. Results demonstrate that DMMU significantly outperforms the other methods and exhibits distributed one shot learning.

References

[1]
Bram Bakker. 2002. Reinforcement Learning Memory. Neural Information Processing Systems 14 (2002), 1475--1782.
[2]
Justin Bayer, Daan Wierstra, Julian Togelius, and Jürgen Schmidhuber. 2009. Evolving memory cell structures for sequence learning. Artificial Neural Networks--ICANN 2009 (2009), 755--764.
[3]
Yan Duan, Marcin Andrychowicz, Bradly Stadie, OpenAI Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, and Wojciech Zaremba. 2017. One-shot imitation learning. In Advances in neural information processing systems. 1087--1098.
[4]
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning.MIT Press.
[5]
Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, Vol. 18, 5 (2005), 602--610.
[6]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, Vol. 9, 8 (1997), 1735--1780.
[7]
Jen Jen Chung Khadka, Shauharda and Kagan Tumer. 2019. Neuroevolution of a Modular Memory-Augmented Neural Network for Deep Memory Problems. Evolutionary computation (2019).
[8]
Shauharda Khadka, Jen Jen Chung, and Kagan Tumer. 2017. Evolving Memory-Augmented Neural Architecture for Deep Memory Problems. In In Proceedings of the Genetic and Evolutionary Computation Conference 2017. ACM.
[9]
Shauharda Khadka, Connor Yates, and Kagan Tumer. 2018. A Memory-based Multiagent Framework for Adaptive Decision Making. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 1977--1979.
[10]
Brenden M Lake, Ruslan R Salakhutdinov, and Josh Tenenbaum. 2013. One-shot learning by inverting a compositional causal process. In Advances in neural information processing systems. 2526--2534.
[11]
Benno Lüders, Mikkel Schl"ager, and Sebastian Risi. 2016. Continual learning through evolvable neural turing machines. In NIPS 2016 Workshop on Continual Learning and Deep Networks (CLDL 2016).
[12]
Arvind Narayanan, Joseph Bonneau, Edward Felten, Andrew Miller, and Steven Goldfeder. 2016. Bitcoin and Cryptocurrency Technologies: A Comprehensive Introduction.Princeton University Press.
[13]
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. One-shot learning with memory-augmented neural networks. arXiv preprint arXiv:1605.06065 (2016).
[14]
Jurgen Schmidhuber. 2015. Deep learning in neural networks: An overview. Neural networks, Vol. 61 (2015), 85--117.
[15]
Kenneth O Stanley, Bobby D Bryant, and Risto Miikkulainen. 2003. Evolving adaptive neural networks with and without adaptive synapses. In Evolutionary Computation, 2003. CEC'03. The 2003 Congress on, Vol. 4. IEEE, 2557--2564.
[16]
Melanie Swan. 2015. Blockchain: Blueprint for a new economy." O'Reilly Media, Inc.".
[17]
Kagan Tumer and Adrian Agogino. 2007. Distributed agent-based air traffic flow management. In Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems. ACM, 255.
[18]
Oriol Vinyals, Charles Blundell, Tim Lillicrap, Daan Wierstra, et almbox. 2016. Matching networks for one shot learning. In Advances in Neural Information Processing Systems. 3630--3638.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
AAMAS '19: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems
May 2019
2518 pages
ISBN:9781450363099

Sponsors

Publisher

International Foundation for Autonomous Agents and Multiagent Systems

Richland, SC

Publication History

Published: 08 May 2019

Check for updates

Author Tags

  1. coordination
  2. memory
  3. multiagent system
  4. one-shot-learning
  5. rnn

Qualifiers

  • Research-article

Funding Sources

  • Electric Power Research Institute

Conference

AAMAS '19
Sponsor:

Acceptance Rates

AAMAS '19 Paper Acceptance Rate 193 of 793 submissions, 24%;
Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 61
    Total Downloads
  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)1
Reflects downloads up to 26 Nov 2024

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media