Cited By
View all- Li NMa LYu GXue BZhang MJin Y(2023)Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications, and Open IssuesACM Computing Surveys10.1145/360370456:2(1-34)Online publication date: 15-Sep-2023
Adaptive Genomic Evolution of Neural Network Topologies (AGENT) for State-to-Action Mapping in Autonomous Agents
Pages 9638 - 9644
Abstract
Neuroevolution is a process of training neural networks (NN) through an evolutionary algorithm, usually to serve as a state-to-action mapping model in control or reinforcement learning-type problems. This paper builds on the Neuro Evolution of Augmented Topologies (NEAT) formalism that allows designing topology and weight evolving NNs. Fundamental advancements are made to the neuroevolution process to address premature stagnation and convergence issues, central among which is the incorporation of automated mechanisms to control the population diversity and average fitness improvement within the neuroevolution process. Insights into the performance and efficiency of the new algorithm is obtained by evaluating it on three benchmark problems from the Open AI platform and an Unmanned Aerial Vehicle (UAV) collision avoidance problem.
References
[1]
Dounis, Anastasios I., and Christos Caraiscos. ”Advanced control systems engineering for energy and comfort management in a building environment—A review.” Renewable and Sustainable Energy Reviews 13.6-7 (2009): 1246–1261.
[2]
Turner, Andrew James, and Julian Francis Miller. ”The importance of topology evolution in neuroevolution: a case study using cartesian genetic programming of artificial neural networks.” Research and Development in Intelligent Systems XXX. Springer, Cham, 2013. 213–226.
[3]
Baxter, Jonathan, Andrew Tridgell, and Lex Weaver. ”Knightcap: a chess program that learns by combining td (lambda) with game-tree search.” arXiv preprint cs/9901002 (1999).
[4]
Peters, J., Vijayakumar, S. and Schaal, S., 2003, September. Reinforcement learning for humanoid robotics. In Proceedings of the third IEEE-RAS international conference on humanoid robots (pp. 1–20).
[5]
Mnih, Volodymyr, et al. ”Human-level control through deep reinforcement learning.” Nature 518.7540 (2015): 529.
[6]
Grathwohl, Will, et al. ”Backpropagation through the void: Optimizing control variates for black-box gradient estimation.” arXiv preprint arXiv:1711.00123 (2017).
[7]
Poggio, Tomaso, et al. ”Why and when can deep-but not shallow-networks avoid the curse of dimensionality: a review.” International Journal of Automation and Computing 14.5 (2017): 503–519.
[8]
Lillicrap, Timothy P., et al. ”Continuous control with deep reinforcement learning.” arXiv preprint arXiv:1509.02971 (2015).
[9]
Floreano, Dario, Peter Dürr, and Claudio Mattiussi. ”Neuroevolution: from architectures to learning.” Evolutionary Intelligence 1.1 (2008): 47–62.
[10]
Liu, Teng, et al. ”Parallel reinforcement learning: a framework and case study.” IEEE/CAA Journal of Automatica Sinica 5.4 (2018): 827–835.
[11]
Salimans, Tim, et al. ”Evolution strategies as a scalable alternative to reinforcement learning.” arXiv preprint arXiv:1703.03864 (2017).
[12]
Schmidhuber, Juergen, and Jieyu Zhao. ”Direct policy search and uncertain policy evaluation.” Aaai spring symposium on search under uncertain and incomplete information, stanford univ. 1998.
[13]
Risi, Sebastian, and Julian Togelius. ”Neuroevolution in games: State of the art and open challenges.” IEEE Transactions on Computational Intelligence and AI in Games 9.1 (2017): 25–41.
[14]
Hansen, Nikolaus, Sibylle D. Müller, and Petros Koumoutsakos. ”Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES).” Evolutionary computation 11.1 (2003): 1–18.
[15]
Ronald, Edmund, and Marc Schoenauer. ”Genetic Lander: An experiment in accurate neuro-genetic control.” International Conference on Parallel Problem Solving from Nature. Springer, Berlin, Heidelberg, 1994.
[16]
Stanley, Kenneth O., and Risto Miikkulainen. ”Efficient reinforcement learning through evolving neural network topologies.” Proceedings of the 4th Annual Conference on Genetic and Evolutionary Computation. Morgan Kaufmann Publishers Inc., 2002.
[17]
Stanley, Kenneth O., David B. D’Ambrosio, and Jason Gauci. ”A hypercube-based encoding for evolving large-scale neural networks.” Artificial life 15.2 (2009): 185–212.
[18]
Hausknecht, Matthew, et al. ”HyperNEAT-GGP: A HyperNEAT-based Atari general game player.” Proceedings of the 14th annual conference on Genetic and evolutionary computation. ACM, 2012.
[19]
Yosinski, Jason, et al. ”Evolving robot gaits in hardware: the Hyper-NEAT generative encoding vs. parameter optimization.” ECAL. 2011.
[20]
Wang, Guochang, Guojian Cheng, and Timothy R. Carr. ”The application of improved NeuroEvolution of Augmenting Topologies neural network in Marcellus Shale lithofacies prediction.” Computers & geosciences 54 (2013): 50–65.
[21]
Nadkarni, João, and Rui Ferreira Neves. ”Combining NeuroEvolution and Principal Component Analysis to trade in the financial markets.” Expert Systems with Applications 103 (2018): 184–195.
[22]
Such, Felipe Petroski, et al. ”Deep neuroevolution: genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning.” arXiv preprint arXiv:1712.06567 (2017).
[23]
James, Derek, and Philip Tucker. ”A comparative analysis of simplification and complexification in the evolution of neural network topologies.” Proc. of Genetic and Evolutionary Computation Conference. 2004.
[24]
Chidambaran, Sharat, Amir Behjat, and Souma Chowdhury. ”Multi-criteria Evolution of Neural Network Topologies: Balancing Experience and Performance in Autonomous Systems.” arXiv preprint arXiv:1807.07979 (2018).
[25]
Vargas, Danilo Vasconcellos, and Junichi Murata. ”Spectrum-diverse neuroevolution with unified neural models.” IEEE transactions on neural networks and learning systems 28.8 (2017): 1759–1773.
[26]
Kalyanmoy, Deb. Multi objective optimization using evolutionary algorithms. John Wiley and Sons, 2001.
[27]
Kruskal, Joseph B. ”On the shortest spanning subtree of a graph and the traveling salesman problem.” Proceedings of the American Mathematical society 7.1 (1956): 48–50.
[28]
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J. and Zaremba, W., 2016. Openai gym. arXiv preprint arXiv:1606.01540.
[29]
Moore, Andrew William. ”Efficient memory-based learning for robot control.” (1990).
[30]
de Broissia, Arnaud de Froissard, and Olivier Sigaud. ”Actor-critic versus direct policy search: a comparison based on sample complexity.” arXiv preprint arXiv:1606.09152 (2016).
[31]
Liu, Ruishan, and James Zou. ”The effects of memory replay in reinforcement learning.” arXiv preprint arXiv:1710.06574 (2017).
[32]
Wan, Tracy, and Neil Xu. ”Advances in Experience Replay.” arXiv preprint arXiv:1805.05536 (2018).
[33]
Iacono, Massimiliano, and Antonio Sgorbissa. ”Path following and obstacle avoidance for an autonomous UAV using a depth camera.” Robotics and Autonomous Systems 106 (2018): 38–46.
[34]
S. Tang, V. Kumar, Translating paths into optimal trajectories for safe coordination of teams of dynamic robots, Robotics: Science and Systems(RSS),Robotics: Science and Systems (RSS),Workshop on On-line Decision-Making in Multi-robot Coordination, ann Arbor, Michigan. (June 2016).
[35]
Paul, Steve. A Bio-inspired Neural System for Energy Optimal Collision Avoidance by Unmanned Aerial Vehicles. Diss. State University of New York at Buffalo, 2017.
[36]
Chowdhury, Souma, et al. ”A mixed-discrete particle swarm optimization algorithm with explicit diversity-preservation.” Structural and Multidisciplinary Optimization 47.3 (2013): 367–388.
Recommendations
Efficient evolution of neural network topologies
CEC '02: Proceedings of the Evolutionary Computation on 2002. CEC '02. Proceedings of the 2002 Congress - Volume 02Neuroevolution, i.e. evolving artificial neural networks with genetic algorithms, has been highly effective in reinforcement learning tasks, particularly those with hidden state information. An important question in neuroevolution is how to gain an ...
Efficient reinforcement learning through evolving neural network topologies
GECCO'02: Proceedings of the 4th Annual Conference on Genetic and Evolutionary ComputationNeuroevolution is currently the strongest method on the pole-balancing benchmark reinforcement learning tasks. Although earlier studies suggested that there was an advantage in evolving the network topology as well as connection weights, the leading ...
Autonomous evolution of topographic regularities in artificial neural networks
Looking to nature as inspiration, for at least the past 25 years, researchers in the field of neuroevolution (NE) have developed evolutionary algorithms designed specifically to evolve artificial neural networks (ANNs). Yet the ANNs evolved through NE ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2019
7095 pages
Copyright © 2019.
Publisher
IEEE Press
Publication History
Published: 20 May 2019
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Nov 2024
Other Metrics
Citations
Cited By
View all- Li NMa LYu GXue BZhang MJin Y(2023)Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications, and Open IssuesACM Computing Surveys10.1145/360370456:2(1-34)Online publication date: 15-Sep-2023
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in