tutorial

Evolving neural networks

Author:

Kenneth O. StanleyAuthors Info & Claims

GECCO '12: Proceedings of the 14th annual conference companion on Genetic and evolutionary computation

Pages 805 - 826

https://doi.org/10.1145/2330784.2330917

Published: 07 July 2012 Publication History

Abstract

Neuroevolution, i.e. evolution of artificial neural networks, has recently emerged as a powerful technique for solving challenging reinforcement learning problems. Compared to traditional (e.g. value-function based) methods, neuroevolution is especially strong in domains where the state of the world is not fully known: the state can be disambiguated through recurrence, and novel situations handled through pattern matching. In this tutorial, we will review (1) neuroevolution methods that evolve fixed-topology networks, network topologies, and network construction processes, (2) ways of combining traditional neural network learning algorithms with evolutionary methods, and (3) applications of neuroevolution to control, robotics, artificial life, and games.

References

[1]

Aaltonen et al. (over 100 authors) (2009). Measurement of the top quark mass with dilepton events selected using neuroevolution at CDF. Physical Review Letters, 102(15):2001.

[2]

Agogino, A., Tumer, K., and Miikkulainen, R. (2005). Efficient credit assignment through evaluation function decomposition. In Proceedings of the Genetic and Evolutionary Computation Conference.

Digital Library

[3]

Angeline, P. J., Saunders, G. M., and Pollack, J. B. (1993). An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5:54--65.

Digital Library

[4]

Angeline, P. J., Saunders, G. M., and Pollack, J. B. (2000). An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks. In press.

[5]

Cliff, D., Harvey, I., and Husbands, P. (1993). Explorations in evolutionary robotics. Adaptive Behavior, 2:73--110.

Digital Library

[6]

Clune, J., Pennock, R. T., and Ofria, C. (2009). The sensitivity of HyperNEAT to different geomet- ric representations of a problem. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2009). New York, NY, USA: ACM Press.

Digital Library

[7]

Clune, J., Stanley, K. O., Pennock, R. T., and Ofria, C. (2011). On the performance of indirect encoding across the continuum of regularity. IEEE Transactions on Evolutionary Computation. To appear.

Digital Library

[8]

D'Ambrosio, D., Lehman, J., Risi, S., and Stanley, K. O. (2010). Evolving policy geometry for scal- able multiagent learning. In Proceedings of the Ninth International Conference on Autonomous Agents and Multiagent Systems (AAMAS-2010), 731--738. International Foundation for Autonomous Agents and Multiagent System.

Digital Library

[9]

D'Ambrosio, D., Lehman, J., Risi, S., and Stanley, K. O. (2011). Task switching in multiagent learning through indirect encoding. In Proceedings of the International Conference on Intelligent Robots and Systems (IROS 2011). Piscataway, NJ: IEEE.

[10]

D'Ambrosio, D. B., and Stanley, K. O. (2008). Generative encoding for multiagent learning. In Pro- ceedings of the Genetic and Evolutionary Computation Conference (GECCO 2008). New York, NY: ACM Press.

Digital Library

[11]

Desai, N. S., and Miikkulainen, R. (2000). Neuro-evolution and natural deduction. In Proceedings of The First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks, 64--69. Piscataway, NJ: IEEE.

[12]

Dubbin, G., and Stanley, K. O. (2010). Learning to dance through interactive evolution. In Proceed- ings of the Eigth European Event on Evolutionary and Biologically Inspired Music, Sound, Art and Design (EvoMUSART 2010). New York, NY: Springer.

Digital Library

[13]

Fagerlund, M. (2003--2006). DelphiNEAT homepage. http://www.cambrianlabs.com/mattias/DelphiNEAT/. {14} Floreano, D., Durr, P., and Mattiussi, C. (2008). Neuroevolution: from architectures to learning. Evolutionary Intelligence, 1:47--62.

[14]

Floreano, D., and Mondada, F. (1998). Evolutionary neurocontrollers for autonomous mobile robots. Neural Networks, 11:1461--1478.

Digital Library

[15]

Floreano, D., and Urzelai, J. (2000). Evolutionary robots with on-line self-organization and behavioral fitness. Neural Networks, 13:431--4434.

Digital Library

[16]

Fullmer, B., and Miikkulainen, R. (1992). Using marker-based genetic encoding of neural networks to evolve finite-state behaviour. In Varela, F. J., and Bourgine, P., editors, Toward a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life, 255--262. Cambridge, MA: MIT Press.

[17]

Gauci, J., and Stanley, K. O. (2008). A case study on the critical role of geometric regularity in machine learning. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI-2008). Menlo Park, CA: AAAI Press.

Digital Library

[18]

Gauci, J., and Stanley, K. O. (2010). Autonomous evolution of topographic regularities in artificial neural networks. Neural Computation, 22(7):1860--1898.

Digital Library

[19]

Gomez, F. (2003). Robust Non-Linear Control Through Neuroevolution. PhD thesis, Department of Com- puter Sciences, The University of Texas at Austin.

Digital Library

[20]

Gomez, F., Burger, D., and Miikkulainen, R. (2001). A neuroevolution method for dynamic resource allocation on a chip multiprocessor. In Proceedings of the INNS-IEEE International Joint Conference on Neural Networks, 2355--2361. Piscataway, NJ: IEEE.

[21]

Gomez, F., and Miikkulainen, R. (2003). Active guidance for a finless rocket using neuroevolution. In Proceedings of the Genetic and Evolutionary Computation Conference, 2084--2095. San Francisco: Kaufmann.

Digital Library

[22]

Greer, B., Hakonen, H., Lahdelma, R., and Miikkulainen, R. (2002). Numerical optimization with neuroevolution. In Proceedings of the 2002 Congress on Evolutionary Computation, 361--401. Piscataway, NJ: IEEE.

[23]

Gruau, F., and Whitley, D. (1993). Adding learning to the cellular development of neural networks: Evolution and the Baldwin effect. Evolutionary Computation, 1:213--233.

Digital Library

[24]

Gruau, F., Whitley, D., and Pyeatt, L. (1996). A comparison between cellular encoding and direct encoding for genetic neural networks. In Koza, J. R., Goldberg, D. E., Fogel, D. B., and Riolo, R. L., editors, Genetic Programming 1996: Proceedings of the First Annual Conference, 81--89. Cambridge, MA: MIT Press.

Digital Library

[25]

Hastings, E., Guha, R., and Stanley, K. O. (2007). Neat particles: Design, representation, and animation of particle system effects. In Proceedings of the IEEE Symposium on Computational Intelligence and Games (CIG-07). Piscataway, NJ: IEEE Press.

Digital Library

[26]

Hastings, E. J., Guha, R. K., and Stanley, K. O. (2010). Automatic content generation in the galactic arms race video game. IEEE Transactions on Computational Intelligence and AI in Games, 1(4):245--263.

[27]

Hoover, A. K., and Stanley, K. O. (2009). Exploiting functional relationships in musical composition. Connection Science Special Issue on Music, Brain, and Cognition, 21(2 and 3):227--251.

Digital Library

[28]

Hoover, A. K., Szerlip, P. A., and Stanley, K. O. (2011). Interactively evolving harmonies through functional scaffolding. In GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation, 387--394. Dublin, Ireland: ACM.

Digital Library

[29]

Hornby, G. S., and Pollack, J. B. (2002). Creating high-level components with a generative representation for body-brain evolution. Artificial Life, 8(3).

Digital Library

[30]

Hornby, G. S., Takamura, S., Yokono, J., Hanagata, O., Fujita, M., and Pollack, J. (2000). Evolution of controllers from a high-level simulator to a high DOF robot. In Evolvable Systems: From Biology to Hardware; Proceedings of the Third International Conference, 80--89. Berlin: Springer.

Digital Library

[31]

Igel, C. (2003). Neuroevolution for reinforcement learning using evolution strategies. In Sarker, R., Reynolds, R., Abbass, H., Tan, K. C., McKay, B., Essam, D., and Gedeon, T., editors, Proceedings of the 2003 Congress on Evolutionary Computation, 2588--2595. Piscataway, NJ: IEEE Press.

[32]

James, D., and Tucker, P. (2005). Evolving a neural network active vision system for shape discrimina- tion. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2005) Late Breaking Papers. New York, NY: ACM Press.

[33]

Kohl, N., Stanley, K., Miikkulainen, R., Samples, M., and Sherony, R. (2006). Evolving a real-world vehicle warning system. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2006), 1681--1688.

Digital Library

[34]

Lehman, J., and Stanley, K. O. (2008). Exploiting open-endedness to solve problems through the search for novelty. In Bullock, S., Noble, J., Watson, R., and Bedau, M., editors, Proceedings of the Eleventh International Conference on Artificial Life (Alife XI). Cambridge, MA: MIT Press.

[35]

Lehman, J., and Stanley, K. O. (2011). Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation, 19(2):189--223.

Digital Library

[36]

Mattiussi, C., and Floreano, D. (2006). Analog Genetic Encoding for the Evolution of Circuits and Networks. IEEE Transactions on Evolutionary Computation, 11(5):596--607.

Digital Library

[37]

McDonnell, J. R., and Waagen, D. (1994). Evolving recurrent perceptrons for time-series modeling. IEEE Transactions on Evolutionary Computation, 5:24--38.

Digital Library

[38]

Mjolsness, E., Sharp, D. H., and Alpert, B. K. (1989). Scaling, machine learning, and genetic neural nets. Advances in Applied Mathematics, 10:137--163.

Digital Library

[39]

Montana, D. J., and Davis, L. (1989). Training feedforward neural networks using genetic algorithms. In Proceedings of the 11th International Joint Conference on Artificial Intelligence, 762--767. San Francisco: Kaufmann.

Digital Library

[40]

Moriarty, D. E. (1997). Symbiotic Evolution of Neural Networks in Sequential Decision Tasks. PhD thesis, Department of Computer Sciences, The University of Texas at Austin. Technical Report UT-AI97--257.

Digital Library

[41]

Moriarty, D. E., and Miikkulainen, R. (1996). Evolving obstacle avoidance behavior in a robot arm. In Maes, P., Mataric, M. J., Meyer, J.-A., Pollack, J., and Wilson, S. W., editors, From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, 468--475. Cambridge, MA: MIT Press.

[42]

Moriarty, D. E., and Miikkulainen, R. (1997). Forming neural networks through efficient and adaptive co-evolution. Evolutionary Computation, 5:373--399.

Digital Library

[43]

Mouret, J.-B., and Doncieux, S. (2009). Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC-2009, 1161--1168. IEEE.

Digital Library

[44]

Nolfi, S., and Floreano, D. (2000). Evolutionary Robotics. Cambridge: MIT Press.

[45]

Potter, M. A., and Jong, K. A. D. (2000). Cooperative coevolution: An architecture for evolving coad- apted subcomponents. Evolutionary Computation, 8:1--29.

Digital Library

[46]

Pujol, J. C. F., and Poli, R. (1998). Evolving the topology and the weights of neural networks using a dual representation. Applied Intelligence Journal, 8(1):73--84. Special Issue on Evolutionary Learning.

Digital Library

[47]

Radcliffe, N. J. (1993). Genetic set recombination and its application to neural network topology optimization. Neural computing and applications, 1(1):67--90.

[48]

Risi, S., and Stanley, K. O. (2010). Indirectly encoding neural plasticity as a pattern of local rules. In Proceedings of the 11th International Conference on Simulation of Adaptive Behavior (SAB2010). Berlin: Springer.

Digital Library

[49]

Risi, S., and Stanley, K. O. (2011). Enhancing ES-HyperNEAT to evolve more complex regular neural networks. In GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation, 1539--1546. Dublin, Ireland: ACM.

Digital Library

[50]

Schaffer, J. D., Whitley, D., and Eshelman, L. J. (1992). Combinations of genetic algorithms and neural networks: A survey of the state of the art. In Whitley, D., and Schaffer, J., editors, Proceedings of the International Workshop on Combinations of Genetic Algorithms and Neural Networks, 1--37. Los Alamitos, CA: IEEE Computer Society Press.

[51]

Secretan, J., Beato, N., D.Ambrosio, D. B., Rodriguez, A., Campbell, A., Folsom-Kovarik, J. T., and Stanley, K. O. (2011). Picbreeder: A case study in collaborative evolutionary exploration of design space. Evolutionary Computation, 19(3):345--371.

Digital Library

[52]

Secretan, J., Beato, N., D'Ambrosio, D. B., Rodriguez, A., Campbell, A., and Stanley, K. O. (2008). Picbreeder: Evolving pictures collaboratively online. In CHI '08: Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, 1759--1768. New York, NY, USA: ACM.

Digital Library

[53]

Seys, C. W., and Beer, R. D. (2004). Evolving walking: The anatomy of an evolutionary search. In Schaal, S., Ijspeert, A., Billard, A., Vijayakumar, S., Hallam, J., and Meyer, J.-A., editors, From Animals to Animats 8: Proceedings of the Eight International Conference on Simulation of Adaptive Behavior, 357--363. Cambridge, MA: MIT Press.

[54]

Siddiqi, A. A., and Lucas, S. M. (1998). A comparison of matrix rewriting versus direct encoding for evolving neural networks. In Proceedings of IEEE International Conference on Evolutionary Computation, 392--397. Piscataway, NJ: IEEE.

[55]

Sit, Y. F., and Miikkulainen, R. (2005). Learning basic navigation for personal satellite assistant using neuroevolution. In Proceedings of the Genetic and Evolutionary Computation Conference.

Digital Library

[56]

Stanley, K. O. (2003). Efficient Evolution of Neural Networks Through Complexification. PhD thesis, De- partment of Computer Sciences, The University of Texas at Austin, Austin, TX.

Digital Library

[57]

Stanley, K. O. (2007). Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines Special Issue on Developmental Systems, 8(2):131--162.

Digital Library

[58]

Stanley, K. O., Bryant, B. D., and Miikkulainen, R. (2005). Real-time neuroevolution in the NERO video game. IEEE Transactions on Evolutionary Computation Special Issue on Evolutionary Computation and Games, 9(6):653--668.

Digital Library

[59]

Stanley, K. O., D'Ambrosio, D. B., and Gauci, J. (2009). A hypercube-based indirect encoding for evolving large-scale neural networks. Artificial Life, 15(2):185--212.

Digital Library

[60]

Stanley, K. O., and Miikkulainen, R. (2002). Evolving neural networks through augmenting topologies. Evolutionary Computation, 10:99--127.

Digital Library

[61]

Stanley, K. O., and Miikkulainen, R. (2003). A taxonomy for artificial embryogeny. Artificial Life, 9(2):93--130.

Digital Library

[62]

Stanley, K. O., and Miikkulainen, R. (2004). Competitive coevolution through evolutionary complexi- fication. Journal of Artificial Intelligence Research, 21:63--100.

Digital Library

[63]

Stanley, K. O., and Miikkulainen, R. (2004). Evolving a roving eye for Go. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004). Berlin: Springer Verlag.

[64]

Sutton, R. S., and Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.

[65]

Taylor, M. E., Whiteson, S., and Stone, P. (2006). Comparing evolutionary and temporal difference methods in a reinforcement learning domain. In GECCO 2006: Proceedings of the Genetic and Evolutionary Computation Conference, 1321--1328.

Digital Library

[66]

v. E. Conradie, A., Miikkulainen, R., and Aldrich, C. (2002). Adaptive control utilising neural swarming. In Proceedings of the Genetic and Evolutionary Computation Conference. San Francisco: Kaufmann.

Digital Library

[67]

v. E. Conradie, A., Miikkulainen, R., and Aldrich, C. (2002). Intelligent process control utilizing sym- biotic memetic neuro-evolution. In Proceedings of the 2002 Congress on Evolutionary Computation.

[68]

Valsalam, V. K., and Miikkulainen, R. (2008). Modular neuroevolution for multilegged locomotion. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2008). New York, NY: ACM Press.

Digital Library

[69]

Verbancsics, P., and Stanley, K. O. (2010). Evolving static representations for task transfer. Journal of Machine Learning Research (JMLR), 11:1737--1769.

Digital Library

[70]

Verbancsics, P., and Stanley, K. O. (2011). Constraining connectivity to encourage modularity in Hyper- NEAT. In GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation, 1483--1490. Dublin, Ireland: ACM.

Digital Library

[71]

Whitley, D., Dominic, S., Das, R., and Anderson, C. W. (1993). Genetic reinforcement learning for neurocontrol problems. Machine Learning, 13:259--284.

Digital Library

[72]

Wieland, A. P. (1990). Evolving controls for unstable systems. In Touretzky, D. S., Elman, J. L., Se- jnowski, T. J., and Hinton, G. E., editors, Connectionist Models: Proceedings of the 1990 Summer School, 91--102. San Francisco: Kaufmann.

[73]

Woolley, B. G., and Stanley, K. O. (2011). On the deleterious effects of a priori objectives on evolution and representation. In GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation, 957--964. Dublin, Ireland: ACM.

Digital Library

[74]

Yao, X. (1999). Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423--1447.

[75]

Zhang, B.-T., and Muhlenbein, H. (1993). Evolving optimal neural networks using genetic algorithms with Occam's razor. Complex Systems, 7:199--220.

Cited By

Meyer-Nieberg SLeopold NUhlig TMeyer-Nieberg SLeopold NUhlig T(2019)Natural Computing and OptimizationNatural Computing for Simulation-Based Optimization and Beyond10.1007/978-3-030-26215-0_2(9-30)Online publication date: 27-Jul-2019
https://doi.org/10.1007/978-3-030-26215-0_2
Fairey JSoule TArnold D(2014)Evolution of communication and cooperationProceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation10.1145/2576768.2598377(169-176)Online publication date: 12-Jul-2014
https://dl.acm.org/doi/10.1145/2576768.2598377

Index Terms

Evolving neural networks
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Evolving neural networks
GECCO '10: Proceedings of the 12th annual conference companion on Genetic and evolutionary computation

Neuroevolution, i.e. evolution of artificial neural networks, has recently emerged as a powerful technique for solving challenging reinforcement learning problems. Compared to traditional (e.g. value-function based) methods, neuroevolution is especially ...
Investigating whether hyperNEAT produces modular neural networks
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

HyperNEAT represents a class of neuroevolutionary algorithms that captures some of the power of natural development with a computationally efficient high-level abstraction of development. This class of algorithms is intended to provide many of the ...
Evolving the placement and density of neurons in the hyperneat substrate
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

The Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) approach demonstrated that the pattern of weights across the connectivity of an artificial neural network (ANN) can be generated as a function of its geometry, thereby allowing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '12: Proceedings of the 14th annual conference companion on Genetic and evolutionary computation

July 2012

1586 pages

ISBN:9781450311786

DOI:10.1145/2330784

Editor:
Terence Soule
University of Idaho, USA
,
General Chair:
Jason H. Moore
Dartmouth College, USA

Copyright © 2012 Author.

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

GECCO '12

Sponsor:

SIGEVO

GECCO '12: Genetic and Evolutionary Computation Conference

July 7 - 11, 2012

Pennsylvania, Philadelphia, USA

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
471
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Meyer-Nieberg SLeopold NUhlig TMeyer-Nieberg SLeopold NUhlig T(2019)Natural Computing and OptimizationNatural Computing for Simulation-Based Optimization and Beyond10.1007/978-3-030-26215-0_2(9-30)Online publication date: 27-Jul-2019
https://doi.org/10.1007/978-3-030-26215-0_2
Fairey JSoule TArnold D(2014)Evolution of communication and cooperationProceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation10.1145/2576768.2598377(169-176)Online publication date: 12-Jul-2014
https://dl.acm.org/doi/10.1145/2576768.2598377

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents