poster

Neurogenetic programming framework for explainable reinforcement learning

Authors:

Vadim Liventsev,

Aki Härmä,

Milan PetkovićAuthors Info & Claims

GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 329 - 330

https://doi.org/10.1145/3449726.3459537

Published: 08 July 2021 Publication History

Get Access

Abstract

Automatic programming, the task of generating computer programs compliant with a specification without a human developer, is usually tackled either via genetic programming methods based on mutation and recombination of programs, or via neural language models. We propose a novel method that combines both approaches using a concept of a virtual neuro-genetic programmer, or scrum team. We demonstrate its ability to provide performant and explainable solutions for various OpenAI Gym tasks, as well as inject expert knowledge into the otherwise data-driven search for solutions.

References

[1]

[n. d.]. Algorithm Synthesis: Deep Learning and Genetic Programming. http://iao.hfuu.edu.cn/blogs/33-algorithm-synthesis-deep-learning-and-genetic-programming. ([n. d.]). (Accessed on 02/03/2021).

Google Scholar

[2]

Daniel A. Abolafia, Mohammad Norouzi, Jonathan Shen, Rui Zhao, and Quoc V. Le. 2018. Neural Program Synthesis with Priority Queue Training. arXiv preprint arXiv:1801.03526 (2018). arXiv:1801.03526 http://arxiv.org/abs/1801.03526

Google Scholar

[3]

Uri Alon, Roy Sadaka, Omer Levy, and Eran Yahav. 2020. Structural language models of code. In International Conference on Machine Learning. PMLR, 245--256.

Google Scholar

[4]

Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2016. DeepCoder: Learning to Write Programs. CoRR abs/1611.01989 (2016). arXiv:1611.01989 http://arxiv.org/abs/1611.01989

Google Scholar

[5]

Wolfgang Banzhaf, Peter Nordin, Robert E Keller, and Frank D Francone. 1998. Genetic programming: an introduction. Vol. 1. Morgan Kaufmann Publishers San Francisco.

Digital Library

Google Scholar

[6]

Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner, Marc Parizeau, and Christian Gagné. 2012. DEAP: Evolutionary Algorithms Made Easy. Journal of Machine Learning Research 13 (jul 2012), 2171--2175.

Digital Library

Google Scholar

[7]

John R Koza. 1992. Genetic programming: on the programming of computers by means of natural selection. Vol. 1. MIT press.

Digital Library

Google Scholar

[8]

Volodymyr Kuleshov and Doina Precup. 2014. Algorithms for multi-armed bandit problems. CoRR abs/1402.6028 (2014). arXiv:1402.6028 http://arxiv.org/abs/1402.6028

Google Scholar

[9]

Yuxi Li. 2017. Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274 (2017).

Google Scholar

[10]

Vadim Liventsev, Aki Härmä, and Milan Petković. 2021. BF++:a language for general-purpose neural program synthesis. (2021). arXiv:cs.AI/2101.09571

Google Scholar

[11]

Thomas Hunt Morgan. 1916. A Critique of the Theory of Evolution. Princeton University Press.

Google Scholar

[12]

Herbert Robbins. 1952. Some aspects of the sequential design of experiments. Bull. Amer. Math. Soc. 58, 5 (1952), 527--535.

Crossref

Google Scholar

[13]

Ken Schwaber and Mike Beedle. 2002. Agile software development with Scrum. Vol. 1. Prentice Hall Upper Saddle River.

Digital Library

Google Scholar

[14]

Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.

Digital Library

Google Scholar

Cited By

View all

Eberhardinger MRupp FMaucher JMaghsudi S(2024)Unveiling the Decision-Making Process in Reinforcement Learning with Genetic ProgrammingAdvances in Swarm Intelligence10.1007/978-981-97-7181-3_28(349-365)Online publication date: 22-Aug-2024
https://dl.acm.org/doi/10.1007/978-981-97-7181-3_28
林晨(2022)Theoretical and Applied Research on Reinforcement Learning MethodsComputer Science and Application10.12677/CSA.2022.12305612:03(554-564)Online publication date: 2022
https://doi.org/10.12677/CSA.2022.123056
Liventsev VHärmä APetković M(2021)Towards Effective Patient SimulatorsFrontiers in Artificial Intelligence10.3389/frai.2021.7986594Online publication date: 15-Dec-2021
https://doi.org/10.3389/frai.2021.798659

Index Terms

Neurogenetic programming framework for explainable reinforcement learning
1. Software and its engineering
  1. Software creation and management
    1. Software development techniques
      1. Automatic programming
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Reinforcement learning

Recommendations

Generating interpretable reinforcement learning policies using genetic programming
GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference Companion

The search for interpretable reinforcement learning policies is of high academic and industrial interest. Especially for industrial systems, domain experts are more likely to deploy autonomously learned controllers if they are understandable and ...
Multi-objective Genetic Programming for Explainable Reinforcement Learning
Genetic Programming
Abstract
Deep reinforcement learning has met noticeable successes recently for a wide range of control problems. However, this is typically based on thousands of weights and non-linearities, making solutions complex, not easily reproducible, ...
Genetic programming methods for reinforcement learning
GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference

Reinforcement Learning (RL) algorithms can be used to optimally solve dynamic decision-making and control problems. With continuous-valued state and input variables, RL algorithms must rely on function approximators to represent the value function and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2021

2047 pages

ISBN:9781450383516

DOI:10.1145/3449726

Editor:
Francisco Chicano
University of Malaga
,
General Chair:
Krzysztof Krawiec
Poznan University of Technology

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2021

Check for updates

Author Tags

Qualifiers

Poster

Funding Sources

European Union

Conference

GECCO '21

Sponsor:

SIGEVO

GECCO '21: Genetic and Evolutionary Computation Conference

July 10 - 14, 2021

Lille, France

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
79
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Eberhardinger MRupp FMaucher JMaghsudi S(2024)Unveiling the Decision-Making Process in Reinforcement Learning with Genetic ProgrammingAdvances in Swarm Intelligence10.1007/978-981-97-7181-3_28(349-365)Online publication date: 22-Aug-2024
https://dl.acm.org/doi/10.1007/978-981-97-7181-3_28
林晨(2022)Theoretical and Applied Research on Reinforcement Learning MethodsComputer Science and Application10.12677/CSA.2022.12305612:03(554-564)Online publication date: 2022
https://doi.org/10.12677/CSA.2022.123056
Liventsev VHärmä APetković M(2021)Towards Effective Patient SimulatorsFrontiers in Artificial Intelligence10.3389/frai.2021.7986594Online publication date: 15-Dec-2021
https://doi.org/10.3389/frai.2021.798659

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Generating interpretable reinforcement learning policies using genetic programming

Multi-objective Genetic Programming for Explainable Reinforcement Learning

Genetic programming methods for reinforcement learning