research-article

Generating interpretable fuzzy controllers using particle swarm optimization and genetic programming

Authors:

Steffen Udluft,

Thomas A. RunklerAuthors Info & Claims

GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 1268 - 1275

https://doi.org/10.1145/3205651.3208277

Published: 06 July 2018 Publication History

Abstract

Autonomously training interpretable control strategies, called policies, using pre-existing plant trajectory data is of great interest in industrial applications. Fuzzy controllers have been used in industry for decades as interpretable and efficient system controllers. In this study, we introduce a fuzzy genetic programming (GP) approach called fuzzy GP reinforcement learning (FGPRL) that can select the relevant state features, determine the size of the required fuzzy rule set, and automatically adjust all the controller parameters simultaneously. Each GP individual's fitness is computed using model-based batch reinforcement learning (RL), which first trains a model using available system samples and subsequently performs Monte Carlo rollouts to predict each policy candidate's performance. We compare FGPRL to an extended version of a related method called fuzzy particle swarm reinforcement learning (FPSRL), which uses swarm intelligence to tune the fuzzy policy parameters. Experiments using an industrial benchmark show that FGPRL is able to autonomously learn interpretable fuzzy policies with high control performance.

References

[1]

E. Alba, C. Cotta, and J.M. Troya. 1996. Type-constrained genetic programming for rule-base definition in fuzzy logic controllers. In Proceedings of the 1st annual conference on genetic programming. MIT Press, 255--260.

Digital Library

[2]

E. Alba, C. Cotta, and J.M. Troya. 1999. Evolutionary design of fuzzy logic controllers using strongly-typed GP. Mathware and Soft Computing 6, 1 (1999), 109--124.

[3]

F.J. Berlanga, A.J. Rivera, M.J. del Jesús, and F. Herrera. 2010. GP-COACH: Genetic Programming-based learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems. Information Sciences 180, 8 (2010), 1183--1200.

Digital Library

[4]

T. Blickle and L. Thiele. 1995. A Mathematical Analysis of Tournament Selection. In ICGA. 9--16.

Digital Library

[5]

L. Busoniu, R. Babuska, B. De Schutter, and D. Ernst. 2010. Reinforcement Learning and Dynamic Programming Using Function Approximators. CRC Press.

Digital Library

[6]

B.-C. Chien, J.Y. Lin, and T.-P. Hong. 2002. Learning discriminant functions with fuzzy attributes for classification using genetic programming. Expert Systems with Applications 23, 1 (2002), 31--37.

[7]

O. Cordón, F. Gomide, F. Herrera, F. Hoffmann, and L. Magdalena. 2004. Ten years of genetic fuzzy systems: current framework and new trends. Fuzzy sets and systems 141, 1 (2004), 5--31.

[8]

S. Depeweg, J.M. Hernández-Lobato, F. Doshi-Velez, and S. Udluft. 2016. Learning and policy search in stochastic dynamical systems with Bayesian neural networks. arXiv preprint arXiv: 1605.07127 (2016).

[9]

I. Fantoni and R. Lozano. 2002. Non-linear control for underactuated mechanical systems. Springer.

Digital Library

[10]

A. Geyer-Schulz. 1995. Fuzzy Rule-Based Expert Systems and Genetic Machine Learning. Physica-Verlag, Heidelberg (1995).

[11]

D. Hein, S. Depeweg, M. Tokic, S. Udluft, A. Hentschel, T.A. Runkler, and V. Sterzing. 2017. A benchmark environment motivated by industrial control problems. In 2017 IEEE Symposium Series on Computational Intelligence (SSCI). 1--8.

[12]

D. Hein, A. Hentschel, T.A. Runkler, and S. Udluft. 2016. Reinforcement Learning with Particle Swarm Optimization Policy (PSO-P) in Continuous State and Action Spaces. International Journal of Swarm Intelligence Research (IJSIR) 7, 3 (2016), 23--42.

Digital Library

[13]

D. Hein, A. Hentschel, T.A. Runkler, and S. Udluft. 2017. Particle swarm optimization for generating interpretable fuzzy reinforcement learning policies. Engineering Applications of Artificial Intelligence 65 (2017), 87--98.

Digital Library

[14]

D. Hein, A. Hentschel, T.A. Runkler, and S. Udluft. 2018. Particle swarm optimization for model predictive control in reinforcement learning environments. In Critical Developments and Applications of Swarm Intelligence, Y. Shi (Ed.). IGI Global, Hershey, PA, USA, Chapter 16, 401--427.

[15]

D. Hein, S. Udluft, M. Tokic, A. Hentschel, T. A. Runkler, and V. Sterzing. 2017. Batch reinforcement learning on the industrial benchmark: First experiences. In 2017 International Joint Conference on Neural Networks (IJCNN). 4214--4221.

[16]

F. Hoffmann and O. Nelles. 2001. Genetic programming for model selection of TSK-fuzzy systems. Information Sciences 136, 1--4 (2001), 7--28.

Digital Library

[17]

J. Kennedy and R.C. Eberhart. 1995. Particle swarm optimization. Proceedings of the IEEE International Joint Conference on Neural Networks (1995), 1942--1948.

[18]

A.S. Koshiyama, T. Escovedo, M.M.B.R. Vellasco, and R. Tanscheit. 2014. GPFIS-Control: A fuzzy Genetic model for Control tasks. In Fuzzy Systems (FUZZ-IEEE), 2014 IEEE International Conference on. IEEE, 1953--1959.

[19]

J.R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA.

Digital Library

[20]

N. Le, H.N. Xuan, A. Brabazon, and T.P. Thi. 2016. Complexity measures in Genetic Programming learning: A brief review. In Evolutionary Computation (CEC), 2016 IEEE Congress on. IEEE, 2409--2416.

[21]

F. Maes, R. Fonteneau, L. Wehenkel, and D. Ernst. 2012. Policy search in a space of simple closed-form formulas: towards interpretability of reinforcement learning. Discovery Science (2012), 37--50.

[22]

E.H. Mamdani and S. Assilian. 1975. An experiment in linguistic synthesis with a fuzzy logic controller. International Journal of Man-Machine Studies 7, 1 (1975), 1--13.

[23]

P. Moscato. 1989. On evolution, search, optimization, genetic algorithms and martial arts: Towards memetic algorithms. Caltech concurrent computation program, C3P Report 826 (1989), 1989.

[24]

L.S. Ramos and J.A.C. González. 2000. A niching scheme for steady state GA-P and its application to fuzzy rule based classifiers induction. Mathware and Soft Computing 7, 2--3 (2000), 337--350.

[25]

E. Rasmussen and C.K.I. Williams. 2006. Gaussian processes for machine learning (adaptive computation and machine learning). Mit Press Ltd.

Digital Library

[26]

L. Sánchez, I. Couso, and J.A. Corrales. 2001. Combining GP operators with SA search to evolve fuzzy rule based classifiers. Information Sciences 136, 1--4 (2001), 175--191.

Digital Library

[27]

H.-P. Schwefel. 1981. Numerical optimization of computer models. John Wiley & Sons, Inc.

Digital Library

[28]

H.-P. Schwefel. 1995. Evolution and optimum seeking. Sixth-generation computer technology series. (1995).

Digital Library

[29]

R.S. Sutton and A.G. Barto. 1998. Reinforcement learning: an introduction. A Bradford book.

Digital Library

[30]

M. Tesmer and P.A. Estévez. 2004. AMIFS: Adaptive feature selection by using mutual information. In Neural Networks, 2004. Proceedings. 2004 IEEE International Joint Conference on, Vol. 1. IEEE, 303--308.

[31]

A. Tsakonas. 2013. Local and global optimization for Takagi-Sugeno fuzzy system by memetic genetic programming. Expert Systems with Applications 40, 8 (2013), 3282--3298.

Digital Library

[32]

E. Tunstel and M. Jamshidi. 1996. On genetic programming of fuzzy rule-based systems for intelligent control. Intelligent Automation & Soft Computing 2, 3 (1996), 271--284.

[33]

L.-X. Wang and J.M. Mendel. 1992. Fuzzy basis functions, universal approximation, and orthogonal least-squares learning. IEEE Transactions on Neural Networks 3, 5 (1992), 807--814.

Digital Library

[34]

L.A. Zadeh. 1965. Fuzzy sets. Information and Control 8 (1965), 338--353.

Cited By

Wang SMei YZhang MLi XHandl J(2024)A Preliminary Counterfactual Explanation Method for Genetic Programming-Evolved Rules: A Case Study on Uncertain Capacitated Arc Routing ProblemProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3654192(547-550)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3654192
Wang SMei YZhang M(2024)Explaining Genetic Programming-Evolved Routing Policies for Uncertain Capacitated Arc Routing ProblemsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.323874128:4(918-932)Online publication date: Aug-2024
https://doi.org/10.1109/TEVC.2023.3238741
Faria RCapron BSecchi ADe Souza M(2024)A data-driven tracking control framework using physics-informed neural networks and deep reinforcement learning for dynamical systemsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107256127(107256)Online publication date: Jan-2024
https://doi.org/10.1016/j.engappai.2023.107256
Show More Cited By

Index Terms

Generating interpretable fuzzy controllers using particle swarm optimization and genetic programming

Recommendations

Generating interpretable reinforcement learning policies using genetic programming
GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference Companion

The search for interpretable reinforcement learning policies is of high academic and industrial interest. Especially for industrial systems, domain experts are more likely to deploy autonomously learned controllers if they are understandable and ...
A fuzzy adaptive turbulent particle swarm optimisation

Particle Swarm Optimisation (PSO) algorithm is a stochastic search technique, which has exhibited good performance across a wide range of applications. However, very often for multimodal problems involving high dimensions, the algorithm tends to suffer ...
Fuzzy logic controllers optimization using genetic algorithms and particle swarm optimization
MICAI'10: Proceedings of the 9th Mexican international conference on Artificial intelligence conference on Advances in soft computing: Part II

In this paper we apply to Bio-inspired and evolutionary optimization methods to design fuzzy logic controllers (FLC) to minimize the steady state error of linear systems. We test the optimal FLC obtained by the genetic algorithms and the PSO applied on ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2018

1968 pages

ISBN:9781450357647

DOI:10.1145/3205651

Editor:
Hernan Aguirre
Shinshu University
,
General Chair:
Keiki Takadama
The University of Electro-Communications

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Bundesministerium für Bildung und Forschung

Conference

GECCO '18

Sponsor:

SIGEVO

GECCO '18: Genetic and Evolutionary Computation Conference

July 15 - 19, 2018

Kyoto, Japan

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
128
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)3

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang SMei YZhang MLi XHandl J(2024)A Preliminary Counterfactual Explanation Method for Genetic Programming-Evolved Rules: A Case Study on Uncertain Capacitated Arc Routing ProblemProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3654192(547-550)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3654192
Wang SMei YZhang M(2024)Explaining Genetic Programming-Evolved Routing Policies for Uncertain Capacitated Arc Routing ProblemsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.323874128:4(918-932)Online publication date: Aug-2024
https://doi.org/10.1109/TEVC.2023.3238741
Faria RCapron BSecchi ADe Souza M(2024)A data-driven tracking control framework using physics-informed neural networks and deep reinforcement learning for dynamical systemsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107256127(107256)Online publication date: Jan-2024
https://doi.org/10.1016/j.engappai.2023.107256
Marchetti FPietropolli GCamerota Verdù FCastelli MMinisci E(2024)Automatic design of interpretable control laws through parametrized Genetic Programming with adjoint state method gradient evaluationApplied Soft Computing10.1016/j.asoc.2024.111654159:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111654
Guo AWang YGuo LZhang RYu YGao S(2023)An adaptive position-guided gravitational search algorithm for function optimization and image threshold segmentationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.106040121(106040)Online publication date: May-2023
https://doi.org/10.1016/j.engappai.2023.106040
Bekkemoen Y(2023)Explainable reinforcement learning (XRL): a systematic literature review and taxonomyMachine Learning10.1007/s10994-023-06479-7Online publication date: 29-Nov-2023
https://doi.org/10.1007/s10994-023-06479-7
Elghamrawy SHassanien A(2022)A hybrid Genetic–Grey Wolf Optimization algorithm for optimizing Takagi–Sugeno–Kang fuzzy systemsNeural Computing and Applications10.1007/s00521-022-07356-534:19(17051-17069)Online publication date: 30-May-2022
https://doi.org/10.1007/s00521-022-07356-5
Azizi MTalatahari SGiaralis A(2021)Active Vibration Control of Seismically Excited Building Structures by Upgraded Grey Wolf OptimizerIEEE Access10.1109/ACCESS.2021.31342029(166658-166673)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3134202
Hein DLimmer SRunkler T(2020)Interpretable Control by Reinforcement LearningIFAC-PapersOnLine10.1016/j.ifacol.2020.12.227753:2(8082-8089)Online publication date: 2020
https://doi.org/10.1016/j.ifacol.2020.12.2277
Masrom MA Ghani NTokhi M(2019)Particle swarm optimization and spiral dynamic algorithm-based interval type-2 fuzzy logic control of triple-link inverted pendulum system: A comparative assessmentJournal of Low Frequency Noise, Vibration and Active Control10.1177/146134841987378040:1(367-382)Online publication date: 14-Sep-2019
https://doi.org/10.1177/1461348419873780

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents