research-article

Transformation-interaction-rational representation for symbolic regression

Author:

Fabrício Olivetti de FrançaAuthors Info & Claims

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference

Pages 920 - 928

https://doi.org/10.1145/3512290.3528695

Published: 08 July 2022 Publication History

Abstract

Symbolic Regression searches for a function form that approximates a dataset often using Genetic Programming. Since there is usually no restriction to what form the function can have, Genetic Programming may return a hard to understand model due to non-linear function chaining or long expressions. A novel representation called Interaction-Transformation was recently proposed to alleviate this problem. In this representation, the function form is restricted to an affine combination of terms generated as the application of a single univariate function to the interaction of selected variables. This representation obtained competing solutions on standard benchmarks. Despite the initial success, a broader set of benchmarking functions revealed the limitations of the constrained representation. In this paper we propose an extension to this representation, called Transformation-Interaction-Rational representation that defines a new function form as the rational of two Interaction-Transformation functions. Additionally, the target variable can also be transformed with an univariate function. The main goal is to improve the approximation power while still constraining the overall complexity of the expression. We tested this representation with a standard Genetic Programming with crossover and mutation. The results show a great improvement when compared to its predecessor and a state-of-the-art performance for a large benchmark.

References

[1]

Guilherme Seidyo Imai Aldeia and Fabricio Olivetti de Franca. 2020. A Parametric Study of Interaction-Transformation Evolutionary Algorithm for Symbolic Regression. In 2020 IEEE Congress on Evolutionary Computation (CEC). IEEE, New York, 8 pages.

Digital Library

[2]

Guilherme Seidyo Imai Aldeia and Fabrício Olivetti de França. 2018. Lightweight Symbolic Regression with the Interaction - Transformation Representation. In 2018 IEEE Congress on Evolutionary Computation (CEC). IEEE, New York, 8 pages.

Digital Library

[3]

Ignacio Arnaldo, Krzysztof Krawiec, and Una-May O'Reilly. 2014. Multiple regression genetic programming. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation. ACM, 879--886.

Digital Library

[4]

Bogdan Burlacu, Gabriel Kronberger, and Michael Kommenda. 2020. Operon C++ an eficient genetic programming framework for symbolic regression. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. 1562--1570.

Digital Library

[5]

F. O. de Franca and G. S. I. Aldeia. 2020. Interaction-Transformation Evolutionary Algorithm for Symbolic Regression. Evolutionary Computation (12 2020), 1--25. arXiv:https://direct.mit.edu/evco/article-pdf/doi/10.1162/evco_a_00285/1888497/evco_a_00285.pdf

[6]

Fabricio Olivetti de Franca and Maira Zabuscha de Lima. 2021. Interaction-transformation symbolic regression with extreme learning machine. Neurocomputing 423 (2021), 609--619.

[7]

Grant Dick. 2014. Bloat and generalisation in symbolic regression. In Asia-Pacific Conference on Simulated Evolution and Learning. Springer, 491--502.

Digital Library

[8]

Norbert Gaffke and Berthold Heiligers. 1996. 30 Approximate designs for polynomial regression: Invariance, admissibility, and optimality. Handbook of Statistics 13 (1996), 1149--1199.

[9]

Andrew Gelman, Jennifer Hill, and Aki Vehtari. 2020. Regression and other stories. Cambridge University Press.

[10]

Alexandre Goldsztejn. 2008. Modal intervals revisited part 1: A generalized interval natural extension. (2008).

[11]

Frank E Harrell. 2017. Regression modeling strategies. Bios 330, 2018 (2017), 14.

[12]

Timothy Hickey, Qun Ju, and Maarten H Van Emden. 2001. Interval arithmetic: From principles to implementation. Journal of the ACM (JACM) 48, 5 (2001), 1038--1068.

Digital Library

[13]

Kevin Jamieson and Ameet Talwalkar. 2016. Non-stochastic best arm identification and hyperparameter optimization. In Artificial Intelligence and Statistics. PMLR, 240--248.

[14]

Daniel Kantor, Fernando J Von Zuben, and Fabricio Olivetti de Franca. 2021. Simulated annealing for symbolic regression. In Proceedings of the Genetic and Evolutionary Computation Conference. 592--599.

Digital Library

[15]

Robert E Kass. 1990. Nonlinear regression analysis and its applications. J. Amer. Statist. Assoc. 85, 410 (1990), 594--596.

[16]

Maarten Keijzer. 2004. Scaled symbolic regression. Genetic Programming and Evolvable Machines 5, 3 (2004), 259--269.

Digital Library

[17]

Michael Kommenda, Bogdan Burlacu, Gabriel Kronberger, and Michael Affenzeller. 2020. Parameter identification for symbolic regression using nonlinear least squares. Genetic Programming and Evolvable Machines 21, 3 (2020), 471--501.

Digital Library

[18]

John R Koza et al. 1994. Genetic programming II. Vol. 17. MIT press Cambridge, MA.

[19]

William La Cava and Jason H. Moore. 2019. Semantic Variation Operators for Multidimensional Genetic Programming. In Proceedings of the Genetic and Evolutionary Computation Conference (Prague, Czech Republic) (GECCO '19). ACM, New York, NY, USA, 1056--1064.

Digital Library

[20]

William La Cava, Patryk Orzechowski, Bogdan Burlacu, Fabricio Olivetti de França, Marco Virgolin, Ying Jin, Michael Kommenda, and Jason H. Moore. 2021. Contemporary Symbolic Regression Methods and their Relative Performance. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. https://openreview.net/pdf?id=xVQMrDLyGst

[21]

William La Cava, Tilak Raj Singh, James Taggart, Srinivas Suri, and Jason Moore. 2019. Learning concise representations for regression by evolving networks of trees. In International Conference on Learning Representations. https://openreview.net/forum?id=Hke-JhA9Y7

[22]

William B Langdon. 1999. Size fair and homologous tree genetic programming crossovers. In Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation-Volume 2. Morgan Kaufmann Publishers Inc., 1092--1097.

[23]

SH Alizadeh Moghaddam, M Mokhtarzade, A Alizadeh Naeini, and SA Alizadeh Moghaddama. 2017. Statistical method to overcome overfitting issue in rational function models. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 42, 4/W4 (2017).

[24]

Fabrício Olivetti de França. 2018. A greedy search tree heuristic for symbolic regression. Information Sciences 442-443 (2018), 18 -- 32.

Digital Library

[25]

Riccardo Poli, William B Langdon, Nicholas F McPhee, and John R Koza. 2008. A field guide to genetic programming. Lulu. com.

[26]

Veli-Matti Taavitsainen. 2010. Ridge and PLS based rational function regression. Journal of chemometrics 24, 11-12 (2010), 665--673.

[27]

Veli-Matti Taavitsainen. 2013. Rational function ridge regression in kinetic modeling: A case study. Chemometrics and Intelligent Laboratory Systems 120 (2013), 136--141.

Cited By

Kronberger GOlivetti de Franca FDesmond HBartlett DKammerer L(2024)The Inefficiency of Genetic Programming for Symbolic RegressionParallel Problem Solving from Nature – PPSN XVIII10.1007/978-3-031-70055-2_17(273-289)Online publication date: 7-Sep-2024
https://doi.org/10.1007/978-3-031-70055-2_17
Wang GWang ELi ZZhou JSun Z(2024)Exploring the mathematic equations behind the materials science data using interpretable symbolic regressionInterdisciplinary Materials10.1002/idm2.12180Online publication date: 29-May-2024
https://doi.org/10.1002/idm2.12180
De França F(2023)Transformation-Interaction-Rational Representation for Symbolic Regression: A Detailed Analysis of SRBench ResultsACM Transactions on Evolutionary Learning and Optimization10.1145/35973123:2(1-19)Online publication date: 28-Jun-2023
https://dl.acm.org/doi/10.1145/3597312
Show More Cited By

Index Terms

Transformation-interaction-rational representation for symbolic regression
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Bio-inspired approaches
        Genetic programming

Recommendations

Transformation-Interaction-Rational Representation for Symbolic Regression: A Detailed Analysis of SRBench Results
Special Issue on the Best of GECCO 2022: Part I
Symbolic Regression searches for a parametric model with the optimal value of the parameters that best fits a set of samples to a measured target. The desired solution has a balance between accuracy and interpretability. Commonly, there is no constraint ...
Hybrid Single Node Genetic Programming for Symbolic Regression
Transactions on Computational Collective Intelligence XXIV - Volume 9770

This paper presents a first step of our research on designing an effective and efficient GP-based method for symbolic regression. First, we propose three extensions of the standard Single Node GP, namely 1 a selection strategy for choosing nodes to be ...
LGP-VEC: A Vectorial Linear Genetic Programming for Symbolic Regression
GECCO '23 Companion: Proceedings of the Companion Conference on Genetic and Evolutionary Computation

Symbolic regression (SR) is a well-known regression problem, that aims to find a symbolic expression that best fits a given dataset. Linear Genetic Programming (LGP) is a good and powerful candidate for solving symbolic regression problems. However, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference

July 2022

1472 pages

ISBN:9781450392372

DOI:10.1145/3512290

Editor:
Jonathan E. Fieldsend
University of Exeter
,
General Chair:
Markus Wagner
The University of Adelaide

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '22

Sponsor:

SIGEVO

GECCO '22: Genetic and Evolutionary Computation Conference

July 9 - 13, 2022

Massachusetts, Boston

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
91
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)1

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kronberger GOlivetti de Franca FDesmond HBartlett DKammerer L(2024)The Inefficiency of Genetic Programming for Symbolic RegressionParallel Problem Solving from Nature – PPSN XVIII10.1007/978-3-031-70055-2_17(273-289)Online publication date: 7-Sep-2024
https://doi.org/10.1007/978-3-031-70055-2_17
Wang GWang ELi ZZhou JSun Z(2024)Exploring the mathematic equations behind the materials science data using interpretable symbolic regressionInterdisciplinary Materials10.1002/idm2.12180Online publication date: 29-May-2024
https://doi.org/10.1002/idm2.12180
De França F(2023)Transformation-Interaction-Rational Representation for Symbolic Regression: A Detailed Analysis of SRBench ResultsACM Transactions on Evolutionary Learning and Optimization10.1145/35973123:2(1-19)Online publication date: 28-Jun-2023
https://dl.acm.org/doi/10.1145/3597312
De Franca FSilva SPaquete L(2023)Fighting Underspecification in Symbolic Regression with Fitness SharingProceedings of the Companion Conference on Genetic and Evolutionary Computation10.1145/3583133.3590525(551-554)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583133.3590525
de França F(2023)Alleviating overfitting in transformation-interaction-rational symbolic regression with multi-objective optimizationGenetic Programming and Evolvable Machines10.1007/s10710-023-09461-324:2Online publication date: 20-Oct-2023
https://dl.acm.org/doi/10.1007/s10710-023-09461-3

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents