tutorial

Overfitting detection and adaptive covariant parsimony pressure for symbolic regression

Authors:

Gabriel Kronberger,

Michael Kommenda,

Michael AffenzellerAuthors Info & Claims

GECCO '11: Proceedings of the 13th annual conference companion on Genetic and evolutionary computation

Pages 631 - 638

https://doi.org/10.1145/2001858.2002060

Published: 12 July 2011 Publication History

Abstract

Covariant parsimony pressure is a theoretically motivated method primarily aimed to control bloat. In this contribution we describe an adaptive method to control covariant parsimony pressure that is aimed to reduce overfitting in symbolic regression. The method is based on the assumption that overfitting can be reduced by controlling the evolution of program length. Additionally, we propose an overfitting detection criterion that is based on the correlation of the fitness values on the training set and a validation set of all models in the population.

The proposed method uses covariant parsimony pressure to decrease the average program length when overfitting occurs and allows an increase of the average program length in the absence of overfitting. The proposed approach is applied on two real world datasets. The experimental results show that the correlation of training and validation fitness can be used as an indicator for overfitting and that the proposed method of covariant parsimony pressure adaption alleviates overfitting in symbolic regression experiments with the two datasets.

References

[1]

H. Akaike. Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory, pages 267--281. 1973.

[2]

R. M. A. Azad and C. Ryan. Abstract functions and lifetime learning in genetic programming for symbolic regression. In Proceedings of the 12th annual conference on Genetic and evolutionary computation, GECCO '10, pages 893--900, New York, NY, USA, 2010. ACM.

Digital Library

[3]

S. Dignum and R. Poli. Generalisation of the limiting distribution of program sizes in tree-based genetic programming and analysis of its effects on bloat. In GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation, volume 2, pages 1588--1595, London, 7-11 July 2007. ACM Press.

Digital Library

[4]

A. Frank and A. Asuncion. UCI machine learning repository, 2010.

[5]

C. Gagne, M. Schoenauer, M. Parizeau, and M. Tomassini. Genetic programming, validation sets, and parsimony pressure. In Genetic Programming, 9th European Conference, EuroGP2006, volume 3905 of Lecture Notes in Computer Science, pages 109--120, Berlin, Heidelberg, New York, 2006. Springer.

Digital Library

[6]

T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning - Data Mining, Inference, and Prediction. Springer, 2009. Second Edition.

[7]

M. Keijzer. Scaled symbolic regression. Genetic Programming and Evolvable Machines, 5(3):259--269, Sept. 2004.

Digital Library

[8]

S. Luke. Two fast tree-creation algorithms for genetic programming. IEEE Transactions on Evolutionary Computation, 4(3):274--283, Sept. 2000.

Digital Library

[9]

R. Poli and N. F. McPhee. Covariant parsimony pressure for genetic programming. Technical Report CES-480, Department of Computing and Electronic Systems, University of Essex, UK, 2008.

[10]

J. Rissanen. A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11:416--431, 1983.

[11]

M. Schmidt and H. Lipson. Symbolic regression of implicit equations. In Genetic Programming Theory and Practice VII, Genetic and Evolutionary Computation, pages 73--85. Springer US, 2010.

[12]

G. E. Schwarz. Estimating the dimension of a model. Annals of Statistics, 6(2):461--464, 1978.

[13]

S. Silva and S. Dignum. Extending operator equalisation: Fitness based self adaptive length distribution for bloat free GP. In Proceedings of the 12th European Conference on Genetic Programming, EuroGP 2009, volume 5481 of LNCS, pages 159--170, Tuebingen, Apr. 15-17 2009. Springer.

Digital Library

[14]

S. Silva and L. Vanneschi. Operator equalisation, bloat and overfitting: a study on human oral bioavailability prediction. In GECCO '09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pages 1115--1122, Montreal, 8-12 July 2009. ACM.

Digital Library

[15]

G. F. Smits and M. Kotanchek. Pareto-front exploitation in symbolic regression. In Genetic Programming in Theory and Practice II, pages 283--299. Springer, 2005.

[16]

C. Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72--101, 1904.

[17]

L. Vanneschi, M. Castelli, and S. Silva. Measuring bloat, overfitting and functional complexity in genetic programming. In Proc. GECCO'10, pages 877--884, July 7-11 2010.

Digital Library

[18]

V. Vapnik. The Nature of Statistical Learning Theory. Springer, New York, 1996.

Digital Library

[19]

E. J. Vladislavleva, G. F. Smits, and D. den Hertog. Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Transactions on Evolutionary Computation, 13(2):333--349, 2009.

Digital Library

[20]

S. Wagner. Heuristic Optimization Software Systems - Modeling of Heuristic Optimization Algorithms in the HeuristicLab Software Environment. PhD thesis, Institute for Formal Models and Verification, Johannes Kepler University, Linz, Austria, 2009.

[21]

S. Winkler, M. Affenzeller, and S. Wagner. Using enhanced genetic programming techniques for evolving classifiers in the context of medical diagnosis. Genetic Programming and Evolvable Machines, 10(2):111--140, 2009.

Digital Library

Cited By

Li HRajbahadur GLin DBezemer CJiang Z(2024)Keeping Deep Learning Models in Check: A History-Based Approach to Mitigate OverfittingIEEE Access10.1109/ACCESS.2024.340254312(70676-70689)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3402543
Keren LLiberzon ALazebnik T(2023)A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledgeScientific Reports10.1038/s41598-023-28328-213:1Online publication date: 23-Jan-2023
https://doi.org/10.1038/s41598-023-28328-2
de França F(2023)Alleviating overfitting in transformation-interaction-rational symbolic regression with multi-objective optimizationGenetic Programming and Evolvable Machines10.1007/s10710-023-09461-324:2Online publication date: 20-Oct-2023
https://doi.org/10.1007/s10710-023-09461-3
Show More Cited By

Index Terms

Overfitting detection and adaptive covariant parsimony pressure for symbolic regression
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
      1. Heuristic function construction

Recommendations

Early stopping criteria to counteract overfitting in genetic programming
GECCO '11: Proceedings of the 13th annual conference companion on Genetic and evolutionary computation

Early stopping typically stops training the first time validation fitness disimproves. This may not be the best strategy given that validation fitness can subsequently increase or decrease. We examine the effects of stopping subsequent to the first ...
Does overfitting affect performance in estimation of distribution algorithms
GECCO '06: Proceedings of the 8th annual conference on Genetic and evolutionary computation

Estimation of Distribution Algorithms (EDAs) are a class of evolutionary algorithms that use machine learning techniques to solve optimization problems. Machine learning is used to learn probabilistic models of the selected population. This model is ...
On the effect of function set to the generalisation of symbolic regression models
GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Supervised learning by means of Genetic Programming aims at the evolutionary synthesis of a model that achieves a balance between approximating the target function on the training data and generalising on new data. In this study we benchmark the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '11: Proceedings of the 13th annual conference companion on Genetic and evolutionary computation

July 2011

1548 pages

ISBN:9781450306904

DOI:10.1145/2001858

Editor:
Natalio Krasnogor
University of Nottingham, UK
,
General Chair:
Pier Luca Lanzi
Politecnico di Milano, Italy

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

GECCO '11

Sponsor:

SIGEVO

GECCO '11: Genetic and Evolutionary Computation Conference

July 12 - 16, 2011

Dublin, Ireland

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
201
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)3

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li HRajbahadur GLin DBezemer CJiang Z(2024)Keeping Deep Learning Models in Check: A History-Based Approach to Mitigate OverfittingIEEE Access10.1109/ACCESS.2024.340254312(70676-70689)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3402543
Keren LLiberzon ALazebnik T(2023)A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledgeScientific Reports10.1038/s41598-023-28328-213:1Online publication date: 23-Jan-2023
https://doi.org/10.1038/s41598-023-28328-2
de França F(2023)Alleviating overfitting in transformation-interaction-rational symbolic regression with multi-objective optimizationGenetic Programming and Evolvable Machines10.1007/s10710-023-09461-324:2Online publication date: 20-Oct-2023
https://doi.org/10.1007/s10710-023-09461-3
Weber GPinz MGhosh S(2022)Machine learning-enabled self-consistent parametrically-upscaled crystal plasticity model for Ni-based superalloysComputer Methods in Applied Mechanics and Engineering10.1016/j.cma.2022.115384402(115384)Online publication date: Dec-2022
https://doi.org/10.1016/j.cma.2022.115384
Vanneschi LCastelli M(2021)Soft target and functional complexity reduction: A hybrid regularization method for genetic programmingExpert Systems with Applications10.1016/j.eswa.2021.114929177(114929)Online publication date: Sep-2021
https://doi.org/10.1016/j.eswa.2021.114929
Liu SLehman CAlRegib G(2020)Robustness And Overfitting Behavior Of Implicit Background Models2020 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP40778.2020.9191361(3274-3278)Online publication date: Oct-2020
https://doi.org/10.1109/ICIP40778.2020.9191361
Weber GPinz MGhosh S(2020)Machine Learning-Aided Parametrically Homogenized Crystal Plasticity Model (PHCPM) for Single Crystal Ni-Based SuperalloysJOM10.1007/s11837-020-04344-9Online publication date: 16-Sep-2020
https://doi.org/10.1007/s11837-020-04344-9
Agapitos ALoughran RNicolau MLucas SO'Neill MBrabazon A(2019)A Survey of Statistical Machine Learning Elements in Genetic ProgrammingIEEE Transactions on Evolutionary Computation10.1109/TEVC.2019.290091623:6(1029-1048)Online publication date: Dec-2019
https://doi.org/10.1109/TEVC.2019.2900916

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents