Evolving artificial neural networks for nonlinear feature construction
Pages 1037 - 1044
Abstract
We use neuroevolution to construct nonlinear transformation functions for feature construction that map points in the original feature space to augmented pattern vectors and improve the performance of generic classifiers. Our research demonstrates that we can apply evolutionary algorithms to both adapt the weights of a fully connected standard multi-layer perceptron (MLP), and optimize the topology of a generalized multi-layer perceptron (GMLP). The evaluation of the MLPs on four commonly used data sets shows an improvement in classification accuracy ranging from 4 to 13 percentage points over the performance on the original pattern set. The GMLPs obtain a slightly better accuracy and conserve 14% to 54% of all neurons and between 40% and 89% of all connections compared to the standard MLP.
References
[1]
C. Aggarwal. The Generalized Dimensionality Reduction Problem. In Proc. SDM, pages 607--618. SIAM, 2010.
[2]
A. Aizerman, E. M. Braverman, and L. I. Rozoner. Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning. Automat. and Rem. Contr., 25:821--837, 1964.
[3]
T. Back. Evolutionary Algorithms in Theory and Practice. Oxford University Press, 1996.
[4]
Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy Layer-Wise Training of Deep Networks. In Proc. NIPS, pages 153--160, 2006.
[5]
T. Berka and H. A. Mayer. Nonlinear Feature Construction with Evolved Neural Networks for Classification Problems. In Proc. ICPRAM. INSTICC, 2012.
[6]
C. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, Inc., USA, 1995.
[7]
T.-J. Chin and D. Suter. Incremental Kernel PCA for Efficient Non-linear Feature Extraction. In Proc. BMVC, pages 939--948. BMVA, 2006.
[8]
A. Coelho, D. Weingaertner, and F. J. von Zuben. Evolving Heteregeneous Neural Networks for Classification Problems. In Proc. GECCO, pages 266--273, USA, July 2001. Morgan Kaufmann.
[9]
P. Cortez, A. Cerdeira, F. Almeida, T. Matos, and J. Reis. Modeling Wine Preferences by Data Mining from Physicochemical Properties. Decis. Support Syst., 47(4):547--553, 2009.
[10]
M. Dragoni, A. Azzini, and A. Tettamanzi. A Neuro-Evolutionary Approach to Electrocardiographic Signal Classification. In Proc. WIVACE, pages 1--11, Italy, 2012. Universit`a degli Studi di Parma, Dipartimento di Scienze Sociali.
[11]
A. Frank and A. Asuncion. UCI Machine Learning Repository, 2010.
[12]
H. Guo and A. K. Nandi. Breast Cancer Diagnosis using Genetic Programming Generated Feature. Pattern Recogn., 39(5):980--987, 2006.
[13]
G. E. Hinton, S. Osindero, and Y. W. Teh. A Fast Learning Algorithm for Deep Belief Nets. Neural Comput., 18:1527--1554, 2006.
[14]
G.-B. Huang, D. H. Wang, and Y. Lan. Extreme Learning Machines: A Survey. Int. J. Mach. Learn. & Cyber., 2(2):107--122, 2011.
[15]
G.-B. Huang, H. Zhou, X. Ding, and R. Zhang. Extreme Learning Machine for Regression and Multiclass Classification. IEEE Trans. Syst. Man Cybern. B. Cybern., 42(2):513--529, 2012.
[16]
R. Huber, R. Schwaiger, and H. A. Mayer. On the Role of Regularization Parameters in Fitness Functions for Evolutionary Designed Artificial Neural Networks. In Proc. WCNN, pages 1063--1066. INNS, Lawrence Erlbaum Associates, September 1996.
[17]
A. Ittner and M. Schlosser. Discovery of Relevant New Features by Generating Non-Linear Decision Trees. In Proc. KDD, pages 108--113. AAAI, 1996.
[18]
G. H. John, R. Kohavi, and K. Pfleger. Irrelevant Features and the Subset Selection Problem. In Proc. ICML, pages 121--129, 1994.
[19]
K. I. Kim, M. O. Franz, and B. Scholkopf. Iterative Kernel Principal Component Analysis for Image Modeling. IEEE Trans. Pattern Anal. Mach. Intell., 27(9):1351--1366, 2005.
[20]
A. Mayer and H. A. Mayer. Multi--Chromosomal Representations in Neuroevolution. In Proc. CI. ACTA Press, November 2006.
[21]
H. A. Mayer and R. Schwaiger. Differentiation of Neuron Types by Evolving Activation Function Templates for Artificial Neural Networks. In Proc. WCCI, pages 1773--1778. IEEE, May 2002.
[22]
K. Neshatian, M. Zhang, and P. Andreae. A Filter Approach to Multiple Feature Construction for Symbolic Learning Classifiers Using Genetic Programming. IEEE Trans. Evol. Comput., 16(5):645--661, Oct. 2012.
[23]
M. Ranzato, Y.-l. Boureau, S. Chopra, and Y. Lecun. A Unified Energy-Based Framework for Unsupervised Learning. In Proc. AISTATS, volume 2, pages 860--867, 2007.
[24]
D. E. Rumelhart, B. Widrow, and M. A. Lehr. The Basic Ideas in Neural Networks. Comm. ACM, 37(3):87--92, March 1994.
[25]
B. Scholkopf, A. Smola, and K.-R. Muller. Nonlinear Component Analysis as a Kernel Eigenvalue Problem. Neural Comput., 10(5):1299--1319, 1998.
[26]
M. J. J. Scott, M. Niranjan, and R. W. Prager. Realisable Classifiers: Improving Operating Performance on Variable Cost Problems. In Proc. BMVC. BMVA, 1998.
[27]
J. R. Sherrah, R. E. Bogner, and A. Bouzerdoum. The Evolutionary Pre-Processor: Automatic Feature Extraction for Supervised Classification using Genetic Programming. In Proc. GP, pages 304--312. Morgan Kaufmann, July 1997.
[28]
V. G. Sigillito, S. P. Wing, L. V. Hutton, and K. B. Baker. Classification of Radar Returns from the Ionosphere using Neural Networks. Johns Hopkins APL Tech. Dig., 10:262--266, 1989.
[29]
W. N. Street, W. H. Wolberg, and O. L. Mangasarian. Nuclear Feature Extraction for Breast Tumor Diagnosis. In Proc. Electronic Imaging, volume 1905, pages 861--870. SPIE, 1993.
[30]
V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, USA, 1995.
[31]
X. Yao. Evolving Artificial Neural Networks. Proc. IEEE, 87(9):1423--1447, 1999.
[32]
Y. Zhang and P. I. Rockett. A Generic Optimising Feature Extraction Method Using Multiobjective Genetic Programming. Appl. Soft Comput., 11(1):1087--1097, Jan. 2011.
[33]
Q.-Y. Zhu, A. Qin, P. Suganthan, and G.-B. Huang. Evolutionary Extreme Learning Machine. Pattern Recognition, 38(10):1759--1763, 2005.
[34]
T. Ziemke, J. Carlsson, and M. Boden. An Experimental Comparison of Weight Evolution in Neural Control Architectures for a 'Garbage-Collecting' Khepera Robot. In Proc. Khepera Workshop. HNI--Verlagsschriftenreihe, 1999.
Index Terms
- Evolving artificial neural networks for nonlinear feature construction
Recommendations
Noise Suppression by Artificial Neural Networks
AWICT 2017: Proceedings of the Second International Conference on Advanced Wireless Information, Data, and Communication TechnologiesIn this research paper, noise suppression ability of associative memories (e.g. Hopfield neural network) is briefly summarized. Motivated by this fact, noise suppression abilities of trained convolutional network are discussed. Introducing the concept ...
A Genetically Optimized Artificial Neural Network Structure for Feature Extraction and Classification of Vascular Tissue Fluorescence Spectrums
CAMP '00: Proceedings of the Fifth IEEE International Workshop on Computer Architectures for Machine Perception (CAMP'00)The optimization of Neural Network structures for feature extraction and classification by employing Genetic Algorithms is addressed here. More precisely, a non-linear filter based on High Order Neural Networks (HONN) whose weights are updated by stable ...
Using Feature Construction to Improve the Performance of Neural Networks
Recent years have seen the growth in popularity of neural networks for business decision support because of their capabilities for modeling, estimating, and classifying. Compared to other AI methods for problem solving such as expert systems, neural ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2013
1672 pages
Copyright © 2013 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 July 2013
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
GECCO '13
Sponsor:
GECCO '13: Genetic and Evolutionary Computation Conference
July 6 - 10, 2013
Amsterdam, The Netherlands
Acceptance Rates
GECCO '13 Paper Acceptance Rate 204 of 570 submissions, 36%;
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 121Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 21 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in