Article

A differential evolution based incremental training method for RBF networks

Authors:

Jouni LampinenAuthors Info & Claims

GECCO '05: Proceedings of the 7th annual conference on Genetic and evolutionary computation

Pages 881 - 888

https://doi.org/10.1145/1068009.1068157

Published: 25 June 2005 Publication History

Abstract

The Differential Evolution (DE) is a floating-point encoded evolutionary strategy for global optimization. It has been demonstrated to be an efficient, effective, and robust optimization method, especially for problems containing continuous variables. This paper concerns applying a DE-based algorithm to training Radial Basis Function (RBF) networks with variables including centres, weights, and widths of RBFs. The proposed algorithm consists of three steps: the first step is the initial tuning, which focuses on searching for the center, weight, and width of a one-node RBF network, the second step is the local tuning, which optimizes the three variables of the one-node RBF network --- its centre, weight, and width, and the third step is the global tuning, which optimizes all the parameters of the whole network together. The second step and the third step both use the cycling scheme to find the parameters of RBF network. The Mean Square Error from the desired to actual outputs is applied as the objective function to be minimized. Training the networks is demonstrated by approximating a set of functions, using different strategies of DE. A comparison of the net performances with several approaches reported in the literature is given and shows the resulting network performs better in the tested functions. The results show that proposed method improves the compared approximation results.

References

[1]

A. Alexandridis, H. Sarimveis, and G. Bafas. A new algorithm for online structure and parameter adaptation of rbf networks. Neural Networks, 16:1003--1017, 2003.

Digital Library

[2]

S. A. Billings and G. L. Zheng. Radial basis function networks configuration using genetic algorithms. Neural Networks, 8(6):877--890, 1995.

Digital Library

[3]

A. G. Bors and I. Pitas. Median radial basis function neural network. IEEE Trans. on Neural Network, 7(6):1351--1364, 1996.

Digital Library

[4]

D. S. Broomhead and D. Lowe. Multivariable functional interpolation and adaptive networks. Complex Systems, 2:321--355, 1998.

[5]

A. Esposito, M. Marinaro, D. Oricchio, and S. Scarpetta. Approximation of continuous and discontinuous mappings by a growing neural rbf-based algorithm. Neural Networks, 13:651--665, 2000.

Digital Library

[6]

C. F. Fung, S. A. Billings, and W. Luo. On-line supervised adaptive training using radial basis function networks. Neural Networks, 9(9):1597--1617, 1996.

Digital Library

[7]

S. Haykin. Neural networks: a comprehensive foundation. Macmillan College Publishing Company, New York, 1994.

Digital Library

[8]

J. Lampinen and I. Zelinka. On stagnation of the differential evolution algorithm. In Proc. of 6th Int'l Conf. on Soft Computing (MENDEL), pages 76--83. Brno, Czech Republic, June 7-9, 2000.

[9]

A. Leonardis and H. Bischof. An efficient mdl-based construction of rbf networks. Neural Networks, 11:963--973, 1998.

Digital Library

[10]

J. Liu, S. Kukkonen, and J. Lampinen. Function approximation with pruned radial basis function networks using a de-based algorithm: an initial investigation. In Proc. of 10th Int'l Conf. on Soft Computing (MENDEL), pages 139--144. Brno, Czech Republic, June 16-18, 2004.

[11]

J. Liu and J. Lampinen. On setting the control parameters of the differential evolution algorithm. In Proc. of 8th Int'l Conf. on Soft Computing (MENDEL), pages 11--18. Brno, Czech Republic, June 5-7, 2002.

[12]

J. Liu and J. Lampinen. Growing rbf networks for function approxiamtion by a de-based method. In Proc. of 1st Int'l Symposium on Computational and Information Science, pages 399--406. Shanghai, China, Dec 16-18, 2004.

Digital Library

[13]

J. Liu and J. Lampinen. A fuzzy differential evolution algorithm. Soft Computing, (to appear).

Digital Library

[14]

M. Marinaro and S. Scarpetta. On-line learning in rbf neural networks: a stochastic approach. Neural Networks, 13:719--729, 2000.

Digital Library

[15]

M. T. Musavi, W. Ahmed, K. H. Chan, K. B. Faris, and D. M. Hummels. On the training of radial basis function classifiers. Neural Networks, 5:595--603, 1992.

Digital Library

[16]

J. Park and J. W. Sandberg. Universal approximation using radial-basis-function networks. Neural Computation, 3:246--257, 1991.

[17]

V. P. Plagianakos and M. N. Vrahatis. Neural network training with constrained integer weights. In Proc. 1999 Congress on Evolutionary Computation, vol. 3, pages 2007--2013. Washington, DC USA, July 6-9, 1999.

[18]

T. Poggio and F. Girosi. Networks for approximation and learning. In Proc. IEEE, vol. 78, no. 9, pages 1481--1497. MIT, Cambridge, MA, USA, September 1990.

[19]

K. Price, R. Storn, and J. A. Lampinen. Differential evolution: a practical approach to global optimization. Springer Berlin/Heidelberg, Germany, (in print).

Digital Library

[20]

G. P. J. Schmitz and C. Aldrich. Combinatorial evolution of regression nodes in feedforward neural networks. Neural Networks, 12:175--189, 1999.

Digital Library

[21]

R. Storn and K. Price. On the usage of differential evolution for function optimization. In Proc. of 1996 Biennial Conference of the North American Fuzzy Information Processing Society (NAFIPS), pages 519--523. Berkeley, CA USA, June 19-22, 1996.

[22]

R. Storn and K. Price. Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4):341--359, December 1997.

Digital Library

[23]

R. Storn and K. Price. Differential evolution - a simple evolution strategy for fast optimization. Dr. Dobb's Journal, 22(4):18--24 and 78, April 1997.

[24]

V. Vinod and S. Ghose. Growing nonuniform feedforward networks for continuous mapping. Neural computing, 10:55--69, 1996.

[25]

Z. Wang and T. Zhu. An efficient learning algorithm for improving generalization performance of radial basis function neural networks. Neural Networks, 13:545--553, 2000.

Digital Library

[26]

F. Yong and T. W. S. Chow. Neural network adaptive wavelets for function approximation. In Proc. of European Symposium on Artificial Neural Networks, pages 345--350. Bruges, Belgium, D-Facto public., Apr 16-18, 1997.

[27]

Q. Zhu, Y. Cai, and L. Liu. A global learning algorithm for a rbf network. Neural Networks, 12:527--540, 1999.

Digital Library

Cited By

Charilogis VTsoulos ITzallas AKarvounis E(2022)Modifications for the Differential Evolution AlgorithmSymmetry10.3390/sym1403044714:3(447)Online publication date: 23-Feb-2022
https://doi.org/10.3390/sym14030447
Zhang YGong CFang HSu HLi CRonch A(2019)An efficient space division---based width optimization method for RBF network using fuzzy clustering algorithmsStructural and Multidisciplinary Optimization10.1007/s00158-019-02217-760:2(461-480)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1007/s00158-019-02217-7
Dash CSahoo PDehuri SCho S(2015)An Empirical Analysis of Evolved Radial Basis Function Networks and Support Vector Machines with Mixture of KernelsInternational Journal on Artificial Intelligence Tools10.1142/s021821301550013x24:04(1550013)Online publication date: 21-Aug-2015
https://doi.org/10.1142/s021821301550013x
Show More Cited By

Index Terms

A differential evolution based incremental training method for RBF networks

Recommendations

Differential Evolution Training Algorithm for Feed-Forward Neural Networks

An evolutionary optimization method over continuous search spaces, differential evolution, has recently been successfully applied to real world and artificial optimization problems and proposed also for neural network training. However, differential ...
Nonlinear system identification using memetic differential evolution trained neural networks

Several gradient-based approaches such as back propagation (BP) and Levenberg Marquardt (LM) methods have been developed for training the neural network (NN) based systems. But, for multimodal cost functions these procedures may lead to local minima, ...
Training RBF networks with selective backpropagation

Backpropagation with selective training (BST) is applied on training radial basis function (RBF) networks. It improves the performance of the RBF network substantially, in terms of convergence speed and recognition error. Three drawbacks of the basic ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '05: Proceedings of the 7th annual conference on Genetic and evolutionary computation

June 2005

2272 pages

ISBN:1595930108

DOI:10.1145/1068009

Editor:
Hans-Georg Beyer
Vorarlberg University of Applied Sciences, Austria
,
General Chair:
Una-May O'Reilly
CSAIL, MIT

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

GECCO05

Sponsor:

GECCO05: Genetic and Evolutionary Computation Conference

June 25 - 29, 2005

Washington DC, USA

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
518
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Charilogis VTsoulos ITzallas AKarvounis E(2022)Modifications for the Differential Evolution AlgorithmSymmetry10.3390/sym1403044714:3(447)Online publication date: 23-Feb-2022
https://doi.org/10.3390/sym14030447
Zhang YGong CFang HSu HLi CRonch A(2019)An efficient space division---based width optimization method for RBF network using fuzzy clustering algorithmsStructural and Multidisciplinary Optimization10.1007/s00158-019-02217-760:2(461-480)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1007/s00158-019-02217-7
Dash CSahoo PDehuri SCho S(2015)An Empirical Analysis of Evolved Radial Basis Function Networks and Support Vector Machines with Mixture of KernelsInternational Journal on Artificial Intelligence Tools10.1142/s021821301550013x24:04(1550013)Online publication date: 21-Aug-2015
https://doi.org/10.1142/s021821301550013x
Sossa HCortés GGuevara E(2014)New Radial Basis Function Neural Network Architecture for Pattern Classification: First ResultsAdvanced Information Systems Engineering10.1007/978-3-319-12568-8_86(706-713)Online publication date: 2014
https://doi.org/10.1007/978-3-319-12568-8_86
Dash CBehera ADehuri SCho S(2013)Differential Evolution-Based Optimization of Kernel Parameters in Radial Basis Function Networks for ClassificationInternational Journal of Applied Evolutionary Computation10.4018/jaec.20130101044:1(56-80)Online publication date: 1-Jan-2013
https://doi.org/10.4018/jaec.2013010104
Wen Yao Xiaoqian Chen Yong Zhao van Tooren M(2012)Concurrent Subspace Width Optimization Method for RBF Neural Network ModelingIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2011.217856023:2(247-259)Online publication date: Feb-2012
https://doi.org/10.1109/TNNLS.2011.2178560
Wu JXu XZuo LLi ZWang J(2011)Adaptive kernel-width selection for kernel-based least-squares policy iteration algorithmProceedings of the 8th international conference on Advances in neural networks - Volume Part II10.5555/2009324.2009402(611-619)Online publication date: 29-May-2011
https://dl.acm.org/doi/10.5555/2009324.2009402
Wu JXu XZuo LLi ZWang J(2011)Adaptive Kernel-Width Selection for Kernel-Based Least-Squares Policy Iteration AlgorithmAdvances in Neural Networks – ISNN 201110.1007/978-3-642-21090-7_70(611-619)Online publication date: 2011
https://doi.org/10.1007/978-3-642-21090-7_70
Yao WChen Xvan Tooren MWei Y(2010)Euclidean distance and second derivative based widths optimization of radial basis function neural networksThe 2010 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2010.5596528(1-8)Online publication date: Jul-2010
https://doi.org/10.1109/IJCNN.2010.5596528
Luo LZhou L(2010)Application of Radial Basis Function Neural Network in Modeling Wastewater Sludge Recycle SystemLife System Modeling and Intelligent Computing10.1007/978-3-642-15859-9_17(117-122)Online publication date: 2010
https://doi.org/10.1007/978-3-642-15859-9_17
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents