Abstract
The hybridization of evolutionary algorithms and local search techniques as, e.g., mathematical programming techniques, also referred to as memetic algorithms, has caught the interest of many researchers in the recent past. Reasons for this include that the resulting algorithms are typically robust and reliable since they take the best of both worlds. However, one crucial drawback of such hybrids is the relatively high cost of the local search techniques since many of them require the gradient or even the Hessian at each candidate solution. Here, we propose an alternative way to compute search directions by exploiting the neighborhood information. That is, for a given point within a population \(\mathcal{P}\), the neighboring solutions in \(\mathcal{P}\) are used to compute the most greedy search direction out of the given data. The method is hence particularly interesting for the usage within population-based search strategies since the search directions come ideally for free in terms of additional function evaluations. In this study, we analyze the novel method first as a stand-alone algorithm and show further on its benefit as a local searcher within differential evolution.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Problem g12 was discarded since it is not a continuous problem.
References
Auger A, Hansen N, Zerpa Perez J M, Ros R, Schoenauer M (2009) Experimental comparisons of derivative free optimization algorithms. In: Experimental algorithms, Springer, pp 3–15
Bäck T, Schwefel HP (1993) An overview of evolutionary algorithms for parameter optimization. Evol Comput 1(1):1–23
Bao Y, Hu Z, Xiong T (2013) A PSO and pattern search based memetic algorithm for SVMs parameters optimization. Neurocomputing 117:98–106
Beyer HG, Finck S (2012) Happycat a simple function class where well-known direct search algorithms do fail. In: Coello Coello CA, et al., (ed) Parallel problem solving from nature—PPSN XII, vol 7491 of Lecture Notes in Computer Science, Springer, Berlin, pp 367–376
Beyer HG, Schwefel HP (2002) Evol strategies: a comprehensive introduction Nat Comput 1(1):3–52
Brent RP (1973) Algorithms for minimization without derivatives, 1st edn. Prentice-Hall, Upper Saddle River, NJ
Brown M, Smith RE (2005) Directed multi-objective optimisation. Int J Comput Syst Signals 6(1):3–17
Caraffini F, Neri F, Iacca G (2013) Parallel memetic structures. Inf Sci 227:60–82
Caraffini F, Neri F, Picinali L (2014) An analysis on separability for memetic computing automatic design. Inf Sci 225:1–22
Chao G, Detong Z (2011) A secant algorithm with line search filter method for nonlinear optimization. Appl Math Model 35(2):879–894
Das S, Suganthan PN (2011) Differential evolution: a survey of the state-of-the-art. IEEE Trans Evol Comput 15(1):4–31
Dennis JE, Schnabel RB (1983) Numerical methods for unconstrained optimization and nonlinear equations. Prentice-Hall, Upper Saddle River
Domínguez IS, Aguirre AH, Valdez SI (2014) A new EDA by a gradient-driven density. In: Parallel problem solving from nature—PPSN XIII—13th international conference, pp 352–361
Durillo JJ, Nebro AJ, Coello Coello CA, Garcia-Nieto J, Luna F, Alba E (2010) A study of multiobjective metaheuristics when solving parameter scalable problems. IEEE Trans Evol Comput 14(4):618–635
Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, NewYork
Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Natural computing series. Springer, NewYork
Gong W, Cai Z, Ling CX (2006) ODE: a fast and robust differential evolution based on orthogonal design. In: Sattar A, Kang BH (eds) AI 2006: advances in artificial intelligence, vol 4304. Lecture Notes in Computer Science, Springer, Berlin, pp 709–718
Griewank A (2000) Evaluating derivatives: principles and techniques of algorithmic differentiation. Number 19 in Frontiers in Applied Mathematics SIAM, Philadelphia, PA
Hazen M, Gupta MR (2006) A multiresolutional estimated gradient architecture for global optimization. In: IEEE international conference on evolutionary computation, CEC, pp 3013–3020
Hooke R, Jeeves TA (1961) Direct search solution of numerical and statistical problems. J ACM 8(2):212–229
Junhua Z, Yan X, Luo L, ZhaoYang D, Yaoyao P (2014) Power system fault diagnosis based on history driven differential evolution and stochastic time domain simulation. Inf Sci 275:13–29
Kleijnen J P C (2015) Response surface methodology. In: Fu MC (ed) Handbook of simulation optimization, vol 216 of international series in operations research & management science, Springer, New York, pp 81–104
Kukkonen S, Lampinen J (2006) Constrained real-parameter optimization with generalized differential evolution. In: Evolutionary computation, 2006. CEC 2006. IEEE Congress on IEEE, pp 207–214
Lara A, Sanchez G, Coello Coello CA, Schütze O (2010) HCS: a new local search strategy for memetic multiobjective evolutionary algorithms. IEEE Trans Evol Comput 14(1):112–132
LaTorre A (2009) A framework for hybrid dynamic evolutionary algorithms: multiple offspring sampling (MOS). Ph.D. thesis
LaTorre A, Muelas S, Pena JM (2011) A mos-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test. Soft Comput 15(11):2187–2199
Li X, Yao X (2012) Cooperatively coevolving particle swarms for large scale optimization. Evol Comput IEEE Trans 16(2):210–224
Liang JJ, Runarsson TP, Mezura-Montes E, Clerc M, Suganthan PN, Coello Coello CA, Deb K (2006) Problem definitions and evaluation criteria for the cec 2006 special session on constrained real-parameter optimization. Technical report, Nanyang Technological University, Singapore
Moscato P (1989) On evolution, search, optimization, genetic algorithms and martial arts: Towards memetic algorithms. Technical report C3P Report 826, California Institute of Technology
Neri F, Cotta C, Moscato P (eds) (2012) Handbook of memetic algorithms, vol 379 of Studies in Computational Intelligence. Springer
Nocedal J, Wright S (2006) Numerical optimization. Springer series in operations research and financial engineering. Springer, NewYork
Omidvar MN, Mei Y, Li X (2014) Effective decomposition of large-scale separable continuous functions for cooperative co-evolutionary algorithms. In: Evolutionary computation (CEC), 2014 IEEE congress on IEEE, pp 1305–1312
Osyczka A, Krenich S (2006) Evolutionary algorithms for global optimization. In: Pintér JD (ed) Global optimization, vol 85. Springer, NewYork, pp 267–300
Polak E, Mayne DQA Robust secant method for optimization problems with inequality constraints. J Optim Theory Appl 33(4):463–477
Qin AK, Suganthan PN (2005) Self-adaptive differential evolution algorithm for numerical optimization. In IEEE congress on evolution computation 2005 (CEC’05), vol 2, pp 1785–1791
Schütze O, Martín A, Lara A, Alvarado S, Salinas E, Coello Coello CA (2015) The directed search method for multi-objective memetic algorithms. Comput Optim Appl 63(2):1–28
Schwefel HP (1993) Evolution and optimum seeking. Wiley, New York, NY
Shiwen Y, Anyong Q (2005) Design of high-power millimeter-wave TM\(_{01}\) - TE\(_{11}\) mode converters by the differential evolution algorithm. IEEE Trans Plasma Sci 33(4):1372–1376
Sivanandam SN, Deepa SN (2007) Introduction to genetic algorithms. Springer, NewYork
Storn R, Price K (1995) Differential evolution: a simple and efficient adaptive scheme for global optimization over continuous spaces. Technical report, International Computer Science Institute, Berkeley, Technical report TR95012
Talbi EG (2002) A taxonomy of hybrid metaheuristics. J. Heuristics 8(5):541–564
Tseng L-Y, Chen C (2008) Multiple trajectory search for large scale global optimization. In: Evolutionary computation, 2008. CEC 2008. (IEEE World Congress on Computational Intelligence). IEEE Congress, pp 3052–3059
Zapotecas Martínez S, Coello Coello CA, (2012) A direct local search mechanism for decomposition-based multi-objective evolutionary algorithms. (2012). In: IEEE congress on evolutionary computation (CEC’2012). IEEE Press, Brisbane
Acknowledgments
The first author acknowledges support from Conacyt Project No. 128554. The third author also acknowledges the financial support from CONCYTEG as part of the ‘Plan Investigadores Jóvenes - DPP-2014’.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
All authors declare that they have no conflict of interest. This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Schütze, O., Alvarado, S., Segura, C. et al. Gradient subspace approximation: a direct search method for memetic computing. Soft Comput 21, 6331–6350 (2017). https://doi.org/10.1007/s00500-016-2187-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-016-2187-x