Nothing Special   »   [go: up one dir, main page]

Skip to main content
Log in

Gradient subspace approximation: a direct search method for memetic computing

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The hybridization of evolutionary algorithms and local search techniques as, e.g., mathematical programming techniques, also referred to as memetic algorithms, has caught the interest of many researchers in the recent past. Reasons for this include that the resulting algorithms are typically robust and reliable since they take the best of both worlds. However, one crucial drawback of such hybrids is the relatively high cost of the local search techniques since many of them require the gradient or even the Hessian at each candidate solution. Here, we propose an alternative way to compute search directions by exploiting the neighborhood information. That is, for a given point within a population \(\mathcal{P}\), the neighboring solutions in \(\mathcal{P}\) are used to compute the most greedy search direction out of the given data. The method is hence particularly interesting for the usage within population-based search strategies since the search directions come ideally for free in terms of additional function evaluations. In this study, we analyze the novel method first as a stand-alone algorithm and show further on its benefit as a local searcher within differential evolution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://mathworks.com.

  2. Problem g12 was discarded since it is not a continuous problem.

References

  • Auger A, Hansen N, Zerpa Perez J M, Ros R, Schoenauer M (2009) Experimental comparisons of derivative free optimization algorithms. In: Experimental algorithms, Springer, pp 3–15

  • Bäck T, Schwefel HP (1993) An overview of evolutionary algorithms for parameter optimization. Evol Comput 1(1):1–23

    Article  Google Scholar 

  • Bao Y, Hu Z, Xiong T (2013) A PSO and pattern search based memetic algorithm for SVMs parameters optimization. Neurocomputing 117:98–106

  • Beyer HG, Finck S (2012) Happycat a simple function class where well-known direct search algorithms do fail. In: Coello Coello CA, et al., (ed) Parallel problem solving from nature—PPSN XII, vol 7491 of Lecture Notes in Computer Science, Springer, Berlin, pp 367–376

  • Beyer HG, Schwefel HP (2002) Evol strategies: a comprehensive introduction Nat Comput 1(1):3–52

    Google Scholar 

  • Brent RP (1973) Algorithms for minimization without derivatives, 1st edn. Prentice-Hall, Upper Saddle River, NJ

    MATH  Google Scholar 

  • Brown M, Smith RE (2005) Directed multi-objective optimisation. Int J Comput Syst Signals 6(1):3–17

    Google Scholar 

  • Caraffini F, Neri F, Iacca G (2013) Parallel memetic structures. Inf Sci 227:60–82

    Article  MathSciNet  Google Scholar 

  • Caraffini F, Neri F, Picinali L (2014) An analysis on separability for memetic computing automatic design. Inf Sci 225:1–22

    Article  MathSciNet  Google Scholar 

  • Chao G, Detong Z (2011) A secant algorithm with line search filter method for nonlinear optimization. Appl Math Model 35(2):879–894

    Article  MathSciNet  MATH  Google Scholar 

  • Das S, Suganthan PN (2011) Differential evolution: a survey of the state-of-the-art. IEEE Trans Evol Comput 15(1):4–31

    Article  Google Scholar 

  • Dennis JE, Schnabel RB (1983) Numerical methods for unconstrained optimization and nonlinear equations. Prentice-Hall, Upper Saddle River

    MATH  Google Scholar 

  • Domínguez IS, Aguirre AH, Valdez SI (2014) A new EDA by a gradient-driven density. In: Parallel problem solving from nature—PPSN XIII—13th international conference, pp 352–361

  • Durillo JJ, Nebro AJ, Coello Coello CA, Garcia-Nieto J, Luna F, Alba E (2010) A study of multiobjective metaheuristics when solving parameter scalable problems. IEEE Trans Evol Comput 14(4):618–635

    Article  MATH  Google Scholar 

  • Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, NewYork

    Book  MATH  Google Scholar 

  • Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Natural computing series. Springer, NewYork

    Book  MATH  Google Scholar 

  • Gong W, Cai Z, Ling CX (2006) ODE: a fast and robust differential evolution based on orthogonal design. In: Sattar A, Kang BH (eds) AI 2006: advances in artificial intelligence, vol 4304. Lecture Notes in Computer Science, Springer, Berlin, pp 709–718

  • Griewank A (2000) Evaluating derivatives: principles and techniques of algorithmic differentiation. Number 19 in Frontiers in Applied Mathematics SIAM, Philadelphia, PA

  • Hazen M, Gupta MR (2006) A multiresolutional estimated gradient architecture for global optimization. In: IEEE international conference on evolutionary computation, CEC, pp 3013–3020

  • Hooke R, Jeeves TA (1961) Direct search solution of numerical and statistical problems. J ACM 8(2):212–229

    Article  MATH  Google Scholar 

  • Junhua Z, Yan X, Luo L, ZhaoYang D, Yaoyao P (2014) Power system fault diagnosis based on history driven differential evolution and stochastic time domain simulation. Inf Sci 275:13–29

    Article  MathSciNet  Google Scholar 

  • Kleijnen J P C (2015) Response surface methodology. In: Fu MC (ed) Handbook of simulation optimization, vol 216 of international series in operations research & management science, Springer, New York, pp 81–104

  • Kukkonen S, Lampinen J (2006) Constrained real-parameter optimization with generalized differential evolution. In: Evolutionary computation, 2006. CEC 2006. IEEE Congress on IEEE, pp 207–214

  • Lara A, Sanchez G, Coello Coello CA, Schütze O (2010) HCS: a new local search strategy for memetic multiobjective evolutionary algorithms. IEEE Trans Evol Comput 14(1):112–132

    Article  Google Scholar 

  • LaTorre A (2009) A framework for hybrid dynamic evolutionary algorithms: multiple offspring sampling (MOS). Ph.D. thesis

  • LaTorre A, Muelas S, Pena JM (2011) A mos-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test. Soft Comput 15(11):2187–2199

    Article  Google Scholar 

  • Li X, Yao X (2012) Cooperatively coevolving particle swarms for large scale optimization. Evol Comput IEEE Trans 16(2):210–224

    Article  MathSciNet  Google Scholar 

  • Liang JJ, Runarsson TP, Mezura-Montes E, Clerc M, Suganthan PN, Coello Coello CA, Deb K (2006) Problem definitions and evaluation criteria for the cec 2006 special session on constrained real-parameter optimization. Technical report, Nanyang Technological University, Singapore

  • Moscato P (1989) On evolution, search, optimization, genetic algorithms and martial arts: Towards memetic algorithms. Technical report C3P Report 826, California Institute of Technology

  • Neri F, Cotta C, Moscato P (eds) (2012) Handbook of memetic algorithms, vol 379 of Studies in Computational Intelligence. Springer

  • Nocedal J, Wright S (2006) Numerical optimization. Springer series in operations research and financial engineering. Springer, NewYork

    Google Scholar 

  • Omidvar MN, Mei Y, Li X (2014) Effective decomposition of large-scale separable continuous functions for cooperative co-evolutionary algorithms. In: Evolutionary computation (CEC), 2014 IEEE congress on IEEE, pp 1305–1312

  • Osyczka A, Krenich S (2006) Evolutionary algorithms for global optimization. In: Pintér JD (ed) Global optimization, vol 85. Springer, NewYork, pp 267–300

    Chapter  Google Scholar 

  • Polak E, Mayne DQA Robust secant method for optimization problems with inequality constraints. J Optim Theory Appl 33(4):463–477

  • Qin AK, Suganthan PN (2005) Self-adaptive differential evolution algorithm for numerical optimization. In IEEE congress on evolution computation 2005 (CEC’05), vol 2, pp 1785–1791

  • Schütze O, Martín A, Lara A, Alvarado S, Salinas E, Coello Coello CA (2015) The directed search method for multi-objective memetic algorithms. Comput Optim Appl 63(2):1–28

  • Schwefel HP (1993) Evolution and optimum seeking. Wiley, New York, NY

    MATH  Google Scholar 

  • Shiwen Y, Anyong Q (2005) Design of high-power millimeter-wave TM\(_{01}\) - TE\(_{11}\) mode converters by the differential evolution algorithm. IEEE Trans Plasma Sci 33(4):1372–1376

    Article  Google Scholar 

  • Sivanandam SN, Deepa SN (2007) Introduction to genetic algorithms. Springer, NewYork

    MATH  Google Scholar 

  • Storn R, Price K (1995) Differential evolution: a simple and efficient adaptive scheme for global optimization over continuous spaces. Technical report, International Computer Science Institute, Berkeley, Technical report TR95012

  • Talbi EG (2002) A taxonomy of hybrid metaheuristics. J. Heuristics 8(5):541–564

    Article  Google Scholar 

  • Tseng L-Y, Chen C (2008) Multiple trajectory search for large scale global optimization. In: Evolutionary computation, 2008. CEC 2008. (IEEE World Congress on Computational Intelligence). IEEE Congress, pp 3052–3059

  • Zapotecas Martínez S, Coello Coello CA, (2012) A direct local search mechanism for decomposition-based multi-objective evolutionary algorithms. (2012). In: IEEE congress on evolutionary computation (CEC’2012). IEEE Press, Brisbane

Download references

Acknowledgments

The first author acknowledges support from Conacyt Project No. 128554. The third author also acknowledges the financial support from CONCYTEG as part of the ‘Plan Investigadores Jóvenes - DPP-2014’.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Oliver Schütze.

Ethics declarations

Conflicts of interest

All authors declare that they have no conflict of interest. This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schütze, O., Alvarado, S., Segura, C. et al. Gradient subspace approximation: a direct search method for memetic computing. Soft Comput 21, 6331–6350 (2017). https://doi.org/10.1007/s00500-016-2187-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2187-x

Keywords

Navigation