short-survey

Evolutionary deep learning: : A survey

Authors:

Jun ZhangAuthors Info & Claims

Volume 483, Issue C

Pages 42 - 58

https://doi.org/10.1016/j.neucom.2022.01.099

Published: 28 April 2022 Publication History

Abstract

As an advanced artificial intelligence technique for solving learning problems, deep learning (DL) has achieved great success in many real-world applications and attracted increasing attention in recent years. However, as the performance of DL depends on many factors such as the architecture and hyperparameters, how to optimize DL has become a hot research topic in the field of DL and artificial intelligence. Evolutionary computation (EC), including evolutionary algorithm and swarm intelligence, is a kind of efficient and intelligent optimization methodology inspired by the mechanisms of biological evolution and behaviors of swarm organisms. Therefore, a large number of researches have proposed EC algorithms to optimize DL, so called evolutionary deep learning (EDL), which have obtained promising results. Given the great progress and rapid development of EDL in recent years, it is quite necessary to review these developments in order to summarize previous research experiences and knowledge, as well as provide references to benefit the development of more researches and applications. For this aim, this paper categorizes existing works in a two-level taxonomy. The higher level includes four categories based on when the EC can be adopted in optimizing the DL, which are the four procedures of the whole DL lifetime, including data processing, model search, model training, and model evaluation and utilization. In the lower level, related works in each category are further classified according to the functionality and the aim of using EC in the corresponding DL procedure, i.e., why using EC in this DL procedure. As a result, the taxonomy can clearly show how an EC algorithm can be used to optimize and improve DL. Moreover, this survey also discusses the potential research directions to provide the prospect of EDL in the future.

References

[1]

Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444,.

[2]

X. Yan, S. Hu, Y. Mao, Y. Ye, H. Yu, Deep multi-view learning methods: A review, Neurocomputing 448 (2021) 106–129,.

[3]

I. Lauriola, A. Lavelli, F. Aiolli, An introduction to deep learning in natural language processing: Models, techniques, and tools, Neurocomputing 470 (2022) 443–456,.

Digital Library

[4]

C. Li, S. Zhang, Y. Qin, E. Estupinan, A systematic review of deep transfer learning for machinery fault diagnosis, Neurocomputing 407 (2020) 121–135,.

[5]

S. Jia, S. Jiang, Z. Lin, N. Li, M. Xu, S. Yu, A survey: Deep learning for hyperspectral image classification with few labeled samples, Neurocomputing 448 (2021) 179–204,.

[6]

Z. Niu, G. Zhong, H. Yu, A review on the attention mechanism of deep learning, Neurocomputing 452 (2021) 48–62,.

[7]

S. Kuutti, R. Bowden, Y. Jin, P. Barber, S. Fallah, A survey of deep learning applications to autonomous vehicle control, IEEE Trans. Intell. Transp. Syst. 22 (2) (2021) 712–733.

[8]

Y. Sun, B. Xue, M. Zhang, G.G. Yen, J. Lv, Automatically designing CNN architectures using the genetic algorithm for image classification, IEEE Trans. Cybern. 50 (9) (2020) 3840–3854.

[9]

Y. Hua, J. Guo, H. Zhao, Deep belief networks and deep learning, Proc. 2015 Int. Conf. Intell. Comput. Internet Things, ICIT 2015. (2015) 1–4. https://doi.org/10.1109/ICAIOT.2015.7111524.

[10]

N. Zeng, H. Li, Y. Peng, A new deep belief network-based multi-task learning for diagnosis of Alzheimer’s disease, Neural Comput. Appl. 6 (2021),.

Digital Library

[11]

N. Zeng, H. Li, Z. Wang, W. Liu, S. Liu, F.E. Alsaadi, X. Liu, Deep-reinforcement-learning-based images segmentation for quantitative analysis of gold immunochromatographic strip, Neurocomputing 425 (2021) 173–180,.

[12]

Y. Sun, B. Xue, M. Zhang, G.G. Yen, A particle swarm optimization-based flexible convolutional autoencoder for image classification, IEEE Trans. Neural Networks Learn. Syst. 30 (8) (2019) 2295–2309,.

[13]

N. Xue, I. Triguero, G.P. Figueredo, D. Landa-Silva, Evolving deep CNN-LSTMs for inventory time series prediction, in: Proc. IEEE Congr. Evol. Comput. CEC (2019) 1517–1524. https://doi.org/10.1109/CEC.2019.8789957.

[14]

C. Paulin, S.A. Selouani, E. Hervet, Speech steganalysis using evolutionary restricted Boltzmann machines, in: Proc. IEEE Congr. Evol. Comput. (2016) 4831–4838. https://doi.org/10.1109/CEC.2016.7744409.

[15]

Z.H. Zhan, L. Shi, K.C. Tan, J. Zhang, A survey on evolutionary computation for complex continuous optimization, Artif. Intell. Rev. 55 (1) (2022) 59–110,.

Digital Library

[16]

Z.H. Zhan, J. Zhang, Y. Lin, J.Y. Li, T. Huang, X.Q. Guo, F.F. Wei, S. Kwong, X.Y. Zhang, R. You, Matrix-based evolutionary computation, IEEE Trans. Emerg. Top. Comput. Intell. (2021),.

[17]

D.B. Fogel, Evolutionary Computation: Toward A New Philosophy of Machine Intelligence, IEEE Press, Piscataway, USA, 1995.

[18]

J. Kennedy, R. Eberhart, Y. Shi, Swarm Intelligence, Morgan Kaufmann, San Mateo, CA, 2001.

[19]

D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Boston, MA, USA, 1989.

Digital Library

[20]

S.C. Liu, Z.G. Chen, Z.H. Zhan, S.W. Jeon, S. Kwong, J. Zhang, Many-objective job-shop scheduling: a multiple populations for multiple objectives-based genetic algorithm approach, IEEE Trans. Cybern. (2021),.

[21]

H.G. Beyer, H.P. Schwefel, Evolution strategies—A comprehensive introduction, Natural Comput. 1 (2002) (2002) 43–52.

[22]

J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press, Cambridge, MA, USA, 1992.

Digital Library

[23]

X. Yao, Y. Liu, G. Lin, Evolutionary programming made faster, IEEE Trans. Evol. Comput. 3 (1999) 82–102,.

Digital Library

[24]

R. Storn, K. Price, Differential evolution – A simple and efficient heuristic for global optimization over continuous spaces, J. Glob. Optim. 11 (1997) 341–359,.

Digital Library

[25]

Z.J. Wang, Z.H. Zhan, Y. Lin, W.J. Yu, H. Wang, S. Kwong, J. Zhang, Automatic niching differential evolution with contour prediction approach for multimodal optimization problems, IEEE Trans. Evol. Comput. 24 (1) (2020) 114–128,.

Digital Library

[26]

J.Y. Li, Z.H. Zhan, K.C. Tan, J. Zhang, A meta-knowledge transfer-based differential evolution for multitask optimization, IEEE Trans. Evol. Comput. (2021),.

[27]

X.F. Liu, Z.H. Zhan, Y. Lin, W.N. Chen, Y.J. Gong, T.L. Gu, H.Q. Yuan, J. Zhang, Historical and heuristic-based adaptive differential evolution, IEEE Trans. Syst. Man Cybern. Syst. 49 (12) (2019) 2623–2635,.

[28]

P. Larrañaga, J.A. Lozano, Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation, Kluwer Academic, Boston, MA, 2002.

[29]

Z.-G. Chen, Y. Lin, Y.-J. Gong, Z.-H. Zhan, J. Zhang, Maximizing lifetime of range-adjustable wireless sensor networks: A neighborhood-based estimation of distribution algorithm, IEEE Trans. Cybern. 51 (11) (2021) 5433–5444,.

[30]

M. Dorigo, L.M. Gambardella, Ant colony system: A cooperative learning approach to the traveling salesman problem, IEEE Trans. Evol. Comput. 1 (1997) 53–66,.

Digital Library

[31]

Z.-G. Chen, Z.-H. Zhan, Y. Lin, Y.-J. Gong, T.-L. Gu, F. Zhao, H.-Q. Yuan, X. Chen, Q. Li, J. Zhang, Multiobjective cloud workflow scheduling: a multiple populations ant colony system approach, IEEE Trans. Cybern. 49 (8) (2019) 2912–2926,.

[32]

S.-Z. Zhou, Z.-H. Zhan, Z.-G. Chen, S. Kwong, J. Zhang, A multi-objective ant colony system algorithm for airline crew rostering problem with fairness and satisfaction, IEEE Trans. Intell. Transp. Syst. 22 (11) (2021) 6784–6798.

[33]

J. Kennedy, R.C. Eberhart, Particle swarm optimization, in: Proc. Int. Conf. Neural Networks, 1995, pp. 1942–1948.

[34]

X. Zhang, K.-J. Du, Z.-H. Zhan, S. Kwong, T.-L. Gu, J. Zhang, Cooperative coevolutionary bare-bones particle swarm optimization with function independent decomposition for large-scale supply chain network design with uncertainties, IEEE Trans. Cybern. 50 (10) (2020) 4454–4468,.

[35]

X.-F. Liu, Z.-H. Zhan, Y. Gao, J. Zhang, S. Kwong, J. Zhang, Coevolutionary particle swarm optimization with bottleneck objective learning strategy for many-objective optimization, IEEE Trans. Evol. Comput. 23 (4) (2019) 587–602,.

[36]

W. Liu, Z. Wang, N. Zeng, Y. Yuan, F.E. Alsaadi, X. Liu, A novel sigmoid-function-based adaptive weighted particle swarm optimizer, IEEE Trans. Cybern. 51 (2021) 1085–1093,.

[37]

N. Zeng, Z. Wang, W. Liu, H. Zhang, K. Hone, X. Liu, A dynamic neighborhood-based switching particle swarm optimization algorithm, IEEE Trans. Cybern. (2020) 1–12,.

[38]

N. Zeng, D. Song, H. Li, Y. You, Y. Liu, F.E. Alsaadi, A competitive mechanism integrated multi-objective whale optimization algorithm with differential evolution, Neurocomputing 432 (2021) 170–182,.

[39]

X. Yao, Evolving artificial neural networks, Proc. IEEE 87 (9) (1999) 1423–1447.

[40]

J. Zhang, Z.H. Zhan, Y. Lin, N.i. Chen, Y.J. Gong, J.H. Zhong, H.S.H. Chung, Y. Li, Y.H. Shi, Evolutionary computation meets machine learning: a survey, IEEE Comput. Intell. Mag. 6 (4) (2011) 68–75,.

Digital Library

[41]

H. Al-Sahaf, Y. Bi, Q.i. Chen, A. Lensen, Y.i. Mei, Y. Sun, B. Tran, B. Xue, M. Zhang, A survey on evolutionary machine learning, J. R. Soc. New Zeal. 49 (2) (2019) 205–228,.

[42]

Z.G. Chen, Z.H. Zhan, W. Shi, W. Chen, J. Zhang, When neural network computation meets evolutionary computation: a survey, Int. Symp. Neural Networks (2016) 603–612,.

[43]

M. Li, Z. Wang, K. Li, X. Liao, K. Hone, X. Liu, Task allocation on layered multiagent systems: When evolutionary many-objective optimization meets deep Q-learning, IEEE Trans. Evol. Comput. 25 (5) (2021) 842–855,.

[44]

Y. Liu, Y. Sun, M. Bing Xue, G.G. Zhang, K.C.T. Yen, A survey on evolutionary neural architecture search, IEEE Trans. Neural Networks Learn. Syst. (2021) 1–11,.

[45]

X. Zhou, A.K. Qin, M. Gong, K.C. Tan, A survey on evolutionary construction of deep neural networks, IEEE Trans. Evol. Comput. 25 (5) (2021) 894–912,.

[46]

X. Zhou, C. Tan, A survey of advances in evolutionary neural architecture search, in: Proc. IEEE Congr. Evol. Comput., 2018, pp. 950–957.

[47]

A. Darwish, A.E. Hassanien, S. Das, A survey of swarm and evolutionary computing approaches for deep learning, Artif. Intell. Rev. 53 (3) (2020) 1767–1812,.

[48]

V. Bharti, B. Biswas, K.K. Shukla, Recent trends in nature inspired computation with applications to deep learning, in: Proc. Conflu. Int. Conf. Cloud Comput. Data Sci. Eng., 2020, pp. 294–299. https://doi.org/10.1109/Confluence47617.2020.9057841.

[49]

K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90.

[50]

D. Silver, A. Huang, C.J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, D. Hassabis, Mastering the game of Go with deep neural networks and tree search, Nature 529 (7587) (2016) 484–489,.

[51]

C. Wang, C. Xu, X. Yao, D. Tao, Evolutionary generative adversarial networks, IEEE Trans. Evol. Comput. 23 (6) (2019) 921–934,.

Digital Library

[52]

W. Liu, Z. Wang, L. Tian, S. Lauria, X. Liu, Melt pool segmentation for additive manufacturing: A generative adversarial network approach, Comput. Electr. Eng. 92 (2021) 107183,.

[53]

W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu, F.E. Alsaadi, A survey of deep neural network architectures and their applications, Neurocomputing 234 (2017) 11–26,.

[54]

E. Papavasileiou, J. Cornelis, B. Jansen, A systematic literature review of the successors of ‘neuroevolution of augmenting topologies’, Evol. Comput. 29 (1) (2021) 1–73,.

[55]

K. Mehta, Z. Kobti, K. Pfaff, S. Fox, Data augmentation using CA evolved GANs, in: Proc. Int. Symp. Comput. Commun., 2019, pp.1087–1092. https://doi.org/10.1109/ISCC47284.2019.8969638.

[56]

P. Dutta, S. Saha, A weak supervision technique with a generative model for improved gene clustering, in: Proc. IEEE Congr. Evol. Comput., 2019, pp. 2521–2528. https://doi.org/10.1109/CEC.2019.8790052.

[57]

J. Ma, D.O. Afolabi, J. Ren, A. Zhen, Predicting seminal quality via imbalanced learning with evolutionary safe-level synthetic minority over-sampling technique, Cognit. Comput. 13 (4) (2021) 833–844,.

[58]

H.L. Le, D. Landa-Silva, M. Galar, S. Garcia, I. Triguero, EUSC: A clustering-based surrogate model to accelerate evolutionary undersampling in imbalanced classification, Appl. Soft Comput. 101 (2021) 107033,.

Digital Library

[59]

S.E. Roshan, S. Asadi, Improvement of Bagging performance for classification of imbalanced datasets using evolutionary multi-objective optimization, Eng. Appl. Artif. Intell. 87 (2020) 103319,.

Digital Library

[60]

Y. Zhu, Y. Yan, Y. Zhang, Y. Zhang, EHSO: Evolutionary hybrid sampling in overlapping scenarios for imbalanced learning, Neurocomputing 417 (2020) 333–346,.

[61]

I. Triguero, M. Galar, H. Bustince, F. Herrera, A first attempt on global evolutionary undersampling for imbalanced big data, in: Proc. IEEE Congr. Evol. Comput. CEC, 2017, pp. 2054–2061. https://doi.org/10.1109/CEC.2017.7969553.

[62]

P.B.C. Castro, B. Krohling, A.G.C. Pacheco, R.A. Krohling, An app to detect melanoma using deep learning: An approach to handle imbalanced data based on evolutionary algorithms, Proc. Int. Jt. Conf. Neural Networks (2020) 1–7,.

[63]

C. Zhang, K.C. Tan, H. Li, G.S. Hong, A cost-sensitive deep belief network for imbalanced classification, IEEE Trans. Neural Networks Learn. Syst. 30 (1) (2019) 109–122,.

[64]

F. Feng, K.-C. Li, J. Shen, Q. Zhou, X. Yang, Using cost-sensitive learning and feature selection algorithms to improve the performance of imbalanced classification, IEEE Access 8 (2020) 69979–69996,.

[65]

W. Zheng, C. Gou, L. Yan, F. Y. Wang, Differential-evolution-based generative adversarial networks for edge detection, in: Proc. Int. Conf. Comput. Vis. Work, 2019, pp. 2999–3008. https://doi.org/10.1109/ICCVW.2019.00362.

[66]

X. Liang, Q. Guo, Y. Qian, W. Ding, Q. Zhang, Evolutionary deep fusion method and its application in chemical structure recognition, IEEE Trans. Evol. Comput. 25 (5) (2021) 883–893,.

[67]

J. Soderlund, A. Blair, Adversarial image generation using evolution and deep learning, Proc. IEEE Congr. Evol. Comput. (2018),.

Digital Library

[68]

S. Jiang, K.S. Chin, L. Wang, G. Qu, K.L. Tsui, Modified genetic algorithm-based feature selection combined with pre-trained deep neural network for demand forecasting in outpatient department, Expert Syst. Appl. 82 (2017) 216–230,.

Digital Library

[69]

T. Elsken, J.H. Metzen, F. Hutter, A survey on neural architecture search, J. Mach. Learn. Res. 20 (2019) 1–21.

[70]

E. Real, S. Moore, A. Selle, S. Saxena, Y.L. Suematsu, J. Tan, Q. V. Le, A. Kurakin, Large-scale evolution of image classifiers, in: Proc. Int. Conf. Mach. Learn., 2017, pp. 4429–4446.

[71]

E. Real, A. Aggarwal, Y. Huang, Q. V. Le, Regularized evolution for image classifier architecture search, in: Proc. AAAI Conf. Artif. Intell., 2019, pp. 4780–4789. https://doi.org/10.1609/aaai.v33i01.33014780.

[72]

C. Saltori, S. Roy, N. Sebe, G. Iacca, Regularized evolutionary algorithm for dynamic neural topology search, Springer International Publishing (2019),.

Digital Library

[73]

P. R. Lorenzo, J. Nalepa, Memetic evolution of deep neural networks, in: Proc. 2018 Genet. Evol. Comput. Conf., 2018, pp. 505–512. https://doi.org/10.1145/3205455.3205631.

[74]

J. Song, Y. Jin, Y. Li, C. Lang, Learning structural similarity with evolutionary-GAN: A New face de-identification method, in: Proc. Int. Conf. Behav. Econ. Socio-Cultural Comput., 2019, pp. 1–8. https://doi.org/10.1109/BESC48373.2019.8962993.

[75]

A. Hadjiivanov, A. Blair, Epigenetic evolution of deep convolutional models, 2019 in: Proc. IEEE Congr. Evol. Comput., 2019, pp. 1478–1486. https://doi.org/10.1109/CEC.2019.8790327.

[76]

X. Wang, Y. Jin, K. Hao, Evolving local plasticity rules for synergistic learning in echo state networks, IEEE Trans. Neural Networks Learn. Syst. 31 (4) (2020) 1363–1374,.

[77]

L. Xie, A. Yuille, Genetic CNN, in: Proc. IEEE Int. Conf. Comput. Vis. 2017-Octob, 2017, pp. 1388–1397. https://doi.org/10.1109/ICCV.2017.154.

[78]

X. Chen, Y. Sun, M. Zhang, D. Peng, Evolving Deep convolutional variational autoencoders for image classification, IEEE Trans. Evol. Comput. 25 (5) (2021) 815–829,.

[79]

T. Zhang, C. Lei, Z. Zhang, X.-B. Meng, C.L.P. Chen, AS-NAS: adaptive scalable neural architecture search with reinforced evolutionary algorithm for deep learning, IEEE Trans. Evol. Comput. 25 (5) (2021) 830–841,.

[80]

A. Martín, R. Lara-Cabrera, F. Fuentes-Hurtado, V. Naranjo, D. Camacho, EvoDeep: A new evolutionary approach for automatic deep neural networks parametrisation, J. Parallel Distrib. Comput. 117 (2018) 180–191,.

Digital Library

[81]

Y. Bi, B. Xue, M. Zhang, Genetic programming with image-related operators and a flexible program structure for feature learning in image classification, IEEE Trans. Evol. Comput. 25 (1) (2021) 87–101.

[82]

Y. Bi, B. Xue, M. Zhang, An evolutionary deep learning approach using genetic programming with convolution operators for image classification, in: Proc.2019 IEEE Congr. Evol. Comput., 2019, pp. 3197–3204. https://doi.org/10.1109/CEC.2019.8790151.

[83]

Y. Sun, B. Xue, M. Zhang, G.G. Yen, Completely automated CNN architecture design based on blocks, IEEE Trans. Neural Networks Learn. Syst. 31 (4) (2020) 1242–1254,.

[84]

D. Song, C. Xu, X. Jia, Y. Chen, C. Xu, Y. Wang, Efficient residual dense block search for image super-resolution, Proc. Thirty-Fourth AAAI Conf. Artif. Intell. (2020) 12007–12014.

[85]

R.A. Viswambaran, G. Chen, B. Xue, M. Nekooei, Evolutionary design of recurrent neural network architecture for human activity recognition, in: Proc. IEEE Congr. Evol. Comput. CEC, 2019, pp. 554–561. https://doi.org/10.1109/CEC.2019.8790050.

[86]

J.K. Kim, Y.S. Han, J.S. Lee, Particle swarm optimization–deep belief network–based rare class prediction model for highly class imbalance problem, Concurr. Comput. Pract. Exp. 29 (11) (2017),.

[87]

W. Liu, Z. Wang, N. Zeng, F.E. Alsaadi, X. Liu, A PSO-based deep learning approach to classifying patients from emergency departments, Int. J. Mach. Learn. Cybern. 12 (7) (2021) 1939–1948,.

[88]

T. Y. Kim, S.B. Cho, Particle swarm optimization-based CNN-LSTM networks for forecasting energy consumption, in: Proc. IEEE Congr. Evol. Comput., 2019, pp. 1510–1516. https://doi.org/10.1109/CEC.2019.8789968.

[89]

R. A. Viswambaran, G. Chen, B. Xue, M. Nekooei, Evolutionary design of recurrent neural network architecture for human activity recognition, in Proc. IEEE Congr. Evol. Comput., 2019, pp. 554–561. https://doi.org/10.1109/CEC.2019.8790050.

[90]

Y. Sun, B. Xue, M. Zhang, G. G. Yen, An experimental study on hyper-parameter optimization for stacked auto-encoders, in Proc. IEEE Congr. Evol. Comput., 2018, pp. 1–8. https://doi.org/10.1109/CEC.2018.8477921.

[91]

F. Huang, A. Jolfaei, A.K. Bashir, Robust multimodal representation learning with evolutionary adversarial attention networks, IEEE Trans. Evol. Comput. 25 (5) (2021) 856–868,.

[92]

J.Y. Li, Z.H. Zhan, J. Xu, S. Kwong, J. Zhang, Surrogate-assisted hybrid-model estimation of distribution algorithm for mixed-variable hyperparameters optimization in convolutional neural networks, IEEE Trans. Neural Networks Learn. Syst. (2021),.

[93]

M. Suganuma, S. Shirakawa, T. Nagao, A genetic programming approach to designing convolutional neural network architectures, in: Proc. Genet. Evol. Comput. Conf., 2017, pp. 497–504. https://doi.org/10.1145/3071178.3071229.

[94]

B. Cao, J. Zhao, Z. Lv, Y.u. Gu, P. Yang, S.K. Halgamuge, Multiobjective evolution of fuzzy rough neural network via distributed parallelism for stock prediction, IEEE Trans. Fuzzy Syst. 28 (5) (2020) 939–952,.

[95]

Z. Lu, I. Whalen, Y. Dhebar, K. Deb, E.D. Goodman, W. Banzhaf, V.N. Boddeti, Multiobjective evolutionary design of deep convolutional neural networks for image classification, IEEE Trans. Evol. Comput. 25 (2) (2021) 277–291,.

[96]

Z. Lu, G. Sreekumar, E. Goodman, W. Banzhaf, K. Deb, V.N. Boddeti, Neural architecture transfer, IEEE Trans. Pattern Anal. Mach. Intell. 43 (9) (2021) 2971–2989,.

[97]

J. Liu, M. Gong, Q. Miao, X. Wang, H. Li, Structure learning for deep neural networks based on multiobjective optimization, IEEE Trans. Neural Networks Learn. Syst. 29 (6) (2018) 2450–2463,.

[98]

Z. Yang, Y. Wang, X. Chen, B. Shi, C. Xu, C. Xu, Q. Tian, C. Xu, CARS: Continuous evolution for efficient neural architecture search, in, Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (2020) 1829–1838.

[99]

J. Liu, Y. Jin, Multi-objective search of robust neural architectures against multiple types of adversarial attacks, Neurocomputing 453 (2021) 73–84,.

Digital Library

[100]

Y.-W. Wen, S.-H. Peng, C.-K. Ting, Two-stage evolutionary neural architecture search for transfer learning, IEEE Trans. Evol. Comput. 25 (5) (2021) 928–940,.

[101]

J.-Y. Li, Z.-H. Zhan, J. Zhang, Evolutionary computation for expensive optimization: A survey, Mach. Intell. Res. 19 (1) (2022) 3–23,.

[102]

M. Gong, J. Liu, A.K. Qin, K. Zhao, K.C. Tan, Evolving deep neural networks via cooperative coevolution with backpropagation, IEEE Trans. Neural Networks Learn. Syst. 32 (1) (2021) 420–434.

[103]

J. Long, S. Zhang, C. Li, Evolving deep Echo state networks for intelligent fault diagnosis, IEEE Trans. Ind. Informatics. 16 (7) (2020) 4928–4937,.

[104]

W.u. Deng, H. Liu, J. Xu, H. Zhao, Y. Song, An improved quantum-inspired differential evolution algorithm for deep belief network, IEEE Trans. Instrum. Meas. 69 (10) (2020) 7319–7327,.

[105]

P. Cui, B. Shabash, K.C. Wiese, EvoDNN – An evolutionary deep neural network with heterogeneous activation functions, in: Proc. IEEE Congr. Evol. Comput., 2019, pp. 2362–2369. https://doi.org/10.1109/CEC.2019.8789964.

[106]

S. Chen, W. Wang, B. Xia, X. You, Q. Peng, Z. Cao, W. Ding, CDE-GAN: Cooperative dual evolution-based generative adversarial network, IEEE Trans. Evol. Comput. 25 (5) (2021) 986–1000,.

[107]

Y. Sun, B. Xue, M. Zhang, G.G. Yen, Evolving deep convolutional neural networks for image classification, IEEE Trans. Evol. Comput. 24 (2) (2020) 394–407,.

[108]

A. Kenny, X. Li, A study on pre-training deep neural networks using particle swarm optimisation, Lect. Notes Comput. Sci. (2017) 361–372,.

[109]

Y. Sun, H. Wang, B. Xue, Y. Jin, G.G. Yen, M. Zhang, Surrogate-assisted evolutionary deep learning using an end-to-end random forest-based performance predictor, IEEE Trans. Evol. Comput. 24 (2) (2020) 350–364,.

[110]

Y. Zhou, Y. Jin, J. Ding, Surrogate-assisted evolutionary search of spiking neural architectures in liquid state machines, Neurocomputing 406 (2020) 12–23,.

[111]

M. Zhang, H. Li, S. Pan, J. Lyu, S. Ling, S. Su, Convolutional neural networks-based lung nodule classification: A surrogate-assisted evolutionary algorithm for hyperparameter optimization, IEEE Trans. Evol. Comput. 25 (5) (2021) 869–882,.

[112]

B. Wang, Y. Sun, B. Xue, M. Zhang, Evolving deep convolutional neural networks by variable-length particle swarm optimization for image classification, in: Proc. IEEE Congr. Evol. Comput., IEEE, 2018, pp. 1–8. https://doi.org/10.1109/CEC.2018.8477735.

[113]

H. Zhu, Y. Jin, Multi-Objective Evolutionary federated learning, IEEE Trans. Neural Networks Learn. Syst. 31 (4) (2020) 1310–1322,.

[114]

W. Irwin-Harris, Y. Sun, B. Xue, M. Zhang, A graph-based encoding for evolutionary convolutional neural network architecture design, Proc: IEEE Congr. Evol. Comput. CEC (2019) 546–553,.

Digital Library

[115]

Z.-H. Zhan, X.-F. Liu, H. Zhang, Z. Yu, J. Weng, Y. Li, T. Gu, J. Zhang, Cloudde: A heterogeneous differential evolution algorithm and its distributed cloud version, IEEE Trans. Parallel Distrib. Syst. 28 (3) (2017) 704–716,.

Digital Library

[116]

J.-Y. Li, Z.-H. Zhan, R.-D. Liu, C. Wang, S. Kwong, J. Zhang, Generation-level parallelism for evolutionary computation: A pipeline-based parallel particle swarm optimization, IEEE Trans. Cybern. 51 (10) (2021) 4848–4859,.

[117]

Z.-J. Wang, Z.-H. Zhan, W.-J. Yu, Y. Lin, J. Zhang, T.-L. Gu, J. Zhang, Dynamic group learning distributed particle swarm optimization for large-scale optimization and its application in cloud workflow scheduling, IEEE Trans. Cybern. 50 (6) (2020) 2715–2729,.

[118]

Z.-H. Zhan, Z.-J. Wang, H.u. Jin, J. Zhang, Adaptive distributed differential evolution, IEEE Trans. Cybern. 50 (11) (2020) 4633–4647,.

[119]

Y. Guo, J.-Y. Li, Z.-H. Zhan, Efficient hyperparameter optimization for convolution neural networks in deep learning: A distributed particle swarm optimization approach, Cybern. Syst. 52 (1) (2021) 36–57,.

[120]

T. Moriya, T. Tanaka, T. Shinozaki, S. Watanabe, K. Duh, Evolution-strategy-based automation of system development for high-performance speech recognition, IEEE/ACM Trans. Audio Speech Lang. Process. 27 (1) (2019) 77–88,.

Digital Library

[121]

S.C. Liu, Z.H. Zhan, K.C. Tan, J. Zhang, A multiobjective framework for many-objective optimization, IEEE Trans. Cybern. (2021) 1–15,.

[122]

X. Zhang, Z.H. Zhan, W. Fang, P. Qian, J. Zhang, Multi population ant colony system with knowledge based local searches for multiobjective supply chain configuration, IEEE Trans. Evol. Comput. (2021),.

[123]

H. Zhao, Z.-H. Zhan, Y. Lin, X. Chen, X.-N. Luo, J. Zhang, S. Kwong, J. Zhang, Local binary pattern-based adaptive differential evolution for multimodal optimization problems, IEEE Trans. Cybern. 50 (7) (2020) 3343–3357,.

[124]

Z.-G. Chen, Z.-H. Zhan, H. Wang, J. Zhang, Distributed individuals for multiple peaks: A novel differential evolution for multimodal optimization problems, IEEE Trans. Evol. Comput. 24 (4) (2020) 708–719,.

[125]

J.Y. Li, K.J. Du, Z.H. Zhan, H. Wang, J. Zhang, Multi-criteria differential evolution: Treating multitask optimization as multi-criteria optimization, Proc. Genet. Evol. Comput. Conf. (2021) 183–184,.

Digital Library

[126]

J. Zhang, C. Li, Adversarial examples: Opportunities and challenges, IEEE Trans. Neural Networks Learn. Syst. 31 (2020) 2578–2593,.

[127]

J. Su, D.V. Vargas, K. Sakurai, One pixel attack for fooling deep neural networks, IEEE Trans. Evol. Comput. 23 (5) (2019) 828–841,.

[128]

P. Vidnerová, R. Neruda, Vulnerability of classifiers to evolutionary generated adversarial examples, Neural Networks 127 (2020) 168–181,.

[129]

Y. Deng, C. Zhang, X. Wang, A multi-objective examples generation approach to fool the deep neural networks in the black-box scenario, in: Proc. IEEE Int. Conf. Data Sci. Cyberspace, 2019, pp. 92–99. https://doi.org/10.1109/DSC.2019.00022.

[130]

Y.J. Gong, J. Zhang, Y. Zhou, Learning multimodal parameters: A bare-bones niching differential evolution approach, IEEE Trans. Neural Networks Learn. Syst. 29 (2018) 2944–2959,.

[131]

C. Zhang, P. Lim, A.K. Qin, K.C. Tan, Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics, IEEE Trans. Neural Networks Learn. Syst. 28 (10) (2017) 2306–2318,.

[132]

C. Zhang, J.H. Sun, K.C. Tan, Deep belief networks ensemble with multi-objective optimization for failure diagnosis, in: Proc. IEEE Int. Conf. Syst. Man, Cybern., 2016, pp. 32–37. https://doi.org/10.1109/SMC.2015.19.

[133]

B. Fielding, T. Lawrence, L. Zhang, Evolving and ensembling deep CNN architectures for image classification, in: Proc. Int. Jt. Conf. Neural Networks, 2019, pp. 1–8. https://doi.org/10.1109/IJCNN.2019.8852369.

[134]

Y. Zhou, G.G. Yen, Z. Yi, Evolutionary compression of deep neural networks for biomedical image segmentation, IEEE Trans. Neural Networks Learn. Syst. 31 (8) (2020) 2916–2929,.

[135]

H. Shu, Y. Wang, X. Jia, K. Han, H. Chen, C. Xu, Q. Tian, C. Xu, Co-evolutionary compression for unpaired image translation, in: Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 3234–3243. https://doi.org/10.1109/ICCV.2019.00333.

[136]

T. Wu, J. Shi, D. Zhou, Y. Lei, M. Gong, A multi-objective particle swarm optimization for neural networks pruning, in: IEEE Congr. Evol. Comput., 2019, pp.570–577. https://doi.org/10.1109/CEC.2019.8790145.

[137]

D. Liang, Z.-H. Zhan, Y. Zhang, J. Zhang, An efficient ant colony system approach for new energy vehicle dispatch problem, IEEE Trans. Intell. Transp. Syst. 21 (11) (2020) 4784–4797,.

[138]

X.-F. Liu, Z.-H. Zhan, J.D. Deng, Y. Li, T. Gu, J. Zhang, An energy efficient ant colony system for virtual machine placement in cloud computing, IEEE Trans. Evol. Comput. 22 (1) (2018) 113–128,.

[139]

S. Xie, A. Kirillov, R. Girshick, K. He, Exploring randomly wired neural networks for image recognition, in: Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 1284–1293. https://doi.org/10.1109/ICCV.2019.00137.

[140]

Y. Duan, X. Chen, H. Xu, Z. Chen, X. Liang, T. Zhang, Z. Li, TransNAS-Bench-101: Improving transferability and generalizability of cross-task neural architecture search, Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (2021) 5251–5260.

[141]

J.-R. Jian, Z.-H. Zhan, J. Zhang, Large-scale evolutionary optimization: a survey and experimental comparative study, Int. J. Mach. Learn. Cybern. 11 (3) (2020) 729–745,.

[142]

X.-F. Liu, Z.-H. Zhan, T.-L. Gu, S. Kwong, Z. Lu, H.-L. Duh, J. Zhang, Neural network-based information transfer for dynamic optimization, IEEE Trans. Neural Networks Learn. Syst. 31 (5) (2020) 1557–1570,.

[143]

Y. Jiang, Z.H. Zhan, K.C. Tan, J. Zhang, Optimizing niching centers for multimodal optimization, IEEE Trans. Evol. Comput. 22 (2018) 894–908,.

[144]

Z.-J. Wang, Z.-H. Zhan, Y. Lin, W.-J. Yu, H.-Q. Yuan, T.-L. Gu, S. Kwong, J. Zhang, Dual-strategy differential evolution with affinity propagation clustering for multimodal optimization problems, IEEE Trans. Evol. Comput. 22 (6) (2018) 894–908,.

Digital Library

[145]

T. Elsken, B. Staffler, J.H. Metzen, F. Hutter, Meta-learning of neural architectures for few-shot learning, in: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn., 2020, pp. 12362–12372,.

[146]

S. Xue, H. Chen, C. Xie, B. Zhang, X. Gong, D. Doermann, Fast and unsupervised neural architecture evolution for visual representation learning, IEEE Comput. Intell. Mag. 16 (3) (2021) 22–32.

[147]

C. Wei, Y. Tang, C.N. Chuang Niu, H. Hu, Y. Wang, J. Liang, Self-supervised representation learning for evolutionary neural architecture search, IEEE Comput. Intell. Mag. 16 (3) (2021) 33–49.

[148]

X.F. Liu, Z.H. Zhan, J. Zhang, Resource-aware distributed differential evolution for training expensive neural-network-based controller in power electronic circuit, IEEE Trans. Neural Networks Learn. Syst. (2021) 1–11,.

[149]

Z. Tan, H. Wang, S. Liu, Multi-stage dimension reduction for expensive sparse multi-objective optimization problems, Neurocomputing 440 (2021) 159–174,.

[150]

S.-H. Wu, Z.-H. Zhan, J. Zhang, SAFE: Scale-adaptive fitness evaluation method for expensive optimization problems, IEEE Trans. Evol. Comput. 25 (3) (2021) 478–491,.

Digital Library

[151]

Y. Jin, H. Wang, T. Chugh, D. Guo, K. Miettinen, Data-driven evolutionary optimization: An overview and case studies, IEEE Trans. Evol. Comput. 23 (3) (2019) 442–458,.

[152]

Y. Jin, H. Wang, C. Sun, Data-Driven Evolutionary Optimization, Springer, 2021.

[153]

J.-Y. Li, Z.-H. Zhan, C. Wang, H.u. Jin, J. Zhang, Boosting data-driven evolutionary algorithm with localized data generation, IEEE Trans. Evol. Comput. 24 (5) (2020) 923–937,.

[154]

J.-Y. Li, Z.-H. Zhan, H. Wang, J. Zhang, Data-driven evolutionary algorithm with perturbation-based ensemble surrogates, IEEE Trans. Cybern. 51 (8) (2021) 3925–3937,.

[155]

Z. Zhang, L. Li, J. Lu, Gradient-based fly immune visual recurrent neural network solving large-scale global optimization, Neurocomputing 454 (2021) 238–253,.

[156]

J.-R. Jian, Z.-G. Chen, Z.-H. Zhan, J. Zhang, Region encoding helps evolutionary computation evolve faster: A new solution encoding scheme in particle swarm for large-scale optimization, IEEE Trans. Evol. Comput. 25 (4) (2021) 779–793,.

Digital Library

[157]

Y. Han, X. Gu, Cooperative hybrid evolutionary algorithm for large scale multi-stage multi-product batch plants scheduling problem, Neurocomputing 419 (2021) 80–96,.

[158]

Y.-F. Ge, W.-J. Yu, Y. Lin, Y.-J. Gong, Z.-H. Zhan, W.-N. Chen, J. Zhang, Distributed differential evolution based on adaptive mergence and split for large-scale optimization, IEEE Trans. Cybern. 48 (7) (2018) 2166–2180,.

[159]

Z.-J. Wang, Z.-H. Zhan, S. Kwong, H.u. Jin, J. Zhang, Adaptive granularity learning distributed particle swarm optimization for large-scale optimization, IEEE Trans. Cybern. 51 (3) (2021) 1175–1188,.

[160]

B. Evans, H. Al-Sahaf, B. Xue, M. Zhang, Evolutionary deep learning: A genetic programming approach to image classification, in: Proc. IEEE Congr. Evol. Comput. CEC, 2018, pp. 1-8. https://doi.org/10.1109/CEC.2018.8477933.

Cited By

Larrañaga PBielza C(2024)Estimation of Distribution Algorithms in Machine Learning: A SurveyIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.331410528:5(1301-1321)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3314105
Bi YXue BZhang M(2024)Genetic Programming-Based Evolutionary Deep Learning for Data-Efficient Image ClassificationIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.321450328:2(307-322)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TEVC.2022.3214503
Wang YDai DLiu DXia SWang G(2024)BTSCCAAI Transactions on Intelligence Technology10.1049/cit2.123289:5(1331-1345)Online publication date: 31-Mar-2024
https://dl.acm.org/doi/10.1049/cit2.12328
Show More Cited By

Index Terms

Evolutionary deep learning: A survey
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Evolutionary Machine Learning: A Survey
Evolutionary Computation (EC) approaches are inspired by nature and solve optimization problems in a stochastic manner. They can offer a reliable and effective approach to address complex problems in real-world applications. EC algorithms have recently ...
Comparative study of diversity based parallel dual population genetic algorithm for unconstrained function optimisations

The genetic algorithms GAs metaheuristic deals with large scale combinatorial optimisation problems. It is biologically inspired by the method, based on the principle of survival of the fittest. In GAs, the concept of multiple populations offers an ...
Niching without niching parameters: particle swarm optimization using a ring topology

Niching is an important technique for multimodal optimization. Most existing niching methods require specification of certain niching parameters in order to perform well. These niching parameters, often used to inform a niching algorithm how far apart ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 483, Issue C

Apr 2022

516 pages

ISSN:0925-2312

Issue’s Table of Contents

Copyright © 2022.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 28 April 2022

Author Tags

Qualifiers

Short-survey

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Larrañaga PBielza C(2024)Estimation of Distribution Algorithms in Machine Learning: A SurveyIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.331410528:5(1301-1321)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3314105
Bi YXue BZhang M(2024)Genetic Programming-Based Evolutionary Deep Learning for Data-Efficient Image ClassificationIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.321450328:2(307-322)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TEVC.2022.3214503
Wang YDai DLiu DXia SWang G(2024)BTSCCAAI Transactions on Intelligence Technology10.1049/cit2.123289:5(1331-1345)Online publication date: 31-Mar-2024
https://dl.acm.org/doi/10.1049/cit2.12328
Granato GBaldassarre G(2024)Bridging flexible goal-directed cognition and consciousnessNeural Networks10.1016/j.neunet.2024.106292176:COnline publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106292
Zhang HLiu WLuo DShi JGuo QGe YChang SWang HHe JHuang Q(2024)Honest-GEInformation Sciences: an International Journal10.1016/j.ins.2024.120941677:COnline publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120941
Triguero IMolina DPoyatos JDel Ser JHerrera F(2024)General Purpose Artificial Intelligence Systems (GPAIS)Information Fusion10.1016/j.inffus.2023.102135103:COnline publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1016/j.inffus.2023.102135
Yaghoubi EYaghoubi EKhamees ARazmi DLu T(2024)A systematic review and meta-analysis of machine learning, deep learning, and ensemble learning approaches in predicting EV charging behaviorEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.108789135:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.108789
Ferri PLomonaco VPassaro LFélix-De Castro ASánchez-Cuesta PSáez CGarcía-Gómez J(2024)Deep continual learning for medical call incidents text classification under the presence of dataset shiftsComputers in Biology and Medicine10.1016/j.compbiomed.2024.108548175:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2024.108548
Yaghoubi EYaghoubi EKhamees AVakili A(2024)A systematic review and meta-analysis of artificial neural network, machine learning, deep learning, and ensemble learning approaches in field of geotechnical engineeringNeural Computing and Applications10.1007/s00521-024-09893-736:21(12655-12699)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00521-024-09893-7
Huang P(2023)A Multiperspective Intervention Path Study of Employment Expectancy in the Era of Big DataWireless Communications & Mobile Computing10.1155/2023/14095452023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/1409545
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents