Abstract
Parameters optimization is a research hotspot of SVM and has gained increasing interest from various research fields. Compared with other optimization algorithms, genetic-based evolutionary algorithms that have achieved optimization according to the laws of separation and free combination in genetics are gradually attracted much attention. Also, due to the characteristics of self-organization and self-adaptation, these algorithms often enable SVM to obtain appropriate parameters, so that the model can be applied to more applications. Additionally, many improvements have been proposed in the past two decades in order to allow the optimized SVM model to obtain better performance. This work focuses on reviewing the current state of genetic-based evolutionary algorithms used to optimize parameters of SVM and its variants. First, we introduce the principles of SVM and provide a survey on optimization methods of its parameters. Then we propose a taxonomy of improving genetic-based evolutionary algorithms according to code mechanism, parameters control, population structure, evolutionary strategy, operation mechanism, operators, and many other hybrid approaches. Furthermore, this paper analyzes and compares the advantages and disadvantages of the above algorithms explicitly, and provides their applicable scenarios as well. Finally, we highlight the existing problems of genetic-based evolutionary algorithms used for parameters optimization of SVM and prospect development trends of this field in the future.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Vapnik V, Izmailov R (2017) Knowledge transfer in SVM and neural networks. Ann Math Artif Intell 81(1):3–19. https://doi.org/10.1007/s10472-017-9538-x
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297. https://doi.org/10.1023/A:1022627411411
Fayed HA, Atiya AF (2019) Speed up grid-search for parameter selection of support vector machines. Appl Soft Comput J 80:202–210. https://doi.org/10.1016/j.asoc.2019.03.037
Yang M, Zhang B, Song YL (2018) Application of support vector machine based on optimized kernel function in people detection. Laser Optoelectron Prog 55(04):107–114
Kari T, Gao WS, Zhang ZW, Mo WX, Wang HB, Cui YP (2018) Power transformer fault diagnosis based on a support vector machine and a genetic algorithm. J Tsinghua Univ (Sci Technol) 58(07):623–629
Liao ZY, Wang YT, Xie XL, Liu JM (2017) Face recognition by support vector machine based on particle swarm optimization. Comput Eng 43(12):248–254
Peng Z, Jiang Y, Yang X, Zhao Z, Zhang L, Wang Y (2018) Bus arrival time prediction based on pca-ga-svm. Neural Netw World 28(1):87–104. https://doi.org/10.14311/NNW.2018.28.005
Li K, Wang L, Wu J, Zhang Q, Liao G, Su L (2018) Using ga-svm for defect inspection of flip chips based on vibration signals. Microelectron Reliab 81:159–166. https://doi.org/10.1016/j.microrel.2017.12.032
Yang B (2019) Dynamic risk identification safety model based on fuzzy support vector machine and immune optimization algorithm. Saf Sci 118:205–211. https://doi.org/10.1016/j.ssci.2019.05.022
Zhang Y, Yu J, Xia C, Yang K, Cao H, Wu Q (2019) Research on ga-svm based head-motion classification via mechanomyography feature analysis. Sensors (Basel, Switzerland) 19(9):1986. https://doi.org/10.3390/s19091986
Wang M, Wan Y, Ye Z, Lai X (2017) Remote sensing image classification based on the optimal support vector machine and modified binary coded ant colony optimization algorithm. Inform Sci 402:50–68. https://doi.org/10.1016/j.ins.2017.03.027
Yan X, Jia M (2018) A novel optimized svm classification algorithm with multi-domain feature and its application to fault diagnosis of rolling bearing. Neurocomputing 313:47–64. https://doi.org/10.1016/j.neucom.2018.05.002
Zhang Z, He X, Sun X, Guo L, Wang J, Wang F (2015) Image recognition of maize leaf disease based on ga-svm. Chem Eng Trans 46:199–204. https://doi.org/10.3303/CET1546034
Tang X, Hong H, Shu Y, Tang H, Li J, Liu W (2019) Urban waterlogging susceptibility assessment based on a pso-svm method using a novel repeatedly random sampling idea to select negative samples. J Hydrol 576:583–595. https://doi.org/10.1016/j.jhydrol.2019.06.058
Ye F (2018) Evolving the svm model based on a hybrid method using swarm optimization techniques in combination with a genetic algorithm for medical diagnosis. Multimed Tools Appl 77(3):3889–3918. https://doi.org/10.1007/s11042-016-4233-1
Li S, Yuan ZG, Wang C, Chen TE, Guo ZC (2018) Optimization of support vector machine parameters based on group intelligence algorithm[J]. CAAI Trans Intell Syst 13(01):70–80
Dong H, Jian G (2015) Parameter selection of a support vector machine, based on a chaotic particle swarm optimization algorithm. Cybern Inf Technol 15(3):140–149. https://doi.org/10.1515/cait-2015-0047
Dorigo M, Gambardella LM (1997) Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans Evolut Comput 1(1):53–66. https://doi.org/10.1109/4235.585892
Li XL, Shao ZJ, Qian JX (2002) An optimizing method based on autonomous animats: fish-swarm algorithm. Syst Eng Theory Pract 11:32–38
Li XL, Qian JX (2003) Studies on artificial fish swarm optimization algorithm based on decomposition and coordination techniques. J Circuits Syst 01:1–6
Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. Technical Report-TR06. Erciyes University
Yang X, Deb S, Fong S, He X, Zhao Y (2016) From swarm intelligence to metaheuristics: nature-inspired optimization algorithms. Computer 49(9):52–59. https://doi.org/10.1109/MC.2016.292
Ye F, Lou XY, Sun LF (2017) An improved chaotic fruit fly optimization based on a mutation strategy for simultaneous feature selection and parameter optimization for svm and its applications. PLoS ONE 12(4):e173516. https://doi.org/10.1371/journal.pone.0173516
Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007
Jiang M, Luo J, Jiang D, Xiong J, Song H, Shen J (2016) A cuckoo search-support vector machine model for predicting dynamic measurement errors of sensors. IEEE Access 4:5030–5037. https://doi.org/10.1109/ACCESS.2016.2605041
Dantas Dias ML, Rocha Neto AR (2017) Training soft margin support vector machines by simulated annealing: a dual approach. Expert Syst Appl 87:157–169. https://doi.org/10.1016/j.eswa.2017.06.016
Sartakhti JS, Afrabandpey H, Saraee M (2017) Simulated annealing least squares twin support vector machine (sa-lstsvm) for pattern classification. Soft Comput 21(15):4361–4373. https://doi.org/10.1007/s00500-016-2067-4
Seifollahi S, Bagirov A, Zare Borzeshi E, Piccardi M (2019) A simulated annealing-based maximum-margin clustering algorithm. Comput Intell-Us 35(1):23–41. https://doi.org/10.1111/coin.12187
Rajathi GI, Jiji GW (2019) Chronic liver disease classification using hybrid whale optimization with simulated annealing and ensemble classifier. Symmetry 11(1):33. https://doi.org/10.3390/sym11010033
Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor
Gozali AA, Fujimura S (2019) Dm-limga: dual migration localized island model genetic algorithm—a better diversity preserver island model. Evolut Intell 12(4):527–539. https://doi.org/10.1007/s12065-019-00253-2
Fernandez M, Caballero J, Fernandez L, Sarai A (2011) Genetic algorithm optimization in drug design qsar: bayesian-regularized genetic neural networks (brgnn) and genetic algorithm-optimized support vectors machines (ga-svm). Mol Divers 15(1):269–289. https://doi.org/10.1007/s11030-010-9234-9
Martins M, Costa L, Frizera A, Ceres R, Santos C (2013) Hybridization between multi-objective genetic algorithm and support vector machine for feature selection in walker-assisted gait. Comput Methods Prog Biol 113(3):736–748. https://doi.org/10.1016/j.cmpb.2013.12.005
Tao Z, Huiling L, Wenwen W, Xia Y (2019) Ga-svm based feature selection and parameter optimization in hospitalization expense modeling. Appl Soft Comput J 75:323–332. https://doi.org/10.1016/j.asoc.2018.11.001
Xalf L, Xian C (2002) Choosing multiple parameters for svm based on genetic algorithm. In: 2002 6th International conference on signal processing proceedings, Beijing, China
Huang C, Wang C (2006) A ga-based feature selection and parameters optimization for support vector machines. Expert Syst Appl 31(2):231–240. https://doi.org/10.1016/j.eswa.2005.09.024
Subhashini KR, Chinta P (2019) An augmented animal migration optimization algorithm using worst solution elimination approach in the backdrop of differential evolution. Evolut Intell 12(2):273–303. https://doi.org/10.1007/s12065-019-00223-8
Zhong Y, Cao Q, Zhao J, Ma A, Zhao B, Zhang L (2017) Optimal decision fusion for urban land-use/land-cover classification based on adaptive differential evolution using hyperspectral and lidar data. Remote Sens Basel. https://doi.org/10.3390/rs9080868
Wang L, Pan Q, Suganthan PN, Wang W, Wang Y (2010) A novel hybrid discrete differential evolution algorithm for blocking flow shop scheduling problems. Comput Oper Res 37(3):509–520. https://doi.org/10.1016/j.cor.2008.12.004
Annepu V, Rajesh A (2019) Implementation of self adaptive mutation factor and cross-over probability based differential evolution algorithm for node localization in wireless sensor networks. Evolut Intell 12(3):469–478. https://doi.org/10.1007/s12065-019-00239-0
Sarkar S, Das S, Chaudhuri SS (2016) Hyper-spectral image segmentation using rényi entropy based multi-level thresholding aided with differential evolution. Expert Syst Appl 50:120–129. https://doi.org/10.1016/j.eswa.2015.11.016
Aburomman AA, Ibne Reaz MB (2017) A novel weighted support vector machines multiclass classifier based on differential evolution for intrusion detection systems. Inform Sci 414:225–246. https://doi.org/10.1016/j.ins.2017.06.007
Lin LL, Jiang SD, Liu XD (2008) Simultaneous selection of parameters and features for SVM based on the differential evolution algorithm. J Jilin Univ (Eng Technol Ed) 38(S2):255–259
Lin LL, Jiang SD, Liu XD (2009) Parameter selection for an SVM based on a differential evolution algorithm. J Harbin Eng Univ 30(02):138–141
Zhao H, Shen L, Yang JG, Yang LG, Xu HM (2010) The model for calculating ultimate analysis of coal by its proximate analysis based on DE-SVM. J China Coal Soc 35(10):1721–1724
Guo Y, Song AG, Bao JT, Cui JW, Zhang HT (2011) Mobile robot traversability prediction based on differential evolution support vector machine. Robot 33(03):257–264
Zhang J, Niu Q, Li K, Irwin GW (2011) Model selection in svms using differential evolution. IFAC Proc Vol 44(1):14717–14722. https://doi.org/10.3182/20110828-6-IT-1002.00584
Li YH, Zhong YH, Yuan CQ (2013) Application of DE-SVM fusion algorithm in intrusion detection. Comput Eng Appl 49(12):70–73
Yang JW, Xu J, Wu XY, Lu YX, Wei JQ (2016) Evaluation method for operational effectiveness based on support vector machine with differential evolution. J Gun Launch Control 37(01):16–20
Shen SH (2017) Diesel engine fault diagnosis based on support vector machine optimized by differential evolution. Smart Fact 05:85–88
Wang L, Zhou DF, Bai RG (2018) Fault diagnosis of tolerance analog circuits based on differential evolution invasive weed algorithm. Appl Res Comput 35(09):2621–2623
Lv PL, Weng XX, Peng XJ (2019) Public traffic passenger recognition based on differential evolution algorithm SVM. J Guangxi Normal Univ (Nat Sci Edn) 37(01):106–114
Lin BH, Gu XS (2008) Soft sensor modeling based on DE-LSSVM. J Chem Ind Eng (China) 07:1681–1685
Xu SJ, Long W (2012) Parameters selection for LSSVM based on differential evolution to mid-long term runoff prediction. Sci Technol Eng 12(27):6955–6959
García-Nieto PJ, García-Gonzalo E, Fernández JRA, Muñiz CD (2019) Modeling of the algal atypical increase in la barca reservoir using the de optimized least square support vector machine approach with feature selection. Math Comput Simulat 166:461–480. https://doi.org/10.1016/j.matcom.2019.07.011
Cheng M, Hoang N, Wu Y (2013) Hybrid intelligence approach based on ls-svm and differential evolution for construction cost index estimation: a taiwan case study. Automat Constr 35:306–313. https://doi.org/10.1016/j.autcon.2013.05.018
Yue XF, Shao HH (2015) Fault diagnosis method of rolling bearing based on DE-LSSVM. Comput Meas Control 23(12):3933–3935
Jun-hong ZYL (2017) Application of complete ensemble intrinsic time-scale decomposition and least-square svm optimized using hybrid de and pso to fault diagnosis of diesel engines. Front Inf Technol Electron Eng 18(2):272–286
Bao ZY, Yu JZ, Yang S (2018) Intelligent optimization algorithm and its MATLAB example, 2nd edn. Publishing House of Electronics Industry, Beijing
Oliveira DC, Chavarette FR, Lopes MLM (2019) Damage diagnosis in an isotropic structure using an artificial immune system algorithm. J Braz Soc Mech Sci 41(11):1–11. https://doi.org/10.1007/s40430-019-1971-9
Li JW, Ren LH, Ding YS, Chen L (2018) Adaptive integrated classification method based on immune optimization for EEG. J Mech Electr Eng 35(08):873–879
Wu H, Hou Z (2004) A short-term load forecasting approach based on immune support vector machines. Power Syst Technol 28(23):47–51
Li Y, Wu ZS, Li YF, Zhu YJ (2018) Defects classification method of welding joints based on artificial immune and support vector machine. J Sichuan Univ (Eng Sci Edn) 50(04):221–227
Cao YM, Jing DQ, Liu CG (2018) The prediction of the displacement of the arch dam based on the twin support vector machine optimized by immune algorithm. J Yangtze River Sci Res Inst 2018:1–6
Wang C, Ma G, Li J, Dai Z, Liu J (2019) Prediction of corrosion rate of submarine oil and gas pipelines based on ia-svm model. IOP Conf Ser Earth Environ Sci 242:22023. https://doi.org/10.1088/1755-1315/242/2/022023
Gupta P, Mehlawat MK, Mittal G (2012) Asset portfolio optimization using support vector machines and real-coded genetic algorithm. J Global Optim 53(2):297–315. https://doi.org/10.1007/s10898-011-9692-3
Meng T, Zhou XZ, Lei YJ (2016) A parameters optimization method for an SVM based on adaptive genetic algorithm. Comput Meas Control 24(09):215–217
Fu H, Li L (2011) Simulation research of SVM parameters optimization based on immune algorithm of vector distance. Comput Simul 28(05):201–204
de Sampaio WB, Silva AC, de Paiva AC, Gattass M (2015) Detection of masses in mammograms with adaption to breast density using genetic algorithm, phylogenetic trees, lbp and svm. Expert Syst Appl 42(22):8911–8928. https://doi.org/10.1016/j.eswa.2015.07.046
Chen P, Yuan L, He Y, Luo S (2016) An improved SVM classifier based on double chains quantum genetic algorithm and its application in analogue circuit diagnosis. Neurocomputing 211:202–211. https://doi.org/10.1016/j.neucom.2015.12.131
Yan XT, Wu MQ (2009) Adaptive differential evolution algorithm based on least square SVM. J Syst Simul 21(07):1921–1925
Yu X (2017) Disaster prediction model based on support vector machine for regression and improved differential evolution. Nat Hazards 85(2):959–976. https://doi.org/10.1007/s11069-016-2613-5
Tian Y, Ju X, Qi Z (2014) Efficient sparse nonparallel support vector machines for classification. Neural Comput Appl 24(5):1089–1099. https://doi.org/10.1007/s00521-012-1331-5
Devos O, Downey G, Duponchel L (2014) Simultaneous data pre-processing and svm classification model selection based on a parallel genetic algorithm applied to spectroscopic data of olive oils. Food Chem 148:124–130. https://doi.org/10.1016/j.foodchem.2013.10.020
Song XR, Zeng J, Gao S, Chen CB (2018) Target recognition based on differential evolution algorithm of least squares support vector machine. Sci Technol Eng 18(16):68–73
Jiaqiang E, Qian C, Zhu H, Peng Q, Zuo W, Liu G (2017) Parameter-identification investigations on the hysteretic preisach model improved by the fuzzy least square support vector machine based on adaptive variable chaos immune algorithm. J Low Freq Noise Vib Active Control 36(3):227–242. https://doi.org/10.1177/0263092317719634
Adankon MM, Cheriet M (2010) Genetic algorithm-based training for semi-supervised svm. Neural Comput Appl 19(8):1197–1206. https://doi.org/10.1007/s00521-010-0358-8
Zhang J, Li Y, Cao Y, Zhang L (2017) Immune SVM used in wear fault diagnosis of aircraft engine Beijing Hangkong Hangtian Daxue Xuebao. J Beijing Univ Aeronaut Astronaut 43(7):1419–1425. https://doi.org/10.13700/j.bh.1001-5965.2016.0553
Corus D, Oliveto PS (2017) Standard steady state genetic algorithms can Hillclimb faster than mutation-only evolutionary algorithms
Zhang D, Liu W, Xu X, Deng Q (2010)A novel interpolation method based on differential evolution-simplex algorithm optimized parameters for support vector regression, vol 6382. Springer, Berlin, pp 64–75. https://doi.org/10.1007/978-3-642-16493-4_7
Fu H, Feng SC, Liu J, Tang B (2016) The modeling and simulation of gas concentration prediction based on De-EDA-SVM. Chin J Sens Actuators 29(02):285–289
Yang L, Xu Z (2019) Feature extraction by PCA and diagnosis of breast tumors using svm with de-based parameter tuning. Int J Mach Learn Cybern 10(3):591–601. https://doi.org/10.1007/s13042-017-0741-1
Sun W, Liu MH (2015) Short-term load forecasting based on improved least squares-support vector machine. Electric Power Sci Eng 31(12):16–21
Zhai S, Jiang T (2015) A new sense-through-foliage target recognition method based on hybrid differential evolution and self-adaptive particle swarm optimization-based support vector machine. Neurocomputing 149:573–584. https://doi.org/10.1016/j.neucom.2014.08.017
Dai S, Niu D, Han Y (2018) Forecasting of power grid investment in china based on support vector machine optimized by differential evolution algorithm and grey wolf optimization algorithm. Appl Sci 8(4):636. https://doi.org/10.3390/app8040636
Wang Z, Zhang Z, Wang W (2019) Emotion recognition based on framework of badeba-svm. Math Probl Eng 2019:1–9. https://doi.org/10.1155/2019/9875250
Leung CSK, Lau HYK (2016) A hybrid multi-objective immune algorithm for numerical optimization, Porto, Portugal, 2016. In: IJCCI 2016 proceedings of the 8th international joint conference on computational intelligence, SciTePress, pp 105–114
Zhou C, Pan P, Yang P, Huang L (2018) Cloud service selection based on chaos quantum immune algorithm, IEEE, pp 1–6. https://doi.org/10.1109/icmic.2018.8529860
Funding
The funding was provided by National Natural Science Foundation of China (Grand Nos. 41361077, 41561085) and Natural Science Foundation of Jiangxi Province (Grand No. 20161BAB203091).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ji, W., Liu, D., Meng, Y. et al. A review of genetic-based evolutionary algorithms in SVM parameters optimization. Evol. Intel. 14, 1389–1414 (2021). https://doi.org/10.1007/s12065-020-00439-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12065-020-00439-z