融合渐近性的灰狼优化支持向量机模型

摘要/Abstract

摘要： 大数据的发展对数据分类领域的分类准确性有了更高的要求;支持向量机(Support Vector Machine,SVM)的广泛应用需要一种高效的方法来构造一个分类能力强的SVM分类器;SVM的核函数参数与惩罚因子以及特征子集对预测模型的复杂度和预测精度有着重要影响。为提高SVM的分类性能,文中将SVM的渐近性融合到灰狼优化(Grey Wolf Optimization,GWO)算法中,提出了新的SVM分类器模型,该模型对SVM的参数与数据的特征子集同时进行优化,融合SVM渐近性的新灰狼个体将灰狼优化算法的搜索空间导向超参数空间中的最佳区域,能够更快地获得最优解;此外,将获得的分类准确率、所选特征个数和支持向量个数相结合,提出了一种新的适应度函数,新的适应度函数与融合渐近性的灰狼优化算法将搜索引向最优解。采用UCI中的多个经典数据集对所提模型进行验证,将其与网格搜素算法、未融合渐近性的灰狼优化算法以及其他文献中的方法进行对比,其分类准确率在不同数据集上均有不同程度的提升。实验结果表明,所提算法能找到SVM的最优参数与最小特征子集,具有更高的分类准确率和更短的平均处理时间。

关键词: 参数优化, 灰狼优化算法, 渐近性, 特征选择, 支持向量机

Abstract: The development of big data requires higher accuracy of data classification.The wide application of support vector machine (SVM) requires an efficient method to construct an SVM classifier with strong classification ability.The kernel parameter,penalty parameter and feature subsets of dataset have an important impact on the complexity and prediction accuracy of the model.In order to improve the classification performance of SVM,the asymptotic of SVM was integrated into the gray wolf optimization (GWO) algorithm,and a new SVM classifier model was proposed.The model optimizes feature selection and parameter optimization of SVM at the same time.The new grey wolf individual integrating the asymptotic property of SVM directs the search space of grey wolf optimization algorithm to the optimal region in super-parameter space,and can obtain the optimal solution faster.In addition,a new fitness function,which combines the classification accuracy obtained from the method,the number of chosen features and the number of support vectors,was proposed.The new fitness function and GWO fused asymptotic lead the search to the optimal solution.This paper used several classical datasets on UCI to verify the proposed model.Compared with the grid search algorithm,the gray wolf optimization algorithm without asymptotic convergence and other methods in the literature,the classification accuracy of the proposed algorithm has different degrees of improvement on different data sets.The experimental results show that the proposed algorithm can find the optimal parameters and the smallest feature subset of SVM,with higher classification accuracy and less average processing time.

Key words: Asymptotic, Feature selection, Gray wolf optimization algorithm, Parameters optimization, Support vector machines

中图分类号:

TP391

武玉坤,肖杰,李伟,楼吉林. 融合渐近性的灰狼优化支持向量机模型[J]. 计算机科学, 2020, 47(2): 37-43. https://doi.org/10.11896/jsjkx.190100092

WU Yu-kun,XIAO Jie,Wei William LEE,LOU Ji-lin. Support Vector Machine Model Based on Grey Wolf Optimization Fused Asymptotic[J]. Computer Science, 2020, 47(2): 37-43. https://doi.org/10.11896/jsjkx.190100092

参考文献

[1]DRUCKER H,WU H,VAPNIK V N.Support vector machines for spam categorization[J].IEEE Transactions on Neural Networks,1999,10(5) 1048-1054.
[2]VATSA M,SINGH R,NOORE A.Improving biometric recognition accuracy and robustness using dwt and and svm watermarking[J].IEICE Electronics Express,2005,2(12):362-367.
[3]BYVATOV E,SCHNEIDER G.Support vector machine applications1in1bioinformatics[J].Applied bioinformatics,2002,2(2):67-77.
[4]DOUCET J P,BARBAULT F,XIA H,et al.Nonlinear svm approaches to qspr/qsar studies and drug design[J].Current Computer-Aided Drug Design,2007,3(4):263-289.
[5]LIN S W,YING K C,CHEN S C,et al.Particle swarm optimization for parameter determination and feature selection of support vector machines[J].Expert Systems with Applications,2008,35(4):1817-1824.
[6]ZHANG X L,CHEN X F,HE Z J.An aco-based algorithm for parameter optimization of support vector11machines[J].Expert Systems with Applications,2010,37(9):6618-6628.
[7]MIRJALILI S,MIRJALILI S M,LEWIS A.Grey wolf optimizer[J].Advances in Engineering Software,2014,69(5):46-61.
[8]JALILIS M R.How effective is the grey wolf optimizer intrai-ning multilayer perceptrons[J].Applied Intelligence,2015,42(4):608-619.
[9]SONG H M,SULAIMAN M H,MOHAMED M R.An application of grey wolf optimizer for solving combined economic emission dispatch problems [J].International Review on Modelling and Simulations,2014,7(5):838-844.
[10]SULAIMAN M H,MUSTAFFA Z,MOHAMED M R,et al. Using the grey wolf optimizer for solving optimal reactive power dispatch problem[J].Applied Soft Computing,2015,32:286-292.
[11]LONG W,ZHAO D Q,XU S J.Improved grey wolf optimization algorithm for constrained optimization problem[J].Journal of Computer Applications,2015,35(9):2590-2595.
[12]FRIEDRICHSF,IGEL C.Evolutionary tuning of multiple SVM parameters[J].Neurocomputing,2005,64(2):107-117.
[13]HUANG C L,WANG C J.A GA-based feature selection and parameters optimization for support vector machines[J].Expert Systems with Applications,2006,31:231-240.
[14]WU C H,TZENG G H,LIN R H.A novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression[J].Expert Systems with Applications,2009,36(3):4725-4735.
[15]CHEN J Y,XIONG H,ZHENG H B.Parameters Optimization for SVM Based on Particle Swarm Algorithm[J].Computer Science,2018,45(6):197-203.
[16]GUO L,WU Y X,ZHAO L,et al.Classification of Mental Task From EEG Signals Using Immune Feature Weighted Support Vector Machines[J].IEEE Transactions on Magnetics,2011,47 (5):866-869.
[17]THARWAT A.A BA-based algorithm for parameter optimization of Support Vector Machine[J].Pattern Recognition Letters,2017,93:13-22.
[18]ALA′M A,HOSSAM F,JA′FAR A.Evolving Support Vector Machines using Whale Optimization Algorithm for spam profiles detection on online social networks in different lingua contexts[J].Knowledge-based systems,2018,8(153):91-104.
[19]CHAO C F,HORNG M H.The construction of support vector machine classifier using the firefly algorithm[J].Computational Intelligence and Neuroscience,2015,1:1-8.
[20]PRABUKUMAR M,AGILANDEESWARI L,GANESAN K. An intelligent lung cancer diagnosis system using cuckoo search optimization and support vector machine classifier[J].Journal of Ambient Intelligence and Humanized Computing,2019,1(10):267-293.
[21]XUE H X,BAI Y P,HU H P.A Novel Hybrid Model Based on TVIW-PSO-GSA Algorithm and Support Vector Machine for Classification Problems[J].IEEE Access,2019(7):27789-27801.
[22]MAO K Z.Feature subset selection for support vector machines through discriminative function pruning analysis[J].IEEE Transactions on Systems,Man,and Cybernetics,2004,34(1):60-67.
[23]RAYMER M L,PUNCH W F,GOODMAN E D,et al.Dimensionality reduction using genetic algorithms[J].IEEE Transactions on Evolutionary Computation,2000(2):164-171.
[24]YANG J,HONAVAR V.Feature subset selection using a genetic algorithm[J].IEEE Intelligent Systems,1998,13(2):44-49.
[25]GUYON I,WESTON J,BARNHILL S,et al.Gene selection for cancer classi?cation using support vector machines[J].Machine Learning,2002,46(1/2/3):389-422.
[26]LIN S W,YING K C,Chen S C,et al.Particle swarm optimization for parameter determination and feature selection of support vector machines[J].Expert Systems with Applications,2008,35(4):1817-1824.
[27]ZHOU T,LU H L,WANG W W,et al.GA-SVM based feature selection and parameter optimization in hospitalization expense modeling [J].Applied Soft Computing,2019,75 (2):323-332.
[28]FARIS H,HASSONAH M A,AL-ZOUBI,et al.A multi-verse optimizer approach for feature selection and optimizing SVM parameters based on a robust system architecture[J].Neural Computing and Applications,2017,30(8):2355-2369.
[29]ALJARAH I.Simultaneous Feature Selection and Support Vector Machine Optimization Using the Grasshopper Optimization Algorithm[J].Cognitive Computation,2018,10(3):478-495.
[30]CHEN Y,MA H W.Feature selection and parameter optimization of support vector machine based on the bees algorithm[J].Modular Machine Tool&Automatic Manufacturing Technique,2013 (11):41-43.
[31]LIN K C,CHEN S Y,JASON C,et al.Feature Selection and Parameter Optimization of Support Vector Machines Based on Modified Artificial Fish Swarm Algorithms[J].Mathematical Problems in Engineering,2015,Article ID 604108.
[32]SHEN Y L,SONG J,WAN Z C.Improved Fireworks Algorithm for Support Vector Machine Feature Selection and Parameters Optimization[J].Microelectronics&Computer,2018,35(1):21-25.
[33]IKEDA K,AOISHI T.An asymptotic statistical analysis of support vector machines with soft margins[J].Neural Networks,2005,18(3):251-259.
[34]ZHAO M Y.Feature selection and parameter optimization for support vector machines:A new approach based on genetic algorithm with feature chromosomes[J].Expert Systems with Applications,2011,38(5):5197-5204.
[35]KEERTHI S S,LIN C J.Asymptotic behaviors of support vector machines with Gaussian kernel[J].Neural Computation,2003,15:1667-1689.
[36]CHEN Z.A Parallel Genetic Algorithm Based Feature Selection and Parameter Optimization for Support Vector Machine[J].Scientific Programming,2016:1-10.
[37]CHANG C C,LIN C J.LIBSVM:a library for support vector machines[J].ACM Transactions on Intelligent Systems and Technology,2011,2(3).
[38]PHAN A V,NGUYEN M L,BUI L T.Feature weighting and SVM parameters optimization based on genetic algorithms for classification problems[J].Applied Intelligence,2016,46(2):455-469.

相关文章 15

[1]	李斌, 万源. 基于相似度矩阵学习和矩阵校正的无监督多视角特征选择 Unsupervised Multi-view Feature Selection Based on Similarity Matrix Learning and Matrix Alignment 计算机科学, 2022, 49(8): 86-96. https://doi.org/10.11896/jsjkx.210700124
[2]	胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[3]	王兵, 吴洪亮, 牛新征. 基于改进势场法的机器人路径规划 Robot Path Planning Based on Improved Potential Field Method 计算机科学, 2022, 49(7): 196-203. https://doi.org/10.11896/jsjkx.210500020
[4]	侯夏晔, 陈海燕, 张兵, 袁立罡, 贾亦真. 一种基于支持向量机的主动度量学习算法 Active Metric Learning Based on Support Vector Machines 计算机科学, 2022, 49(6A): 113-118. https://doi.org/10.11896/jsjkx.210500034
[5]	康雁, 王海宁, 陶柳, 杨海潇, 杨学昆, 王飞, 李浩. 混合改进的花授粉算法与灰狼算法用于特征选择 Hybrid Improved Flower Pollination Algorithm and Gray Wolf Algorithm for Feature Selection 计算机科学, 2022, 49(6A): 125-132. https://doi.org/10.11896/jsjkx.210600135
[6]	单晓英, 任迎春. 基于改进麻雀搜索优化支持向量机的渔船捕捞方式识别 Fishing Type Identification of Marine Fishing Vessels Based on Support Vector Machine Optimized by Improved Sparrow Search Algorithm 计算机科学, 2022, 49(6A): 211-216. https://doi.org/10.11896/jsjkx.220300216
[7]	陈景年. 一种适于多分类问题的支持向量机加速方法 Acceleration of SVM for Multi-class Classification 计算机科学, 2022, 49(6A): 297-300. https://doi.org/10.11896/jsjkx.210400149
[8]	邢云冰, 龙广玉, 胡春雨, 忽丽莎. 基于SVM的类别增量人体活动识别方法 Human Activity Recognition Method Based on Class Increment SVM 计算机科学, 2022, 49(5): 78-83. https://doi.org/10.11896/jsjkx.210400024
[9]	储安琪, 丁志军. 基于灰狼优化算法的信用评估样本均衡化与特征选择同步处理 Application of Gray Wolf Optimization Algorithm on Synchronous Processing of Sample Equalization and Feature Selection in Credit Evaluation 计算机科学, 2022, 49(4): 134-139. https://doi.org/10.11896/jsjkx.210300075
[10]	孙林, 黄苗苗, 徐久成. 基于邻域粗糙集和Relief的弱标记特征选择方法 Weak Label Feature Selection Method Based on Neighborhood Rough Sets and Relief 计算机科学, 2022, 49(4): 152-160. https://doi.org/10.11896/jsjkx.210300094
[11]	武玉坤, 李伟, 倪敏雅, 许志骋. 单类支持向量机融合深度自编码器的异常检测模型 Anomaly Detection Model Based on One-class Support Vector Machine Fused Deep Auto-encoder 计算机科学, 2022, 49(3): 144-151. https://doi.org/10.11896/jsjkx.210100142
[12]	李宗然, 陈秀宏, 陆赟, 邵政毅. 鲁棒联合稀疏不相关回归 Robust Joint Sparse Uncorrelated Regression 计算机科学, 2022, 49(2): 191-197. https://doi.org/10.11896/jsjkx.210300034
[13]	张叶, 李志华, 王长杰. 基于核密度估计的轻量级物联网异常流量检测方法 Kernel Density Estimation-based Lightweight IoT Anomaly Traffic Detection Method 计算机科学, 2021, 48(9): 337-344. https://doi.org/10.11896/jsjkx.200600108
[14]	杨蕾, 降爱莲, 强彦. 基于自编码器和流形正则的结构保持无监督特征选择 Structure Preserving Unsupervised Feature Selection Based on Autoencoder and Manifold Regularization 计算机科学, 2021, 48(8): 53-59. https://doi.org/10.11896/jsjkx.200700211
[15]	侯春萍, 赵春月, 王致芃. 基于自反馈最优子类挖掘的视频异常检测算法 Video Abnormal Event Detection Algorithm Based on Self-feedback Optimal Subclass Mining 计算机科学, 2021, 48(7): 199-205. https://doi.org/10.11896/jsjkx.200800146

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed