research-article

Hybrid filter-wrapper feature selection using whale optimization algorithm: : A multi-objective approach

Authors:

Abdelouahab Moussaoui,

Djaafar ZouacheAuthors Info & Claims

Volume 183, Issue C

https://doi.org/10.1016/j.eswa.2021.115312

Published: 30 November 2021 Publication History

Highlights

•

A novel multi-objective feature selection algorithm is proposed.

•

It hybridizes filter and wrapper models into whale optimization algorithm.

•

It has been validated through ten datasets and compared against five algorithms.

•

Yields solutions with fewer features and improves the classification accuracy.

Abstract

Feature selection aims at finding the minimum number of features that result in high classification accuracy. Accordingly, the feature selection is considered as a multi-objective problem. However, most existing approaches treat feature selection as single-objective problem, and they are often divided into two main categories: filter and wrapper methods. Filters are known as fast methods but less accurate, while wrappers are computationally expensive but with high classification performance. This paper proposes a novel hybrid filter-wrapper feature selection approach using whale optimization algorithm (WOA). The proposed method is a multi-objective algorithm in which a filter and wrapper fitness functions are optimized simultaneously. Our algorithm’s efficiency is demonstrated through an extensive comparison with seven well-known algorithms on twelve benchmark datasets. Experimental results show the ability of the proposed algorithm to obtain several subsets that include smaller number of features with excellent classification accuracy.

References

[1]

M. Amoozegar, B. Minaei-Bidgoli, Optimizing multi-objective pso based feature selection method using a feature elitism mechanism, Expert Systems with Applications 113 (2018) 499–514.

Digital Library

[2]

A. Auger, J. Bader, D. Brockhoff, E. Zitzler, Theory of the hypervolume indicator: optimal-distributions and the choice of the reference point, in: Proceedings of the tenth ACM SIGEVO workshop on Foundations of genetic algorithms, 2009, pp. 87–102.

[3]

L. Brezočnik, I. Fister, V. Podgorelec, Swarm intelligence algorithms for feature selection: a review, Applied Sciences 8 (2018) 1521.

[4]

Cervante, L., Xue, B., Shang, L., & Zhang, M. (2013). A multi-objective feature selection approach based on binary pso and rough set theory. In European conference on evolutionary computation in combinatorial optimization (pp. 25–36). Springer.

[5]

G. Chandrashekar, F. Sahin, A survey on feature selection methods, Computers & Electrical Engineering 40 (2014) 16–28.

Digital Library

[6]

C.A.C. Coello, G.T. Pulido, M.S. Lechuga, Handling multiple objectives with particle swarm optimization, IEEE Transactions on Evolutionary Computation 8 (2004) 256–279.

[7]

M. Dash, H. Liu, Feature selection for classification, Intelligent Data Analysis 1 (1997) 131–156.

[8]

M.A. El Aziz, A.A. Ewees, A.E. Hassanien, Multi-objective whale optimization algorithm for content-based image retrieval, Multimedia Tools and Applications 77 (2018) 26135–26172.

[9]

E. Emary, W. Yamany, A.E. Hassanien, V. Snasel, Multi-objective gray-wolf optimization for attribute reduction, Procedia Computer Science 65 (2015) 623–632.

[10]

Fister, D., Fister, I., Jagrič, T., & Brest, J. (2018). A novel self-adaptive differential evolution for feature selection using threshold mechanism. In 2018 IEEE symposium series on computational intelligence (SSCI) (pp. 17–24). IEEE.

[11]

Frank, A. (2010). Uci machine learning repository. http://archive. ics. uci. edu/ml.

[12]

A. Got, A. Moussaoui, D. Zouache, A guided population archive whale optimization algorithm for solving multiobjective optimization problems, Expert Systems with Applications 141 (2020).

Digital Library

[13]

Hamdani, T. M., Won, J.-M., Alimi, A. M., & Karray, F. (2007). Multi-objective feature selection with nsga ii. In International conference on adaptive and natural computing algorithms (pp. 240–247). Springer.

[14]

M. Hammami, S. Bechikh, C.-C. Hung, L.B. Said, A multi-objective hybrid filter-wrapper evolutionary approach for feature selection, Memetic Computing 11 (2019) 193–208.

[15]

Hancer, E., Xue, B., Zhang, M., Karaboga, D., & Akay, B. (2015). A multi-objective artificial bee colony approach to feature selection using fuzzy mutual information. In 2015 IEEE congress on evolutionary computation (CEC) (pp. 2420–2427). IEEE.

[16]

E. Hancer, B. Xue, M. Zhang, D. Karaboga, B. Akay, Pareto front feature selection based on artificial bee colony optimization, Information Sciences 422 (2018) 462–479.

Digital Library

[17]

Hussien, A. G., Houssein, E. H., & Hassanien, A. E. (2017). A binary whale optimization algorithm with hyperbolic tangent fitness function for feature selection. In 2017 Eighth international conference on intelligent computing and information systems (ICICIS) (pp. 166–172). IEEE.

[18]

Jović, A., Brkić, K., & Bogunović, N. (2015). A review of feature selection methods with applications. In 2015 38th international convention on information and communication technology, electronics and microelectronics (MIPRO) (pp. 1200–1205). IEEE.

[19]

M. Labani, P. Moradi, M. Jalili, A multi-objective genetic algorithm for text feature selection using the relative discriminative criterion, Expert Systems with Applications 149 (2020).

[20]

J. Li, K. Cheng, S. Wang, F. Morstatter, R.P. Trevino, J. Tang, H. Liu, Feature selection: A data perspective, ACM Computing Surveys (CSUR) 50 (2017) 1–45.

Digital Library

[21]

H. Liu, H. Motoda, Feature extraction, construction and selection: A data mining perspective, Vol. 453, Springer Science & Business Media, 1998.

[22]

M. Mafarja, S. Mirjalili, Whale optimization approaches for wrapper feature selection, Applied Soft Computing 62 (2018) 441–453.

[23]

S. Mirjalili, A. Lewis, The whale optimization algorithm, Advances in Engineering Software 95 (2016) 51–67.

Digital Library

[24]

S. Mirjalili, S. Saremi, S.M. Mirjalili, L.d.S. Coelho, Multi-objective grey wolf optimizer: a novel algorithm for multi-criterion optimization, Expert Systems with Applications 47 (2016) 106–119.

Digital Library

[25]

S. Rathee, S. Ratnoo, Feature selection using multi-objective chc genetic algorithm, Procedia Computer Science 167 (2020) 1656–1664.

Digital Library

[26]

A. Sahoo, S. Chandra, Multi-objective grey wolf optimizer for improved cervix lesion classification, Applied Soft Computing 52 (2017) 64–80.

[27]

C.E. Shannon, A mathematical theory of communication, Bell System Technical Journal 27 (1948) 379–423.

[28]

Sharawi, M., Zawbaa, H. M., & Emary, E. (2017). Feature selection approach based on whale optimization algorithm. In 2017 Ninth International Conference on Advanced Computational Intelligence (ICACI) (pp. 163–168). IEEE.

[29]

R.W. Swiniarski, A. Skowron, Rough set methods in feature selection and recognition, Pattern Recognition Letters 24 (2003) 833–849.

Digital Library

[30]

J. Too, A.R. Abdullah, N. Mohd Saad, Hybrid binary particle swarm optimization differential evolution-based feature selection for emg signals classification, Axioms 8 (2019) 79.

[31]

J. Too, A.R. Abdullah, N. Mohd Saad, N. Mohd Ali, W. Tee, A new competitive binary grey wolf optimizer to solve the feature selection problem in emg signals classification, Computers 7 (2018) 58.

[32]

J. Too, A.R. Abdullah, N. Mohd Saad, W. Tee, Emg feature selection and classification using a pbest-guide binary particle swarm optimization, Computation 7 (2019) 12.

[33]

A. Unler, A. Murat, A discrete particle swarm optimization method for feature selection in binary classification problems, European Journal of Operational Research 206 (2010) 528–539.

[34]

B. Xue, L. Cervante, L. Shang, W.N. Browne, M. Zhang, A multi-objective particle swarm optimisation for filter-based feature selection in classification problems, Connection Science 24 (2012) 91–116.

[35]

B. Xue, L. Cervante, L. Shang, W.N. Browne, M. Zhang, Multi-objective evolutionary algorithms for filter based feature selection in classification, International Journal on Artificial Intelligence Tools 22 (2013) 1350024.

[36]

B. Xue, M. Zhang, W.N. Browne, Particle swarm optimization for feature selection in classification: A multi-objective approach, IEEE Transactions on Cybernetics 43 (2012) 1656–1671.

[37]

B. Xue, M. Zhang, W.N. Browne, X. Yao, A survey on evolutionary computation approaches to feature selection, IEEE Transactions on Evolutionary Computation 20 (2015) 606–626.

Digital Library

[38]

Z. Yong, G. Dun-wei, Z. Wan-qiu, Feature selection of unreliable data using an improved multi-objective pso algorithm, Neurocomputing 171 (2016) 1281–1290.

[39]

Y. Zhang, S. Cheng, Y. Shi, D.-W. Gong, X. Zhao, Cost-sensitive feature selection using two-archive multi-objective artificial bee colony algorithm, Expert Systems with Applications 137 (2019) 46–58.

Digital Library

Cited By

Sağbaş E(2024)A novel two-stage wrapper feature selection approach based on greedy search for text sentiment classificationNeurocomputing10.1016/j.neucom.2024.127729590:COnline publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.127729
SabbaghGol HSaadatfar HKhazaiepoor M(2024)Evolution of the random subset feature selection algorithm for classification problemKnowledge-Based Systems10.1016/j.knosys.2023.111352285:COnline publication date: 15-Feb-2024
https://dl.acm.org/doi/10.1016/j.knosys.2023.111352
Zhang ZZhang FMao LChen CNing H(2024)DFS-WRInformation Fusion10.1016/j.inffus.2023.102191104:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.inffus.2023.102191
Show More Cited By

Index Terms

Hybrid filter-wrapper feature selection using whale optimization algorithm: A multi-objective approach
1. Mathematics of computing

Index terms have been assigned to the content through auto-classification.

Recommendations

Pareto front feature selection based on artificial bee colony optimization

Feature selection has two major conflicting aims, i.e., to maximize the classification performance and to minimize the number of selected features to overcome the curse of dimensionality. To balance their trade-off, feature selection can be handled as a ...
A novel multi-objective forest optimization algorithm for wrapper feature selection
Highlights
- A new multi-objective wrapper method based on Forest Optimization (MOFOA) is proposed.
Abstract
Feature selection is one of the important techniques of dimensionality reduction in data preprocessing because datasets generally have redundant and irrelevant features that adversely affect the performance and complexity of ...
A multi-objective algorithm for multi-label filter feature selection problem
Abstract
Feature selection is an important data preprocessing method before classification. Multi-objective optimization algorithms have been proved an effective way to solve feature selection problems. However, there are few studies on multi-objective ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 183, Issue C

Nov 2021

1520 pages

ISSN:0957-4174

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 30 November 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sağbaş E(2024)A novel two-stage wrapper feature selection approach based on greedy search for text sentiment classificationNeurocomputing10.1016/j.neucom.2024.127729590:COnline publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.127729
SabbaghGol HSaadatfar HKhazaiepoor M(2024)Evolution of the random subset feature selection algorithm for classification problemKnowledge-Based Systems10.1016/j.knosys.2023.111352285:COnline publication date: 15-Feb-2024
https://dl.acm.org/doi/10.1016/j.knosys.2023.111352
Zhang ZZhang FMao LChen CNing H(2024)DFS-WRInformation Fusion10.1016/j.inffus.2023.102191104:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.inffus.2023.102191
Xu ZYang FTang CWang HWang SSun JZhang Y(2024)FG-HFSExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.123069245:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.123069
Li MZhao YLou MDeng SWang L(2024)A multitasking multi-objective differential evolution gene selection algorithm enhanced with new elite and guidance strategies for tumor identificationExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122701241:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122701
Ma HLi MLv SWang LDeng S(2024)Roulette wheel-based level learning evolutionary algorithm for feature selection of high-dimensional dataApplied Soft Computing10.1016/j.asoc.2024.111948163:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111948
Avinash NSinha SShivamurthaiah M(2024)An Improved Gannet Optimization Algorithm Based on Opposition-Based Schemes for Feature Selection Problems in High-Dimensional DatasetsSN Computer Science10.1007/s42979-023-02487-55:1Online publication date: 8-Jan-2024
https://dl.acm.org/doi/10.1007/s42979-023-02487-5
Dhal PAzad C(2024)A fine-tuning deep learning with multi-objective-based feature selection approach for the classification of textNeural Computing and Applications10.1007/s00521-023-09225-136:7(3525-3553)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s00521-023-09225-1
Zhou RZhang YHe K(2023)A novel hybrid binary whale optimization algorithm with chameleon hunting mechanism for wrapper feature selection in QSAR classification modelExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.121015234:COnline publication date: 30-Dec-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.121015
Kusumadewi SRosita LGustri Wahyuni E(2023)Fuzzy linear regression based on a hybrid of fuzzy C-means and the fuzzy inference system for predicting serum iron levels in patients with chronic kidney diseaseExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.120314227:COnline publication date: 11-Jul-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.120314
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents