research-article

A two-stage hybrid ant colony optimization for high-dimensional feature selection

Authors:

Licheng JiaoAuthors Info & Claims

Volume 116, Issue C

https://doi.org/10.1016/j.patcog.2021.107933

Published: 01 August 2021 Publication History

Highlights

•

A two-stage algorithm for high-dimensional feature selection is proposed.

•

A advanced hybrid ant colony optimization algorithm is proposed.

•

Our method has good selection performance with short running time.

Abstract

Ant colony optimization (ACO) is widely used in feature selection owing to its excellent global/local search capabilities and flexible graph representation. However, the current ACO-based feature selection methods are mainly applied to low-dimensional datasets. For thousands of dimensional datasets, the search for the optimal feature subset (OFS) becomes extremely difficult due to the exponential increase of the search space. In this paper, we propose a two-stage hybrid ACO for high-dimensional feature selection (TSHFS-ACO). As an additional stage, it uses the interval strategy to determine the size of OFS for the following OFS search. Compared to the traditional one-stage methods that determine the size of OFS and search for OFS simultaneously, the stage of checking the performance of partial feature number endpoints in advance helps to reduce the complexity of the algorithm and alleviate the algorithm from getting into a local optimum. Moreover, the advanced ACO algorithm embeds the hybrid model, which uses the features’ inherent relevance attributes and the classification performance to guide OFS search. The test results on eleven high-dimensional public datasets show that TSHFS-ACO is suitable for high-dimensional feature selection. The obtained OFS has state-of-the-art performance on most datasets. And compared with other ACO-based feature selection methods, TSHFS-ACO has a shorter running time.

References

[1]

I.A. Gheyas, L.S. Smith, Feature subset selection in large dimensionality domains, Pattern Recognit. 43 (1) (2010) 5–13.

[2]

I. Guyon, A. Elisseeff, An introduction to variable and feature selection, J. Mach. Learn. Res. 3 (1) (2003) 1157–1182.

[3]

J.M. Sotoca, F. Pla, Supervised feature selection by clustering using conditional mutual information-based distances, Pattern Recognit. 43 (6) (2010) 2068–2081.

[4]

S. Sharmin, M. Shoyaib, A.A. Ali, M.A.H. Khan, O. Chae, Simultaneous feature selection and discretization based on mutual information, Pattern Recognit. 91 (2019) 162–174.

[5]

G. Wang, Q. Song, B. Xu, Y. Zhou, Selecting feature subset for high dimensional data via the propositional foil rules, Pattern Recognit. 46 (1) (2013) 199–214.

Digital Library

[6]

T. Chen, C. Guestrin, XGBoost: a scalable tree boosting system, Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining, ACM, 2016, pp. 785–794.

Digital Library

[7]

S. Amini, S. Homayouni, A. Safari, A.A. Darvishsefat, Object-based classification of hyperspectral data using random forest algorithm, Geo-Spatial Inf. Sci. 21 (2) (2018) 127–138.

[8]

M. El Yafrani, B. Ahiod, Efficiently solving the traveling thief problem using hill climbing and simulated annealing, Inf. Sci. 432 (2018) 231–244.

[9]

L. Yu, Z. Bian, L. Xiang, Developing a dynamic neighborhood structure for an adaptive hybrid simulated annealing–tabu search algorithm to solve the symmetrical traveling salesman problem, Appl. Soft Comput. 49 (2016) 937–952.

[10]

L. Abualigah, Group search optimizer: a nature-inspired meta-heuristic optimization algorithm with its results, variants, and applications, Neural Comput. Appl. 32 (2020) 1–24.

[11]

L. Abualigah, A. Diabat, A novel hybrid antlion optimization algorithm for multi-objective task scheduling problems in cloud computing environments, Cluster Comput. 24 (1) (2021) 205–223.

[12]

H. Zhang, X. Bai, J. Zhou, J. Cheng, H. Zhao, Object detection via structural feature selection and shape model, IEEE Trans. Image Process. 22 (12) (2013) 4984–4995.

[13]

H. Zhu, W. Ma, L. Li, L. Jiao, S. Yang, B. Hou, A dual-branch attention fusion deep network for multiresolution remote-sensing image classification, Inf. Fusion 58 (2020) 116–131.

[14]

A. Senawi, H.L. Wei, S.A. Billings, A new maximum relevance-minimum multicollinearity (MRmMC) method for feature selection and ranking, Pattern Recognit. 67 (2017) 47–61.

[15]

U. Kamath, K. De Jong, A. Shehu, Effective automated feature construction and selection for classification of biological sequences, PLoS One 9 (7) (2014) 1.

[16]

L.M.Q. Abualigah, E.S. Hanandeh, Applying genetic algorithms to information retrieval using vector space model, Int. J. Comput. Sci. Eng. Appl. 5 (1) (2015) 19.

[17]

S. Gu, R. Cheng, Y. Jin, Feature selection for high-dimensional classification using a competitive swarm optimizer, Soft Comput. 22 (3) (2018) 811–822.

[18]

K. Zheng, X. Wang, Feature selection method with joint maximal information entropy between features and class, Pattern Recognit. 77 (2018) 20–29.

Digital Library

[19]

L.M. Abualigah, A.T. Khader, Unsupervised text feature selection technique based on hybrid particle swarm optimization algorithm with genetic operators for the text clustering, J. Supercomput. 73 (11) (2017) 4773–4795.

Digital Library

[20]

L.M.Q. Abualigah, Feature Selection and Enhanced Krill Herd Algorithm for Text Document Clustering, Springer, 2019.

[21]

B. Tran, B. Xue, M. Zhang, Variable-length particle swarm optimization for feature selection on high-dimensional classification, IEEE Trans. Evolut. Comput. 23 (3) (2018) 473–487.

[22]

A. Purohit, N.S. Chaudhari, A. Tiwari, Construction of classifier with feature selection based on genetic programming, IEEE Congress on Evolutionary Computation, 2010, pp. 1–5.

[23]

B. Tran, B. Xue, M. Zhang, Genetic programming for multiple-feature construction on high-dimensional classification, Pattern Recognit. 93 (2019) 404–417.

[24]

S. Tabakhi, P. Moradi, F. Akhlaghian, An unsupervised feature selection algorithm based on ant colony optimization, Eng. Appl. Artif. Intell. 32 (2014) 112–123.

[25]

S. Tabakhi, P. Moradi, Relevance–redundancy feature selection based on ant colony optimization, Pattern Recognit. 48 (9) (2015) 2798–2811.

Digital Library

[26]

Z. Manbari, F.A. Tab, C. Salavati, Fast unsupervised feature selection based on the improved binary ant system and mutation strategy, Neural Comput. Appl. (2019) 1–20.

[27]

S. Tabakhi, A. Najafi, R. Ranjbar, P. Moradi, Gene selection for microarray data classification using a novel ant colony optimization, Neurocomputing 168 (2015) 1024–1036.

Digital Library

[28]

Y. Wan, M. Wang, Z. Ye, X. Lai, A feature selection method based on modified binary coded ant colony optimization algorithm, Appl. Soft Comput. 49 (2016) 248–258.

[29]

S. Kashef, H. Nezamabadi-pour, An advanced ACO algorithm for feature subset selection, Neurocomputing 147 (2015) 271–279.

[30]

H.R. Kanan, K. Faez, An improved feature selection method based on ant colony optimization (ACO) evaluated on face recognition system, Appl. Math. Comput. 205 (2) (2008) 716–725.

[31]

M.H. Aghdam, N. Ghasem-Aghaee, M.E. Basiri, Text feature selection using ant colony optimization, Expert Syst. Appl. 36 (3) (2009) 6843–6853.

Digital Library

[32]

B. Xue, M. Zhang, W.N. Browne, X. Yao, A survey on evolutionary computation approaches to feature selection, IEEE Trans. Evolut. Comput. 20 (4) (2015) 606–626.

[33]

B. Xiao, E.R. Hancock, R.C. Wilson, Graph characteristics from the heat kernel trace, Pattern Recognit. 42 (11) (2009) 2589–2606.

[34]

Z. Zhang, L. Bai, Y. Liang, E. Hancock, Joint hypergraph learning and sparse regression for feature selection, Pattern Recognit. 63 (2017) 291–309.

[35]

P. Moradi, M. Rostami, Integration of graph clustering with ant colony optimization for feature selection, Knowl.-Based Syst. 84 (2015) 144–161.

[36]

V.D. Blondel, J.-L. Guillaume, R. Lambiotte, E. Lefebvre, Fast unfolding of communities in large networks, J. Stat. Mech. 2008 (10) (2008) 10008.

[37]

M. Kong, P. Tian, A binary ant colony optimization for the unconstrained function optimization problem, International Conference on Computational and Information Science, 2005, pp. 682–687.

[38]

M. Ghosh, R. Guha, R. Sarkar, A. Abraham, A wrapper-filter feature selection technique based on ant colony optimization, Neural Comput. Appl. 32 (2020) 7839–7857.

[39]

L.M. Abualigah, A.T. Khader, E.S. Hanandeh, A combination of objective functions and hybrid krill herd algorithm for text document clustering analysis, Eng. Appl. Artif. Intell. 73 (AUG.) (2018) 111–125.

[40]

L.M. Abualigah, A.T. Khader, E.S. Hanandeh, A new feature selection method to improve the document clustering using particle swarm optimization algorithm, J. Comput. Sci. 25 (2018) 456–466.

[41]

B. Singh, N. Kushwaha, O.P. Vyas, A feature subset selection technique for high dimensional data using symmetric uncertainty, J. Data Anal. Inf. Process. 02 (4) (2014) 95–105.

[42]

G. Patterson, M. Zhang, Fitness functions in genetic programming for classification with unbalanced data, Australasian Joint Conference on Artificial Intelligence, 2007, pp. 769–775.

[43]

B.H. Nguyen, B. Xue, P. Andreae, H. Ishibuchi, M. Zhang, Multiple reference points based decomposition for multi-objective feature selection in classification: Static and dynamic mechanisms, IEEE Trans. Evolut. Comput. 24 (2019) 170–184.

Cited By

Dorighello RDelgado MLüders RPigatto DLi XHandl J(2024)ACO-Pruning for Deep Neural Networks: A Case Study in CNNsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3664125(1895-1903)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3664125
Cui MWang KDing XXu ZWang XShi P(2024)Multi-view Stable Feature Selection with Adaptive Optimization of View WeightsKnowledge-Based Systems10.1016/j.knosys.2024.111970299:COnline publication date: 5-Sep-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111970
Zhao JWu DZhou YLiang JWei WLi Y(2024)Rough set Theory-Based group incremental approach to feature selectionInformation Sciences: an International Journal10.1016/j.ins.2024.120733675:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120733
Show More Cited By

Index Terms

A two-stage hybrid ant colony optimization for high-dimensional feature selection
1. Mathematics of computing
  1. Mathematical analysis
2. Theory of computation
  1. Design and analysis of algorithms

Index terms have been assigned to the content through auto-classification.

Recommendations

Feature selection using ant colony optimization (ACO): a new method and comparative study in the application of face recognition system
ICDM'07: Proceedings of the 7th industrial conference on Advances in data mining: theoretical aspects and applications

Feature Selection (FS) and reduction of pattern dimensionality is a most important step in pattern recognition systems. One approach in the feature selection area is employing population-based optimization algorithms such as Genetic Algorithm (GA)-based ...
Enriched ant colony optimization and its application in feature selection

This paper presents a new variant of ant colony optimization (ACO), called enRiched Ant Colony Optimization (RACO). This variation tries to consider the previously traversed edges in the earlier executions to adjust the pheromone values appropriately ...
Image feature selection based on ant colony optimization
AI'11: Proceedings of the 24th international conference on Advances in Artificial Intelligence

Image feature selection (FS) is an important task which can affect the performance of image classification and recognition. In this paper, we present a feature selection algorithm based on ant colony optimization (ACO). For <em>n</em> features, most ACO-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Pattern Recognition

Pattern Recognition Volume 116, Issue C

Aug 2021

405 pages

ISSN:0031-3203

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 August 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dorighello RDelgado MLüders RPigatto DLi XHandl J(2024)ACO-Pruning for Deep Neural Networks: A Case Study in CNNsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3664125(1895-1903)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3664125
Cui MWang KDing XXu ZWang XShi P(2024)Multi-view Stable Feature Selection with Adaptive Optimization of View WeightsKnowledge-Based Systems10.1016/j.knosys.2024.111970299:COnline publication date: 5-Sep-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111970
Zhao JWu DZhou YLiang JWei WLi Y(2024)Rough set Theory-Based group incremental approach to feature selectionInformation Sciences: an International Journal10.1016/j.ins.2024.120733675:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120733
Moosavi SSaadat AAbaid ZNi WLi KGuizani M(2024)Feature selection based on dataset variance optimization using Hybrid Sine Cosine – Firehawk Algorithm (HSCFHA)Future Generation Computer Systems10.1016/j.future.2024.02.017155:C(272-286)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.future.2024.02.017
Moslemi A(2024)A tutorial-based survey on feature selectionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107136126:PDOnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107136
Abdel-Salam MHu GÇelik EGharehchopogh FEL-Hasnony I(2024)Chaotic RIME optimization algorithm with adaptive mutualism for feature selection problemsComputers in Biology and Medicine10.1016/j.compbiomed.2024.108803179:COnline publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2024.108803
Li MCao RZhao YLi YDeng S(2024)Population characteristic exploitation-based multi-orientation multi-objective gene selection for microarray data classificationComputers in Biology and Medicine10.1016/j.compbiomed.2024.108089170:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2024.108089
Chu SYuan XPan JLin BLee Z(2024)A multi-strategy surrogate-assisted social learning particle swarm optimization for expensive optimization and applicationsApplied Soft Computing10.1016/j.asoc.2024.111876162:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111876
Cui MChen JQiu XLv WQin HZhang X(2024)Multi-class intrusion detection system in SDN based on hybrid BiLSTM modelCluster Computing10.1007/s10586-024-04477-527:7(9937-9956)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s10586-024-04477-5
Zandvakili AMansouri NJavidi M(2024)A new feature selection algorithm based on fuzzy-pathfinder optimizationNeural Computing and Applications10.1007/s00521-024-10043-236:28(17585-17614)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s00521-024-10043-2
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents