research-article

Improved firefly algorithm for feature selection with the ReliefF-based initialization and the weighted voting mechanism

Authors:

Yue-lin GaoAuthors Info & Claims

Neural Computing and Applications, Volume 35, Issue 1

Pages 275 - 301

https://doi.org/10.1007/s00521-022-07755-8

Published: 09 September 2022 Publication History

Abstract

Feature selection has become popular in data mining tasks currently for its ability of improving the performance of the algorithm and gaining more information about the dataset. Although the firefly algorithm is a well-performed heuristic algorithm, there is still much room for improvement as to the feature selection problem. In this research, an improved firefly algorithm designed for feature selection with the ReliefF-based initialization method and the weighted voting mechanism is proposed. First of all, a feature grouping initialization method that combines the results of the ReliefF algorithm and the cosine similarity is designed to take place of random initialization. Then, the direction of the firefly is modified to move toward the optimal solution. Finally, inspired by the ensemble algorithm, a weighted voter is proposed to build recommended positions for fireflies, which is also integrated with the elite crossover operator and the mutation operator to improve the diversity of the population. Selected from the mixed swarm, a new population is constructed to replace the original population in the next stage. To verify the effectiveness of the algorithm proposed in this paper, 18 datasets are utilized and 9 comparison algorithms (e.g., Black Hole Algorithm, Grey Wolf Optimizer and Pigeon Inspired Optimizer) from state-of-the-art related works are selected for the simulating experiments. The experimental results demonstrate the superiority of the proposed algorithm applied to the feature selection problem.

References

[1]

Alican D and Derya B Machine learning and data mining in manufacturing Expert Syst Appl 2021 166

[2]

Jie C, Jiawei L, Shulin W, and Sheng Y Feature selection in machine learning: a new perspective Neurocomputing 2018 300 70-79

[3]

Mohammad T, Majdi MM, Ali AH, Hossam F, Ibrahim A, Seyedali M, and Hamido F An evolutionary gravitational search-based feature selection Inf Sci 2019 497 219-239

[4]

Lei Y, Huan L (2014) Efficient feature selection via analysis of relevance and redundancy. J Mach Learn Res 5, 1205–1224 (2004). Springer Nature 2021 LATEX template Article Title 19

[5]

Mehrdad R, Kamal B, Elahe N, and Saman F Review of swarm intelligence-based feature selection methods Eng Appl Artif Intell 2021 100

[6]

Gao W, Hu L, Zhang P, and He J Feature selection considering the composition of feature relevancy Pattern Recogn Lett 2018 112 70-74

[7]

Manoranjan D and Huan L Feature selection for classification Intell Data Anal 1997 1 1-4

[8]

Sankalap A and Priyanka A Binary butterfly optimization approaches for feature selection Expert Syst Appl 2019 116 147-160

[9]

Ryan JU, Melissa M, William GLC, Randal SO, and Jason HM Relief-based feature selection: introduction and review J Biomed Inform 2018 85 189-203

[10]

Kira K, Rendell LA (1992) The feature selection problem: Traditional methods and a new algorithm. In: Proceedings of the Tenth National Conference on Artificial Intelligence, AAAI Press, San Jose, California, pp 129–134

[11]

Igor K (1994) Estimating attributes: analysis and extensions of RELIEF. Paper presented at the 94th European Conference on Machine Learning, Catania, Italy, 6–8 April 1994

[12]

Girish C and Ferat S A survey on feature selection methods Comput Electr Eng 2014 40 16-28

[13]

Wan Y, Wang M, Ye Z, and Lai X A feature selection method based on modified binary coded ant colony optimization algorithm Appl Soft Comput 2016 49 248-258

[14]

Ibrahim A, Maria H, Hossam F, Nailah A, Ali AH, Majdi MM, Mohamed EAE, and Seyedali M A dynamic locality multi-objective salp swarm algorithm for feature selection Comput Ind Eng 2020 147

[15]

Emrah H, Bing X, and Mengjie Z Differential evolution for filter feature selection based on information theory and feature ranking Knowl Based Syst 2018 140 103-119

[16]

Yong Z, DunWei G, XiaoZhi G, Tian T, and Xiaoyan S Binary differential evolution with self-learning for multi-objective feature selection Inf Sci 2020 507 67-85

[17]

Ke C, Fengyu Z, and Xianfeng Y Hybrid particle swarm optimization with spiral-shaped mechanism for feature selection Expert Syst Appl 2019 128 140-156

[18]

Maryam A and Behrouz MB Optimizing multi-objective PSO based feature selection method using a feature elitism mechanism Expert Syst Appl 2018 113 499-514

[19]

Bing X, Mengjie Z, Will NB (2012) New Fitness Functions in Binary Particle Swarm Optimisation for Feature Selection. Paper presented at the IEEE Congress on Evolutionary Computation, Brisbane, Australia, 10–15 June 2012

[20]

Bach HN, Bing X, and Peter A PSO with surrogate models for feature selection: static and dynamic clustering-based methods Memetic Comput 2019 10 291-300

[21]

Wang L, Gao Y, Gao S, and Yong X A new feature selection method based on a self-variant genetic algorithm applied to android malware detection Symmetry 2021 13 1290

[22]

Eid E, Hossam MZ, and Aboul EH Binary grey wolf optimization approaches for feature selection Neurocomputing 2016 172 371-381

[23]

Pei H, JengShyang P, and ShuChuan C Improved Binary Grey Wolf Optimizer and Its application for feature selection Knowl Based Syst 2020 195

[24]

Mafarja MM, Ibrahim A, Hossam F, Abdelaziz IH, Ala MA, and Seyedali M Binary grasshopper optimisation algorithm approaches for feature selection problems Expert Syst Appl 2019 117 267-286

[25]

Majdi MM, Ibrahim A, Ali AH, Abdelaziz IH, Hossam F, Ala MA, and Seyedali M Evolutionary population dynamics and grasshopper optimization approaches for feature selection problems Knowl Based Syst 2018 145 25-45

[26]

Gehad IS, Ghada K, and Mohamed HH A novel chaotic salp swarm algorithm for global optimization and feature selection Appl Intell 2018 48 3462-3481

[27]

Hossam F, Majdi MM, Ali AH, Ibrahim A, Ala MA, Seyedali M, and Hamido F An efficient binary salp swarm algorithm with crossover scheme for feature Knowl Based Syst 2018 154 43-67

[28]

Emrah H, Bing X, Mengjie Z, Dervis K, Bahriye A (2015) A multi-objective artificial bee colony approach to feature selection using fuzzy mutual information. Paper presented at the IEEE Congress on Evolutionary Computation, Sendai, Japan, 25–28 May 2015

[29]

Majdi MM and Seyedali M Hybrid whale optimization algorithm with simulated annealing for feature selection Neurocomputing 2017 260 302-312

[30]

Majdi MM and Seyedali M Whale optimization approaches for wrapper feature selection Appl Soft Comput 2018 62 441-453

[31]

Yanan Z, Renjing L, Xin W, Huiling C, and Chengye L Boosted binary harris hawks optimizer and feature selection Eng Comput 2021 37 3741-3770

[32]

Yang XS (2009) Firefly algorithms for multimodal optimization, stochastic algorithms: foundations and applications. SAGA 2009. Lecture notes in computer science. Springer, Berlin, Heidelberg, 5792.

[33]

Jinran W, YouGan W, Kevin B, YuChu T, Brodie L, and Zhe D An improved firefly algorithm for global continuous optimization problems Expert Syst Appl 2020 149

[34]

ChunFeng W and WenXin S A novel firefly algorithm based on gender difference and its convergence Appl Soft Comput 2019 80 107-124

[35]

Aref Y and Cemal K A modified firefly algorithm for global minimum optimization Appl Soft Comput 2018 62 29-44

[36]

Xingsi X A compact firefly algorithm for matching biomedical ontologies Knowl Inf Syst 2020 62 2855-2871

[37]

Asma MA, Abdulqader MM, and Abdullatif G An improved hybrid firefly algorithm for capacitated vehicle routing Appl Soft Comput 2019 84 1568-4946

[38]

Hui W, Wenjun W, Zhihua C, Xinyu Z, Jia Z, and Ya L A new dynamic firefly algorithm for demand estimation of water resources Inf Sci 2018 438 95-106

[39]

Selvakumar B and Muneeswaran K Firefly algorithm based feature selection for network intrusion detection Comput Secur 2019 81 148-155

[40]

Long Z, Linlin S, and Jianhua W Optimal feature selection using distance-based discrete firefly algorithm with mutual information criterion Neural Comput Appl 2017 28 2795-2808

[41]

Yong Z, Xianfang S, and DunWei G A return-cost-based binary firefly algorithm for feature selection Inf Sci 2017 418 561-574

[42]

Bing X, Mengjie Z, and Will NB Particle swarm optimisation for feature selection in classification:novel initialisation and updating mechanisms Appl Soft Comput 2014 18 261-276

[43]

Bach HN, Bing X, Ivy L, Mengjie Z (2014) PSO and statistical clustering for feature selection: A new representation. Paper presented at the 10th SEAL International Conference, Dunedin, New Zealand,15–18 December 2014

[44]

Elnaz P and Nizamettin A Binary black hole algorithm for feature selection and classification on biological data Appl Soft Comput 2017 56 94-106

[45]

Hui W, Zhihua C, Hui S, Shahryar R, and XinShe Y Randomly attracted firefly algorithm with neighborhood search and dynamic parameter adjustment mechanism Soft Comput 2017 21 5325-5339

[46]

Bach HN, Bing X, and Mengjie Z A survey on swarm intelligence approaches to feature selection in data mining Swarm Evol Comput 2020 54 100663

[47]

Mohamed AB, Doaa E, Ibrahim ME, Victor HCA, and Seyedali M A new fusion of grey wolf optimizer algorithm with a two-phase mutation for feature selection Expert Syst Appl 2020 139 112824

[48]

Dua D, Graff C (2019) UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science

[49]

Yudong Z, Shuihua W, Preetha P, and Genlin J Binary PSO with mutation operator for feature selection using decision tree applied to spam detection Appl Soft Comput 2014 64 22-31

[50]

Hussien AG, Hassanien AE, Houssein EH, Bhattacharyya S, and Amin M Bhattacharyya S, Mukherjee A, Bhaumik H, Das S, and Yoshida K S-shaped binary whale optimization algorithm for feature selection Recent trends in signal and image processing 2019 Singapore Springer 79-87

[51]

Hadeel A, Ahmad S, and Khair ES A feature selection algorithm for intrusion detection system based on pigeon inspired optimizer Expert Syst Appl 2020 148

[52]

Benjamini Y and Hochberg Y Controlling the false discovery rate: a practical and powerful approach to multiple testing J Roy Stat Soc: Ser B (Methodol) 1995 57 289-300

Cited By

Wang ZChen YCai ZHeidari ALiu LChen H(2024)Weighted mean of vectors algorithm with neighborhood information interaction and vertical and horizontal crossover mechanism for feature selectionApplied Intelligence10.1007/s10489-024-05889-x55:1Online publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1007/s10489-024-05889-x

Index Terms

Improved firefly algorithm for feature selection with the ReliefF-based initialization and the weighted voting mechanism
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
  2. Machine learning
    1. Machine learning algorithms

Index terms have been assigned to the content through auto-classification.

Recommendations

Optimal feature selection using distance-based discrete firefly algorithm with mutual information criterion

In this paper, we investigate feature subset selection problem by a new self-adaptive firefly algorithm (FA), which is denoted as DbFAFS. In classical FA, it uses constant control parameters to solve different problems, which results in the premature of ...
A return-cost-based binary firefly algorithm for feature selection

Various real-world applications can be formulated as feature selection problems, which have been known to be NP-hard. In this paper, we propose an effective feature selection method based on firefly algorithm (FFA), called return-cost-based binary FFA (...
Feature Selection by Firefly Algorithm with Improved Initialization Strategy
ECBS 2021: 7th Conference on the Engineering of Computer Based Systems

In this paper, the application of the swarm intelligence algorithm for feature selection challenges in the field of machine learning is presented. Feature selection, which is part of the dimension reduction process, helps in selecting those features ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neural Computing and Applications

Neural Computing and Applications Volume 35, Issue 1

Jan 2023

1023 pages

ISSN:0941-0643

EISSN:1433-3058

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2022. Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 09 September 2022

Accepted: 17 August 2022

Received: 21 March 2022

Author Tags

Qualifiers

Research-article

Funding Sources

Key Project of Ningxia Natural Science Foundation
Major scientific Research Project of Northern University for Nationalities
Natural Science Foundation of NingXia Hui Autonomous Region
Research Startup Foundation of North Minzu University
National Natural Science Foundation of China
First-class Discipline Construction Fund project of Ningxia Higher Education

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang ZChen YCai ZHeidari ALiu LChen H(2024)Weighted mean of vectors algorithm with neighborhood information interaction and vertical and horizontal crossover mechanism for feature selectionApplied Intelligence10.1007/s10489-024-05889-x55:1Online publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1007/s10489-024-05889-x

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents