research-article

Co-operative Co-evolutionary Many-objective Embedded Multi-label Feature Selection with Decomposition-based PSO

Authors:

Mengjie ZhangAuthors Info & Claims

GECCO '23: Proceedings of the Genetic and Evolutionary Computation Conference

Pages 438 - 446

https://doi.org/10.1145/3583131.3590373

Published: 12 July 2023 Publication History

Abstract

Multi-label classification is an emerging machine-learning problem involving the prediction of a set of class labels based on the instance's features. In real-world problems, there are hundreds or thousands of features, many of which are irrelevant or redundant, resulting in a large search space for feature selection, i.e. to find small and discriminate feature subsets that improve the classification performance. Feature selection for multi-label classification is a many-objective optimisation problem with more than three main conflicting objectives: one of which is to reduce the number of selected features. There are many metrics for measuring multi-label classification performance, each of which can conflict with one another depending on the task. Hence, multi-label feature selection is a many-objective optimisation problem when three or more classification metrics and the number of selected features are optimised. In this paper, we propose to combine multi-label feature selection with evolutionary many-objective optimisation to address the above challenges and handle the trade-offs between multiple classification metrics and the number of selected features, using a decomposition-based algorithm. The results demonstrate that our proposed method is capable of finding discriminative and small feature subsets that can significantly improve the classification performances in comparison with other many-objective feature selection approaches.

References

[1]

J. Blank, K. Deb, Y. Dhebar, S. Bandaru, and H. Seada. In press. Generating Well-Spaced Points on a Unit Simplex for Evolutionary Many-Objective Optimization. IEEE Transactions on Evolutionary Computation (In press).

[2]

Matthew Boutell, Jiebo Luo, Xipeng Shen, and Christopher Brown. 2004. Learning multi-label scene classification. Pattern Recognition 37 (2004), 1757--1771.

[3]

Ali Braytee, Wei Liu, Daniel R Catchpoole, and Paul J Kennedy. 2017. Multi-label feature selection using correlation information. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1649--1656.

Digital Library

[4]

M. Dash and H. Liu. 1997. Feature selection for classification. Intelligent Data Analysis 1, 1 (1997), 131 -- 156.

[5]

Kalyanmoy Deb and Himanshu Jain. 2014. An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints. IEEE Transactions on Evolutionary Computation 18, 4 (2014), 577--601.

[6]

K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 2 (April 2002), 182--197.

Digital Library

[7]

Kaan Demir, Bach Hoai Nguyen, Bing Xue, and Mengjie Zhang. 2020. A Decomposition based Multi-objective Evolutionary Algorithm with ReliefF based Local Search and Solution Repair Mechanism for Feature Selection. In 2020 IEEE Congress on Evolutionary Computation (CEC). 1--8.

Digital Library

[8]

Kaan Demir, Bach Hoai Nguyen, Bing Xue, and Mengjie Zhang. 2021. Multi-objective Multi-label Feature Selection with an Aggregated Performance Metric and Dominance-based Initialisation. In 2021 IEEE Congress on Evolutionary Computation (CEC). 760--767.

Digital Library

[9]

Hongbin Dong, Jing Sun, Xiaohang Sun, and Rui Ding. 2020. A many-objective feature selection for multi-label classification. Knowledge-Based Systems 208 (2020).

[10]

Richard O Duda, Peter E Hart, and David G Stork. 2012. Pattern classification. John Wiley & Sons.

Digital Library

[11]

André Elisseeff and Jason Weston. 2001. A Kernel Method for Multi-Labelled Classification. In Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic (Vancouver, British Columbia, Canada) (NIPS'01). MIT Press, Cambridge, MA, USA, 681--687.

Digital Library

[12]

Juncheng Hu, Yonghao Li, Gaochao Xu, and Wanfu Gao. 2022. Dynamic subspace dual-graph regularized multi-label feature selection. Neurocomputing 467 (2022), 184--196.

Digital Library

[13]

Ling Jian, Jundong Li, Kai Shu, and Huan Liu. 2016. Multi-Label Informed Feature Selection. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (New York, New York, USA) (IJCAI'16). AAAI Press, 1627--1633.

Digital Library

[14]

Jing Yin, Tengfei Tao, and Jianhua Xu. 2015. A Multi-label feature selection algorithm based on multi-objective optimization. In 2015 International Joint Conference on Neural Networks (IJCNN). 1--7.

[15]

Gizem Karagoz, Adnan Yazici, Tansel Dökeroğlu, and Ahmet Cosar. 2021. A New Framework of Multi-objective Evolutionary Algorithms for Feature Selection and Multi-label Classification of Video Data. International Journal of Machine Learning and Cybernetics 12 (2021).

[16]

Jaesung Lee and Dae-Won Kim. 2015. Memetic feature selection algorithm for multi-label classification. Information Sciences 293 (2015), 80--96.

[17]

Xiaodong Li. 2003. A Non-dominated Sorting Particle Swarm Optimizer for Multiobjective Optimization. In Genetic and Evolutionary Computation --- GECCO 2003, Erick Cantú-Paz, James A. Foster, Kalyanmoy Deb, Lawrence David Davis, Rajkumar Roy, Una-May O'Reilly, Hans-Georg Beyer, Russell Standish, Graham Kendall, Stewart Wilson, Mark Harman, Joachim Wegener, Dipankar Dasgupta, Mitch A. Potter, Alan C. Schultz, Kathryn A. Dowsland, Natasha Jonoska, and Julian Miller (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 37--48.

[18]

Bach Nguyen, Bing Xue, Ivy Liu, Peter Andreae, and Mengjie Zhang. 2016. New mechanism for archive maintenance in PSO-based multi-objective feature selection. Soft Computing 20 (03 2016).

[19]

B. H. Nguyen, B. Xue, P. Andreae, H. Ishibuchi, and M. Zhang. 2019. Multiple Reference Points based Decomposition for Multi-objective Feature Selection in Classification: Static and Dynamic Mechanisms. IEEE Transactions on Evolutionary Computation (2019), 1--1.

[20]

Bach Hoai Nguyen, Bing Xue, and Mengjie Zhang. 2020. A survey on swarm intelligence approaches to feature selection in data mining. Swarm and Evolutionary Computation 54 (2020).

[21]

Hoai Bach Nguyen, Bing Xue, Hisao Ishibuchi, Peter Andreae, and Mengjie Zhang. 2017. Multiple Reference Points MOEA/D for Feature Selection. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (Berlin, Germany) (GECCO '17). Association for Computing Machinery, New York, NY, USA, 157--158.

Digital Library

[22]

Feiping Nie, Heng Huang, Xiao Cai, and Chris H. Ding. 2010. Efficient and Robust Feature Selection via Joint ℓ_2,1-Norms Minimization. In Advances in Neural Information Processing Systems 23. Curran Associates, Inc., 1813--1821.

Digital Library

[23]

Rafael B. Pereira, Alexandre Plastino, Bianca Zadrozny, and Luiz H.C. Merschmann. 2018. Correlation analysis of performance measures for multi-label classification. Information Processing Management 54, 3 (2018), 359--369.

Digital Library

[24]

Mitchell A Potter and Kenneth A De Jong. 2000. Cooperative coevolution: An architecture for evolving coadapted subcomponents. Evolutionary computation 8, 1 (2000), 1--29.

[25]

Konstantinos Sechidis, Grigorios Tsoumakas, and Ioannis Vlahavas. 2011. On the Stratification of Multi-label Data. In Machine Learning and Knowledge Discovery in Databases. Springer Berlin Heidelberg, 145--158.

Digital Library

[26]

Mohammad S Sorower. 2010. A literature survey on algorithms for multi-label learning. Oregon State University, Corvallis 18 (2010), 1--25.

[27]

Newton Spolaôr, Everton Alvares Cherman, Maria Carolina Monard, and Huei Diana Lee. 2013. A Comparison of Multi-label Feature Selection Methods using the Problem Transformation Approach. Electronic Notes in Theoretical Computer Science 292 (2013), 135 -- 151. Proceedings of the XXXVIII Latin American Conference in Informatics (CLEI).

Digital Library

[28]

Binh Tran, Bing Xue, Mengjie Zhang, and Su Nguyen. 2016. Investigation on particle swarm optimisation for feature selection on high-dimensional data: local search and selection bias. Connection Science (2016), 1--25.

[29]

Anupam Trivedi, Dipti Srinivasan, Krishnendu Sanyal, and Abhiroop Ghosh. 2017. A Survey of Multiobjective Evolutionary Algorithms Based on Decomposition. IEEE Transactions on Evolutionary Computation 21, 3 (2017), 440--462.

Digital Library

[30]

Konstantinos Trohidis, Grigorios Tsoumakas, George Kalliris, and I. Vlahavas. 2008. Multi-Label Classification of Music into Emotions. In ISMIR.

[31]

Alper Unler and Alper Murat. 2010. A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research 206, 3 (2010), 528 -- 539.

[32]

B. Xue, M. Zhang, and W. N. Browne. 2012. Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach. IEEE Transactions on Cybernetics 43, 6 (2012), 1656--1671.

[33]

Bing Xue, Mengjie Zhang, Will N Browne, and Xin Yao. 2015. A survey on evolutionary computation approaches to feature selection. IEEE Transactions on Evolutionary Computation 20, 4 (2015), 606--626.

Digital Library

[34]

B. Xue, M. Zhang, W. N. Browne, and X. Yao. 2015. A Survey on Evolutionary Computation Approaches to Feature Selection. IEEE Transactions on Evolutionary Computation 20, 4 (2015), 606--626.

Digital Library

[35]

Min-Ling Zhang and Zhi-Hua Zhou. 2007. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition 40, 7 (2007), 2038 -- 2048.

Digital Library

[36]

Q. Zhang and H. Li. 2007. MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition. IEEE Transactions on Evolutionary Computation 11, 6 (Dec 2007), 712--731.

Cited By

Seo WPark JLee SMoon AKim DLee J(2024)Memetic multilabel feature selection using pruned refinement processJournal of Big Data10.1186/s40537-024-00961-211:1Online publication date: 6-Aug-2024
https://doi.org/10.1186/s40537-024-00961-2
Saadatmand HAkbarzadeh-T M(2024)Many-Objective Jaccard-Based Evolutionary Feature Selection for High-Dimensional Imbalanced Data ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.341619646:12(8820-8835)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3416196
Hancer EXue BZhang M(2024)A Multimodal Multiobjective Evolutionary Algorithm for Filter Feature Selection in Multilabel ClassificationIEEE Transactions on Artificial Intelligence10.1109/TAI.2024.33805905:9(4428-4442)Online publication date: Sep-2024
https://doi.org/10.1109/TAI.2024.3380590

Index Terms

Co-operative Co-evolutionary Many-objective Embedded Multi-label Feature Selection with Decomposition-based PSO
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
  2. Machine learning
    1. Learning paradigms
2. Theory of computation
  1. Design and analysis of algorithms
    1. Mathematical optimization

Index terms have been assigned to the content through auto-classification.

Recommendations

Label correlations-based multi-label feature selection with label enhancement
Abstract
Feature selection, as an important pre-processing technique, can efficiently mitigate the issue of “the curse of dimensionality” by selecting discriminative features especially for multi-label learning, a discriminative feature subset can improve ...
Multi-Label Feature Selection using Correlation Information
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

High-dimensional multi-labeled data contain instances, where each instance is associated with a set of class labels and has a large number of noisy and irrelevant features. Feature selection has been shown to have great benefits in improving the ...
Distinguishing two types of labels for multi-label feature selection
Highlights
- We categorize labels into two groups: independent labels and dependent labels.
- ...
Abstract
Multi-label feature selection plays an important role in pattern recognition, which can improve multi-label classification performance. In traditional multi-label feature selection methods based on information theory, feature relevance ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '23: Proceedings of the Genetic and Evolutionary Computation Conference

July 2023

1667 pages

ISBN:9798400701191

DOI:10.1145/3583131

Chair:
Sara Silva,
Program Chair:
Luís Paquete

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '23

Sponsor:

SIGEVO

GECCO '23: Genetic and Evolutionary Computation Conference

July 15 - 19, 2023

Lisbon, Portugal

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
153
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)3

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Seo WPark JLee SMoon AKim DLee J(2024)Memetic multilabel feature selection using pruned refinement processJournal of Big Data10.1186/s40537-024-00961-211:1Online publication date: 6-Aug-2024
https://doi.org/10.1186/s40537-024-00961-2
Saadatmand HAkbarzadeh-T M(2024)Many-Objective Jaccard-Based Evolutionary Feature Selection for High-Dimensional Imbalanced Data ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.341619646:12(8820-8835)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3416196
Hancer EXue BZhang M(2024)A Multimodal Multiobjective Evolutionary Algorithm for Filter Feature Selection in Multilabel ClassificationIEEE Transactions on Artificial Intelligence10.1109/TAI.2024.33805905:9(4428-4442)Online publication date: Sep-2024
https://doi.org/10.1109/TAI.2024.3380590

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten