tutorial

PCA for improving the performance of XCSR in classification of high-dimensional problems

Authors:

Mohammad Behdad,

Mohammed BennamounAuthors Info & Claims

GECCO '11: Proceedings of the 13th annual conference companion on Genetic and evolutionary computation

Pages 361 - 368

https://doi.org/10.1145/2001858.2002020

Published: 12 July 2011 Publication History

Abstract

XCSR is an accuracy-based learning classifier system (LCS) which can handle classification problems with real-value features. However, as the number of features increases, a high classification accuracy comes at the cost of more resources: larger population sizes and longer computational running times. In this research, we present a PCA-enhanced LCS, which uses principal component analysis (PCA) as a preprocessing step for XCSR, and examine how it performs on complex multi-dimensional real-world data. The experiments show that this technique, in addition to significantly reducing the computational resources and time requirements of XCSR, maintains its high accuracy and even occasionally improves it. In addition to that, it reduces the required population size needed by XCSR.

References

[1]

H. Abdi and L. J. Williams. Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics, 2(4):433--459, 2010.

Digital Library

[2]

M. Behdad, L. Barone, T. French, and M. Bennamoun. An investigation of real-valued accuracy-based learning classifier systems for electronic fraud detection. In GECCO (Companion), pages 1893--1900, 2010.

Digital Library

[3]

L. Bull. Applications of Learning Classifier Systems. Springer, 2004.

Digital Library

[4]

M. V. Butz and S. W. Wilson. An algorithmic descrtipion of xcs, 2000. Technical report 2000017, Illinois Genetics Algorithms Laboratory.

[5]

F. Ferrandi, P. Lanzi, D. Sciuto, and M. Tanelli. System-level metrics for hardware/software architectural mapping. In Proceedings of the Second IEEE International Workshop on Electronic Design, Test and Applications, DELTA '04, pages 231--236, Washington, DC, USA, 2004. IEEE Computer Society.

Digital Library

[6]

S. Hettich and S. D. Bay. The UCI KDD archive {http://kdd.ics.uci.edu}. Irvine, CA: University of California, Department of Information and Computer Science, 1999.

[7]

J. H. Holland. Adaptation. Progress in Theoretical Biology, 4:263--293, 1976.

[8]

T. Howley, M. G. Madden, M.-L. O'Connell, and A. G. Ryder. The effect of principal component analysis on machine learning accuracy with high-dimensional spectral data. Knowledge-Based Systems, 19(5):363--370, 2006.

Digital Library

[9]

M. Kirley and M.Abedini. CoXCS: a coevolutionary learning classifier based on feature space partitioning. In Lecture Notes in Computer Science, volume 5866, pages 360--369, 2009.

Digital Library

[10]

J. Marín-Blázquez and G. Martínez Pérez. Intrusion detection using a linguistic hedged fuzzy-XCS classifier system. Soft Computing - A Fusion of Foundations, Methodologies and Applications, 13:273--290, 2009.

Digital Library

[11]

B. Moore. Principal component analysis in linear systems: Controllability, observability, and model reduction. IEEE transactions on automatic control, 26:17--32, 1981.

[12]

A. J. O'Toole, H. Abdi, K. A. Deffenbacher, and D. Valentin. Low-dimensional representation of faces in higher dimensions of the face space. Journal of the Optical Society of America, 10(3):405--411, Mar 1993.

[13]

B. Ravichandran, A. Gandhe, R. Smith, and R. Mehra. Robust automatic target recognition using learning classifier systems. Information Fusion, 8(3):252--265, 2007.

Digital Library

[14]

K. Shafi, T. Kovacs, H. A. Abbass, and W. Zhu. Intrusion detection with evolutionary learning classifier systems. Natural Computing, 8(1):3--27, 2009.

Digital Library

[15]

O. Sigaud and S. W. Wilson. Learning classifier systems: a survey. Soft Computing - A Fusion of Foundations, Methodologies and Applications, 11(11):1065--1078, 2007.

Digital Library

[16]

P. Stalph, X. Llora, D. Goldberg, and M. V. Butz. Resource management and scalability of the XCSF learning classifier system. Theoretical Computer Science, 2010.

Digital Library

[17]

C. Stone and L. Bull. For real! XCS with continuous-valued inputs. Evolutionary Computation, 11(3):299--336, 2003.

Digital Library

[18]

K. Tamee, P. Rojanavasu, S. Udomthanapong, and O. Pinngern. Using self-organizing maps with learning classifier system for intrusion detection. In PRICAI '08, pages 1071--1076. Springer, 2008.

Digital Library

[19]

A. N. Toosi and M. Kahani. A new approach to intrusion detection based on an evolutionary soft computing model using neuro-fuzzy classifiers. Computer Communications, 30(10):2201--2212, 2007.

Digital Library

[20]

S. Wilson. Function approximation with a classifier system. In Proceedings of GECCO'01, pages 974--981, 2001.

[21]

S. W. Wilson. Classifier fitness based on accuracy. Evolutionary Computation, 3(2):149--175, 1995.

Digital Library

[22]

S. W. Wilson. Get real! XCS with continuous-valued inputs. In Learning Classifier Systems, From Foundations to Applications, pages 209--222. Springer, 2000.

Digital Library

[23]

X. Xu and X. Wang. An adaptive network intrusion detection method based on pca and support vector machines. In X. Li, S. Wang, and Z. Dong, editors, Advanced Data Mining and Applications, volume 3584 of Lecture Notes in Computer Science, pages 731--731. Springer, 2005.

Cited By

Debie EShafi K(2019)Implications of the curse of dimensionality for supervised learning classifier systemsPattern Analysis & Applications10.1007/s10044-017-0649-022:2(519-536)Online publication date: 1-May-2019
https://dl.acm.org/doi/10.1007/s10044-017-0649-0
Yin YTian YWang WDuan FWu ZZhou M(2013)Classification Based on LBP and SVM for Human Embryo Microscope ImagesHuman-Computer Interaction. Towards Intelligent and Implicit Interaction10.1007/978-3-642-39342-6_31(280-288)Online publication date: 2013
https://doi.org/10.1007/978-3-642-39342-6_31

Index Terms

PCA for improving the performance of XCSR in classification of high-dimensional problems
1. Computing methodologies
  1. Machine learning

Recommendations

On the problems of using learning classifier systems for fraud detection
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

Fraud detection problems have some uniquely challenging properties which make them difficult. In this paper, we investigate the fraud detection problem by describing the common properties of electronic fraud and examining how learning classifier systems ...
XCSR with computed continuous action
AI'12: Proceedings of the 25th Australasian joint conference on Advances in Artificial Intelligence

Wilson extended XCS with interval based conditions to XCSR to handle real valued inputs. However, the possible actions must always be determined in advance. Yet domains such as robot control require numerical actions, so that neither XCS nor XCSR with ...
Evaluating indirect and direct classification techniques for network intrusion detection

The network intrusion detection domain has seen increased research that exploit data mining and machine learning techniques and principles. Typically, multi-category classification models are built to classify network traffic instances either as normal ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '11: Proceedings of the 13th annual conference companion on Genetic and evolutionary computation

July 2011

1548 pages

ISBN:9781450306904

DOI:10.1145/2001858

Editor:
Natalio Krasnogor
University of Nottingham, UK
,
General Chair:
Pier Luca Lanzi
Politecnico di Milano, Italy

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

GECCO '11

Sponsor:

SIGEVO

GECCO '11: Genetic and Evolutionary Computation Conference

July 12 - 16, 2011

Dublin, Ireland

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
136
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Debie EShafi K(2019)Implications of the curse of dimensionality for supervised learning classifier systemsPattern Analysis & Applications10.1007/s10044-017-0649-022:2(519-536)Online publication date: 1-May-2019
https://dl.acm.org/doi/10.1007/s10044-017-0649-0
Yin YTian YWang WDuan FWu ZZhou M(2013)Classification Based on LBP and SVM for Human Embryo Microscope ImagesHuman-Computer Interaction. Towards Intelligent and Implicit Interaction10.1007/978-3-642-39342-6_31(280-288)Online publication date: 2013
https://doi.org/10.1007/978-3-642-39342-6_31

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten