Article

Local Fisher discriminant analysis for supervised dimensionality reduction

Author:

Masashi SugiyamaAuthors Info & Claims

ICML '06: Proceedings of the 23rd international conference on Machine learning

Pages 905 - 912

https://doi.org/10.1145/1143844.1143958

Published: 25 June 2006 Publication History

Get Access

Abstract

Dimensionality reduction is one of the important preprocessing steps in high-dimensional data analysis. In this paper, we consider the supervised dimensionality reduction problem where samples are accompanied with class labels. Traditional Fisher discriminant analysis is a popular and powerful method for this purpose. However, it tends to give undesired results if samples in some class form several separate clusters, i.e., multimodal. In this paper, we propose a new dimensionality reduction method called local Fisher discriminant analysis (LFDA), which is a localized variant of Fisher discriminant analysis. LFDA takes local structure of the data into account so the multimodal data can be embedded appropriately. We also show that LFDA can be extended to non-linear dimensionality reduction scenarios by the kernel trick.

References

[1]

Belkin, M., & Niyogi, P. (2003). Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15, 1373--1396.

Digital Library

Google Scholar

[2]

Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7, 179--188.

Crossref

Google Scholar

[3]

Fukunaga, K. (1990). Introduction to statistical pattern recognition. Boston: Academic Press, Inc. Second edition.

Digital Library

Google Scholar

[4]

Globerson, A., & Roweis, S. (2006). Metric learning by collapsing classes. Advances in Neural Information Processing Systems 18 (pp. 451--458). Cambridge, MA: MIT Press.

Google Scholar

[5]

Goldberger, J., Roweis, S., Hinton, G., & Salakhutdinov, R. (2005). Neighbourhood components analysis. In L. K. Saul, Y. Weiss and L. Bottou (Eds.), Advances in neural information processing systems 17, 513--520. Cambridge, MA: MIT Press.

Google Scholar

[6]

He, X., & Niyogi, P. (2004). Locality preserving projections. In S. Thrun, L. Saul and B. Schöölkopf (Eds.), Advances in neural information processing systems 16. Cambridge, MA: MIT Press.

Google Scholar

[7]

Mika, S., Rätsch, G., Weston, J., Schölkopf, B., Smola, A., & Müüller, K.-R. (2003). Constructing descriptive and discriminative nonlinear features: Rayleigh coefficients in kernel feature spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 623--628.

Digital Library

Google Scholar

[8]

Schölkopf, B., & Smola, A. J. (2002). Learning with kernels. Cambridge, MA: MIT Press.

Google Scholar

[9]

Zelnik-Manor, L., & Perona, P. (2005). Self-tuning spectral clustering. In L. K. Saul, Y. Weiss and L. Bottou (Eds.), Advances in neural information processing systems 17, 1601--1608. Cambridge, MA: MIT Press.

Google Scholar

Cited By

View all

Qu LPei Y(2024)A Comprehensive Review on Discriminant Analysis for Addressing Challenges of Class-Level Limitations, Small Sample Size, and RobustnessProcesses10.3390/pr1207138212:7(1382)Online publication date: 2-Jul-2024
https://doi.org/10.3390/pr12071382
Wang SNie FWang ZWang RLi X(2024)Robust Principal Component Analysis via Joint Reconstruction and ProjectionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.321430735:5(7175-7189)Online publication date: May-2024
https://doi.org/10.1109/TNNLS.2022.3214307
Zhao DYin HZhou HCai LQin Y(2024)A Zero-Sample Fault Diagnosis Method Based on Transfer LearningIEEE Transactions on Industrial Informatics10.1109/TII.2024.340563420:10(11542-11552)Online publication date: Oct-2024
https://doi.org/10.1109/TII.2024.3405634
Show More Cited By

Index Terms

Local Fisher discriminant analysis for supervised dimensionality reduction
1. Computing methodologies
  1. Machine learning
2. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Statistical graphics

Recommendations

Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis

Reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high-dimensional data analysis. Fisher discriminant analysis (FDA) is a traditional technique for supervised dimensionality reduction, but it ...
Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis

Reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high-dimensional data analysis. Fisher discriminant analysis (FDA) is a traditional technique for supervised dimensionality reduction, but it ...
Local Weighted Semi-supervised Discriminant Analysis for Dimensionality Reduction
IHMSC '15: Proceedings of the 2015 7th International Conference on Intelligent Human-Machine Systems and Cybernetics - Volume 01

In this paper, we present a novel weighted version of semi-supervised discriminant analysis method by assigning weights to each labeled samples. The proposed within-class weight can detect the outliers and between-class weight can discover the support ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICML '06: Proceedings of the 23rd international conference on Machine learning

June 2006

1154 pages

ISBN:1595933832

DOI:10.1145/1143844

Program Chairs:
William Cohen,
Andrew Moore

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

ICML '06 Paper Acceptance Rate 140 of 548 submissions, 26%;

Overall Acceptance Rate 140 of 548 submissions, 26%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

262
Total Citations
View Citations
1,544
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)4

Reflects downloads up to 17 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Qu LPei Y(2024)A Comprehensive Review on Discriminant Analysis for Addressing Challenges of Class-Level Limitations, Small Sample Size, and RobustnessProcesses10.3390/pr1207138212:7(1382)Online publication date: 2-Jul-2024
https://doi.org/10.3390/pr12071382
Wang SNie FWang ZWang RLi X(2024)Robust Principal Component Analysis via Joint Reconstruction and ProjectionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.321430735:5(7175-7189)Online publication date: May-2024
https://doi.org/10.1109/TNNLS.2022.3214307
Zhao DYin HZhou HCai LQin Y(2024)A Zero-Sample Fault Diagnosis Method Based on Transfer LearningIEEE Transactions on Industrial Informatics10.1109/TII.2024.340563420:10(11542-11552)Online publication date: Oct-2024
https://doi.org/10.1109/TII.2024.3405634
Wang ZLi QNie FWang RWang FLi X(2024)Efficient Local Coherent Structure Learning via Self-Evolution Bipartite GraphIEEE Transactions on Cybernetics10.1109/TCYB.2023.332184354:8(4527-4538)Online publication date: Aug-2024
https://doi.org/10.1109/TCYB.2023.3321843
Rizk HHashima S(2024)RISense: 6G-Enhanced Human Activity Recognition System with RIS and Deep LDA2024 25th IEEE International Conference on Mobile Data Management (MDM)10.1109/MDM61037.2024.00035(119-128)Online publication date: 24-Jun-2024
https://doi.org/10.1109/MDM61037.2024.00035
Wang JYin HNie FLi X(2024)Adaptive and fuzzy locality discriminant analysis for dimensionality reductionPattern Recognition10.1016/j.patcog.2024.110382151(110382)Online publication date: Jul-2024
https://doi.org/10.1016/j.patcog.2024.110382
Yang XCao CZhou KPeng SWang ZLin LNie F(2024)A Novel Linear Discriminant Analysis Based on Alternate Ratio Sum MinimizationInformation Sciences10.1016/j.ins.2024.121444(121444)Online publication date: Sep-2024
https://doi.org/10.1016/j.ins.2024.121444
Gong JZou BXu CXu JYou X(2024)Hybrid Learning Based on Fisher Linear DiscriminantInformation Sciences10.1016/j.ins.2024.120465(120465)Online publication date: Mar-2024
https://doi.org/10.1016/j.ins.2024.120465
Pan JLe Capitaine H(2024)Metric learning with multi-relational dataInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02430-xOnline publication date: 17-Nov-2024
https://doi.org/10.1007/s13042-024-02430-x
Chen ZSun YHu DBian YWang SZhang XTao X(2024)Semi-supervised Kernel Fisher discriminant analysis based on exponential-adjusted geometric distanceNeural Computing and Applications10.1007/s00521-024-09768-x36:24(14825-14855)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s00521-024-09768-x
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis

Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis

Local Weighted Semi-supervised Discriminant Analysis for Dimensionality Reduction