Article

A subspace kernel for nonlinear feature extraction

Authors:

Mingrui Wu,

Jason FarquharAuthors Info & Claims

IJCAI'07: Proceedings of the 20th international joint conference on Artifical intelligence

Pages 1125 - 1130

Published: 06 January 2007 Publication History

Abstract

Kernel based nonlinear Feature Extraction (KFE) or dimensionality reduction is a widely used preprocessing step in pattern classification and data mining tasks. Given a positive definite kernel function, it is well known that the input data are implicitly mapped to a feature space with usually very high dimensionality. The goal of KFE is to find a low dimensional subspace of this feature space, which retains most of the information needed for classification or data analysis. In this paper, we propose a subspace kernel based on which the feature extraction problem is transformed to a kernel parameter learning problem. The key observation is that when projecting data into a low dimensional subspace of the feature space, the parameters that are used for describing this subspace can be regarded as the parameters of the kernel function between the projected data. Therefore current kernel parameter learning methods can be adapted to optimize this parameterized kernel function. Experimental results are provided to validate the effectiveness of the proposed approach.

References

[1]

{Chapelle et al., 2002} O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3):131-159, 2002.

Digital Library

Google Scholar

[2]

{Cristianini et al., 2002} N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola. On kernel-target alignment. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, Cambridge, MA, 2002. MIT Press.

Google Scholar

[3]

{Kim et al., 2005} H. Kim, P. Howland, and H. Park. Dimension reduction in text classification with support vector machines. Journal of Machine Learning Research, 6:37-53, 2005.

Digital Library

Google Scholar

[4]

{Mika et al., 2001} S. Mika, G. Rätsch, and K. R. Müller. A mathematical programming approach to the kernel fisher algorithm. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, Cambridge, MA, 2001. The MIT Press.

Google Scholar

[5]

{Park and Park, 2004} C. H. Park and H. Park. Nonlinear feature extraction based on centroids and kernel functions. Pattern Recognition, 37:801-810, 2004.

Crossref

Google Scholar

[6]

{Rosipal and Trejo, 2001} R. Rosipal and L. J. Trejo. Kernel partial least squares regression in reproducing kernel hilbert space. Journal of Machine Learning Research, 2:97-123, 2001.

Digital Library

Google Scholar

[7]

{Schölkopf and Smola, 2002} B. Schölkopf and A. J. Smola. Learning with Kernels. The MIT Press, Cambridge, MA, 2002.

Google Scholar

[8]

{Vapnik, 1995} V. Vapnik. The Nature of Statistical Learning Theory. Springer Verlag, New York, 1995.

Digital Library

Google Scholar

[9]

{Weston et al., 2000} J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik. Feature selection for svms. In Advances in Neural Information Processing Systems 12, Cambridge, MA, 2000. MIT Press.

Google Scholar

[10]

{Wold, 1975} H. Wold. Soft modeling by latent variables; the nonlinear iterative partial least squares approach. In J. Gani, editor, Perspectives in Probability and Statistics, pages 520-540, London, 1975. Academic Press.

Google Scholar

[11]

{Wu et al., 2005} M.Wu, B. Schölkopf, and G. Bakir. Building sparse large margin classifiers. In L. D. Raedt and S. Wrobel, editors, Proc. 22th International Conference on Machine Learning, pages 1001-1008. ACM, 2005.

Digital Library

Google Scholar

[12]

{Xiong et al., 2005} T. Xiong, J. Ye, Q. Li, R. Janardan, and V. Cherkassky. Efficient kernel discriminant analysis via QR decomposition. In L. K. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17, pages 1529-1536. MIT Press, Cambridge, MA, 2005.

Google Scholar

Cited By

View all

Chen JJi SCeran BLi QWu MYe JLi YLiu BSarawagi S(2008)Learning subspace kernels for classificationProceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining10.1145/1401890.1401908(106-114)Online publication date: 24-Aug-2008
https://dl.acm.org/doi/10.1145/1401890.1401908
Nguyen CHo T(2008)An efficient kernel matrix evaluation measurePattern Recognition10.1016/j.patcog.2008.04.00541:11(3366-3372)Online publication date: 1-Nov-2008
https://dl.acm.org/doi/10.1016/j.patcog.2008.04.005

A subspace kernel for nonlinear feature extraction
1. Computing methodologies
  1. Machine learning
    1. Machine learning algorithms
2. Information systems
  1. Information systems applications

Recommendations

Bilinear Analysis for Kernel Selection and Nonlinear Feature Extraction

This paper presents a unified criterion, Fisher kernel criterion (FKC), for feature extraction and recognition. This new criterion is intended to extract the most discriminant features in different nonlinear spaces, and then, fuse these features under a ...
Kernel Self-optimized Locality Preserving Discriminant Analysis for feature extraction and recognition

We propose Kernel Self-optimized Locality Preserving Discriminant Analysis (KSLPDA) for feature extraction and recognition. The procedure of KSLPDA is divided into two stages, i.e., one is to solve the optimal expansion of the data-dependent kernel with ...
Feature space locality constraint for kernel based nonlinear discriminant analysis

Subspace learning is an important approach in pattern recognition. Nonlinear discriminant analysis (NDA), due to its capability of describing nonlinear manifold structure of samples, is considered to be more powerful to undertake classification tasks in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IJCAI'07: Proceedings of the 20th international joint conference on Artifical intelligence

January 2007

2953 pages

Editors:
Rajeev Sangal
International Institute of Information Technology, Hyderabad
,
Harish Mehta
Onward Technologies Limited
,
R. K. Bagga
International Institute of Information Technology, Hyderabad

Publisher

Morgan Kaufmann Publishers Inc.

San Francisco, CA, United States

Publication History

Published: 06 January 2007

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Chen JJi SCeran BLi QWu MYe JLi YLiu BSarawagi S(2008)Learning subspace kernels for classificationProceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining10.1145/1401890.1401908(106-114)Online publication date: 24-Aug-2008
https://dl.acm.org/doi/10.1145/1401890.1401908
Nguyen CHo T(2008)An efficient kernel matrix evaluation measurePattern Recognition10.1016/j.patcog.2008.04.00541:11(3366-3372)Online publication date: 1-Nov-2008
https://dl.acm.org/doi/10.1016/j.patcog.2008.04.005

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Bilinear Analysis for Kernel Selection and Nonlinear Feature Extraction

Kernel Self-optimized Locality Preserving Discriminant Analysis for feature extraction and recognition

Feature space locality constraint for kernel based nonlinear discriminant analysis