Article

Two-stage learning kernel algorithms

Authors:

Corinna Cortes,

Afshin RostamizadehAuthors Info & Claims

ICML'10: Proceedings of the 27th International Conference on International Conference on Machine Learning

Pages 239 - 246

Published: 21 June 2010 Publication History

Abstract

This paper examines two-stage techniques for learning kernels based on a notion of alignment. It presents a number of novel theoretical, algorithmic, and empirical results for alignment-based techniques. Our results build on previous work by Cristianini et al. (2001), but we adopt a different definition of kernel alignment and significantly extend that work in several directions: we give a novel and simple concentration bound for alignment between kernel matrices; show the existence of good predictors for kernels with high alignment, both for classification and for regression; give algorithms for learning a maximum alignment kernel by showing that the problem can be reduced to a simple QP; and report the results of extensive experiments with this alignment-based method in classification and regression tasks, which show an improvement both over the uniform combination of kernels and over other state-of-the-art learning kernel methods.

References

[1]

Bach, F. Exploring large feature spaces with hierarchical multiple kernel learning. In NIPS, 2008.

Digital Library

[2]

Balcan, M.-F. and Blum, A. On a theory of learning with similarity functions. In ICML, pp. 73-80,2006.

Digital Library

[3]

Cortes, C. Invited talk: Can learning kernels help performance? In ICML 2009, pp. 161,2009.

Digital Library

[4]

Cortes, C. and Vapnik, Vladimir. Support-Vector Networks. Machine Learning, 20(3), 1995.

Digital Library

[5]

Cortes, C., Mohri, A., and Rostamizadeh, A. regularization for learning kernels. In UAI, 2009a.

Digital Library

[6]

Cortes, C, Mohri, A., and Rostamizadeh, A. Learning nonlinear combinations of kernels. In NIPS, 2009b.

Digital Library

[7]

Cortes, C, Mohri, A., and Rostamizadeh, A. Generalization bounds for learning kernels. In ICML '10,2010.

[8]

Cristianini, N., Shawe-Taylor, J., Elisseeff, A., and Kandola, J. S. On kernel-target alignment. In NIPS, 2001.

Digital Library

[9]

Cristianini, N., Kandola, J. S., Elisseeff, A., and Shawe-Taylor, J. On kernel target alignment. www.support-vector.net/papers/alignment_JMLR.ps, unpublish., 2002.

[10]

Kandola, J. S., Shawe-Taylor, J., and Cristianini, N. On the extensions of kernel alignment. tech. report 120, Dept. of Computer Science, Univ. of London, UK, 2002a.

[11]

Kandola, J. S., Shawe-Taylor, J., and Cristianini, N. Optimizing kernel alignment over combinations of kernels. tech. report 121, Univ. of London, UK, 2002b.

[12]

Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L. El, and Jordan, M. Learning the kernel matrix with semidefinite programming. JMLR, 5, 2004.

Digital Library

[13]

Meila, M. Data centering in feature space. AISTATS, 2003.

[14]

Mcchelli, C. and Pontil, M. Learning the kernel function via regularization. JMLR, 6, 2005.

Digital Library

[15]

Ong, C. S., Smola, A., and Williamson, R. Learning the kernel with hyperkernels. JMLR, 6, 2005.

Digital Library

[16]

Pothin, J.-B. and Richard, C. Optimizing kernel alignment by data translation in feature space. In ICASSP, 2008.

[17]

Shawe-Taylor, J. and Cristianini, N. Kernel Methods for Pattern Analysis. Cambridge Univ. Press, 2004.

Digital Library

[18]

Srebro, N. and Ben-David, S. Learning bounds for support vector machines with learned kernels. In COLT, 2006.

Digital Library

Cited By

Huusari RKadri H(2019)Entangled kernelsProceedings of the 28th International Joint Conference on Artificial Intelligence10.5555/3367243.3367398(2578-2584)Online publication date: 10-Aug-2019
https://dl.acm.org/doi/10.5555/3367243.3367398
Zhu CWang PMiao DZhou RBalinsky ATan ATarng W(2019)Rank-consistency-based multi-view learning with UniversumProceedings of the 1st International Conference on Advanced Information Science and System10.1145/3373477.3373700(1-6)Online publication date: 15-Nov-2019
https://dl.acm.org/doi/10.1145/3373477.3373700
Zhang LZhen XShao LSong J(2019)Learning Match Kernels on Grassmann Manifolds for Action RecognitionIEEE Transactions on Image Processing10.1109/TIP.2018.286668828:1(205-215)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1109/TIP.2018.2866688
Show More Cited By

Index Terms

Two-stage learning kernel algorithms
1. Computing methodologies
  1. Machine learning
2. Theory of computation
  1. Design and analysis of algorithms

Recommendations

Two-stage multiple kernel learning with multiclass kernel polarization

The success of kernel methods is very much dependent on the choice of kernels. Multiple kernel learning (MKL) aims at learning a combination of different kernels in order to better match the underlying problem instead of using a single fixed kernel. In ...
Multiple Kernel Learning Algorithms

In recent years, several methods have been proposed to combine multiple kernels instead of using a single one. These different kernels may correspond to using different notions of similarity or may be using information coming from multiple sources (...
A binary classification framework for two-stage multiple kernel learning
ICML'12: Proceedings of the 29th International Coference on International Conference on Machine Learning

With the advent of kernel methods, automating the task of specifying a suitable kernel has become increasingly important. In this context, the Multiple Kernel Learning (MKL) problem of finding a combination of prespecified base kernels that is suitable ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

ICML'10: Proceedings of the 27th International Conference on International Conference on Machine Learning

June 2010

1262 pages

ISBN:9781605589077

Sponsors

NSF: National Science Foundation
Xerox
Microsoft Research: Microsoft Research
Yahoo!
IBM: IBM

Publisher

Omnipress

Madison, WI, United States

Publication History

Published: 21 June 2010

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huusari RKadri H(2019)Entangled kernelsProceedings of the 28th International Joint Conference on Artificial Intelligence10.5555/3367243.3367398(2578-2584)Online publication date: 10-Aug-2019
https://dl.acm.org/doi/10.5555/3367243.3367398
Zhu CWang PMiao DZhou RBalinsky ATan ATarng W(2019)Rank-consistency-based multi-view learning with UniversumProceedings of the 1st International Conference on Advanced Information Science and System10.1145/3373477.3373700(1-6)Online publication date: 15-Nov-2019
https://dl.acm.org/doi/10.1145/3373477.3373700
Zhang LZhen XShao LSong J(2019)Learning Match Kernels on Grassmann Manifolds for Action RecognitionIEEE Transactions on Image Processing10.1109/TIP.2018.286668828:1(205-215)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1109/TIP.2018.2866688
Chen LChen DWang H(2019)Alignment Based Kernel Selection for Multi-Label LearningNeural Processing Letters10.1007/s11063-018-9863-z49:3(1157-1177)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s11063-018-9863-z
Nguyen KLe TNguyen TPhung DWebb GGuo YFarooq F(2018)Robust Bayesian Kernel Machine via Stein Variational Gradient Descent for Big DataProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3219819.3220015(2003-2011)Online publication date: 19-Jul-2018
https://dl.acm.org/doi/10.1145/3219819.3220015
Liu HZheng QLi ZQin TZhu L(2018)An efficient multi-feature SVM solver for complex event detectionMultimedia Tools and Applications10.1007/s11042-017-5166-z77:3(3509-3532)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1007/s11042-017-5166-z
Liu JCoull B(2017)Robust hypothesis test for nonlinear effect with Gaussian processesProceedings of the 31st International Conference on Neural Information Processing Systems10.5555/3294771.3294847(795-803)Online publication date: 4-Dec-2017
https://dl.acm.org/doi/10.5555/3294771.3294847
Li JLiu YLin HYue YWang W(2017)Efficient kernel selection via spectral analysisProceedings of the 26th International Joint Conference on Artificial Intelligence10.5555/3172077.3172183(2124-2130)Online publication date: 19-Aug-2017
https://dl.acm.org/doi/10.5555/3172077.3172183
(2017)Low-rank decomposition meets kernel learningArtificial Intelligence10.1016/j.artint.2017.05.001250:C(1-15)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1016/j.artint.2017.05.001
Chen NHoi SLi SXiao XBennett PJosifovski VNeville JRadlinski F(2016)Mobile App TaggingProceedings of the Ninth ACM International Conference on Web Search and Data Mining10.1145/2835776.2835812(63-72)Online publication date: 8-Feb-2016
https://dl.acm.org/doi/10.1145/2835776.2835812
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents