Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Distance metric learning with local multiple kernel embedding

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Distance metric learning aims to learn a data-dependent similarity measure, which is widely employed in machine learning. Recently, metric learning algorithms that incorporate multiple kernel learning have shown promising outcomes for classification tasks. However, the multiple kernel learning part of the existing metric learning with multiple kernel just uses a linear combination form of different kernel functions, where each kernel shares the same weight in the entire input space, thus the potential local structure of samples located at different locations in the input space is ignored. To address the aforementioned issues, in this paper, we propose a distance metric learning approach with local multiple kernel embedding (DMLLMK) for small datasets. The weight of each kernel function in DMLLMK is assigned locally, so that there are many different values of weight in each kernel space. This local weight method enables metric learning to capture more information in the data. Our proposed DMLLMK adjusts the kernel weight by using a gating function; moreover, the kernel weight locally depends on the input data. The metric of metric learning and the parameters of the gating function are optimized simultaneously by an alternating learning process. The DMLLMK makes metric learning applicable to small datasets by constructing constraints on the set of similar pairs and dissimilar pairs such that some data are reused, and they produce different constraints on the model. In addition, regularization techniques are used to keep DMLLMK more conservative and prevent overfitting on small data. The experimental results of our proposed method when compared with other metric learning methods on the benchmark dataset show that our proposed DMLLMK is effective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Al-Obaidi SAR, Zabihzadeh D, Hajiabadi H (2020) Robust metric learning based on the rescaled hinge loss. Int J Mach Learn Cybern 11:2515–2528

    Article  Google Scholar 

  2. Cai X, Wang C, Xiao B, Chen X, Zhou J (2012) Deep nonlinear metric learning with independent subspace analysis for face verification. In: Proceedings of the 20th ACM international conference on multimedia, pp 749–752

  3. Cao Q, Ying Y, Li P (2013) Similarity metric learning for face recognition. In: Proceedings of the IEEE international conference on computer vision, pp 2408–2415

  4. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    Article  MATH  Google Scholar 

  5. Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27

    Article  MATH  Google Scholar 

  6. Davis JV, Kulis B, Jain P, Sra S, Dhillon IS (2007) Information-theoretic metric learning. In: Proceedings of the 24th international conference on machine learning, ACM, pp 209–216

  7. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MATH  Google Scholar 

  8. Geng B, Tao D, Xu C (2011) Daml: domain adaptation metric learning. IEEE Trans Image Process 20(10):2980–2989

    Article  MATH  Google Scholar 

  9. Gönen M, Alpaydın E (2013) Localized algorithms for multiple kernel learning. Pattern Recogn 46(3):795–807

    Article  MATH  Google Scholar 

  10. Han Y, Yang K, Ma Y, Liu G (2014) Localized multiple kernel learning via sample-wise alternating optimization. IEEE Trans Cybern 44(1):137–148

    Article  Google Scholar 

  11. Han Y, Yang K, Yang Y, Ma Y (2018) Localized multiple kernel learning with dynamical clustering and matrix regularization. IEEE Trans Neural Netw Learn Syst 29(2):486–499

    Article  Google Scholar 

  12. He Y, Chen W, Chen Y, Mao Y (2013) Kernel density metric learning. In: 2013 IEEE 13th international conference on data mining, IEEE, pp 271–280

  13. He Y, Mao Y, Chen W, Chen Y (2015) Nonlinear metric learning with kernel density estimation. IEEE Trans Knowl Data Eng 27(6):1602–1614

    Article  Google Scholar 

  14. Hekler EB, Klasnja P, Chevance G, Golaszewski NM, Lewis D, Sim I (2019) Why we need a small data paradigm. BMC Med 17(1):1–9

    Article  Google Scholar 

  15. Hu M, Tsang ECC, Guo Y, Chen D, Xu W (2021) A novel approach to attribute reduction based on weighted neighborhood rough sets. Knowl-Based Syst 220:106908

  16. Hu M, Tsang ECC, Guo Y, Xu W (2021) Fast and robust attribute reduction based on the separability in fuzzy decision systems. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2020.3040803

    Article  Google Scholar 

  17. Huang K, Ying Y, Campbell C (2009) Gsml: A unified framework for sparse metric learning. In: 2009 Ninth IEEE international conference on data mining, IEEE, pp 189–198

  18. Jain AK (2010) Data clustering: 50 years beyond k-means. Pattern Recogn Lett 31(8):651–666

    Article  Google Scholar 

  19. Jain P, Kulis B, Dhillon IS (2010) Inductive regularized learning of kernel functions. Neural Inf Process Syst pp 946–954

  20. Jiang S, Xu Y, Wang T, Yang H, Qiu S, Yu H, Song H (2019) Multi-label metric transfer learning jointly considering instance space and label space distribution divergence. IEEE Access 7:10362–10373

    Article  Google Scholar 

  21. Kedem D, Tyree S, Sha F, Lanckriet GR, Weinberger KQ (2012) Non-linear metric learning. In: Neural information processing systems, Citeseer, pp 2573–2581

  22. Kloft M, Brefeld U, Sonnenburg S, Zien A (2011) Lp-norm multiple kernel learning. J Mach Learn Res 12:953–997

    MATH  Google Scholar 

  23. Li X, Shen C, Shi Q, Dick A, Van Den Hengel A (2012) Non-sparse linear representations for visual tracking with online reservoir metric learning. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp 1760–1767

  24. Lim D, Lanckriet G (2014) Efficient learning of mahalanobis metrics for ranking. In: International conference on machine learning, PMLR, pp 1980–1988

  25. Lu J, Wang G, Deng W, Jia K (2015) Reconstruction-based metric learning for unconstrained face verification. IEEE Trans Inf Forensics Secur 10(1):79–89

    Article  Google Scholar 

  26. Lu X, Wang Y, Zhou X, Ling Z (2016) A method for metric learning with multiple-kernel embedding. Neural Process Lett 43(3):905–921

    Article  Google Scholar 

  27. Mensink T, Verbeek J, Perronnin F, Csurka G (2012) Metric learning for large scale image classification: Generalizing to new classes at near-zero cost. In: European Conference on Computer Vision, Springer, New York, pp 488–501

  28. Parameswaran S, Weinberger KQ (2010) Large margin multi-task metric learning. Neural Inf Process Syst 23:1867–1875

    Google Scholar 

  29. Rakotomamonjy A, Bach F, Canu S, Grandvalet Y (2007) More efficiency in multiple kernel learning. In: Proceedings of the 24th international conference on machine learning, ACM, pp 775–782

  30. Rakotomamonjy A, Bach F, Canu S, Grandvalet Y (2008) Simplemkl. J Mach Learn Res 9:2491–2521

    MATH  Google Scholar 

  31. Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. arXiv preprint arXiv:1703.05175

  32. Sung F, Yang Y, Zhang L, Xiang T, Torr PH, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1199–1208

  33. Vinyals O, Blundell C, Lillicrap T, Wierstra D et al (2016) Matching networks for one shot learning. Adv Neural Inf Process Syst 29:3630–3638

    Google Scholar 

  34. Wang F, Zuo W, Zhang L, Meng D, Zhang D (2015) A kernel classification framework for metric learning. IEEE Trans Neural Netw Learn Syst 26(9):1950–1962

    Article  Google Scholar 

  35. Wang J, Do H, Woznica A, Kalousis A (2011) Metric learning with multiple kernels. Neural Inf Process Syst 24:1170–1178

    Google Scholar 

  36. Wang J, Deng Z, Choi KS, Jiang Y, Luo X, Chung FL, Wang S (2016) Distance metric learning for soft subspace clustering in composite kernel space. Pattern Recogn 52:113–134

    Article  MATH  Google Scholar 

  37. Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244

    MATH  Google Scholar 

  38. Xing EP, Ng AY, Jordan MI, Russell S (2002) Distance metric learning with application to clustering with side-information. In: Neural information processing systems, Citeseer, pp 521–528

  39. Xu S, Yang M, Zhou Y, Zheng R, Liu W, He J (2020) Partial label metric learning by collapsing classes. Int J Mach Learn Cybern 11:2453–2460

    Article  Google Scholar 

  40. Xu X, Tsang IW, Xu D (2013) Soft margin multiple kernel learning. IEEE Trans Neural Netw Learn Syst 24(5):749–761

    Article  Google Scholar 

  41. Xu Y, Min H, Song H, Wu Q (2016) Multi-instance multi-label distance metric learning for genome-wide protein function prediction. Comput Biol Chem 63:30–40

    Article  Google Scholar 

  42. Xu Z, Weinberger KQ, Chapelle O (2012) Distance metric learning for kernel machines. arXiv preprint arXiv:1208.3422

  43. Xun Y, Meng W, Luming Z, Tao D (2016) Empirical risk minimization for metric learning using privileged information. In: IJCAI international joint conference on artificial intelligence, pp 2266–2272

  44. Ying Y, Li P (2012) Distance metric learning with eigenvalue optimization. J Mach Learn Res 13(1):1–26

    MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by the Macao Science and Technology Development Funds (0019/2019/A1 and 0075/2019/A2), National Natural Science Foundation of China (No. 62072024, No. 62106148), Beijing Municipal Education Commission Science and Technology General Project (KM202110016001), Scientific Research Foundation of Beijing University of Civil Engineering and Architecture (No. KYJJ2017017, Y19-19) and Opening Fund of Hebei Key Laboratory of Machine Learning and Computational Intelligence (2019-2021-A).

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Macau University of Science and Technology, Taipa, Macao, China

    Qingshuo Zhang, Eric C. C. Tsang & Meng Hu

  2. School of Science, Beijing University of Civil Engineering and Architecture, Beijing, 100044, China

    Qiang He

Authors
  1. Qingshuo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric C. C. Tsang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiang He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Meng Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eric C. C. Tsang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Q., Tsang, E.C.C., He, Q. et al. Distance metric learning with local multiple kernel embedding. Int. J. Mach. Learn. & Cyber. 14, 79–92 (2023). https://doi.org/10.1007/s13042-021-01487-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01487-2

Keywords

Search

Navigation