Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Regularized partial least squares for multi-label learning

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In reality, data objects often belong to several different categories simultaneously, which are semantically correlated to each other. Multi-label learning can handle and extract useful information from such kind of data effectively. Since it has a great variety of potential applications, multi-label learning has attracted widespread attention from many domains. However, two major challenges still remain for multi-label learning: high dimensionality and correlations of data. In this paper, we address the problems by using the technique of partial least squares (PLS) and propose a new multi-label learning method called rPLSML (regularized Partial Least Squares for Multi-label Learning). Specifically, we exploit PLS discriminant analysis to identify a latent and common space from the variable and label spaces of data, and then construct a learning model based on the latent space. To tackle the multi-collinearity problem raised from the high dimensionality, a \(\ell _2\)-norm penalty is further exerted on the optimization problem. The experimental results on public data sets show that rPLSML has better performance than the state-of-the-art multi-label learning algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://www.computationalmedicine.org/challenge/.

  2. http://mlkd.csd.auth.gr/multilabel.html.

  3. http://www.public.asu.edu/~jye02/Software/CCA/index.html.

References

  1. Boutell MR, Luo J, Shen X, Brown C (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771

    Article  Google Scholar 

  2. Cheng W, Hüllermeier E (2009) Combining instance-based learning and logistic regression for multilabel classification. Mach Learn 76(2–3):211–225

    Article  Google Scholar 

  3. Chu D, Liao LZ, Ng MK, Zhang X (2013) Sparse canonical correlation analysis: new formulation and algorithm. IEEE Trans Pattern Anal Mach Intell 35(12):3050–3065

    Article  Google Scholar 

  4. Gibaja E, Ventura S (2015) A tutorial on multilabel learning. ACM Computing Surveys 47(3):Article No.52

  5. Huang SJ, Zhou ZH (2012) Multi-label learning by exploiting label correlations locally. In: Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI’12), Toronto, Canada, pp 949–955

  6. Huang SJ, Yu Y, Zhou ZH (2012) Multi-label hypothesis reuse. In: Proceedings of the 18th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2012), Beijing, China, pp 525–533

  7. Ji S, Tang L, Yu S, Ye J (2010) A shared-subspace learning framework for multi-label classification. ACM Trans on Knowl Disco from Data 4(2):Article No.8

  8. Krishnan A, Williams LJ, McIntosh AR, Abdi H (2011) Patial least squares (pls) methods for neuroimaging: a tutorial and review. NeuroImage 56(1):455–475

    Article  Google Scholar 

  9. Lee D, Lee Y, Pawitan Y, Lee W (2013) Sparse partial least-squares regression for high-throughput survival data analysis. Stat Med 32(30):5340–5352

    Article  MathSciNet  Google Scholar 

  10. Lee J, Kim DW (2015) Mutual information-based multi-label feature selection using interaction information. Expert Syst Appl 42(4):2013–2025

    Article  Google Scholar 

  11. Li L, Liu H, Ma Z, Mo Y, Duan Z, Zhou J, Zhao J (2014) Multi-label feature selection via information gain. In: Proceedings of ADMA’14, pp 345–355

  12. Liu H, Wu X, Zhang S (2014a) A new supervised feature selection method for pattern classification. Comput Intell 30(2):342–361

    Article  MathSciNet  Google Scholar 

  13. Liu H, Zhang S, Wu X (2014b) Mlslr: multilabel learning via sparse logistic regression. Inf Sci 281:310–320

    Article  MathSciNet  MATH  Google Scholar 

  14. Lo HY, Wang JC, Wang HM, Lin SD (2011) Cost-sensitive multi-label learning for audio tag annotation and retrieval. IEEE Trans Multimed 13(3):518–529

    Article  Google Scholar 

  15. Ma H, Chen E, Xu L, Xiong H (2012) Capturing correlations of multiple labels: a generative probabilistic model for multi-label learning. Neurocomputing 92:116–123

    Article  Google Scholar 

  16. Madjarov G, Kocev D, Gjorgjevikj D, Dzeroski S (2012) An extensive experimental comparison of methods for multi-label learning. Pattern Recognit 45(9):3084–3104

    Article  Google Scholar 

  17. Montanes E, Senge R, Barranquero J, Quevedo JR, Coz JJ, Hullermeier E (2014) Dependent binary relevance models for multi-label classification. Pattern Recognit 47(3):1494–1508

    Article  Google Scholar 

  18. Musa AB (2014) A comparison of l\(_1\)-regularizion, pca, kpca and ica for dimensionality reduction in logistic regression. Int J Mach Learn Cybern 5(6):861–873

    Article  Google Scholar 

  19. Pan W, Ma P, Hu Q, Su X, Ma C (2014) Comparative analysis on margin based feature selection algorithms. Int J Mach Learn Cybern 5(3):339–367

    Article  Google Scholar 

  20. Read J (2008) A pruned problem transformation method for multi-label classification. In: Proceedings of the 2008 New Zealand Computer Science Research Student Conference (NZCSRS 2008), pp 143–150

  21. Read J, Pfahringer B, Holmes G, Franks E (2009) Classifier chains for multi-label classification. In: Proceedings of ECML/PKDD 2009, pp 254–269

  22. Rivas-Perea P, Cota-Ruiz J, Rosiles JG (2014) A nonlinear least squares quasi-newton strategy for lp-svr hyper-parameters selection. Int J Mach Learn Cybern 5(4):579–597

    Article  Google Scholar 

  23. Sharma A, Paliwal KK (2015) Linear discriminant analysis for the small sample size problem: an overview. Int J Mach Learn Cybern 6(3):443–454

    Article  Google Scholar 

  24. Song Y, Zhang L, Giles C (2011) Automatic tag recommendation algorithms for social recommender systems. ACM Trans Web 5(1):4:1–4:31

  25. Spolaor N, Cherman E, Monard M, Lee H (2013) A comparison of multi-label feature selection methods using the problem transformation approach. Electron Notes Theor Comput Sci 292:135–151

    Article  Google Scholar 

  26. Sun L, Ji S, Ye J (2011) Canonical correlation analysis for multilabel classification: a least-squares formulation, extensions, and analysis. IEEE Trans Pattern Anal Mach Intell 33(1):194–200

    Article  Google Scholar 

  27. Tai F, Lin HT (2012) Multi-label classification with principal label space transformation. Neural Comput 24(9):2508–2542

    Article  MathSciNet  MATH  Google Scholar 

  28. Tsoumakas G, Dimou A, Spyromitros E, Mezaris V, Kompatsiaris I, Vlahavas I (2009) Correlation-based pruning of stacked binary relevance models for multi-label learning. In: Proceedings of the Workshop on Learning from Multi-Label Data (MLD’09), pp 101–116

  29. Tsoumakas G, Katakis I, Vlahavas I (2010) Data mining and knowledge discovery handbook, chap Mining Multi-label Data, pp 667–686

  30. Xie Z, Xu Y (2014) Sparse group LASSO based uncertain feature selection. Int J Mach Learn Cybern 5(2):201–210

    Article  Google Scholar 

  31. Xu J (2014) Multi-label core vector machine with a zero label. Pattern Recognit 47(7):2542–2557

    Article  MATH  Google Scholar 

  32. Xu J, Jagadeesh V, Manjunath BS (2014) Multi-label learning with fused multimodal bi-relational graph. IEEE Trans Multimed 16(2):403–412

    Article  Google Scholar 

  33. Xu Y, Jiao L, Wang S, Wei J, Fan Y, Lai M, Chang EIC (2013) Multi-label classification for colon cancer using histopathological images. Microsc Res Tech 76(12):1266–1277

    Article  Google Scholar 

  34. Zhang M, Wu L (2015) Lift: Multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120

    Article  MathSciNet  Google Scholar 

  35. Zhang M, Pena J, Robles V (2009) Feature selection for multi-label naive bayes classification. Inf Sci 179:3218–3229

    Article  MATH  Google Scholar 

  36. Zhang ML, Zhang K (2010) Multi-label learning by exploiting label dependency. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’10), pp 999–1007

  37. Zhang ML, Zhou ZH (2006) Multi-label neural network with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351

    Article  Google Scholar 

  38. Zhang ML, Zhou ZH (2007) Ml-\(k\)nn: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048

    Article  MATH  Google Scholar 

  39. Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  40. Zhou ZH, Zhang ML (2007) Multi-instance multi-label learning with application to scene classification. In: Advances in Neural Information Processing Systems (NIPS’06), pp 1609–1616

  41. Zhou ZH, Zhang ML, Huang SJ, Li YF (2012) Multi-instance multi-label learning. Artif Intell 176(1):2291–2320

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The authors are grateful to the anonymous referees for their valuable comments and suggestions, which can substantially improve this paper. This work was partially supported by the National NSF of China (61572443, 61272007, 61170109 and 61170108), the NSF of Zhejiang province (LY14F020008 and LY14F020012).

Author information

Authors and Affiliations

  1. Department of Computer Science, Zhejiang Normal University, Jinhua, China

    Huawen Liu, Jianmin Han, Zhongyu Chen & Zhonglong Zheng

  2. Institute for Integrated and Intelligent Systems, Griffith University, Brisbane, Australia

    Zongjie Ma

Authors
  1. Huawen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zongjie Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianmin Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhongyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhonglong Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huawen Liu.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (XLS 84 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, H., Ma, Z., Han, J. et al. Regularized partial least squares for multi-label learning. Int. J. Mach. Learn. & Cyber. 9, 335–346 (2018). https://doi.org/10.1007/s13042-016-0500-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-016-0500-8

Keywords

Search

Navigation