Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Concave-Convex Programming for Ramp Loss-Based Maximum Margin and Minimum Volume Twin Spheres Machine

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Twin hyper-sphere support vector machine (THSVM) classifies two classes of samples via two hyper-spheres instead of a pair of nonparallel hyper-planes in the conventional twin support vector machine. It avoids the matrix inverse operation. However, the THSVM uses hinge loss which easily leads to sensitivity of the noises. In this paper, we propose a ramp loss-based maximum margin and minimum volume twin spheres machine (\(\hbox {RM}^{3}\hbox {TSM}\)) to enhance the ability of noise resistance. \(\hbox {RM}^{3}\hbox {TSM}\) is robust because of introducing the ramp loss function, but it is non-convex. \(\hbox {RM}^{3}\hbox {TSM}\) has several admirable advantages. For example, it can explicitly incorporate noises in the training process. In addition, the ramp loss function can be decomposed into the difference of a convex hinge loss and another convex loss. Then the concave-convex programming procedure is employed to efficiently solve the non-convex problems of \(\hbox {RM}^{3}\hbox {TSM}\) by solving a sequence of convex programs iteratively. Experimental results on one artificial data set, 14 imbalanced binary datasets and 3 multiclass datasets indicate that our proposed \(\hbox {RM}^{3}\hbox {TSM}\) yields a good generalization performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Vapnik V (1995) The nature of statistical learning theory. Springer, New York

    Book  Google Scholar 

  2. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  3. Tian Y, Qi Z, Ju X, Shi Y, Liu X (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44(7):1067–1079

    Article  Google Scholar 

  4. Peng X (2010) A \(\nu \)-twin support vector machine (\(\nu \)-TSVM) classifier and its geometric algorithms. Inf Sci 180(20):3863–3875

    Google Scholar 

  5. Xu Y, Wang L, Zhong P (2012) A rough margin-based \(\nu \)-twin support vector machine. Neural Comput Appl 21(6):1307–1317

    Google Scholar 

  6. Peng X (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23(3):365–372

    Article  Google Scholar 

  7. Xu Y, Wang L (2012) A weighted twin support vector regression. Knowl Based Syst 33:92–101

    Article  MathSciNet  Google Scholar 

  8. Xu Y, Yang Z, Pan X (2017) A novel twin support-vector machine with pinball loss. IEEE Trans Neural Netw Learn Syst 28(2):359–370

    Article  MathSciNet  Google Scholar 

  9. Wang XZ, He Q, Chen DG, Yeung D (2005) A genetic algorithm for solving the inverse problem of support vector machines. Neurocomputing 68:225–238

    Article  Google Scholar 

  10. Lu SX, Wang XZ, Zhang GQ, Zhou X (2015) Effective algorithms of the Moore-Penrose inverse matrices for extreme learning machine. Intell Data Anal 19(4):743–760

    Article  Google Scholar 

  11. Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221:12–27

    Article  MathSciNet  Google Scholar 

  12. Xu Y, Yang Z, Zhang Y, Pan X, Wang L (2016) A maximum margin and minimum volume hyper-spheres machine with pinball loss for imbalanced data classification. Knowl Based Syst 95:75–85

    Article  Google Scholar 

  13. Mason L, Baxter J, Bartlett PL, Frean M (1999) Boosting algorithms as gradient descent in function space (Technical Report), RSISE. Australian National University, Canberra

    Google Scholar 

  14. Suykens J, DeBrabanter J, Lukas L (2002) Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48(1):85–105

    Article  Google Scholar 

  15. Lin C, Wang S (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(2):464–471

    Article  Google Scholar 

  16. Huang XL, Shi L, Suykens JAK (2014) Ramp loss linear programming support vector machine. J Mach Learn Res 15(1):2185–2211

    MathSciNet  MATH  Google Scholar 

  17. Liu D, Shi Y, Tian Y (2015) Ramp loss nonparallel support vector machine for pattern classification. Knowl Based Syst 85:224–233

    Article  Google Scholar 

  18. Liu D, Shi Y, Tian Y, Huang X (2016) Ramp loss least squares support vector machine. J Comput Sci 14:61–68

    Article  MathSciNet  Google Scholar 

  19. Collobert R, Sinz F, Weston J, Bottou L (2006) Trading convexity for scalability. In: Proceedings of the 23\(^{\rm rd}\) ICML, Pittsburgh, pp 201–208

  20. Yuille AL, Rangarajan A (2003) The concave-convex procedure. Neural Comput 15(4):915–936

    Article  Google Scholar 

  21. LeCun Y, Bottou L, Orr GB (1998) Efficient BackProp. Neural Netw Tricks Trade 1524(1):9–50

    Article  Google Scholar 

  22. Platt JC (1999) Fast training of support vector machines using sequential minimal optimization, advances in kernel methods. MIT Press, Cambridge, pp 185–208

    Google Scholar 

  23. Chang WC, Lee CP, Lin CJ (2013) A revisit to support vector data description. Department of Computer Science, National Taiwan University, Taipei, Taiwan, Technical Report

  24. Wu M, Ye J (2009) A small sphere and large margin approach for novelty detection using training data with outliers. IEEE Trans Pattern Anal Mach Intell 31(11):2088–2092

    Article  Google Scholar 

  25. Xu Y (2017) Maximum margin of twin spheres support vector machine for imbalanced data classification. IEEE Trans Cybern 47(6):1540–1550

    Article  Google Scholar 

  26. Xu Y, Wang Q, Pang X, Tian Y (2018) Maximum margin of twin spheres machine with pinball loss for imbalanced data classification. Appl Intell 48(1):23–34

    Article  Google Scholar 

  27. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    Book  Google Scholar 

  28. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge

    Book  Google Scholar 

  29. Khemchandani R, Jayadeva, Chandra S (2009) Optimal kernel selection in twin support vector machines. Optim Lett 3(1):77–88

    Article  MathSciNet  Google Scholar 

  30. Jayadeva, Khemchandani R, Chandra S (2017) Twin support vector machines: models, extensions and applications, Springer series on studies in computational intelligence. Springer, Berlin

    Book  Google Scholar 

  31. Maldonado S, Weber R, Famili F (2014) Feature selection for high-dimensional class-imbalanced data sets using support vector machines. Inf Sci 286:228–246

    Article  Google Scholar 

  32. Sokolova M, Japkowicz N, Szpakowicz S (2006) Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation, advances in artificial intelligence. Springer, Berlin, pp 1015–1021

    Google Scholar 

  33. Jurman G, Riccadonna S, Furlanello C (2012) A comparison of MCC and CEN error measures in multiclass prediction. PLoS ONE 7(8):e41882

    Article  Google Scholar 

  34. Hand DJ, Till RJ (2001) A simple generalisation of area under the ROC curve for multiple class classification problems. Mach Learn 45(2):171–186

    Article  Google Scholar 

  35. Ar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  Google Scholar 

  36. Gar C, Salvador A (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180(10):2044–2064

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation. This work was supported in part by the Beijing Natural Science Foundation (No. 4172035) and National Natural Science Foundation of China (No. 11671010).

Author information

Authors and Affiliations

  1. College of Science, China Agricultural University, Beijing, 100083, China

    Qian Wang & Yitian Xu

Authors
  1. Qian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yitian Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yitian Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Q., Xu, Y. Concave-Convex Programming for Ramp Loss-Based Maximum Margin and Minimum Volume Twin Spheres Machine. Neural Process Lett 50, 1093–1114 (2019). https://doi.org/10.1007/s11063-018-9903-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-018-9903-8

Keywords

Search

Navigation