Nothing Special   »   [go: up one dir, main page]

Skip to main content

Fuzzy Semi-supervised Support Vector Machines

  • Conference paper
Machine Learning and Data Mining in Pattern Recognition (MLDM 2011)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 6871))

Abstract

In this paper, a fuzzy semi-supervised support vector machines (FSS-SVM) algorithm is proposed. It tries to overcome the need for a large labelled training set to learn accurate classifiers. For this, it uses both labelled and unlabelled data for training. It also modulates the effect of the unlabelled data in the learning process. Empirical evaluations showed that by additionally using unlabelled data, FSS-SVM requires less labelled training data than its supervised version, support vector machines, to achieve the same level of classification performance. Also, the incorporated fuzzy membership values of the unlabelled training patterns in the learning process have positively influenced the classification performance in comparison with its crisp variant.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Aamodt, A., Plaza, E.: Case-Based Reasoning. In: Proc. MLnet Summer School on Machine Learning and Knowledge Acquisition, pp. 1–58 (1994)

    Google Scholar 

  2. Balcan, M.F., Blum, A.: An augmented PAC model for semi-supervised learning (2005) (manuscript)

    Google Scholar 

  3. Baluja, S.: Probabilistic modeling for face orientation discrimination: Learning from labeled and unlabeled data. Advances in Neural Information Processing Systems 11, 854–860 (1998)

    Google Scholar 

  4. Bennett, K., Demiriz, A.: Semi-supervised support vector machines. Advances in Neural Information Processing Systems 11, 368–374 (1998)

    Google Scholar 

  5. Bensaid, A.M., Hall, L.O., et al.: Partially supervised clustering for image segmentation. Pattern Recognition 29(5), 859–871 (1996)

    Article  Google Scholar 

  6. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer Academic Publishers, Norwell (1981)

    Book  MATH  Google Scholar 

  7. Blum, A., Mitchell, T.: Combining labeled and unlabeled data with cotraining. In: Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pp. 92–100 (1998)

    Google Scholar 

  8. Bobrowski, L., Bezdek, J.C.: C-means clustering with the l l and l8 Norms. IEEE Transactions on, Systems, Man and Cybernetics 21(3), 545–554 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  9. Booker, L.B., Goldberg, D.E., et al.: Classifier systems and genetic algorithms. Artificial Intelligence 40(1-3), 235–282 (1989)

    Article  Google Scholar 

  10. Cao, L.J., Lee, H.P., et al.: Modified support vector novelty detector using training data with outliers. Pattern Recognition Letters 24(14), 2479–2487 (2003)

    Article  MATH  Google Scholar 

  11. Chapelle, O., Sindhwani, V., et al.: Branch and Bound for Semi- Supervised Support Vector Machines. In: Advances in Neural Information Processing Systems, (NIPS) (2006)

    Google Scholar 

  12. Cheeseman, P., Freeman, D., et al.: Autoclass: A Bayesian classiciation system. In: Proceedings of the Fifth International Conference on Machine Learning, pp. 54–64 (1988)

    Google Scholar 

  13. Collins, M., Singer, Y.: Unsupervised models for named entity classification. In: Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pp. 189–196 (1999)

    Google Scholar 

  14. Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13(1), 21–27 (1967)

    Article  MATH  Google Scholar 

  15. Cozman, F.G., Cohen, I., et al.: Semi-Supervised Learning of Mixture Models and Bayesian Networks. In: Proceedings of the Twentieth International Conference of Machine Learning (2003)

    Google Scholar 

  16. Dagan, I., Engelson, S.P.: Committee-based sampling for training probabilistic classifiers. In: Proceedings of the Twelfth International Conference on Machine Learning, pp. 150–157 (1995)

    Google Scholar 

  17. Day, W.H.E., Edelsbrunner, H.: Efficient algorithms for agglomerative hierarchical clustering methods. Journal of Classification 1(1), 7–24 (1984)

    Article  MATH  Google Scholar 

  18. Dempster, A.P., Laird, N.M., et al.: Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39(1), 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  19. Ester, M., Kriegel, H.P., et al.: A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In: Proc. 2nd Int. Conf. on Knowledge Discovery and Data Mining, pp. 226–231. AAAI Press, Portland (1996)

    Google Scholar 

  20. Freund, Y., Seung, H.S., et al.: Selective Sampling Using the Query by Committee Algorithm. Machine Learning 28(2), 133–168 (1997)

    Article  MATH  Google Scholar 

  21. Fung, G., Mangasarian, O.: Semi-supervised support vector machines for unlabeled data classification (Technical Report 99-05). Data mining Institute, University of Wisconsin at Madison, Madison, WI (1999)

    Google Scholar 

  22. Guyon, I., Matic, N., et al.: Discovering informative patterns and data cleaning. Advances in Knowledge Discovery and Data Mining Table of Contents, 181–203 (1996)

    Google Scholar 

  23. Huang, W., Lippmann, R.: Neural net and traditional classifiers. Neural Information Processing Systems, 387–396 (1988)

    Google Scholar 

  24. Joachims, T.: Making large-Scale SVM Learning Practical. In: Scoelkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods-Support Vector Learning (1999)

    Google Scholar 

  25. Joachims, T.: Transductive inference for text classification using support vector machines. In: Proceedings of the Sixteenth International Conference on Machine Learning, pp. 200–209 (1999)

    Google Scholar 

  26. Jones, R.: Learning to Extract Entities from Labeled and Unlabeled Text, University of Utah (2005)

    Google Scholar 

  27. Kaelbling, L.P., Littman, M.L., et al.: Reinforcement Learning: A Survey. Arxiv preprint cs.AI/9605103 (1996)

    Google Scholar 

  28. Kockelkorn, M., Luneburg, A., et al.: Using transduction and multi-view learning to answer emails. In: Proceedings of the European Conference on Principle and Practice of Knowledge Discovery in Databases, pp. 266–277 (2003)

    Google Scholar 

  29. Kohonen, T.: Self-organized formation of topologically correct feature maps. Biological Cybernetics 43(1), 59–69 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  30. Koller, D., Sahami, M.: Hierarchically classifying documents using very few words. In: Proceedings of the Fourteenth International Conference on Machine Learning, pp. 170–178 (1997)

    Google Scholar 

  31. Langley, P., Simon, H.A.: Applications of machine learning and rule induction. Communications of the ACM 38(11), 54–64 (1995)

    Article  Google Scholar 

  32. Letters, P.R.: Training algorithms for fuzzy support vector machines with noisy data. Pattern Recognition Letters 25(14), 1647–1656 (2004)

    Article  Google Scholar 

  33. Liere, R., Tadepalli, P.: Active learning with committees for text categorization. In: Proceedings of the Fourteenth National Conference on Artificial Intelligence, pp. 591–596 (1997)

    Google Scholar 

  34. Lin, C.F., Wang, S.D.: Fuzzy support vector machines. IEEE Transactions on Neural Networks 13(2), 464–471 (2002)

    Article  Google Scholar 

  35. Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning 2(4), 285–318 (1988)

    Google Scholar 

  36. McCallum, A., Nigam, K.: Employing EM in pool-based active learning for text classification. In: Proceedings of ICML 1998, 15th International Conference on Machine Learning, pp. 350–358 (1998)

    Google Scholar 

  37. Mitchell, T.: The role of unlabeled data in supervised learning. In: Proceedings of the Sixth International Colloquium on Cognitive Science (1999)

    Google Scholar 

  38. Mladenic, D.: Learning word normalization using word suffix and context from unlabeled data. In: Proceedings of the Nineteenth International Conference on Machine Learning Table of Contents, pp. 427–434 (2002)

    Google Scholar 

  39. Nigam, K., Ghani, R.: Analyzing the effectiveness and applicability of co-training. In: Proceedings of the Ninth International Conference on Information and Knowledge Management, pp. 86–93 (2000)

    Google Scholar 

  40. Nigam, K., McCallum, A.K., et al.: Text Classification from Labeled and Unlabeled Documents using EM. Machine Learning 39(2), 103–134 (2000)

    Article  MATH  Google Scholar 

  41. Quinlan, J.R.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)

    Google Scholar 

  42. Rosenberg, C., Hebert, M., et al.: Semi-supervised selftraining of object detection models. In: Seventh IEEE Workshop on Applications of Computer Vision, vol. 1, pp. 29–36 (2005)

    Google Scholar 

  43. Schohn, G., Cohn, D.: Less is more: Active learning with support vector machines. In: Proceedings of the Seventeenth International Conference on Machine Learning, vol. 282, pp. 285–286 (2000)

    Google Scholar 

  44. Seung, H.S., Opper, M., et al.: Query by committee. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 287–294 (1992)

    Google Scholar 

  45. Sheng-de Wang, C.L.: Training algorithms for fuzzy support vector machines with noisy data. In: IEEE 13th Workshop on, Neural Networks for Signal Processing NNSP 2003, pp. 517–526 (2003)

    Google Scholar 

  46. Tong, S., Koller, D.: Support vector machine active learning with applications to text classification. The Journal of Machine Learning Research 2, 45–66 (2002)

    MATH  Google Scholar 

  47. Vapnik, V., Golowich, S., et al.: Support vector method for function approximation, regression estimation, and signal processing. Advances in Neural Information Processing Systems 9, 281–287 (1997)

    Google Scholar 

  48. Vapnik, V.N.: Statistical learning theory. Wiley, New York (1998)

    MATH  Google Scholar 

  49. Wang, W., Yang, J., et al.: STING: A Statistical Information Grid Approach to Spatial Data Mining. In: Proceedings of the 23rd International Conference on Very Large Data Bases, pp. 186–195 (1997)

    Google Scholar 

  50. Xu, L., Schuurmans, D.: Unsupervised and Semi-supervised Multiclass Support Vector Machines. In: AAAI 2005, Pittsburgh, PA, pp. 904–910 (2005)

    Google Scholar 

  51. Yarowsky, D.: Unsupervised word sense disambiguation rivalling supervised methods. In: Proceedings of the 33rd conference on Association for Computational Linguistics, pp. 189–196 (1995)

    Google Scholar 

  52. Zhang, X.: Using class-center vectors to build support vector machines. In: Proceedings of the 1999 IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing IX, pp. 3–11 (1999)

    Google Scholar 

  53. Zhou, Z.-H., Zhan, D.-C., Yang, Q.: Semi-supervised learning with very few labeled training examples. In: Twenty-Second AAAI Conference on Artificial Intelligence, (AAAI 2007) (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Benbrahim, H. (2011). Fuzzy Semi-supervised Support Vector Machines. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2011. Lecture Notes in Computer Science(), vol 6871. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23199-5_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-23199-5_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-23198-8

  • Online ISBN: 978-3-642-23199-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics