Nothing Special   »   [go: up one dir, main page]

Skip to main content

Error-Correcting Output Codes as a Transformation from Multi-Class to Multi-Label Prediction

  • Conference paper
Discovery Science (DS 2012)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7569))

Included in the following conference series:

  • 985 Accesses

Abstract

In this paper, we reinterpret error-correcting output codes (ECOCs) as a framework for converting multi-class classification problems into multi-label prediction problems. Different well-known multi-label learning approaches can be mapped upon particular ways of dealing with the original multi-class problem. For example, the label powerset approach obviously constitutes the inverse transformation from multi-label back to multi-class, whereas binary relevance learning may be viewed as the conventional way of dealing with ECOCs, in which each classifier is learned independently of the others. Consequently, we evaluate whether alternative choices for solving the multi-label problem may result in improved performance. This question is interesting because it is not clear whether approaches that do not treat the bits of the code words independently have sufficient error-correcting properties. Our results indicate that a slight but consistent advantage can be obtained with the use of multi-label methods, in particular when longer codes are employed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2000)

    MathSciNet  Google Scholar 

  2. Armano, G., Chira, C., Hatami, N.: Error-correcting output codes for multi-label text categorization. In: Proceedings of the 3rd Italian Information Retrieval Workshop, Bari, Italy, pp. 26–37 (2012)

    Google Scholar 

  3. Berger, A.: Error-correcting output coding for text classification. In: Proceedings of the IJCAI 1999 Workshop on Machine Learning for Information Filtering, Stockholm, Sweden (1999)

    Google Scholar 

  4. Bose, R.C., Ray-Chaudhuri, D.K.: On a class of error correcting binary group codes. Information and Control 3(1), 68–79 (1960)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bouckaert, R.R., Frank, E., Hall, M., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: WEKA — Experiences with a Java open-source project. Journal of Machine Learning Research 11, 2533–2541 (2010)

    MATH  Google Scholar 

  6. Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recognition 37(9), 1757–1771 (2004)

    Article  Google Scholar 

  7. Brinker, K., Hüllermeier, E.: Case-based multilabel ranking. In: Veloso, M.M. (ed.) Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), Hyderabad, India, pp. 702–707 (2007)

    Google Scholar 

  8. Dembczynski, K., Cheng, W., Hüllermeier, E.: Bayes optimal multilabel classification via probabilistic classifier chains. In: Fürnkranz, J., Joachims, T. (eds.) Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, Israel, pp. 279–286 (2010)

    Google Scholar 

  9. Dembczyński, K., Waegeman, W., Cheng, W., Hüllermeier, E.: On label dependence in multi-label classification. In: Zhang, M.L., Tsoumakas, G., Zhou, Z.H. (eds.) Proceedings of the ICML 2010 Workshop on Learning from Multi-Label Data, Haifa, Israel, pp. 5–12 (2010)

    Google Scholar 

  10. Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)

    MATH  Google Scholar 

  11. Diplaris, S., Tsoumakas, G., Mitkas, P.A., Vlahavas, I.P.: Protein Classification with Multiple Algorithms. In: Bozanis, P., Houstis, E.N. (eds.) PCI 2005. LNCS, vol. 3746, pp. 448–456. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  12. Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14 (NIPS 2001), pp. 681–687 (2002)

    Google Scholar 

  13. Ferng, C.-S., Lin, H.-T.: Multi-label classification with error-correcting codes. Journal of Machine Learning Research – Proceedings Track 20, 281–295 (2011)

    Google Scholar 

  14. Fürnkranz, J., Hüllermeier, E., Loza Mencía, E., Brinker, K.: Multilabel classification via calibrated label ranking. Machine Learning 73(2), 133–153 (2008)

    Article  Google Scholar 

  15. Gama, J., Brazdil, P.: Cascade generalization. Machine Learning 41(3), 315–343 (2000)

    Article  MATH  Google Scholar 

  16. Ghani, R.: Using error-correcting codes for text classification. In: Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 303–310. Morgan Kaufmann (2000)

    Google Scholar 

  17. Hocquenghem, A.: Codes correcteurs d’erreurs. Chiffres 2, 147–156 (1959) (in French)

    MathSciNet  MATH  Google Scholar 

  18. Ji, S., Sun, L., Jin, R., Ye, J.: Multi-label multiple kernel learning. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 21 (NIPS 2008), pp. 777–784. Curran Associates, Inc., Vancouver (2009)

    Google Scholar 

  19. Kang, F., Jin, R., Sukthankar, R.: Correlated label propagation with application to multi-label learning. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), pp. 1719–1726. IEEE Computer Society (2006)

    Google Scholar 

  20. Kittler, J., Ghaderi, R., Windeatt, T., Matas, J.: Face verification via error correcting output codes. Image and Vision Computing 21(13-14), 1163–1169 (2003)

    Article  Google Scholar 

  21. Kong, E.B., Dietterich, T.G.: Error-correcting output coding corrects bias and variance. In: Proceedings of the 12th International Conference on Machine Learning (ICML 1995), pp. 313–321. Morgan Kaufmann (1995)

    Google Scholar 

  22. Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research 5, 361–397 (2004)

    Google Scholar 

  23. Loza Mencía, E., Fürnkranz, J.: Efficient Pairwise Multilabel Classification for Large-Scale Problems in the Legal Domain. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part II. LNCS (LNAI), vol. 5212, pp. 50–65. Springer, Heidelberg (2008), accompanying EUR-Lex dataset available at http://www.ke.tu-darmstadt.de/resources/eurlex

    Chapter  Google Scholar 

  24. MacWilliams, F.J., Sloane, N.J.A.: The Theory of Error-Correcting Codes. North-Holland Mathematical Library. North Holland (January 1983)

    Google Scholar 

  25. Madjarov, G., Gjorgjevikj, D., Deroski, S.: Two stage architecture for multi-label learning. Pattern Recognition 45(3), 1019–1034 (2012)

    Article  Google Scholar 

  26. Melvin, I., Ie, E., Weston, J., Noble, W.S., Leslie, C.: Multi-class protein classification using adaptive codes. Journal of Machine Learning Research 8, 1557–1581 (2007)

    MathSciNet  MATH  Google Scholar 

  27. Park, S.H., Fürnkranz, J.: Multi-label classification with label constraints. In: Hüllermeier, E., Fürnkranz, J. (eds.) Proceedings of the ECML PKDD 2008 Workshop on Preference Learning (PL 2008), Antwerp, Belgium, pp. 157–171 (2008)

    Google Scholar 

  28. Park, S.-H., Weizsäcker, L., Fürnkranz, J.: Exploiting Code Redundancies in ECOC. In: Pfahringer, B., Holmes, G., Hoffmann, A. (eds.) DS 2010. LNCS, vol. 6332, pp. 266–280. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  29. Park, S.H., Fürnkranz, J.: Efficient prediction algorithms for binary decomposition techniques. Data Mining and Knowledge Discovery 24(1), 40–77 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  30. Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Machine Learning 85(3), 333–359 (2011)

    Article  Google Scholar 

  31. Snoek, C.G., Worring, M., van Gemert, J.C., Geusebroek, J.M., Smeulders, A.W.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proceedings of ACM Multimedia, Santa Barbara, CA, pp. 421–430 (2006)

    Google Scholar 

  32. Tsoumakas, G., Katakis, I.: Multi-label classification: An overview. International Journal of Data Warehousing and Mining 3(3), 1–17 (2007)

    Article  Google Scholar 

  33. Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Mining multi-label data. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, 2nd edn., pp. 667–685. Springer (2010)

    Google Scholar 

  34. Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Random k-labelsets for multilabel classification. IEEE Transactions on Knowledge and Data Engineering 23(7), 1079–1089 (2011)

    Article  Google Scholar 

  35. Tsoumakas, G., Spyromitros Xioufis, E., Vilcek, J., Vlahavas, I.P.: Mulan: A Java library for multi-label learning. Journal of Machine Learning Research 12, 2411–2414 (2011), http://mulan.sourceforge.net/

    MathSciNet  Google Scholar 

  36. Windeatt, T., Ghaderi, R.: Coding and decoding strategies for multi-class learning problems. Information Fusion 4(1), 11–21 (2003)

    Article  Google Scholar 

  37. Wolpert, D.H.: Stacked generalization. Neural Networks 5(2), 241–260 (1992)

    Article  MathSciNet  Google Scholar 

  38. Zaragoza, J.H., Sucar, J.E., Morales, E.F., Bielza, C.: Larrañaga: Bayesian chain classifiers for multidimensional classification. In: Walsh, T. (ed.) Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI 2011), Barcelona, Spain, pp. 2192–2197 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Fürnkranz, J., Park, SH. (2012). Error-Correcting Output Codes as a Transformation from Multi-Class to Multi-Label Prediction. In: Ganascia, JG., Lenca, P., Petit, JM. (eds) Discovery Science. DS 2012. Lecture Notes in Computer Science(), vol 7569. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33492-4_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-33492-4_21

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33491-7

  • Online ISBN: 978-3-642-33492-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics