Abstract
In this paper, we reinterpret error-correcting output codes (ECOCs) as a framework for converting multi-class classification problems into multi-label prediction problems. Different well-known multi-label learning approaches can be mapped upon particular ways of dealing with the original multi-class problem. For example, the label powerset approach obviously constitutes the inverse transformation from multi-label back to multi-class, whereas binary relevance learning may be viewed as the conventional way of dealing with ECOCs, in which each classifier is learned independently of the others. Consequently, we evaluate whether alternative choices for solving the multi-label problem may result in improved performance. This question is interesting because it is not clear whether approaches that do not treat the bits of the code words independently have sufficient error-correcting properties. Our results indicate that a slight but consistent advantage can be obtained with the use of multi-label methods, in particular when longer codes are employed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2000)
Armano, G., Chira, C., Hatami, N.: Error-correcting output codes for multi-label text categorization. In: Proceedings of the 3rd Italian Information Retrieval Workshop, Bari, Italy, pp. 26–37 (2012)
Berger, A.: Error-correcting output coding for text classification. In: Proceedings of the IJCAI 1999 Workshop on Machine Learning for Information Filtering, Stockholm, Sweden (1999)
Bose, R.C., Ray-Chaudhuri, D.K.: On a class of error correcting binary group codes. Information and Control 3(1), 68–79 (1960)
Bouckaert, R.R., Frank, E., Hall, M., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: WEKA — Experiences with a Java open-source project. Journal of Machine Learning Research 11, 2533–2541 (2010)
Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recognition 37(9), 1757–1771 (2004)
Brinker, K., Hüllermeier, E.: Case-based multilabel ranking. In: Veloso, M.M. (ed.) Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), Hyderabad, India, pp. 702–707 (2007)
Dembczynski, K., Cheng, W., Hüllermeier, E.: Bayes optimal multilabel classification via probabilistic classifier chains. In: Fürnkranz, J., Joachims, T. (eds.) Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, Israel, pp. 279–286 (2010)
Dembczyński, K., Waegeman, W., Cheng, W., Hüllermeier, E.: On label dependence in multi-label classification. In: Zhang, M.L., Tsoumakas, G., Zhou, Z.H. (eds.) Proceedings of the ICML 2010 Workshop on Learning from Multi-Label Data, Haifa, Israel, pp. 5–12 (2010)
Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)
Diplaris, S., Tsoumakas, G., Mitkas, P.A., Vlahavas, I.P.: Protein Classification with Multiple Algorithms. In: Bozanis, P., Houstis, E.N. (eds.) PCI 2005. LNCS, vol. 3746, pp. 448–456. Springer, Heidelberg (2005)
Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14 (NIPS 2001), pp. 681–687 (2002)
Ferng, C.-S., Lin, H.-T.: Multi-label classification with error-correcting codes. Journal of Machine Learning Research – Proceedings Track 20, 281–295 (2011)
Fürnkranz, J., Hüllermeier, E., Loza Mencía, E., Brinker, K.: Multilabel classification via calibrated label ranking. Machine Learning 73(2), 133–153 (2008)
Gama, J., Brazdil, P.: Cascade generalization. Machine Learning 41(3), 315–343 (2000)
Ghani, R.: Using error-correcting codes for text classification. In: Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 303–310. Morgan Kaufmann (2000)
Hocquenghem, A.: Codes correcteurs d’erreurs. Chiffres 2, 147–156 (1959) (in French)
Ji, S., Sun, L., Jin, R., Ye, J.: Multi-label multiple kernel learning. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 21 (NIPS 2008), pp. 777–784. Curran Associates, Inc., Vancouver (2009)
Kang, F., Jin, R., Sukthankar, R.: Correlated label propagation with application to multi-label learning. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), pp. 1719–1726. IEEE Computer Society (2006)
Kittler, J., Ghaderi, R., Windeatt, T., Matas, J.: Face verification via error correcting output codes. Image and Vision Computing 21(13-14), 1163–1169 (2003)
Kong, E.B., Dietterich, T.G.: Error-correcting output coding corrects bias and variance. In: Proceedings of the 12th International Conference on Machine Learning (ICML 1995), pp. 313–321. Morgan Kaufmann (1995)
Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research 5, 361–397 (2004)
Loza Mencía, E., Fürnkranz, J.: Efficient Pairwise Multilabel Classification for Large-Scale Problems in the Legal Domain. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part II. LNCS (LNAI), vol. 5212, pp. 50–65. Springer, Heidelberg (2008), accompanying EUR-Lex dataset available at http://www.ke.tu-darmstadt.de/resources/eurlex
MacWilliams, F.J., Sloane, N.J.A.: The Theory of Error-Correcting Codes. North-Holland Mathematical Library. North Holland (January 1983)
Madjarov, G., Gjorgjevikj, D., Deroski, S.: Two stage architecture for multi-label learning. Pattern Recognition 45(3), 1019–1034 (2012)
Melvin, I., Ie, E., Weston, J., Noble, W.S., Leslie, C.: Multi-class protein classification using adaptive codes. Journal of Machine Learning Research 8, 1557–1581 (2007)
Park, S.H., Fürnkranz, J.: Multi-label classification with label constraints. In: Hüllermeier, E., Fürnkranz, J. (eds.) Proceedings of the ECML PKDD 2008 Workshop on Preference Learning (PL 2008), Antwerp, Belgium, pp. 157–171 (2008)
Park, S.-H., Weizsäcker, L., Fürnkranz, J.: Exploiting Code Redundancies in ECOC. In: Pfahringer, B., Holmes, G., Hoffmann, A. (eds.) DS 2010. LNCS, vol. 6332, pp. 266–280. Springer, Heidelberg (2010)
Park, S.H., Fürnkranz, J.: Efficient prediction algorithms for binary decomposition techniques. Data Mining and Knowledge Discovery 24(1), 40–77 (2012)
Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Machine Learning 85(3), 333–359 (2011)
Snoek, C.G., Worring, M., van Gemert, J.C., Geusebroek, J.M., Smeulders, A.W.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proceedings of ACM Multimedia, Santa Barbara, CA, pp. 421–430 (2006)
Tsoumakas, G., Katakis, I.: Multi-label classification: An overview. International Journal of Data Warehousing and Mining 3(3), 1–17 (2007)
Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Mining multi-label data. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, 2nd edn., pp. 667–685. Springer (2010)
Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Random k-labelsets for multilabel classification. IEEE Transactions on Knowledge and Data Engineering 23(7), 1079–1089 (2011)
Tsoumakas, G., Spyromitros Xioufis, E., Vilcek, J., Vlahavas, I.P.: Mulan: A Java library for multi-label learning. Journal of Machine Learning Research 12, 2411–2414 (2011), http://mulan.sourceforge.net/
Windeatt, T., Ghaderi, R.: Coding and decoding strategies for multi-class learning problems. Information Fusion 4(1), 11–21 (2003)
Wolpert, D.H.: Stacked generalization. Neural Networks 5(2), 241–260 (1992)
Zaragoza, J.H., Sucar, J.E., Morales, E.F., Bielza, C.: Larrañaga: Bayesian chain classifiers for multidimensional classification. In: Walsh, T. (ed.) Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI 2011), Barcelona, Spain, pp. 2192–2197 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fürnkranz, J., Park, SH. (2012). Error-Correcting Output Codes as a Transformation from Multi-Class to Multi-Label Prediction. In: Ganascia, JG., Lenca, P., Petit, JM. (eds) Discovery Science. DS 2012. Lecture Notes in Computer Science(), vol 7569. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33492-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-33492-4_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33491-7
Online ISBN: 978-3-642-33492-4
eBook Packages: Computer ScienceComputer Science (R0)