Abstract
The classical training algorithms of support vector machines (SVM) are supervised learning algorithms which based on large-scale labeled samples, while these labeled samples are not easy to be acquired or labeled costly and class–unbalanced dataset, meanwhile these SVM algorithms are originally designed for the solution of two-class problems. To solve these problems of SVM, An Active learning algorithm based on decision directed acyclic graph (DDAG) for SVM is proposed in the paper, which train the multiclass SVMs using as few labeled instances as possible while maintaining the same SVM performance, or achieving the generalization performance of SVM classification as good as possible. The experimental results on the UCI data show that the proposed approach can achieve higher clasification accuracy, but using less labeled samples, while improving generalization performance and ruducing the marking costs of SVM training.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Vapnik, V.N.: Statistical learning theory. Wiley, New York (1998)
Kreßel, U.: Advances in kernel methods-support vector learning. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Pairwise classification and support vector machines, pp. 255–268. MIT Press, Cambridge (1999)
Dietterich, T.G., Bakiri, G.: Solving multi-class learning problem via error correcting output codes. J. Artif. Intell. Res. 2, 263–286 (1995)
Bennett, K. P., Blue, J. A.: A support vector machine approachto decision trees. In: Proceedings of the JCNN’98. Alaska: Anchorage, pp. 12396–24011, (1997)
Platt, J., Cristianini, N., Shawe-Taylor, J.: Large margin DAGs for multiclass classification. In: Sollas, A., Leen, T.K., Mulle, K.R. (eds.) Advances in neural information processing systems 12, pp. 547–553. MIT Press, Cambridge (2000)
Tong, S.: Active learning: theory and applications. Stanford University, USA (2001)
Liu, W., Wang, T.: Active learning for online spam filtering. Asia Information Retrieval Symposium (AIRS). Springer, Berlin/Heidelberg, pp. 555–560, (2008)
Brinker, K.: On multiclass active learning with support vector machines. In: Proceedings of the European Conference on Artificial Intelligence, pp. 969–970, (2004)
Jain, P., Kapoor, A.: Active learning for large multi-class problems. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Minmi, FL, USA., pp. 762–769, (2009)
Yan, R., Yang, Y., Hauptmann, A.: Automatically labeling video data using multi-class active learning. In: Proceedings of the Ninth IEEE International Conference on Computer Vision (ICCV 2003). Washington: IEEE Computer Society, pp. 516–523, (2003)
Yang, B., Sun, J., Wang, T. et al.: Effective multi-label active learning for text classification. In: Proceedings of the 15th ACM SIGKDD International Conference On Knowledge Discovery And Data Mining. ACM New York, NY, USA, pp 917–926, (2009)
Yan, R., Hauptmann, A.: Multi-class active learning for video semantic feature extraction. In: Proceedings of the IEEE International Conference On Mutimedia and Expo(ICME), specela session On “Active Learning on Multimedia Retrieval”. Taipei, TaiWan, pp. 27–30, (2004)
Alex Holub, Pietro Perona, Michael C. Burl.Entropy-based active learning for object recognition. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Anchorage, AK, pp. 1–8, (2008)
Qi, G.-J., Hua, X.-S., Rui, Y., et al.: Two-dimensional multilabel active learning with an efficient online adaptation model for image classification. IEEE Transact. Pattern Anal. Mach. Intell. 31(10), 1880–1897 (2008)
Joshi, A. J., Porikli, F., Papanikolopoulos, N.: Multi-class active learning for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C]. Minmi, FL, USA, pp. 2372–2379, (2009)
Singh, M., Brew, A., Greene, D., et al.: Score normalization and aggregation for active learning in multi-label classification. UCD-CSI-2010-2. University College Dublin, Dublin (2010)
Vijayanarasimhan, S., Grauman, K.: What’s it going to cost you? predicting effort vs. informativeness for multi-label image annotations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Minmi, FL, USA., pp. 2262–2269, (2009)
Hsu, C.W., Lin, C.J.: A comparison of methods for multi-class support vector machine. IEEE Transact. Neural. Netw. 13(2), 415–425 (2002)
Takahashi, F., Shigeo, A.: Decision-tree-based multi-class support vector machines. In: Proceedings of the ICONIP Annual Conference of the Institute of Systems, Control and Information Engineers. Japan, pp. 483–484, (2002)
LI, X.R., Zhao, G., Zhao, L.Y.: Design of decision-tree-based support vector machines multi-class classifier based on vector projection. Control Decis. 23(7), 745–750 (2008)
Faraoun, K.M., Boukelif, A.: Genetic programming approach for multi-category pattern classification applied to network intrusions detection. Int. J. Comput. Intell. 3(1), 79–90 (2007)
Jing, H.U.: Machine learning and optimization design of neural network classifier. Hefei University of Technology, Hefei (2007)
Hu, J., GAO, J., Yang, J.: The approach to active leaning on fuzzy neural network classifier. J. Univ. Sci. Technol. China 38(3), 241–246 (2008)
Gong, X.: Research on bayesian learning theory and its application. Institute of Computing Technology Chinese Academy of Science, Beijing (2002)
Feng, C., Chen, Z., Huang, H.: Organization names recognition with active learning. Mini-micro Syst. 27(4), 710–714 (2006)
Baram, Y., Yaniv, R.E., Luz, L.: Online choice of active learning algorithms. J. Mach. Learn. Res. 5, 255–291 (2004)
Lin, H.-T., Lin, C.-J., Weng, R.C.: A note on Platt’s probabilistic outputs for support vector machines. Mach. Learn. 68(3), 267–276 (2007)
Xun, Y.U.A.N., Xiu-qing, W.U., Ri-chang, H.O.N.G., et al.: Video genre categorization using SVM classifiers with active learning. J. Univ. Sci. Technol. China 39(5), 474–478 (2009)
Ting-Fan, W., Lin, C.-J., Weng, R.C.: Probability estimates for multi-class classification by pairwise coupling. J. Mach. Learn. Res. 5, 975–1005 (2004)
Hai-long, X.U., Xiao-feng, B.I.E., Hui, F.E.N.G., et al.: Active learning algorithm for SVM based on QBC. Syst. Eng. Electron. 38(12), 2865–2871 (2015)
Hai-long, X.U., Xiao-dan, W.A.N.G., Yong, L.I.A.O., et al.: Incremental training algorithm of SVM based on active learning. Control Decis. 25(2), 282–286 (2010)
Hai-long, X.U., Xiao-dan, W.A.N.G., Yong, L.I.A.O., et al.: New approach for optimizing model of RBF-SVM based on PSO. Control Decis. 25(3), 367–370 (2010)
Acknowledgements
The authors wish to express their gratitude to the referees for their helpful comments and kind suggestions in revising this paper. This work is substantially supported by Grants from the China Postdoctoral Science Foundation (Nos.: 2017M613415).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xu, H., Bie, X., Feng, H. et al. Multiclass SVM active learning algorithm based on decision directed acyclic graph and one versus one. Cluster Comput 22 (Suppl 3), 6241–6251 (2019). https://doi.org/10.1007/s10586-018-1951-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10586-018-1951-3