Abstract
The dynamic Web, which contains huge number of digital documents, is expanding day by day. Thus, it has become a tough challenge to search for a particular document from such a large volume of collections. Text classification is a technique which can speed up the search and retrieval tasks and hence is the need of the hour. Aiming in this direction, this study proposes an efficient technique that uses the concept of connected component (CC) of a graph and Wordnet along with four established feature selection techniques [e.g., TF-IDF, Chi-square, Bi-Normal Separation (BNS) and Information Gain (IG)] to select the best features from a given input dataset in order to prepare an efficient training feature vector. Next, multilayer extreme learning machine (ML-ELM) (which is based on the architecture of deep learning) and other state-of-the-art classifiers are trained on this efficient training feature vector for classification of text data. The experimental work has been carried out on DMOZ and 20-Newsgroups datasets. We have studied the behavior and compared the results of different classifiers using these four important feature selection techniques used for classification process and observed that ML-ELM achieved the maximum overall F-measure of 72.28 % on DMOZ dataset using TF-IDF as the feature selection technique and 81.53 % on 20-Newsgroups dataset using BNS as the feature selection technique compared to other state-of-the-art classifiers which signifies the usefulness of deep learning used by ML-ELM for classifying the text data. Experimental results on these benchmark datasets show the stability and effectiveness of our approach over other competing approaches.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Determined through experiment, iterating the values over a range and considered the value at which best results were obtained.
References
Bai Z, Huang G-B, Wang D, Wang H, Westover MB (2014) Sparse extreme learning machine for classification. IEEE Trans Cybern 44(10):1858–1870
Chen R-C, Hsieh C-H (2006) Web page classification based on a support vector machine using a weighted vote schema. Expert Syst Appl 31(2):427–435
Ding S, Xu X, Nie R (2014) Extreme learning machine and its applications. Neural Comput Appl 25(3–4):549–556
Ding S, Zhang N, Xu X, Guo L, Zhang J (2015) Deep extreme learning machine and its application in EEG classification. Math Probl Eng 2015
Forman G (2003) An extensive empirical study of feature selection metrics for text classification. J Mach Learn Res 3:1289–1305
Gomez JC, Moens M-F (2012) Hierarchical classification of web documents by stratified discriminant analysis. In: Multidisciplinary information retrieval. Springer, pp 94–108
Gopal S, Yang Y (2013) Recursive regularization for large-scale classification with hierarchical and graphical dependencies. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 257–265
Guo G, Wang H, Bell D, Bi Y, Greer K (2003) Knn model-based approach in classification. In: On the move to meaningful internet systems, (2003) CoopIS, DOA, and ODBASE. Springer, pp 986–996
Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Huang G-B (2003) Learning capability and storage capacity of two-hidden-layer feedforward networks. IEEE Trans Neural Netw 14(2):274–281
Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489–501
Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42:513–529
Im Kim K, Park HR (2009) Svd-lda: a combined model for text classification. J Inf Process Syst 5(1):5–10
Kasun LLC, Zhou H, Huang G-B, Vong CM (2013) Representational learning with extreme learning machine for big data. IEEE Intell Syst 28(6):31–34
Ke W (2012) Least information document representation for automated text classification. Proc Am Soc Inf Sci Technol 49(1):1–10
Klassen M, Paturi N (2010) Web document classification by keywords using random forests. In: Networked digital technologies. Springer, pp 256–261
Liang N-Y, Huang G-B, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423
Lingras P, Butz C (2007) Rough set based 1-v-1 and 1-vr approaches to support vector machine multi-classification. Inf Sci 177(18):3782–3798
Li L, Song D, Liao L (2012) Vertical classification of web pages for structured data extraction. In: Information retrieval technology. Springer, pp 486–495
Liu X, Gao C, Li P (2012) A comparative analysis of support vector machines and extreme learning machines. Neural Netw 33:58–66
Manning CD, Raghavan P, Schütze H et al (2008) Introduction to information retrieval, vol 1. Cambridge university press, Cambridge
Mirza B, Kok S, Dong F (2016) Multi-layer online sequential extreme learning machine for image classification. In: Proceedings of ELM-2015 vol 1. Springer, pp 39–49
Oh H-S, Choi Y, Myaeng S-H (2011) Text classification for a large-scale taxonomy using dynamically mixed local and global models for a node. In: Advances in information retrieval. Springer, pp 7–18
Rifkin R, Yeo G, Poggio T (2003) Regularized least-squares classification. Nato Sci Ser Sub Ser III Comput Syst Sci 190:131–154
Rujiang B, Xiaoyue W, Zewen H (2011) A novel web pages classification model based on integrated ontology. In: Software engineering, business continuity, and education. Springer, pp 1–10
Song Y, Roth D (2014) On dataless hierarchical text classification. In: AAAI. pp 1579–1585
Tang J, Deng C, Huang G-B, Hou J (2014) A fast learning algorithm for multi-layer extreme learning machine. In: International Conference on IEEE Image Processing (ICIP). pp 175–178
Wan CH, Lee LH, Rajkumar R, Isa D (2012) A hybrid text classification approach with low dependency on parameter by integrating k-nearest neighbor and support vector machine. Expert Syst Appl 39(15):11 880–11 888
Xue G-R, Xing D, Yang Q, Yu Y (2008) Deep classification in large-scale text hierarchies. In: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 619–626
Yang Y, Pedersen JO (1997) A comparative study on feature selection in text categorization. ICML 97:412–420
Yang Y, Wu Q (2015) Multilayer extreme learning machine with subnetwork nodes for representation learning. IEEE Trans Cybern 99:1–14
Zhang J, Niu Y, Nie H (2009) Web document classification based on fuzzy k-nn algorithm,” In: CIS’09. International conference on computational intelligence and security, 2009, vol 1. IEEE, pp 193–196
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Human and animal rights
This article does not contain any studies with human participants or animals performed by any of the authors.
Conflict of interest
None.
Additional information
Communicated by A. Di Nola.
Rights and permissions
About this article
Cite this article
Roul, R.K., Asthana, S.R. & Kumar, G. Study on suitability and importance of multilayer extreme learning machine for classification of text data. Soft Comput 21, 4239–4256 (2017). https://doi.org/10.1007/s00500-016-2189-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-016-2189-8