Abstract
This paper addresses text categorization problem that training data may be derived from a different time period than test data. We present a method for text categorization that minimizes the impact of temporal effects by using term smoothing and transfer learning techniques. We first used a technique called Temporal-based Term Smoothing (TTS) to replace those time sensitive features with representative terms, then applied boosting based transfer learning algorithm called TrAdaBoost for categorization. The results using a 21-year Japanese Mainichi Newspaper corpus showed that integrating term smoothing and transfer learning improves overall performance, especially it is effective when the creation time period of the test data differs greatly from the training data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Blitzer, J., McDonald, R., Pereira, F.: Domain adaptation with structural correspondence learning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 120–128 (2006)
Dai, W., Xue, G.R., Yang, Q., Yu, Y.: Co-clustering based classification for out-of-domain documents. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 210–219 (2007)
Dai, W., Yang, Q., Xue, G.R., Yu, Y.: Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 193–200 (2007)
Daume III, H.: Frustratingly easy domain adaptation. In: Proceedings of the 45th Annual Meeting of the Association of computational Linguistics, pp. 256–263 (2007)
Elkan, C., Noto, K.: Learning classifiers from only positive and unlabeled data. In: Proceedings of the KDD 2008, pp. 213–220 (2008)
Folino, G., Pizzuti, C., Spezzano, G.: An adaptive distributed ensemble approach to mine concept-drifting data streams. In: Proceedings of the 19th IEEE International Conference on Tools with Artificial Intelligence, pp. 183–188 (2007)
Forman, G.: An extensive empirical study of feature selection metrics for text classification. Mach. Learn. Res. 3, 1289–1305 (2003)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)
Fan, F.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: a library for large linear classification. Mach. Learn. 9, 1871–1874 (2008)
Glorot, X., Bordes, A., Bengio, Y.: Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th International Conference on Machine Learning, pp. 97–110 (2011)
He, D., Parker, D.S.: Topic dynamics: an alternative model of bursts in streams of topics. In: Proceedings of the 16th ACM SIGKDD Conference on Knowledge discovery and Data Mining, pp. 443–452 (2010)
Joachims, T.: SVM Light Support Vector Machine. Department of Computer Science Cornell University (1998)
Kleinberg, M.: Bursty and hierarchical structure in streams. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 91–101 (2002)
Klinkenberg, R., Joachims, T.: Detecting concept drift with support vector machines. In: Proceedings of the 17th International Conference on Machine Learning, pp. 487–494 (2000)
Kudo, T., Matsumoto, Y.: Fast methods for kernel-based text analysis. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pp. 24–31 (2003)
Lazarescu, M.M., Venkatesh, S., Bui, H.H.: Using multiple windows to track concept drift. Intel. Data Anal. 8(1), 25–59 (2004)
Li, Y., Yang, M., Zhang, Z.: Scientific articles recommendation. In: Proceedings of the ACM International Conference on Information and Knowledge Management CIKM 2013, pp. 1147–1156 (2013)
Liu, B., Dai, Y., Li, X., Lee, W.S., Yu, P.S.: Building text classifiers using positive and unlabeled examples. In: Proceedings of the ICDM 2003, pp. 179–188 (2003)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the International Conference on Learning Representations Workshop (2013)
Mourao, F., Rocha, L., Araujo, R., Couto, T., Goncalves, M., Meira, Jr., W.: Understanding temporal aspects in document classification. In: Proceedings of the 1st ACM International Conference on Web Search and Data Mining, pp. 159–169 (2008)
Murphy, J.: Technical Analysis of the Financial Markets, Prentice Hall, New Jersey (1999)
Raina, R., Ng, A.Y., Koller, D.: Constructing informative priors using transfer learning. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 713–720 (2006)
Salles, T., Rocha, L., Pappa, G.L.: Temporally-aware algorithms for document classification. In: Proceedings of the 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 307–314 (2010)
Sparinnapakorn, K., Kubat, M.: Combining subclassifiers in text categorization: a dst-based solution and a case study. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 210–219 (2007)
Wang, C., Blei, D., Heckerman, D.: Continuous time dynamic topic models. In: Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, pp. 579–586 (2008)
Siao, M., Guo, Y.: Domain adaptation for sequence labeling tasks with a probabilistic language adaptation model. In: Proceedings of the 30th International Conference on Machine Learning, pp. 293–301 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Fukumoto, F., Suzuki, Y. (2015). Smoothing Temporal Difference for Text Categorization. In: Zuccon, G., Geva, S., Joho, H., Scholer, F., Sun, A., Zhang, P. (eds) Information Retrieval Technology. AIRS 2015. Lecture Notes in Computer Science(), vol 9460. Springer, Cham. https://doi.org/10.1007/978-3-319-28940-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-28940-3_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28939-7
Online ISBN: 978-3-319-28940-3
eBook Packages: Computer ScienceComputer Science (R0)