short-paper

Chinese Document Classification with Bi-directional Convolutional Language Model

Authors:

Guosheng YinAuthors Info & Claims

SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1785 - 1788

https://doi.org/10.1145/3397271.3401248

Published: 25 July 2020 Publication History

Abstract

By setting a typeface, each character of the Chinese text can be converted to a glyph pixel matrix. We propose to conduct text classification with such glyph features using bi-directional convolution. Although the pixel embedding can be applied to all languages, it is much more convenient to be used to represent Chinese scripts due to the square shape of Chinese characters. We extract both the forward and backward n-gram features of the text via bi-directional convolutional operations and then concatenate them. A subsequent 1-dimensional max-over-time pooling is applied to the bi-directional feature maps, and then three fully connected layers are used for conducting text classification. The proposed model has a light-weight architecture that only contains a single-layer convolutional neural network. Experiments on several Chinese text classification datasets demonstrate surprisingly excellent results for the training speed and superior performance of the proposed model in comparison with traditional methods.

Supplementary Material

MP4 File (3397271.3401248.mp4)

Video file

Download
14.80 MB

References

[1]

Ronan Collobert, Jason Weston, L?on Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa. 2011. Natural Language Processing (Almost) from Scratch. Journal of Machine Learning Research, Vol. 12 (2011), 2493--2537.

Digital Library

[2]

Yoav Goldberg. 2016. A primer on neural network models for natural language processing. Journal of Artificial Intelligence Research, Vol. 57 (2016), 345--420.

Digital Library

[3]

Alon Jacovi, Oren Sar Shalom, and Yoav Goldberg. 2018. Understanding Convolutional Neural Networks for Text Classification. In Empirical Methods in Natural Language Processing. 56--65.

[4]

Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of Tricks for Efficient Text Classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, Vol. 2. 427--431.

[5]

Yuanzhi Ke and Masafumi Hagiwara. 2017. Radical-level Ideograph Encoder for RNN-based Sentiment Analysis of Chinese and Japanese. In Asian Conference on Machine Learning. 561--573.

[6]

Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1746--1751.

[7]

Yee Leung, Jiang-She Zhang, and Zong-Ben Xu. 2000. Clustering by scale-space filtering. IEEE Transactions on pattern analysis and machine intelligence, Vol. 22, 12 (2000), 1396--1410.

Digital Library

[8]

Hui Li, Ye Liu, Nikos Mamoulis, and David S. Rosenblum. 2019. Translation-Based Sequential Recommendation for Complex Users on Sparse Data. IEEE Trans. Knowl. Data Eng. (2019).

[9]

Jingyang Li and Maosong Sun. 2007. Scalable Term Selection for Text Categorization. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). 774--782.

[10]

Frederick Liu, Han Lu, Chieh Lo, and Graham Neubig. 2017. Learning Character-level Compositionality with Visual Features. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 2059--2068.

[11]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems 26. 3111--3119.

[12]

Daiki Shimada, Ryunosuke Kotani, and Hitoshi Iyatomi. 2016. Document classification through image-based character embedding and wildcard training. In 2016 IEEE International Conference on Big Data (Big Data). IEEE, 3922--3927.

[13]

Tzuray Su and Hungyi Lee. 2017. Learning Chinese Word Representations From Glyphs Of Characters. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 264--273.

[14]

Chi Sun, Xipeng Qiu, and Xuanjing Huang. 2019. VCWE: Visual Character-enhanced Word Embeddings. In NAACL-HLT 2019: Annual Conference of the North American Chapter of the Association for Computational Linguistics. 2710--2719.

[15]

Xiang Zhang and Yann LeCun. 2017. Which Encoding is the Best for Text Classification in Chinese, English, Japanese and Korean? arXiv preprint arXiv:1708.02657 (2017).

[16]

Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in neural information processing systems. 649--657.

[17]

Liyuan Zheng, Yajie Hu, Bin Liu, and Wei Deng. 2020. Learning robust word representation over a semantic manifold. Knowledge-Based Systems, Vol. 192 (2020), 105358.

Cited By

Wagner TGuhl DLanghals B(2024)The Impact of Data Preparation and Model Complexity on the Natural Language Classification of Chinese News HeadlinesAlgorithms10.3390/a1704013217:4(132)Online publication date: 22-Mar-2024
https://doi.org/10.3390/a17040132
Mozumder MArmand TImtiyaj Uddin SAthar ASumon RHussain AKim H(2023)Metaverse for Digital Anti-Aging Healthcare: An Overview of Potential Use Cases Based on Artificial Intelligence, Blockchain, IoT Technologies, Its Challenges, and Future DirectionsApplied Sciences10.3390/app1308512713:8(5127)Online publication date: 20-Apr-2023
https://doi.org/10.3390/app13085127
Gokce Narin N(2023)The Role of Artificial Intelligence and Robotic Solution Technologies in Metaverse DesignMetaverse10.1007/978-981-99-4641-9_4(45-63)Online publication date: 13-Oct-2023
https://doi.org/10.1007/978-981-99-4641-9_4
Show More Cited By

Index Terms

Chinese Document Classification with Bi-directional Convolutional Language Model
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing

Recommendations

Chinese Text Classification Based on Hybrid Model of CNN and LSTM
DSIT 2020: Proceedings of the 3rd International Conference on Data Science and Information Technology

Text classification is one of the basic tasks of natural language processing. In recent years, deep learning has been widely used in text classification tasks. The representative one is the convolutional neural network. The convolutional neural network(...
Performance analysis of chinese cursive character recognition based on convolutional neural network
RACS '19: Proceedings of the Conference on Research in Adaptive and Convergent Systems

Chinese cursive characters written in old books are more difficult to recognize than other Chinese Characters such as handwritten Chinese character because they have many various styles. For this reason, it needs a software-based recognition model or ...
Chinese Text Feature Extraction and Classification Based on Deep Learning
CSAE '19: Proceedings of the 3rd International Conference on Computer Science and Application Engineering

With the rapid development of deep learning, neural networks have been widely used in natural language processing tasks and achieved good results. Since convolutional neural networks can acquire high-level features that can better represent textual ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2020

2548 pages

ISBN:9781450380164

DOI:10.1145/3397271

General Chairs:
Jimmy Huang
York University, Canada
,
Yi Chang
Jilin University, China
,
Xueqi Cheng
Chinese Academy of Sciences, China
,
Program Chairs:
Jaap Kamps
University of Amsterdam, Netherlands
,
Vanessa Murdock
Amazon, U.S.A.
,
Ji-Rong Wen
Renmin University of China, China
,
Yiqun Liu
Tsinghua University, China

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Fundamental Research Funds for the Central Universities
Research Grants Council of Hong Kong

Conference

SIGIR '20

Sponsor:

SIGIR

SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval

July 25 - 30, 2020

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
270
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wagner TGuhl DLanghals B(2024)The Impact of Data Preparation and Model Complexity on the Natural Language Classification of Chinese News HeadlinesAlgorithms10.3390/a1704013217:4(132)Online publication date: 22-Mar-2024
https://doi.org/10.3390/a17040132
Mozumder MArmand TImtiyaj Uddin SAthar ASumon RHussain AKim H(2023)Metaverse for Digital Anti-Aging Healthcare: An Overview of Potential Use Cases Based on Artificial Intelligence, Blockchain, IoT Technologies, Its Challenges, and Future DirectionsApplied Sciences10.3390/app1308512713:8(5127)Online publication date: 20-Apr-2023
https://doi.org/10.3390/app13085127
Gokce Narin N(2023)The Role of Artificial Intelligence and Robotic Solution Technologies in Metaverse DesignMetaverse10.1007/978-981-99-4641-9_4(45-63)Online publication date: 13-Oct-2023
https://doi.org/10.1007/978-981-99-4641-9_4
The TPham QPham XDo‐Duy TReddy Gadekallu T(2023)AI and Computer Vision Technologies for MetaverseMetaverse Communication and Computing Networks10.1002/9781394160013.ch5(85-124)Online publication date: 6-Oct-2023
https://doi.org/10.1002/9781394160013.ch5

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents