research-article

Incorporating Multi-Level User Preference into Document-Level Sentiment Classification

Authors:

Chengqing ZongAuthors Info & Claims

ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), Volume 18, Issue 1

Article No.: 7, Pages 1 - 17

https://doi.org/10.1145/3234512

Published: 19 November 2018 Publication History

Abstract

Document-level sentiment classification aims to predict a user’s sentiment polarity in a document about a product. Most existing methods only focus on review contents and ignore users who post reviews. In fact, when reviewing a product, different users have different word-using habits to express opinions (i.e., word-level user preference), care about different attributes of the product (i.e., aspect-level user preference), and have different characteristics to score the review (i.e., polarity-level user preference). These preferences have great influence on interpreting the sentiment of text. To address this issue, we propose a model called Hierarchical User Attention Network (HUAN), which incorporates multi-level user preference into a hierarchical neural network to perform document-level sentiment classification. Specifically, HUAN encodes different kinds of information (word, sentence, aspect, and document) in a hierarchical structure and imports user embedding and user attention mechanism to model these preferences. Empirical results on two real-world datasets show that HUAN achieves state-of-the-art performance. Furthermore, HUAN can also mine important attributes of products for different users.

References

[1]

Mohammad Al Boni, Keira Qi Zhou, Hongning Wang, and Matthew S. Gerber. 2015. Model adaptation for personalized opinion analysis. In Proceedings of the 2015 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 769--774.

[2]

Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, and Zhiyuan Liu. 2016. Neural sentiment classification with user and product attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1650--1659.

[3]

Dan C. Ciresan, Ueli Meier, and Jürgen Schmidhuber. 2012. Multi-column deep neural networks for image classification. In Proceedings of the 2012 Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 3642--3649.

Digital Library

[4]

Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12 (2011), 2493--2537.

Digital Library

[5]

George E. Dahl, Dong Yu, Li Deng, and Alex Acero. 2011. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Transactions on Audio Speech and Language Processing 20, 1 (2011), 30--42.

Digital Library

[6]

Zi-Yi Dou. 2017. Capturing user and product information for document level sentiment analysis with deep memory network. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 532--537.

[7]

Wenliang Gao, Naoki Yoshinaga, Nobuhiro Kaji, and Masaru Kitsuregawa. 2013. Modeling user leniency and product popularity for sentiment classification. In Proceedings of the 2016 International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, 1107--1111.

[8]

Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T. Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1602--1613.

[9]

Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. 2017. An unsupervised neural attention model for aspect extraction. In Proceedings of the 2017 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 388--397.

[10]

Sepp Hochreiter and Jãijrgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735--1780.

Digital Library

[11]

Xia Hu, Lei Tang, Jiliang Tang, and Huan Liu. 2013. Exploiting social relations for sentiment analysis in microblogging. In Proceedings of the 2013 International Conference on Web Search and Data Mining. ACM, 537--546.

Digital Library

[12]

Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent Twitter sentiment classification. In Proceedings of the 2011 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 151--160.

Digital Library

[13]

Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1746--1751.

[14]

Svetlana Kiritchenko, Xiaodan Zhu, and Saif M Mohammad. 2014. Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research 50 (2014), 723--762.

[15]

Jiwei Li, Thang Luong, Dan Jurafsky, and Eduard Hovy. 2015. When are tree structures necessary for deep learning of representations. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2304--2314.

[16]

Junjie Li, Haitong Yang, and Chengqing Zong. 2016. Sentiment classification of social media text considering user attributes. In Proceedings of the 2016 Conference on Natural Language Processing and Chinese Computing. Springer, 583--594.

[17]

Shoushan Li, Chu-Ren Huang, and Chengqing Zong. 2011. Multi-domain sentiment classification with classifier combination. Journal of Computer Science and Technology 26, 1 (2011), 25--33.

Digital Library

[18]

Shoushan Li, Shengfeng Ju, Guodong Zhou, and Xiaojun Li. 2012. Active learning for imbalanced sentiment classification. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, 139--148.

Digital Library

[19]

Shoushan Li, Sophia Yat Mei Lee, Ying Chen, Chu-Ren Huang, and Guodong Zhou. 2010. Sentiment classification and polarity shifting. In Proceedings of the 2010 International Conference on Computational Linguistics. COLING 2010 Organizing Committee, 635--643.

Digital Library

[20]

Shoushan Li, Yunxia Xue, Zhongqing Wang, and Guodong Zhou. 2013. Active learning for cross-domain sentiment classification. In Proceedings of the 2013 International Joint Conference on Artificial Intelligence. AAAI Press, 2127--2133.

Digital Library

[21]

Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies 5, 1 (2012), 1--167.

[22]

Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of the 2014 Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 55--60.

[23]

Tomas Mikolov, Greg Corrado, Kai Chen, Jeffrey Dean, Tomas Mikolov, Greg Corrado, Kai Chen, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781 (2013).

[24]

Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval 2, 1--2 (2008), 1--135.

Digital Library

[25]

Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 79--86.

Digital Library

[26]

Martin F. Porter. 1980. An algorithm for suffix stripping. Program 14, 3 (1980), 130--137.

[27]

Lizhen Qu, Georgiana Ifrim, and Gerhard Weikum. 2010. The bag-of-opinions method for review rating prediction from sparse text patterns. In Proceedings of the 2010 International Conference on Computational Linguistics. COLING 2010 Organizing Committee, 913--921.

Digital Library

[28]

Richard Socher, Brody Huval, Christopher D. Manning, and Andrew Y. Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, 1201--1211.

Digital Library

[29]

Richard Socher, Jeffrey Pennington, Eric H. Huang, Andrew Y. Ng, and Christopher D. Manning. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 151--161.

Digital Library

[30]

Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1631--1642.

[31]

Kaisong Song, Shi Feng, Wei Gao, Daling Wang, Ge Yu, and Kam Fai Wong. 2015. Personalized sentiment classification based on latent individuality of microblog users. In Proceedings of the 2015 International Joint Conference on Artificial Intelligence. AAAI Press, 2277--2283.

Digital Library

[32]

Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the 2015 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1556--1566.

[33]

Chenhao Tan, Lillian Lee, Jie Tang, Long Jiang, Ming Zhou, and Ping Li. 2011. User-level sentiment analysis incorporating social networks. In Proceedings of the 2011 International Conference on Knowledge Discovery and Data Mining. ACM, 1397--1405.

Digital Library

[34]

Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1422--1432.

[35]

Duyu Tang, Bing Qin, and Ting Liu. 2015. Learning semantic representations of users and products for document level sentiment classification. In Proceedings of the 2015 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1014--1023.

[36]

Duyu Tang, Bing Qin, Yuekui Yang, and Yuekui Yang. 2015. User modeling with neural network for review rating prediction. In Proceedings of the 2015 International Joint Conference on Artificial Intelligence. AAAI Press, 1340--1346.

Digital Library

[37]

Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. Learning sentiment-specific word embedding for Twitter sentiment classification. In Proceedings of the 2014 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1555--1565.

[38]

Hongning Wang, Yue Lu, and Chengxiang Zhai. 2010. Latent aspect rating analysis on review text data: A rating regression approach. In Proceedings of the 2010 International Conference on Knowledge Discovery and Data Mining. ACM, 783--792.

Digital Library

[39]

Lu Wang and Wang Ling. 2016. Neural network-based abstract generation for opinions and arguments. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 47--57.

[40]

Linlin Wang, Kang Liu, Zhu Cao, Jun Zhao, and Gerard de Melo. 2015. Sentiment-aspect extraction based on restricted Boltzmann machines. In Proceedings of the 2015 Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 616--625.

[41]

Rui Xia, Feng Xu, Jianfei Yu, Yong Qi, and Erik Cambria. 2016. Polarity shift detection, elimination and ensemble: A three-stage model for document-level sentiment analysis. Information Processing and Management 52, 1 (2016), 36--45.

Digital Library

[42]

Rui Xia, Feng Xu, Chengqing Zong, Qianmu Li, Yong Qi, and Tao Li. 2015. Dual sentiment analysis: Considering two sides of one review. IEEE Transactions on Knowledge and Data Engineering 27, 8 (2015), 2120--2133.

[43]

Rui Xia and Chengqing Zong. 2010. Exploring the use of word relation features for sentiment classification. In Proceedings of the 2010 International Conference on Computational Linguistics. Association for Computational Linguistics, 1336--1344.

Digital Library

[44]

Rui Xia, Chengqing Zong, and Shoushan Li. 2011. Ensemble of feature sets and classification algorithms for sentiment classification. Information Sciences 181, 6 (2011), 1138--1152.

Digital Library

[45]

Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 1480--1489.

[46]

Matthew D. Zeiler. 2012. ADADELTA: An adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012).

Cited By

Damade VArya E(2023)A Study of Literature on Deep Learning-Driven Sentimental Evaluation in Text2023 2nd International Conference on Futuristic Technologies (INCOFT)10.1109/INCOFT60753.2023.10425003(1-7)Online publication date: 24-Nov-2023
https://doi.org/10.1109/INCOFT60753.2023.10425003
Fu MPan L(2022)Sentiment Analysis of Tourist Scenic Spots Internet Comments Based on LSTMMathematical Problems in Engineering10.1155/2022/59449542022(1-9)Online publication date: 18-Jul-2022
https://doi.org/10.1155/2022/5944954
Wang XZhou TWang XFang Y(2022)Harshness-aware sentiment mining framework for product reviewExpert Systems with Applications10.1016/j.eswa.2021.115887187(115887)Online publication date: Jan-2022
https://doi.org/10.1016/j.eswa.2021.115887
Show More Cited By

Index Terms

Incorporating Multi-Level User Preference into Document-Level Sentiment Classification
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Sentiment analysis

Recommendations

Aspect-level Sentiment Classification with HEAT (HiErarchical ATtention) Network
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Aspect-level sentiment classification is a fine-grained sentiment analysis task, which aims to predict the sentiment of a text in different aspects. One key point of this task is to allocate the appropriate sentiment words for the given aspect.Recent ...
Sentence-level Sentiment Classification with Weak Supervision
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

Sentence-level sentiment classification is important to understand users' fine-grained opinions. Existing methods for sentence-level sentiment classification are mainly based on supervised learning. However, it is difficult to obtain sentiment labels of ...
Exploring overall opinions for document level sentiment classification with structural SVM

As a fundamental task of sentiment analysis, document level sentiment classification aims to predict user's overall sentiment (e.g., positive or negative) towards the target in a document. The document usually consists of various opinion sentences ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Asian and Low-Resource Language Information Processing

ACM Transactions on Asian and Low-Resource Language Information Processing Volume 18, Issue 1

March 2019

196 pages

ISSN:2375-4699

EISSN:2375-4702

DOI:10.1145/3292011

Editor:
Nianwen Xue
Brandeis University, Waltham, USA

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 November 2018

Accepted: 01 June 2018

Revised: 01 March 2018

Received: 01 November 2017

Published in TALLIP Volume 18, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Key Research and Development Program of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
351
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)3

Reflects downloads up to 28 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Damade VArya E(2023)A Study of Literature on Deep Learning-Driven Sentimental Evaluation in Text2023 2nd International Conference on Futuristic Technologies (INCOFT)10.1109/INCOFT60753.2023.10425003(1-7)Online publication date: 24-Nov-2023
https://doi.org/10.1109/INCOFT60753.2023.10425003
Fu MPan L(2022)Sentiment Analysis of Tourist Scenic Spots Internet Comments Based on LSTMMathematical Problems in Engineering10.1155/2022/59449542022(1-9)Online publication date: 18-Jul-2022
https://doi.org/10.1155/2022/5944954
Wang XZhou TWang XFang Y(2022)Harshness-aware sentiment mining framework for product reviewExpert Systems with Applications10.1016/j.eswa.2021.115887187(115887)Online publication date: Jan-2022
https://doi.org/10.1016/j.eswa.2021.115887
Gu XXia KJiang YJolfaei A(2021)Multi-task Fuzzy Clustering–Based Multi-task TSK Fuzzy System for Text Sentiment ClassificationACM Transactions on Asian and Low-Resource Language Information Processing10.1145/347610321:2(1-24)Online publication date: 18-Nov-2021
https://dl.acm.org/doi/10.1145/3476103

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents