research-article

Aspect-level Sentiment Classification with HEAT (HiErarchical ATtention) Network

Authors:

Hui WangAuthors Info & Claims

CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Pages 97 - 106

https://doi.org/10.1145/3132847.3133037

Published: 06 November 2017 Publication History

Abstract

Aspect-level sentiment classification is a fine-grained sentiment analysis task, which aims to predict the sentiment of a text in different aspects. One key point of this task is to allocate the appropriate sentiment words for the given aspect.Recent work exploits attention neural networks to allocate sentiment words and achieves the state-of-the-art performance. However, the prior work only attends to the sentiment information and ignores the aspect-related information in the text, which may cause mismatching between the sentiment words and the aspects when an unrelated sentiment word is semantically meaningful for the given aspect. To solve this problem, we propose a HiErarchical ATtention (HEAT) network for aspect-level sentiment classification. The HEAT network contains a hierarchical attention module, consisting of aspect attention and sentiment attention. The aspect attention extracts the aspect-related information to guide the sentiment attention to better allocate aspect-specific sentiment words of the text. Moreover, the HEAT network supports to extract the aspect terms together with aspect-level sentiment classification by introducing the Bernoulli attention mechanism. To verify the proposed method, we conduct experiments on restaurant and laptop review data sets from SemEval at both the sentence level and the review level. The experimental results show that our model better allocates appropriate sentiment expressions for a given aspect benefiting from the guidance of aspect terms. Moreover, our method achieves better performance on aspect-level sentiment classification than state-of-the-art models.

References

[1]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. Computer Science (2014).

[2]

Danushka Bollegala, David Weir, and John Carroll. 2013. Cross-domain sentiment classification using a sentiment sensitive thesaurus. Knowledge and Data Engineering, IEEE Transactions on 25, 8 (2013), 1719--1731.

Digital Library

[3]

Jiajun Cheng, Xin Zhang, Pei Li, Sheng Zhang, Zhaoyun Ding, and Hui Wang. 2016. Exploring sentiment parsing of microblogging texts for opinion polling on chinese public figures. Applied Intelligence (2016), 1--14.

[4]

Kyunghyun Cho, Bart Van MerriÃÃnboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).

[5]

Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).

Digital Library

[6]

David Golub and Xiaodong He. 2016. Character-level question answering with attention. arXiv preprint arXiv:1604.00727 (2016).

[7]

Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 507--517.

Digital Library

[8]

Sepp Hochreiter and Jijrgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780.

Digital Library

[9]

Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 151--160.

Digital Library

[10]

Rafal Jozefowicz, Wojciech Zaremba, and Ilya Sutskever. 2015. An Empirical Exploration of Recurrent Network Architectures. In Proceedings of the 32nd International Conference on Machine Learning (ICML). 2342--2350.

Digital Library

[11]

Yoon Kim, Carl Denton, Luong Hoang, and Alexander M Rush. 2017. Structured attention networks. arXiv preprint arXiv:1702.00887 (2017).

[12]

Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif Mohammad. 2014. NRC-Canada-2014: Detecting aspects and sentiment in customer reviews. In SemEval 2014. 437--442.

[13]

Svetlana Kiritchenko, Xiaodan Zhu, and Saif M Mohammad. 2014. Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research 50 (2014), 723--762.

[14]

Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, and Richard Socher. 2016. Ask Me Anything: Dynamic Memory Networks for Natural Language Processing. In Proceedings of The 33rd International Conference on Machine Learning. 1378--1387.

Digital Library

[15]

Siwei Lai, Kang Liu, Shizhu He, and Jun Zhao. 2016. How to generate a good word embedding. IEEE Intelligent Systems 31, 6 (2016), 5--14.

Digital Library

[16]

Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A Structured Self-attentive Sentence Embedding. arXiv preprint arXiv:1703.03130 (2017).

[17]

Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies 5, 1 (2012), 1--167.

[18]

Minh Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. Computer Science (2015).

[19]

Haitao Mi, Zhiguo Wang, and Abe Ittycheriah. 2016. Supervised attentions for neural machine translation. arXiv preprint arXiv:1608.00112 (2016).

[20]

P. Nema, M. Khapra, A. Laha, and B. Ravindran. 2017. Diversity driven Attention Model for Query-based Abstractive Summarization. ArXiv e-prints arXiv:1704.08300 (April 2017).

[21]

Thien Hai Nguyen and Kiyoaki Shirai. PhraseRNN: Phrase Recursive Neural Network for Aspect-based Sentiment Analysis. In EMNLP. 2509--2514.

[22]

Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammad Al-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, and Orphee De Clercq. SemEval-2016 Task 5: Aspect Based Sentiment Analysis. In International Workshop on Semantic Evaluation.

[23]

Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. SemEval-2015 Task 12: Aspect Based Sentiment Analysis. (2015).

[24]

Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 Task 4: Aspect Based Sentiment Analysis. Proceedings of International Workshop on Semantic Evaluation at (2014), 27--35.

[25]

Guang Qiu, Bing Liu, Jiajun Bu, and Chun Chen. 2011. Opinion word expansion and target extraction through double propagation. Computational linguistics 37, 1 (2011), 9--27.

Digital Library

[26]

Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A Neural Attention Model for Abstractive Sentence Summarization. Computer Science (2015).

[27]

Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, Vol. 1631. Cite- seer, 1642.

[28]

Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. 2015. End-To-End Memory Networks. Computer Science (2015).

[29]

Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1422--1432.

[30]

Duyu Tang, Bing Qin, and Ting Liu. 2016. Aspect level sentiment classification with deep memory network. arXiv preprint arXiv:1605.08900 (2016).

[31]

Wenya Wang, Sinno Jialin Pan, Daniel Dahlmeier, and Xiaokui Xiao. Coupled Multi-layer Attentions for Co-extraction of Aspect and Opinion Terms. In 31st AAAI Conference on Artificial Intelligence (AAAI-17). AAAI Press.

[32]

Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016. Attention-based LSTM for Aspect-level Sentiment Classification. In Conference on Empirical Methods in Natural Language Processing. 606--615.

[33]

Jason Weston, Sumit Chopra, and Antoine Bordes. 2014. Memory networks. arXiv preprint arXiv:1410.3916 (2014).

[34]

Haibing Wu and Xiaodong Gu. 2014. Reducing Over-Weighting in Supervised Term Weighting for Sentiment Analysis. In COLING. 1322--1330.

[35]

Haibing Wu, Yiwei Gu, Shangdi Sun, and Xiaodong Gu. 2016. Aspect-based Opinion Summarization with Convolutional Neural Networks. In 2016 International Joint Conference on Neural Networks (IJCNN). IEEE, 3157--3163.

[36]

Yuanbin Wu, Qi Zhang, Xuanjing Huang, and Lide Wu. 2009. Phrase dependency parsing for opinion mining. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3-Volume 3. Association for Computational Linguistics, 1533--1541.

Digital Library

[37]

Caiming Xiong, Victor Zhong, and Richard Socher. 2016. Dynamic Coattention Networks For Question Answering. arXiv preprint arXiv:1611.01604 (2016).

[38]

Bishan Yang and Claire Cardie. 2013. Joint Inference for Fine-grained Opinion Extraction. In ACL (1). 1640--1649.

[39]

Wenpeng Yin, Mo Yu, Bing Xiang, Bowen Zhou, and Hinrich Schütze. 2016. Simple Question Answering by Attentive Convolutional Neural Network. arXiv preprint arXiv:1606.03391 (2016).

[40]

Jianfei Yu and Jing Jiang. 2016. Learning sentence embeddings with auxiliary tasks for cross-domain sentiment classification. Association for Computational Linguistics.

[41]

Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017. Dynamic Key-Value Memory Networks for Knowledge Tracing. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 765--774.

Digital Library

Cited By

Mamta Ekbal A(2025)Quality achhi hai (is good), satisfied! Towards aspect based sentiment analysis in code-mixed languageComputer Speech & Language10.1016/j.csl.2024.10166889(101668)Online publication date: Jan-2025
https://doi.org/10.1016/j.csl.2024.101668
Liu YLi J(2024)An Online Hotel Selection Method With Three-Dimensional Analysis of Reviews' HelpfulnessInternational Journal of Fuzzy System Applications10.4018/IJFSA.34349013:1(1-25)Online publication date: 15-May-2024
https://doi.org/10.4018/IJFSA.343490
Gao YZhang LXu Y(2024)CKG: Improving ABSA with text augmentation using ChatGPT and knowledge-enhanced gated attention graph convolutional networksPLOS ONE10.1371/journal.pone.030150819:6(e0301508)Online publication date: 27-Jun-2024
https://doi.org/10.1371/journal.pone.0301508
Show More Cited By

Index Terms

Aspect-level Sentiment Classification with HEAT (HiErarchical ATtention) Network
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Sentiment analysis
  2. Information systems applications
    1. Data mining

Recommendations

Sentence-level Sentiment Classification with Weak Supervision
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

Sentence-level sentiment classification is important to understand users' fine-grained opinions. Existing methods for sentence-level sentiment classification are mainly based on supervised learning. However, it is difficult to obtain sentiment labels of ...
Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification
Abstract
Aspect-level sentiment classification aims to distinguish the sentiment polarities over one or more aspect terms in a sentence. Existing approaches mostly model different aspects in one sentence independently, which ignore the sentiment ...
Domain customization for aspect-oriented opinion analysis with multi-level latent sentiment clues
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

Aspect-oriented opinion mining detects the reviewers' sentiment orientation (e.g. positive, negative or neutral) towards different product-features. Domain customization is a big challenge for opinion mining due to the accuracy loss across domains. In ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

November 2017

2604 pages

ISBN:9781450349185

DOI:10.1145/3132847

General Chairs:
Ee-Peng Lim
Singapore Management University, Singapore
,
Marianne Winslett
University of Illinois at Urbana-Champaign, USA, and Advanced Digital Sciences Center, Singapore
,
Program Chairs:
Mark Sanderson
RMIT, Australia
,
Ada Fu
Chinese University of Hong Kong, Hong Kong
,
Jimeng Sun
Georgia Tech, USA
,
Shane Culpepper
RMIT, Australia
,
Eric Lo
Chinese University of Hong Kong, Hong Kong
,
Joyce Ho
Emory University, USA
,
Debora Donato
Mix Tech, Inc., USA
,
Rakesh Agrawal
Data Insights Laboratories, USA
,
Yu Zheng
Microsoft Research Asia, China
,
Carlos Castillo
Qatar Computing Research Institute, Qatar
,
Aixin Sun
Nanyang Technological University, Singapore
,
Vincent S. Tseng
National Cheng Kung University, Taiwan
,
Chenliang Li
Wuhan University, China

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Conference

CIKM '17

Sponsor:

CIKM '17: ACM Conference on Information and Knowledge Management

November 6 - 10, 2017

Singapore, Singapore

Acceptance Rates

CIKM '17 Paper Acceptance Rate 171 of 855 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

89
Total Citations
View Citations
1,432
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)4

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mamta Ekbal A(2025)Quality achhi hai (is good), satisfied! Towards aspect based sentiment analysis in code-mixed languageComputer Speech & Language10.1016/j.csl.2024.10166889(101668)Online publication date: Jan-2025
https://doi.org/10.1016/j.csl.2024.101668
Liu YLi J(2024)An Online Hotel Selection Method With Three-Dimensional Analysis of Reviews' HelpfulnessInternational Journal of Fuzzy System Applications10.4018/IJFSA.34349013:1(1-25)Online publication date: 15-May-2024
https://doi.org/10.4018/IJFSA.343490
Gao YZhang LXu Y(2024)CKG: Improving ABSA with text augmentation using ChatGPT and knowledge-enhanced gated attention graph convolutional networksPLOS ONE10.1371/journal.pone.030150819:6(e0301508)Online publication date: 27-Jun-2024
https://doi.org/10.1371/journal.pone.0301508
Pan T(2024)Cyberbullying detection based on aspect-level sentiment analysisProceedings of the 2024 3rd International Conference on Cryptography, Network Security and Communication Technology10.1145/3673277.3673312(200-204)Online publication date: 19-Jan-2024
https://dl.acm.org/doi/10.1145/3673277.3673312
Xu JXie JCai YLin ZLeung HLi QChua T(2024)Context-Aware Dynamic Word Embeddings for Aspect Term ExtractionIEEE Transactions on Affective Computing10.1109/TAFFC.2023.326294115:1(144-156)Online publication date: Jan-2024
https://doi.org/10.1109/TAFFC.2023.3262941
Yu MPeng FZhao YZhang WYu JZhao M(2024)IDSV-GCN: Integrating Dual Syntactic Views Graph Convolutional Network for aspect-based sentiment analysisKnowledge-Based Systems10.1016/j.knosys.2024.112656305(112656)Online publication date: Dec-2024
https://doi.org/10.1016/j.knosys.2024.112656
Jian ZLi JWu QYao J(2024)Retrieval Contrastive Learning for Aspect-Level Sentiment ClassificationInformation Processing & Management10.1016/j.ipm.2023.10353961:1(103539)Online publication date: Jan-2024
https://doi.org/10.1016/j.ipm.2023.103539
You LPeng JJin HClaramunt CZeng HZhang Z(2024)DRGAT: Dual-Relational Graph Attention Networks for Aspect-based Sentiment ClassificationInformation Sciences10.1016/j.ins.2024.120531(120531)Online publication date: Mar-2024
https://doi.org/10.1016/j.ins.2024.120531
Wankhade MKulkarni CRao A(2024)A survey on aspect base sentiment analysis methods and challengesApplied Soft Computing10.1016/j.asoc.2024.112249167(112249)Online publication date: Dec-2024
https://doi.org/10.1016/j.asoc.2024.112249
Jatain DNiranjanamurthy MDayananda P(2024)A Hybrid Bio-inspired Fuzzy Feature Selection Approach for Opinion Mining of Learner CommentsSN Computer Science10.1007/s42979-023-02526-15:1Online publication date: 2-Jan-2024
https://doi.org/10.1007/s42979-023-02526-1
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents