research-article

Aspect term extraction for opinion mining using a Hierarchical Self-Attention Network

Authors:

Aditya Srikanth Veerubhotla,

Vishnu Teja Narapareddy,

Lalita Bhanu Murthy Neti,

Aruna MalapatiAuthors Info & Claims

Volume 465, Issue C

Pages 195 - 204

https://doi.org/10.1016/j.neucom.2021.08.133

Published: 20 November 2021 Publication History

Graphical abstract

Display Omitted

Highlights

•

We present a novel HSAN model for aspect identification task.

•

Compared with existing state-of-the-art models, HSAN takes significantly lesser training time.

•

Experimental results show that HSAN outperforms the state-of-the-art models.

•

We evaluate the impact of each attention layers on the performance of proposed HSAN.

Abstract

Aspect identification is one of the important sub-tasks in opinion mining and this task can be considered as a token-level sequencing problem. Most recent approaches employ BERT based network to identify the aspect term, which is often complex, consumes a lot of memory, and needs more training time. In this paper, we propose a novel Hierarchical Self-Attention Network (HSAN) which performs well, needs lesser memory and training time. HSAN hierarchically applies a self-attention mechanism to first capture the importance of each word in the context of the overall meaning of the sentence and then it explores the internal dependency of the words in the same sentence to identify interdependent collocated words. A fusion of these two-attention mechanisms helps HSAN to predict multiple aspect terms effectively in the given sentence along with multi-token aspect terms. Our proposed network uses word embeddings, which is a combination of general-purpose embeddings and domain-specific embeddings. We evaluate the performance of HSAN on SemEval-2014 datasets, experimental results demonstrate the efficiency and effectiveness of our model.

References

[1]

D. Bahdanau, K. Cho, Y. Bengio, Neural machine translation by jointly learning to align and translate, 2014, arXiv preprint arXiv:1409.0473.

[2]

S. Chen, J. Liu, Y. Wang, W. Zhang, Z. Chi, Synchronous double-channel recurrent network for aspect-opinion pair extraction, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 6515–6524.

[3]

Z. Chen, T. Qian, Enhancing aspect term extraction with soft prototypes, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 2107–2117.

[4]

M. Chernyshevich, Ihs r&d belarus: Cross-domain extraction of product features using conditional random fields, 2014.

[5]

H. Dai, Y. Song, Neural aspect and opinion term extraction with mined rules as weak supervision, 2019, arXiv preprint arXiv:1907.03750.

[6]

J. Devlin, M.W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, 2018, arXiv preprint arXiv:1810.04805.

[7]

A. Giannakopoulos, C. Musat, A. Hossmann, M. Baeriswyl, Unsupervised aspect term extraction with b-lstm & crf using automatically labelled datasets, 2017, arXiv preprint arXiv:1709.05094.

[8]

P. He, W. Huang, Y. Qiao, C.C. Loy, X. Tang, Reading scene text in deep convolutional sequences, in: Thirtieth AAAI Conference on Artificial Intelligence, 2016.

[9]

R. He, W.S. Lee, H.T. Ng, D. Dahlmeier, An unsupervised neural attention model for aspect extraction, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 388–397.

[10]

R. He, W.S. Lee, H.T. Ng, D. Dahlmeier, Exploiting document knowledge for aspect-level sentiment classification, 2018, arXiv preprint arXiv:1806.04346.

[11]

R. He, J. McAuley, Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering, in: Proceedings of the 25th International Conference on World Wide Web, 2016, pp. 507–517.

Digital Library

[12]

K.M. Hermann, T. Kocisky, E. Grefenstette, L. Espeholt, W. Kay, M. Suleyman, P. Blunsom, Teaching machines to read and comprehend, in: Advances in Neural Information Processing Systems, 2015, pp. 1693–1701.

[13]

M. Hu, B. Liu, Mining and summarizing customer reviews, in: Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004, pp. 168–177.

Digital Library

[14]

M. Hu, Y. Peng, Z. Huang, D. Li, Y. Lv, Open-domain targeted sentiment analysis via span-based extraction and classification, 2019, arXiv preprint arXiv:1906.03820.

[15]

S. Jebbara, P. Cimiano, Aspect-based relational sentiment analysis using a stacked neural network architecture, in: Proceedings of the Twenty-second European Conference on Artificial Intelligence, IOS Press, 2016, pp. 1123–1131.

[16]

J. Lafferty, A. McCallum, F. PEREIRA, Crf: Probalistic models for segmenting and labeling sequence data, in: Proceedings of the Eighteenth International Conference on Machine Learning (ICML-2001): IMCL, 2001.

[17]

X. Li, L. Bing, P. Li, W. Lam, A unified model for opinion target extraction and target sentiment prediction, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 6714–6721.

[18]

X. Li, L. Bing, P. Li, W. Lam, Z. Yang, Aspect term extraction with history attention and selective transformation, 2018, arXiv preprint arXiv:1805.00760.

[19]

X. Li, L. Bing, W. Zhang, W. Lam, Exploiting bert for end-to-end aspect-based sentiment analysis, 2019, arXiv preprint arXiv:1910.00883.

[20]

Z. Lin, M. Feng, C.N.d. Santos, M. Yu, B. Xiang, B. Zhou, Y. Bengio, A structured self-attentive sentence embedding, 2017, arXiv preprint arXiv:1703.03130.

[21]

P. Liu, S. Joty, H. Meng, Fine-grained opinion mining with recurrent neural networks and word embeddings, in: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, pp. 1433–1443.

[22]

T. Mikolov, E. Grave, P. Bojanowski, C. Puhrsch, A. Joulin, Advances in pre-training distributed word representations, 2017, arXiv preprint arXiv:1712.09405.

[23]

V. Mnih, N. Heess, A. Graves, et al., Recurrent models of visual attention, in: Advances in Neural Information Processing Systems, 2014, pp. 2204–2212.

[24]

J. Pennington, R. Socher, C.D. Manning, Glove: Global vectors for word representation, in: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp. 1532–1543.

[25]

M. Pontiki, D. Galanis, H. Papageorgiou, S. Manandhar, I. Androutsopoulos, Semeval-2015 task 12: Aspect based sentiment analysis, in: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), 2015, pp. 486–495.

[26]

S. Poria, E. Cambria, A. Gelbukh, Aspect extraction for opinion mining with a deep convolutional neural network, Knowledge-Based Systems 108 (2016) 42–49.

Digital Library

[27]

G. Qiu, B. Liu, J. Bu, C. Chen, Opinion word expansion and target extraction through double propagation, Computational Linguistics 37 (2011) 9–27.

Digital Library

[28]

A.M. Rush, S. Chopra, J. Weston, A neural attention model for abstractive sentence summarization, 2015, arXiv preprint arXiv:1509.00685.

[29]

A.H. Tai, W.K. Ching, L.Y. Chan, Detection of machine failure: Hidden markov model approach, Computers & Industrial Engineering 57 (2009) 608–619.

[30]

I. Titov, R. McDonald, A joint model of text and aspect ratings for sentiment summarization, in: Proceedings of ACL-08: HLT, 2008, pp. 308–316.

[31]

Z. Toh, W. Wang, Dlirec: Aspect term extraction and term polarity classification system, Association for Computational Linguistics and Dublin City University, Citeseer, 2014.

[32]

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: Advances in Neural Information Processing Systems, 2017, pp. 5998–6008.

[33]

W. Wang, S.J. Pan, D. Dahlmeier, X. Xiao, Recursive neural conditional random fields for aspect-based sentiment analysis, 2016, arXiv preprint arXiv:1603.06679.

[34]

W. Wang, S.J. Pan, D. Dahlmeier, X. Xiao, Coupled multi-layer attentions for co-extraction of aspect and opinion terms, in: Thirty-First AAAI Conference on Artificial Intelligence, 2017.

[35]

H. Xu, B. Liu, L. Shu, P.S. Yu, Double embeddings and cnn-based sequence labeling for aspect extraction, 2018, arXiv preprint arXiv:1805.04601.

[36]

H. Xu, B. Liu, L. Shu, P.S. Yu, Bert post-training for review reading comprehension and aspect-based sentiment analysis, 2019, arXiv preprint arXiv:1904.02232.

[37]

B. Yang, C. Cardie, Context-aware learning for sentence-level sentiment analysis with posterior regularization, in: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2014, pp. 325–335.

[38]

Y. Yin, F. Wei, L. Dong, K. Xu, M. Zhang, M. Zhou, Unsupervised word and dependency path embeddings for aspect term extraction, 2016, arXiv preprint arXiv:1605.07843.

[39]

M.D. Zeiler, Adadelta: an adaptive learning rate method, 2012, arXiv preprint arXiv:1212.5701.

Cited By

Chu CSo RKwong EChan A(2023)Leveraging rule-based model and machine learning transformer for mining aspect-based financial opinions in colloquial languageProceedings of the 2023 7th International Conference on Software and e-Business10.1145/3641067.3641075(71-78)Online publication date: 21-Dec-2023
https://dl.acm.org/doi/10.1145/3641067.3641075
Yuan LWang JYu LZhang X(2023)Encoding Syntactic Information into Transformers for Aspect-Based Sentiment Triplet ExtractionIEEE Transactions on Affective Computing10.1109/TAFFC.2023.329173015:2(722-735)Online publication date: 7-Jul-2023
https://dl.acm.org/doi/10.1109/TAFFC.2023.3291730
Liu BLin TLi M(2023)Improving aspect term extraction via span-level tag data augmentationApplied Intelligence10.1007/s10489-022-03558-553:3(3207-3220)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1007/s10489-022-03558-5
Show More Cited By

Index Terms

Aspect term extraction for opinion mining using a Hierarchical Self-Attention Network
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
  2. Machine learning
    1. Learning paradigms
    2. Machine learning approaches
2. Information systems
  1. Information systems applications
    1. Data mining

Index terms have been assigned to the content through auto-classification.

Recommendations

Post-processing method with aspect term error correction for enhancing aspect term extraction
Abstract
Aspect Term Extraction (ATE), which aims to extract aspect terms from review sentences, is an important subtask of sentiment analysis. Existing studies have proposed many sequence taggers, which have achieved impressive progress. However, previous ...
Aspect-based sentiment analysis via multitask learning for online reviews
Abstract
Aspect based sentiment analysis(ABSA) aims to identify aspect terms in online reviews and predict their corresponding sentiment polarity. Sentiment analysis poses a challenging fine-grained task. Two typical subtasks are involved: ...
POS-ATAEPE-BiLSTM: an aspect-based sentiment analysis algorithm considering part-of-speech embedding
Abstract
Aspect-based sentiment analysis (ABSA) is a granular sentiment classification task that involves identifying sentiment polarities toward aspects in a sentence. Performing ABSA on online e-commerce reviews is essential for understanding customers’ ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 465, Issue C

Nov 2021

585 pages

ISSN:0925-2312

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 20 November 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chu CSo RKwong EChan A(2023)Leveraging rule-based model and machine learning transformer for mining aspect-based financial opinions in colloquial languageProceedings of the 2023 7th International Conference on Software and e-Business10.1145/3641067.3641075(71-78)Online publication date: 21-Dec-2023
https://dl.acm.org/doi/10.1145/3641067.3641075
Yuan LWang JYu LZhang X(2023)Encoding Syntactic Information into Transformers for Aspect-Based Sentiment Triplet ExtractionIEEE Transactions on Affective Computing10.1109/TAFFC.2023.329173015:2(722-735)Online publication date: 7-Jul-2023
https://dl.acm.org/doi/10.1109/TAFFC.2023.3291730
Liu BLin TLi M(2023)Improving aspect term extraction via span-level tag data augmentationApplied Intelligence10.1007/s10489-022-03558-553:3(3207-3220)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1007/s10489-022-03558-5
Yin WXu YLiu CZheng DWang QLiu C(2023)Prompt-Oriented Fine-Tuning Dual Bert for Aspect-Based Sentiment AnalysisArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44204-9_42(505-517)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-44204-9_42
Kim TKim H(2022)Opinion Mining-Based Term Extraction Sentiment Classification ModelingMobile Information Systems10.1155/2022/55931472022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/5593147
So RChu CLee C(2022)Extract Aspect-based Financial Opinion Using Natural Language InferenceProceedings of the 2022 International Conference on E-business and Mobile Commerce10.1145/3543106.3543120(83-87)Online publication date: 13-May-2022
https://dl.acm.org/doi/10.1145/3543106.3543120
Kumar ABalan RGupta PNeti LMalapati A(2022)BILEAT: a highly generalized and robust approach for unified aspect-based sentiment analysisApplied Intelligence10.1007/s10489-022-03311-y52:12(14025-14040)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1007/s10489-022-03311-y

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents