Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

A weakly supervised knowledge attentive network for aspect-level sentiment classification

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Deep neural networks have achieved good performance in recent years for aspect-level sentiment classification (ASC), whereas most neural ASC models neglect the commonsense knowledge absent from text but essential for aspect affective understanding, which largely limits the performance of neural ASC. In this paper, we propose a Weakly Supervised Knowledge Attentive Network, which resolves the above problems via knowledge attention and weakly supervised learning. Specifically, we first present a Knowledge Attentive Network (KAN) to capture more aspect-related information by incorporating external commonsense knowledge into the attention mechanism. Then, we propose a weakly supervised learning method, which allows our KAN model to learn more knowledge from the pseudo-samples generated upon the rich-resource document-level sentiment classification datasets. Extensive experiments on four benchmark datasets show the significant advantages of our proposed approach. In particular, we obtain state-of-the-art performance in terms of accuracy on all the datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://www.nltk.org/.

  2. http://alt.qcri.org/semeval2014/task4/.

  3. http://alt.qcri.org/semeval2015/task12/.

  4. http://alt.qcri.org/semeval2016/task5/.

  5. https://www.yelp.ca/dataset/challenge.

  6. http://jmcauley.ucsd.edu/data/amazon/.

  7. https://pytorch.org/.

References

  1. Xu M, Biqing Z, Heng Y, Junlong C, Jiatao C, Hongye L (2022) Combining dynamic local context focus and dependency cluster attention for aspect-level sentiment classification. Neurocomputing 478:49–69

    Article  Google Scholar 

  2. Jiandian Z, Tianyi L, Weijia J, Jiantao Z (2022) Relation construction for aspect-level sentiment classification. Inf Sci 586:209–223

    Article  Google Scholar 

  3. Wang B, Shen T, Long G, Zhou T, Chang Y (2021) Eliminating sentiment bias for aspect-level sentiment classification with unsupervised opinion extraction. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pages 3002–3012. Association for Computational Linguistics,

  4. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) NRC-canada-2014: detecting aspects and sentiment in customer reviews. In SemEval, pages 437–442,

  5. Jiang L, Yu M, Zhou M, Liu X, Zhao T (2011) Target-dependent twitter sentiment classification. In ACL, pages 151–160,

  6. Ming Z, Vasile P, Yan W, Zhicheng J (2021) Attention-based word embeddings using artificial bee colony algorithm for aspect-level sentiment classification. Inf Sci 545:713–738

    Article  Google Scholar 

  7. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. In ACL, pages 2514–2523,

  8. Sepp H, Jürgen S (1997) Long short-term memory. Neural Comput 9(8):1735–1780

  9. Wang Y, Huang M, Zhao L, et al (2016) Attention-based lstm for aspect-level sentiment classification. In EMNLP, pages 606–615,

  10. Chen P, Sun Z, Bing L, Yang W (2017) Recurrent attention network on memory for aspect sentiment analysis. In EMNLP, pages 452–461

  11. Li X, Bing L, Lam W, Shi B (2018) Transformation networks for target-oriented sentiment classification. In ACL, pages 946–956,

  12. Nguyen CV, Le KH, Tran AM, Pham QH, Nguyen BT (2022) Learning for amalgamation: a multi-source transfer learning framework for sentiment classification. Inf Sci 590:1–14

    Article  Google Scholar 

  13. Chen H, Xia R, Yu J (2021) Reinforced counterfactual data augmentation for dual sentiment classification. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 269–278. Association for Computational Linguistics,

  14. Geng B, Yang M, Yuan F, Wang S, Ao X, Xu R (2021) Iterative network pruning with uncertainty regularization for lifelong sentiment classification. In Fernando Diaz, Chirag Shah, Torsten Suel, Pablo Castells, Rosie Jones, and Tetsuya Sakai, editors, SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, pages 1229–1238. ACM,

  15. Ma Y, Peng H, Cambria E (2018) Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm. In AAAI, pages 5876–5883,

  16. He R, Lee WS, Ng HT, Dahlmeier D (2018) Exploiting document knowledge for aspect-level sentiment classification. In ACL, pages 579–585,

  17. Min Y, Wenpeng Y, Qu Q, Tu W, Ying S, Xiaojun C (2021) Neural attentive network for cross-domain aspect-level sentiment classification. IEEE Trans Affect Comput 12(3):761–775

    Article  Google Scholar 

  18. Gichang L, Jaeyun J, Seungwan S, CzangYeob K, Pilsung K (2018) Sentiment classification with word localization based on weakly supervised learning with a convolutional neural network. Knowl Based Syst 152:70–82

    Article  Google Scholar 

  19. Qingchun B, Jie Z, Liang H (2022) PG-RNN: using position-gated recurrent neural networks for aspect-based sentiment classification. J Supercomput 78(3):4073–4094

    Article  Google Scholar 

  20. Naresh Kumar KE, Uma V (2021) Intelligent sentinet-based lexicon for context-aware sentiment analysis: optimized neural network for sentiment classification on social media. J Supercomput 77(11):12801–12825

    Article  Google Scholar 

  21. Su J, Jialong T, Hui J, Lu Z, Yubin G, Linfeng S, Deyi X, Le S, Jiebo L (2021) Enhanced aspect-based sentiment analysis models with progressive self-supervised attention learning. Artif Intell 296:103477

    Article  MathSciNet  MATH  Google Scholar 

  22. Chen F, Yuan Z, Huang Y (2020) Multi-source data fusion for aspect-level sentiment classification. Knowl Based Syst, 187

  23. Tang D, Qin B, Feng X, Liu T (2016) Effective lstms for target-dependent sentiment classification. In COLING, pages 3298–3307

  24. Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In EMNLP, pages 214–224,

  25. Sukhbaatar S, Weston J, Fergus R, et al (2015) End-to-end memory networks. In NIPS, pages 2440–2448,

  26. Dong L, Wei F, Zhou M, Xu K (2015) Question answering over freebase with multi-column convolutional neural networks. In ACL, pages 260–269

  27. Mihaylov T, Frank A (2018) Knowledgeable reader: enhancing cloze-style reading comprehension with external commonsense knowledge. In ACL, pages 821–832,

  28. Zhang X, Zhang C, Li X, Dong XL, Shang J, Faloutsos C, Han J (2022) Oa-mine: open-world attribute mining for e-commerce products with weak supervision. In Frédérique Laforest, Raphaël Troncy, Elena Simperl, Deepak Agarwal, Aristides Gionis, Ivan Herman, and Lionel Médini, editors, WWW ’22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022, pages 3153–3161. ACM,

  29. Li-Ming C, Bao-Xin X, Zhao-Yun D (2022) Multiple weak supervision for short text classification. Appl Intell 52(8):9101–9116

    Article  Google Scholar 

  30. Alexander D, Kusa W, de Vries AP (2022) ORCAS-I: queries annotated with intent using weak supervision. In Enrique Amigó, Pablo Castells, Julio Gonzalo, Ben Carterette, J. Shane Culpepper, and Gabriella Kazai, editors, SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 2022, pages 3057–3066. ACM,

  31. Dehghani M, Zamani H, Severyn A, Kamps J, Croft WB (2017) Neural ranking models with weak supervision. In SIGIR, pages 65–74,

  32. Alexander R, Bach Stephen H, Henry E, Jason F, Wu S, Christopher R (2017) Snorkel: rapid training data creation with weak supervision. VLDB 11(3):269–282

    Google Scholar 

  33. Wu F, Zhang J, Yuan Z, Wu S, Huang Y, Yan J (2017)Sentence-level sentiment classification with weak supervision. In SIGIR, pages 973–976

  34. Meng Y, Shen J, Zhang C, Han J (2019) Weakly-supervised hierarchical text classification. In AAAI, pages 6826–6833

  35. Alex G, Jürgen S (2005) Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Netw 18(5–6):602–610

    Google Scholar 

  36. Cambria E, Poria S, Bajpai R, Schuller B (2016) Senticnet 4: a semantic resource for sentiment analysis based on conceptual primitives. In COLING, pages 2666–2677,

  37. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: online learning of social representations. In SIGKDD, pages 701–710

  38. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111–3119

  39. Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In EMNLP, pages 1532–1543, (2014)

  40. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  41. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. In IJCAI, pages 4068–4074,

  42. Sun C, Huang L, Qiu X (2019) Utilizing bert for aspect-based sentiment analysis via constructing auxiliary sentence. In NAACL, pages 380–385

  43. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

Download references

Acknowledgements

We greatly appreciate anonymous reviewers and the associate editor for their valuable and high quality comments that greatly helped to improve the quality of this article. This research is funded by Shanghai Science and Technology Innovation Action Plan International Cooperation project “Research on international multi language online learning platform and key technologies (No. 20510780100)”, and Open Research Fund of NPPA Key Laboratory of Publishing Integration Development, ECNUP. And also supported by Shanghai Open University, “Research on the willingness, occurrence mechanism and cultivation path of intelligent teaching leadership for university teachers (No. XJ2101)”.

Author information

Authors and Affiliations

  1. Shanghai Engineering Research Center of Open Distance Education, Shanghai Open University, Shanghai, China

    Qingchun Bai & Jun Xiao

  2. NPPA Key Laboratory of Publishing Integration Development, ECNUP, Shanghai, China

    Qingchun Bai

  3. Fudan University, Shanghai, China

    Jie Zhou

Authors
  1. Qingchun Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingchun Bai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bai, Q., Xiao, J. & Zhou, J. A weakly supervised knowledge attentive network for aspect-level sentiment classification. J Supercomput 79, 5403–5420 (2023). https://doi.org/10.1007/s11227-022-04820-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04820-w

Keywords

Search

Navigation