research-article

Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN

Authors:

Xuan WangAuthors Info & Claims

Expert Systems with Applications: An International Journal, Volume 72, Issue C

Pages 221 - 230

https://doi.org/10.1016/j.eswa.2016.10.065

Published: 15 April 2017 Publication History

Abstract

A divide-and-conquer method classifying sentence types before sentiment analysis.Classifying sentence types by the number of opinion targets a sentence contain.A data-driven approach automatically extract features from input sentences. Different types of sentences express sentiment in very different ways. Traditional sentence-level sentiment classification research focuses on one-technique-fits-all solution or only centers on one special type of sentences. In this paper, we propose a divide-and-conquer approach which first classifies sentences into different types, then performs sentiment analysis separately on sentences from each type. Specifically, we find that sentences tend to be more complex if they contain more sentiment targets. Thus, we propose to first apply a neural network based sequence model to classify opinionated sentences into three types according to the number of targets appeared in a sentence. Each group of sentences is then fed into a one-dimensional convolutional neural network separately for sentiment classification. Our approach has been evaluated on four sentiment classification datasets and compared with a wide range of baselines. Experimental results show that: (1) sentence type classification can improve the performance of sentence-level sentiment analysis; (2) the proposed approach achieves state-of-the-art results on several benchmarking datasets.

References

[1]

T. lvarez Lpez, J. Juncal-Martnez, M. Fernndez-Gavilanes, E. Costa-Montenegro, F.J. Gonzlez-Castao, GTI at SemEval-2016 task 5: Svm and crf for aspect detection and unsupervised aspect-based sentiment analysis, Association for Computational Linguistics, San Diego, California, 2016.

[2]

O. Appel, F. Chiclana, J. Carter, H. Fujita, A hybrid approach to the sentiment analysis problem at the sentence level, Knowledge-Based Systems, 108 (2016) 110-124.

Digital Library

[3]

S. Baccianella, A. Esuli, F. Sebastiani, SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining, 2010.

[4]

Y. Bengio, R. Ducharme, P. Vincent, C. Jauvin, A neural probabilistic language model, Journal of Machine Learning Research, 3 (2003) 1137-1155.

Digital Library

[5]

C. Brun, J. Perez, C. Roux, XRCE at SemEval-2016 task 5: Feedbacked ensemble modeling on syntactico-semantic knowledge for aspect based sentiment analysis, Association for Computational Linguistics, San Diego, California, 2016.

[6]

J. Carrillo-de Albornoz, L. Plaza, An emotion-based model of negation, intensifiers, and modality for polarity and intensity classification, Journal of the American Society for Information Science and Technology, 64 (2013) 1618-1633.

[7]

F.S. etin, E. Yldrm, C. zbey, G. Eryiit, TGB at SemEval-2016 task 5: Multi-lingual constraint system for aspect based sentiment analysis, Association for Computational Linguistics, San Diego, California, 2016.

[8]

I. Chaturvedi, Y.-S. Ong, I.W. Tsang, R.E. Welsch, E. Cambria, Learning word dependencies in text by means of a deep recurrent belief network, Knowledge-Based Systems, 108 (2016) 144-154.

Digital Library

[9]

R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, P. Kuksa, Natural language processing (almost) from scratch, The Journal of Machine Learning Research, 12 (2011) 2493-2537.

Digital Library

[10]

D. Crystal, Dictionary of linguistics and phonetics, John Wiley & Sons, 2011.

[11]

L. Dong, F. Wei, C. Tan, D. Tang, M. Zhou, K. Xu, Adaptive recursive neural network for target-dependent twitter sentiment classification, Association for Computational Linguistics, Baltimore, Maryland, 2014.

[12]

J.L. Elman, Finding structure in time, Cognitive Science, 14 (1990) 179-211.

[13]

M. Fernndez-Gavilanes, T. lvarez-Lpez, J. Juncal-Martnez, E. Costa-Montenegro, F.J. Gonzlez-Castao, Unsupervised method for sentiment analysis in online texts, Expert Systems with Applications, 58 (2016) 57-75.

Digital Library

[14]

M. Ganapathibhotla, B. Liu, Mining opinions in comparative sentences, Association for Computational Linguistics, 2008.

Digital Library

[15]

R. Gonzlez-Ibnez, S. Muresan, N. Wacholder, Identifying sarcasm in twitter: a closer look, Association for Computational Linguistics, 2011.

Digital Library

[16]

A. Graves, A.-r. Mohamed, G. Hinton, Speech recognition with deep recurrent neural networks, IEEE, 2013.

[17]

V. Hatzivassiloglou, J.M. Wiebe, Effects of adjective orientation and gradability on sentence subjectivity, Association for Computational Linguistics, 2000.

Digital Library

[18]

T. Hercig, T. Brychcn, L. Svoboda, M. Konkol, UWB at SemEval-2016 task 5: Aspect based sentiment analysis, Association for Computational Linguistics, San Diego, California, 2016.

[19]

G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, R.R. Salakhutdinov, Improving neural networks by preventing co-adaptation of feature detectors, Computing Research Repository (CoRR), abs/1207.0580 (2012).

[20]

S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Computation, 9 (1997) 1735-1780.

Digital Library

[21]

M. Hu, B. Liu, Mining and summarizing customer reviews, ACM, 2004.

Digital Library

[22]

Huang, Z., Xu, W., & Yu, K. (2015). Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991.

[23]

O. Irsoy, C. Cardie, Deep recursive neural networks for compositionality in language, 2014.

[24]

O. Irsoy, C. Cardie, Opinion mining with deep recurrent neural networks, 2014.

[25]

L. Jia, C. Yu, W. Meng, The effect of negation on sentiment analysis and retrieval effectiveness, ACM, 2009.

[26]

L. Jiang, M. Yu, M. Zhou, X. Liu, T. Zhao, Target-dependent twitter sentiment classification, Association for Computational Linguistics, 2011.

Digital Library

[27]

N. Jindal, B. Liu, Identifying comparative sentences in text documents, ACM, 2006.

[28]

N. Jindal, B. Liu, Mining comparative sentences and relations, 2006.

[29]

N. Kalchbrenner, E. Grefenstette, P. Blunsom, A convolutional neural network for modelling sentences, Association for Computational Linguistics, Baltimore, Maryland, 2014.

[30]

W. Kessler, J. Kuhn, A corpus of comparisons in product reviews, 2014.

[31]

Y. Kim, Convolutional neural networks for sentence classification, 2014.

[32]

A. Kumar, S. Kohail, A. Kumar, A. Ekbal, C. Biemann, IIT-TUDA at SemEval-2016 task 5: Beyond sentiment lexicon: Combining domain dependency and distributional semantics features for aspect based sentiment analysis, Association for Computational Linguistics, San Diego, California, 2016.

[33]

J. Lafferty, A. McCallum, F.C. Pereira, Conditional random fields: Probabilistic models for segmenting and labeling sequence data, 2001.

[34]

Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016). Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360.

[35]

Q. Le, T. Mikolov, Distributed representations of sentences and documents, 2014.

[36]

B. Liu, Sentiment analysis and opinion mining, Morgan & Claypool Publishers, 2012.

Digital Library

[37]

B. Liu, Sentiment analysis: Mining opinions, sentiments, and emotions, Cambridge University Press, 2015.

[38]

Liu, P., Qiu, X., & Huang, X. (2016). Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101.

Digital Library

[39]

K. Liu, L. Xu, J. Zhao, Extracting opinion targets and opinion words from online reviews with graph co-ranking, Association for Computational Linguistics, Baltimore, Maryland, 2014.

[40]

Y. Liu, X. Yu, Z. Chen, B. Liu, Sentiment analysis of sentences with modalities, ACM, 2013.

[41]

A. McCallum, Efficiently inducing features of conditional random fields, Morgan Kaufmann, 2003.

[42]

M. Mitchell, J. Aguilar, T. Wilson, B.V. Durme, Open domain targeted sentiment, 2013.

[43]

A. Muhammad, N. Wiratunga, R. Lothian, Contextual sentiment analysis for social media genres, Knowledge-Based Systems, 108 (2016) 92-101.

Digital Library

[44]

T. Nakagawa, K. Inui, S. Kurohashi, Dependency tree-based sentiment classification using CRFs with hidden variables, Association for Computational Linguistics, 2010.

[45]

R. Narayanan, B. Liu, A. Choudhary, Sentiment analysis of conditional sentences, Association for Computational Linguistics, 2009.

[46]

Okazaki, N. (2007). Crfsuite: a fast implementation of conditional random fields (CRFs).

[47]

B. Pang, L. Lee, A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts, Association for Computational Linguistics, 2004.

Digital Library

[48]

B. Pang, L. Lee, Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales, Association for Computational Linguistics, 2005.

Digital Library

[49]

B. Pang, L. Lee, S. Vaithyanathan, Thumbs up?: sentiment classification using machine learning techniques, 2002.

[50]

M. Park, Y. Yuan, Linguistic knowledge-driven approach to chinese comparative elements extraction, Association for Computational Linguistics, 2015.

[51]

M. Pontiki, D. Galanis, H. Papageorgiou, I. Androutsopoulos, S. Manandhar, M. AL-Smadi, M. Al-Ayyoub, Y. Zhao, B. Qin, O.D. Clercq, V. Hoste, M. Apidianaki, X. Tannier, N. Loukachevitch, E. Kotelnikov, N. Bel, S.M. Jimnez-Zafra, G. Eryiit, SemEval-2016 task 5: Aspect based sentiment analysis, Association for Computational Linguistics, San Diego, California, 2016.

[52]

A. Popescu, O. Etzioni, Extracting product features and opinions from reviews, Springer, 2005.

[53]

S. Poria, E. Cambria, A. Gelbukh, Aspect extraction for opinion mining with a deep convolutional neural network, Knowledge-Based Systems, 108 (2016) 42-49.

Digital Library

[54]

S. Poria, E. Cambria, G. Winterstein, G.-B. Huang, Sentic patterns: Dependency-based rules for concept-level sentiment analysis, Knowledge-Based Systems, 69 (2014) 45-63.

Digital Library

[55]

G. Qiu, B. Liu, J. Bu, C. Chen, Opinion word expansion and target extraction through double propagation, Computational Linguistics, 37 (2011) 9-27.

Digital Library

[56]

K. Ravi, V. Ravi, A survey on opinion mining and sentiment analysis: tasks, approaches and applications, Knowledge-Based Systems, 89 (2015) 14-46.

Digital Library

[57]

S. Rill, D. Reinel, J. Scheidt, R.V. Zicari, Politwi: Early detection of emerging political topics on twitter and the impact on concept-level sentiment analysis, Knowledge-Based Systems, 69 (2014) 24-33.

Digital Library

[58]

E. Riloff, A. Qadir, P. Surve, L. De Silva, N. Gilbert, R. Huang, Sarcasm as contrast between a positive sentiment and negative situation., Association for Computational Linguistics, 2013.

[59]

E. Riloff, J. Wiebe, Learning extraction patterns for subjective expressions, Association for Computational Linguistics, 2003.

[60]

M. Schuster, K.K. Paliwal, Bidirectional recurrent neural networks, IEEE Transactions on Signal Processing, 45 (1997) 2673-2681.

Digital Library

[61]

R. Socher, B. Huval, C.D. Manning, A.Y. Ng, Semantic compositionality through recursive matrix-vector spaces, ACL, 2012.

[62]

R. Socher, J. Pennington, E.H. Huang, A.Y. Ng, C.D. Manning, Semi-supervised recursive autoencoders for predicting sentiment distributions, Association for Computational Linguistics, 2011.

Digital Library

[63]

R. Socher, A. Perelygin, J.Y. Wu, J. Chuang, C.D. Manning, A.Y. Ng, C. Potts, Recursive deep models for semantic compositionality over a sentiment treebank, Citeseer, 2013.

[64]

V. Stoyanov, C. Cardie, Topic identification for fine-grained opinion analysis, Association for Computational Linguistics, 2008.

Digital Library

[65]

Tai, K. S., Socher, R., & Manning, C. D. (2015). Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075.

[66]

Tang, D., Qin, B., Feng, X., & Liu, T. (2015a). Target-dependent sentiment classification with long short term memory. arXiv preprint arXiv:1512.01100.

[67]

D. Tang, B. Qin, T. Liu, Deep learning for sentiment analysis: Successful approaches and future challenges, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 5 (2015) 292-303.

Digital Library

[68]

D. Tang, B. Qin, F. Wei, L. Dong, T. Liu, M. Zhou, A joint segmentation and classification framework for sentence level sentiment classification, IEEE/ACM Transactions on Audio, Speech, and Language Processing, 23 (2015) 1750-1761.

Digital Library

[69]

Z. Toh, J. Su, NLANGP at SemEval-2016 task 5: Improving aspect based sentiment analysis using neural network features, Association for Computational Linguistics, San Diego, California, 2016.

[70]

O. Tsur, D. Davidov, A. Rappoport, Icwsm-a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews, 2010.

[71]

P.D. Turney, Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews, Association for Computational Linguistics, 2002.

Digital Library

[72]

D.-T. Vo, Y. Zhang, Target-dependent twitter sentiment classification with rich automatic features, 2015.

[73]

S. Wang, C.D. Manning, Baselines and bigrams: Simple, good sentiment and topic classification, Association for Computational Linguistics, 2012.

Digital Library

[74]

J.M. Wiebe, R.F. Bruce, T.P. OHara, Development and use of a gold-standard data set for subjectivity classifications, Association for Computational Linguistics, 1999.

Digital Library

[75]

J. Wiebe, T. Wilson, Learning to disambiguate potentially subjective expressions, Association for Computational Linguistics, 2002.

[76]

J. Wiebe, T. Wilson, C. Cardie, Annotating expressions of opinions and emotions in language, Language Resources and Evaluation, 39 (2005) 165-210.

[77]

S. Yang, Y. Ko, Extracting comparative entities and predicates from texts using comparative type classification, Association for Computational Linguistics, 2011.

[78]

H. Yu, V. Hatzivassiloglou, Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences, Association for Computational Linguistics, 2003.

[79]

L. Zhang, S. Ferrari, P. Enjalbert, Opinion analysis: the effect of negation on polarity and intensity, 2012.

[80]

M. Zhang, Y. Zhang, D.-T. Vo, Neural networks for open domain targeted sentiment, 2015.

Cited By

Zhu YFan LXu L(2024)Advances in task-oriented research in sentiment analysis: an overview of methods and analysis of challengesProceedings of the 2nd International Conference on Educational Knowledge and Informatization10.1145/3691720.3691757(223-227)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3691720.3691757
Chen SLi F(2024)Ponzi scheme detection in smart contracts using the integration of deep learning and formal verificationIET Blockchain10.1049/blc2.120564:2(185-196)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1049/blc2.12056
Cao XWang JWu PFang Z(2024)VulMPFFIET Information Security10.1049/2024/43131852024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1049/2024/4313185
Show More Cited By

Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN
1. Computing methodologies
  1. Artificial intelligence
2. Hardware
  1. Power and energy
    1. Power estimation and optimization

Recommendations

Sentence compression for aspect-based sentiment analysis

Sentiment analysis, which addresses the computational treatment of opinion, sentiment, and subjectivity in text, has received considerable attention in recent years. In contrast to the traditional coarse-grained sentiment analysis tasks, such as document-...
Joint sentiment/topic model for sentiment analysis
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Sentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet ...
A joint segmentation and classification framework for sentence level sentiment classification

In this paper, we propose a joint segmentation and classification framework for sentence-level sentiment classification. It is widely recognized that phrasal information is crucial for sentiment classification. However, existing sentiment classification ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 72, Issue C

April 2017

455 pages

ISSN:0957-4174

Issue’s Table of Contents

Copyright © The Authors.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 15 April 2017

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

162
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhu YFan LXu L(2024)Advances in task-oriented research in sentiment analysis: an overview of methods and analysis of challengesProceedings of the 2nd International Conference on Educational Knowledge and Informatization10.1145/3691720.3691757(223-227)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3691720.3691757
Chen SLi F(2024)Ponzi scheme detection in smart contracts using the integration of deep learning and formal verificationIET Blockchain10.1049/blc2.120564:2(185-196)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1049/blc2.12056
Cao XWang JWu PFang Z(2024)VulMPFFIET Information Security10.1049/2024/43131852024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1049/2024/4313185
Mao YLiu QZhang Y(2024)Sentiment analysis methods, applications, and challengesJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2024.10204836:4Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.jksuci.2024.102048
Derakhshi MRazzaghi T(2024)An imbalance-aware BiLSTM for control chart patterns early detectionExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123682249:PCOnline publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123682
Liu SHuang QLi MWei Y(2024)A new LASSO-BiLSTM-based ensemble learning approach for exchange rate forecastingEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107305127:PBOnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107305
Nath DDwivedi S(2024)Aspect-based sentiment analysis: approaches, applications, challenges and trendsKnowledge and Information Systems10.1007/s10115-024-02200-966:12(7261-7303)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s10115-024-02200-9
Huang KWang J(2023)Short-term auto parts demand forecasting based on EEMD—CNN—BiLSTM—Attention—combination modelJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-22422245:4(5449-5465)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/JIFS-224222
Li WZhu LMao RCambria EWilliams BChen YNeville J(2023)SKIERProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i11.26541(13121-13129)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i11.26541
Başarslan MKayaalp F(2023)MBi-GRUMCONV: A novel Multi Bi-GRU and Multi CNN-Based deep learning model for social media sentiment analysisJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-022-00386-312:1Online publication date: 10-Jan-2023
https://dl.acm.org/doi/10.1186/s13677-022-00386-3
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents