research-article

Public Access

Mining Aspect-Specific Opinion using a Holistic Lifelong Topic Model

Authors:

Bing LiuAuthors Info & Claims

WWW '16: Proceedings of the 25th International Conference on World Wide Web

Pages 167 - 176

https://doi.org/10.1145/2872427.2883086

Published: 11 April 2016 Publication History

Abstract

Aspect-level sentiment analysis or opinion mining consists of several core sub-tasks: aspect extraction, opinion identification, polarity classification, and separation of general and aspect-specific opinions. Various topic models have been proposed by researchers to address some of these sub-tasks. However, there is little work on modeling all of them together. In this paper, we first propose a holistic fine-grained topic model, called the JAST (Joint Aspect-based Sentiment Topic) model, that can simultaneously model all of above problems under a unified framework. To further improve it, we incorporate the idea of lifelong machine learning and propose a more advanced model, called the LAST (Lifelong Aspect-based Sentiment Topic) model. LAST automatically mines the prior knowledge of aspect, opinion, and their correspondence from other products or domains. Such knowledge is automatically extracted and incorporated into the proposed LAST model without any human involvement. Our experiments using reviews of a large number of product domains show major improvements of the proposed models over state-of-the-art baselines.

References

[1]

R. Agrawal, R. Srikant, et al. Fast algorithms for mining association rules. In Proc. 20th int. conf. very large data bases, VLDB, volume 1215, pages 487--499, 1994.

Digital Library

[2]

D. Andrzejewski, X. Zhu, and M. Craven. Incorporating domain knowledge into topic modeling via dirichlet forest priors. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 25--32. ACM, 2009.

Digital Library

[3]

D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. the Journal of machine Learning research, 3:993--1022, 2003.

Digital Library

[4]

S. Branavan, H. Chen, J. Eisenstein, and R. Barzilay. Learning document-level semantic properties from free-text annotations. Journal of Artificial Intelligence Research, pages 569--603, 2009.

Digital Library

[5]

S. Brody and N. Elhadad. An unsupervised aspect-sentiment model for online reviews. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 804--812. Association for Computational Linguistics, 2010.

Digital Library

[6]

J. Chang, S. Gerrish, C. Wang, J. L. Boyd-graber, and D. M. Blei. Reading tea leaves: How humans interpret topic models. In Advances in neural information processing systems, pages 288--296, 2009.

Digital Library

[7]

Z. Chen and B. Liu. Mining topics in documents: standing on the shoulders of big data. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1116--1125. ACM, 2014.

Digital Library

[8]

Z. Chen and B. Liu. Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data. In ICML, pages 703--711, 2014.

[9]

Z. Chen, N. Ma, and B. Liu. Lifelong Learning for Sentiment Classification. In ACL, pages 750--756, 2015.

[10]

Q. Diao, M. Qiu, C.-Y. Wu, A. J. Smola, J. Jiang, and C. Wang. Jointly modeling aspects, ratings and sentiments for movie recommendation (jmars). In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 193--202. ACM, 2014.

Digital Library

[11]

L. Fang and M. Huang. Fine granular aspect analysis using latent structural models. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers-Volume 2, pages 333--337. Association for Computational Linguistics, 2012.

Digital Library

[12]

T. L. Griffiths and M. Steyvers. Finding scientific topics. Proceedings of the National Academy of Sciences, 101(suppl 1):5228--5235, 2004.

[13]

T. L. Griffiths, M. Steyvers, D. M. Blei, and J. B. Tenenbaum. Integrating topics and syntax. In Advances in neural information processing systems, pages 537--544, 2004.

[14]

Y. He, C. Lin, and H. Alani. Automatically extracting polarity-bearing topics for cross-domain sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 123--131. Association for Computational Linguistics, 2011.

Digital Library

[15]

G. Heinrich. A generic approach to topic models. In Machine Learning and Knowledge Discovery in Databases, pages 517--532. Springer, 2009.

[16]

M. Hu and B. Liu. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 168--177. ACM, 2004.

Digital Library

[17]

Y. Hu, J. Boyd-Graber, B. Satinoff, and A. Smith. Interactive topic modeling. Machine learning, 95(3):423--469, 2014.

Digital Library

[18]

J. Jagarlamudi, H. Daumé III, and R. Udupa. Incorporating lexical priors into topic models. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 204--213. Association for Computational Linguistics, 2012.

Digital Library

[19]

Y. Jo and A. H. Oh. Aspect and sentiment unification model for online review analysis. In Proceedings of the fourth ACM international conference on Web search and data mining, pages 815--824. ACM, 2011.

Digital Library

[20]

J.-H. Kang, J. Ma, and Y. Liu. Transfer topic modeling with ease and scalability. In SDM, pages 564--575. SIAM, 2012.

[21]

N. Kawamae. Latent interest-topic model: finding the causal relationships behind dyadic data. In Proceedings of the 19th ACM international conference on Information and knowledge management, pages 649--658. ACM, 2010.

Digital Library

[22]

S. Kim, J. Zhang, Z. Chen, A. H. Oh, and S. Liu. A hierarchical aspect-sentiment model for online reviews. In AAAI, 2013.

Digital Library

[23]

A. Lazaridou, I. Titov, and C. Sporleder. A bayesian model for joint unsupervised induction of sentiment, aspect and discourse representations. In ACL (1), pages 1630--1639, 2013.

[24]

P. Li, Y. Wang, W. Gao, and J. Jiang. Generating aspect-oriented multi-document summarization with event-aspect model. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1137--1146. Association for Computational Linguistics, 2011.

Digital Library

[25]

C. Lin and Y. He. Joint sentiment/topic model for sentiment analysis. In Proceedings of the 18th ACM conference on Information and knowledge management, pages 375--384. ACM, 2009.

Digital Library

[26]

B. Liu. Sentiment analysis and opinion mining (synthesis lectures on human language technologies). Morgan & Claypool Publishers, 2012.

[27]

Y. Lu and C. Zhai. Opinion integration through semi-supervised topic modeling. In Proceedings of the 17th international conference on World Wide Web, pages 121--130. ACM, 2008.

Digital Library

[28]

Y. Lu, C. Zhai, and N. Sundaresan. Rated aspect summarization of short comments. In Proceedings of the 18th international conference on World wide web, pages 131--140. ACM, 2009.

Digital Library

[29]

H. Mahmoud. Polya Urn Models. Chapman & Hall/CRC Texts in Statistical Science, 2008.

[30]

Q. Mei, X. Ling, M. Wondra, H. Su, and C. Zhai. Topic sentiment mixture: modeling facets and opinions in weblogs. In Proceedings of the 16th international conference on World Wide Web, pages 171--180. ACM, 2007.

Digital Library

[31]

D. Mimno, H. M. Wallach, E. Talley, M. Leenders, and A. McCallum. Optimizing semantic coherence in topic models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 262--272. Association for Computational Linguistics, 2011.

Digital Library

[32]

S. Moghaddam and M. Ester. The flda model for aspect-based opinion mining: addressing the cold start problem. In Proceedings of the 22nd international conference on World Wide Web, pages 909--918. International World Wide Web Conferences Steering Committee, 2013.

Digital Library

[33]

A. Mukherjee and B. Liu. Aspect extraction through semi-supervised modeling. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 339--348. Association for Computational Linguistics, 2012.

Digital Library

[34]

D. Newman, J. H. Lau, K. Grieser, and T. Baldwin. Automatic evaluation of topic coherence. In HLT-NAACL, pages 100--108, 2010.

Digital Library

[35]

D. Newman, Y. Noh, E. Talley, S. Karimi, and T. Baldwin. Evaluating topic models for digital libraries. In Proceedings of the 10th annual joint conference on Digital libraries, pages 215--224. ACM, 2010.

Digital Library

[36]

S. J. Pan and Q. Yang. A Survey on Transfer Learning. IEEE Trans. Knowl. Data Eng., 22(10):1345--1359, 2010.

Digital Library

[37]

M. Paul and M. Dredze. Factorial lda: Sparse multi-dimensional text models. In Advances in Neural Information Processing Systems, pages 2582--2590, 2012.

[38]

M. Paul and R. Girju. Cross-cultural analysis of blogs and forums with mixed-collection topic models. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3-Volume 3, pages 1408--1417. Association for Computational Linguistics, 2009.

Digital Library

[39]

M. Paul and R. Girju. A two-dimensional topic-aspect model for discovering multi-faceted topics. Urbana, 51:61801, 2010.

[40]

M. J. Paul and M. Dredze. Drug extraction from the web: Summarizing drug experiences with multi-dimensional topic models. In HLT-NAACL, pages 168--178, 2013.

[41]

J. Petterson, W. Buntine, S. M. Narayanamurthy, T. S. Caetano, and A. J. Smola. Word features for latent dirichlet allocation. In Advances in Neural Information Processing Systems, pages 1921--1929, 2010.

[42]

R. Salakhutdinov and G. E. Hinton. Deep boltzmann machines. In International Conference on Artificial Intelligence and Statistics, pages 448--455, 2009.

[43]

C. Sauper and R. Barzilay. Automatic aggregation by joint modeling of aspects and values. Journal of Artificial Intelligence Research, 2013.

Digital Library

[44]

S. Thrun. Lifelong Learning Algorithms. In S. Thrun and L. Pratt, editors, Learning To Learn. Kluwer Academic Publishers, 1998.

[45]

I. Titov and R. T. McDonald. A joint model of text and aspect ratings for sentiment summarization. In ACL, volume 8, pages 308--316. Citeseer, 2008.

[46]

H. Wang, Y. Lu, and C. Zhai. Latent aspect rating analysis on review text data: a rating regression approach. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 783--792. ACM, 2010.

Digital Library

[47]

L. Wang, K. Liu, Z. Cao, J. Zhao, and G. de Melo. Sentiment-aspect extraction based on restricted boltzmann machines. 2015.

[48]

G. Xue, W. Dai, Q. Yang, and Y. Yu. Topic-bridged PLSA for cross-domain text classification. In SIGIR, pages 627--634, 2008.

Digital Library

[49]

B. Yang and C. Cardie. Joint inference for fine-grained opinion extraction. In ACL (1), pages 1640--1649, 2013.

[50]

S.-h. Yang, S. P. Crain, and H. Zha. Bridging the language gap: topic adaptation for documents with different technicality. In International Conference on Artificial Intelligence and Statistics, pages 823--831, 2011.

[51]

C. Zhai, A. Velivelli, and B. Yu. A cross-collection mixture model for comparative text mining. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 743--748. ACM, 2004.

Digital Library

[52]

W. X. Zhao, J. Jiang, H. Yan, and X. Li. Jointly modeling aspects and opinions with a maxent-lda hybrid. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 56--65. Association for Computational Linguistics, 2010.

Digital Library

[53]

G. Zipf. Selective studies and the principle of relative frequencies in language, 1932.

Cited By

Chen LMankad S(2024)A Structural Topic and Sentiment-Discourse Model for Text AnalysisSSRN Electronic Journal10.2139/ssrn.4020651Online publication date: 2024
https://doi.org/10.2139/ssrn.4020651
Xu JXie JCai YLin ZLeung HLi QChua T(2024)Context-Aware Dynamic Word Embeddings for Aspect Term ExtractionIEEE Transactions on Affective Computing10.1109/TAFFC.2023.326294115:1(144-156)Online publication date: Jan-2024
https://doi.org/10.1109/TAFFC.2023.3262941
Abulaish MWasi NSharma S(2024)The role of lifelong machine learning in bridging the gap between human and machine learning: A scientometric analysisWIREs Data Mining and Knowledge Discovery10.1002/widm.152614:2Online publication date: 10-Jan-2024
https://doi.org/10.1002/widm.1526
Show More Cited By

Index Terms

Mining Aspect-Specific Opinion using a Holistic Lifelong Topic Model
1. Information systems
  1. World Wide Web
    1. Web searching and information discovery
      1. Personalization

Recommendations

Joint sentiment/topic model for sentiment analysis
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Sentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet ...
Topic sentiment change analysis
MLDM'11: Proceedings of the 7th international conference on Machine learning and data mining in pattern recognition

Public opinions on a topic may change over time. Topic Sentiment change analysis is a new research problem consisting of two main components: (a) mining opinions on a certain topic, and (b) detect significant changes of sentiment of the opinions on the ...
Twitter Opinion Topic Model: Extracting Product Opinions from Tweets by Leveraging Hashtags and Sentiment Lexicon
CIKM '14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

Aspect-based opinion mining is widely applied to review data to aggregate or summarize opinions of a product, and the current state-of-the-art is achieved with Latent Dirichlet Allocation (LDA)-based model. Although social media data like tweets are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '16: Proceedings of the 25th International Conference on World Wide Web

April 2016

1482 pages

ISBN:9781450341431

General Chairs:
Jacqueline Bourdeau
Tele-university (TELUQ), Montreal, QC, Canada
,
Jim A. Hendler
Rensselaer Polytechnic Institute, Troy, NY, USA
,
Roger Nkambou Nkambou
Université du Québec à Montréal, Montreal, QC, Canada
,
Program Chairs:
Ian Horrocks
University of Oxford, UK
,
Ben Y. Zhao
University of California at Santa Barbara, CA, USA

Copyright © 2016 Copyright is held by the International World Wide Web Conference Committee (IW3C2).

Sponsors

IW3C2: International World Wide Web Conference Committee

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

International World Wide Web Conferences Steering Committee

Republic and Canton of Geneva, Switzerland

Publication History

Published: 11 April 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NCI
NSF
Bosch

Conference

WWW '16

Sponsor:

IW3C2

WWW '16: 25th International World Wide Web Conference

April 11 - 15, 2016

Québec, Montréal, Canada

Acceptance Rates

WWW '16 Paper Acceptance Rate 115 of 727 submissions, 16%;

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

75
Total Citations
View Citations
1,189
Total Downloads

Downloads (Last 12 months)97
Downloads (Last 6 weeks)17

Reflects downloads up to 23 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen LMankad S(2024)A Structural Topic and Sentiment-Discourse Model for Text AnalysisSSRN Electronic Journal10.2139/ssrn.4020651Online publication date: 2024
https://doi.org/10.2139/ssrn.4020651
Xu JXie JCai YLin ZLeung HLi QChua T(2024)Context-Aware Dynamic Word Embeddings for Aspect Term ExtractionIEEE Transactions on Affective Computing10.1109/TAFFC.2023.326294115:1(144-156)Online publication date: Jan-2024
https://doi.org/10.1109/TAFFC.2023.3262941
Abulaish MWasi NSharma S(2024)The role of lifelong machine learning in bridging the gap between human and machine learning: A scientometric analysisWIREs Data Mining and Knowledge Discovery10.1002/widm.152614:2Online publication date: 10-Jan-2024
https://doi.org/10.1002/widm.1526
Gheibi OWeyns D(2023)Dealing with Drift of Adaptation Spaces in Learning-based Self-Adaptive Systems Using Lifelong Self-AdaptationACM Transactions on Autonomous and Adaptive Systems10.1145/363642819:1(1-57)Online publication date: 13-Dec-2023
https://dl.acm.org/doi/10.1145/3636428
Qian YZhang FLiu Z(2023)Policy generation network for zero‐shot policy learningComputational Intelligence10.1111/coin.1259139:5(707-733)Online publication date: 4-Jul-2023
https://doi.org/10.1111/coin.12591
Yang ZZheng JGe Z(2023)Lifelong Bayesian Learning Machines for Streaming Industrial Big DataIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2022.319883353:3(1554-1565)Online publication date: Mar-2023
https://doi.org/10.1109/TSMC.2022.3198833
Sun GCong YGu CTang XDing ZYu H(2023)Hierarchical Lifelong Machine Learning With “Watchdog”IEEE Transactions on Big Data10.1109/TBDATA.2021.31108629:1(63-74)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TBDATA.2021.3110862
Laddha AMukherjee A(2023)Aspect Specific Opinion Expression Extraction Using Attention Based LSTM-CRF NetworkComputational Linguistics and Intelligent Text Processing10.1007/978-3-031-23804-8_34(442-454)Online publication date: 26-Feb-2023
https://doi.org/10.1007/978-3-031-23804-8_34
Khan MAzam NKhalid SAziz F(2022)Hierarchical lifelong topic modeling using rules extracted from network communitiesPLOS ONE10.1371/journal.pone.026448117:3(e0264481)Online publication date: 3-Mar-2022
https://doi.org/10.1371/journal.pone.0264481
Yang ZGe Z(2022)On Paradigm of Industrial Big Data Analytics: From Evolution to RevolutionIEEE Transactions on Industrial Informatics10.1109/TII.2022.319039418:12(8373-8388)Online publication date: Dec-2022
https://doi.org/10.1109/TII.2022.3190394
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents