research-article

MRR: an unsupervised algorithm to rank reviews by relevance

Authors:

Vinicius Woloszyn,

Henrique D. P. dos Santos,

Leandro Krug Wives,

Karin BeckerAuthors Info & Claims

WI '17: Proceedings of the International Conference on Web Intelligence

Pages 877 - 883

https://doi.org/10.1145/3106426.3106444

Published: 23 August 2017 Publication History

Abstract

The automatic detection of relevant reviews plays a major role in tasks such as opinion summarization, opinion-based recommendation, and opinion retrieval. Supervised approaches for ranking reviews by relevance rely on the existence of a significant, domain-dependent training data set. In this work, we propose MRR (Most Relevant Reviews), a new unsupervised algorithm that identifies relevant revisions based on the concept of graph centrality. The intuition behind MRR is that central reviews highlight aspects of a product that many other reviews frequently mention, with similar opinions, as expressed in terms of ratings. MRR constructs a graph where nodes represent reviews, which are connected by edges when a minimum similarity between a pair of reviews is observed, and then employs PageRank to compute the centrality. The minimum similarity is graph-specific, and takes into account how reviews are written in specific domains. The similarity function does not require extensive pre-processing, thus reducing the computational cost. Using reviews from books and electronics products, our approach has outperformed the two unsupervised baselines and shown a comparable performance with two supervised regression models in a specific setting. MRR has also achieved a significantly superior run-time performance in a comparison with the unsupervised baselines.

References

[1]

Li Chen, Guanliang Chen, and Feng Wang. 2015. Recommender systems based on user reviews: the state of the art. User Modeling and User-Adapted Interaction 25, 2 (2015), 99--154.

Digital Library

[2]

Alton Y.K. Chua and Snehasish Banerjee. 2016. Helpfulness of user-generated reviews as a function of review sentiment, product type and information quality. Computers in Human Behavior 54 (2016), 547 -- 554.

Digital Library

[3]

Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS) 20, 4 (2002), 422--446.

Digital Library

[4]

T Jayalakshmi and A Santhakumaran. 2011. Statistical Normalization and Back Propagation for Classification. International Journal of Computer Theory and Engineering 3, 1 (2011), 89.

[5]

Soo-Min Kim, Patrick Pantel, Tim Chklovski, and Marco Pennacchiotti. 2006. Automatically Assessing Review Helpfulness. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP '06). Association for Computational Linguistics, Stroudsburg, PA, USA, 423--430. http://dl.acm.org/citation.cfm?id=1610075.1610135

Digital Library

[6]

Jon M Kleinberg. 1999. Authoritative sources in a hyperlinked environment. Journal of the ACM (JACM) 46, 5 (1999), 604--632.

Digital Library

[7]

Neethu Kurian and Shimmi Asokan. 2015. Summarizing User Opinions: A Method for Labeled-data Scarce Product Domains. Procedia Computer Science 46 (2015), 93 -- 100.

[8]

Xuan Nhat Lam, Thuc Vu, Trong Duc Le, and Anh Duc Duong. 2008. Addressing Cold-start Problem in Recommendation Systems. In Proceedings of the 2Nd International Conference on Ubiquitous Information Management and Communication (ICUIMC '08). ACM, New York, NY, USA, 208--211.

Digital Library

[9]

Julian McAuley, Rahul Pandey, and Jure Leskovec. 2015. Inferring networks of substitutable and complementary products. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 785--794.

Digital Library

[10]

Susan M Mudambi and David Schuff. 2010. What makes a helpful review? A study of customer reviews on Amazon. com. MIS quarterly 34, 1 (2010), 185--200.

Digital Library

[11]

Arjun Mukherjee and Bing Liu. 2012. Modeling Review Comments. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Jeju Island, Korea, 320--329. http://www.aclweb.org/anthology/P12-1034

Digital Library

[12]

Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank citation ranking: bringing order to the web. (1999).

[13]

Duyu Tang, Bing Qin, and Ting Liu. 2015. Learning Semantic Representations of Users and Products for Document Level Sentiment Classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Beijing, China, 1014--1023. http://www.aclweb.org/anthology/P15-1098

[14]

Oren Tsur and Ari Rappoport. 2009. RevRank: A Fully Unsupervised Algorithm for Selecting the Most Helpful Book Reviews. In ICWSM.

[15]

Mikalai Tsytsarau and Themis Palpanas. 2012. Survey on mining subjective data on the web. Data Mining and Knowledge Discovery 24, 3 (2012), 478--514.

Digital Library

[16]

Xiaojun Wan. 2013. Co-Regression for Cross-Language Review Rating Prediction. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, Sofia, Bulgaria, 526--531. http://www.aclweb.org/anthology/P13-2094

[17]

Douglas Brent West and others. 2001. Introduction to graph theory. Vol. 2. Prentice hall Upper Saddle River.

[18]

Frank Wilcoxon, SK Katti, and Roberta A Wilcox. 1970. Critical values and probability levels for the Wilcoxon rank sum test and the Wilcoxon signed rank test. Selected tables in mathematical statistics 1 (1970), 171--259.

[19]

V. Woloszyn, H. D. P. dos Santos, and L. K. Wives. 2016. The influence of readability aspects on the user's perception of helpfulness of online reviews. Revista de Sistemas de Informação da FSMA 18 (2016).

[20]

Jianwei Wu, Bing Xu, and Sheng Li. 2011. An unsupervised approach to rank product reviews. In Fuzzy Systems and Knowledge Discovery (FSKD), 2011 Eighth International Conference on, Vol. 3. IEEE, 1769--1772.

[21]

Wenting Xiong and Diane Litman. 2011. Automatically Predicting Peer-Review Helpfulness. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Portland, Oregon, USA, 502--507. http://www.aclweb.org/anthology/P11-2088

Digital Library

[22]

Yinfei Yang, Yaowei Yan, Minghui Qiu, and Forrest Bao. 2015. Semantic Analysis and Helpfulness Prediction of Text for Online Product Reviews. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Beijing, China, 38--44. http://www.aclweb.org/anthology/P15-2007

[23]

Yi-Ching Zeng and Shih-Hung Wu. 2013. Modeling the Helpful Opinion Mining of Online Consumer Reviews as a Classification Problem. In Proceedings of the IJCNLP 2013 Workshop on NLP for Social Media (SocialNLP). Asian Federation of Natural Language Processing, Nagoya, Japan, 29--35. http://www.aclweb.org/anthology/W13-4205

Cited By

Wang RChen ZZhang MLi ZLiu ZDang ZYu CChen XKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Revamp: Enhancing Accessible Information Seeking Experience of Online Shopping for Blind or Low Vision UsersProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445547(1-14)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445547
Santos HUlbrich AVieira R(2021)Evaluation of a Prescription Outlier Detection System in Hospital’s Pharmacy Services2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM52615.2021.9669703(2862-2868)Online publication date: 9-Dec-2021
https://doi.org/10.1109/BIBM52615.2021.9669703
Santos HUlbrich AWoloszyn VVieira R(2019)DDC-Outlier: Preventing Medication Errors Using Unsupervised LearningIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2018.282802823:2(874-881)Online publication date: Mar-2019
https://doi.org/10.1109/JBHI.2018.2828028
Show More Cited By

Index Terms

MRR: an unsupervised algorithm to rank reviews by relevance
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Domain Specific Opinion Retrieval
AIRS '09: Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology

Opinion retrieval is a novel information retrieval task and has attracted a great deal of attention with the rapid increase of online opinionated information. Most previous work adopts the classical two stage framework, i.e., first retrieving topic ...
Improve the effectiveness of the opinion retrieval and opinion polarity classification
CIKM '08: Proceedings of the 17th ACM conference on Information and knowledge management

Opinion retrieval is a document retrieving and ranking process. A relevant document must be relevant to the query and contain opinions toward the query. Opinion polarity classification is an extension of opinion retrieval. It classifies the retrieved ...
A unified relevance model for opinion retrieval
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Representing the information need is the greatest challenge for opinion retrieval. Typical queries for opinion retrieval are composed of either just content words, or content words with a small number of cue "opinion" words. Both are inadequate for ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WI '17: Proceedings of the International Conference on Web Intelligence

August 2017

1284 pages

ISBN:9781450349512

DOI:10.1145/3106426

Conference Chair:
Amit Sheth
Wright State University GuoROLE@GENERAL CHAIR
,
General Chairs:
Axel Ngonga
Leipzig University, Germany
,
yin Wang
Chongqing University of Posts and Telecommunications, China
,
Elizabeth Chang
The University of New South Wales, Australia
,
Dominik Ślęzak
Infobright Inc. & University of Warsaw, Poland
,
Bogdan Franczyk
Leipzig University, Germany
,
Program Chairs:
Rainer Alt
Leipzig University, Germany
,
Xiaohui Tao
University of Southern Queensland, Australia

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
TCII: IEEE Computer Society Technical Committee on Intelligent Informatics
Web Intelligence Consortium

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 August 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

CAPES and CNPq, Brazil

Conference

WI '17

Sponsor:

SIGAI
TCII

WI '17: International Conference on Web Intelligence 2017

August 23 - 26, 2017

Leipzig, Germany

Acceptance Rates

WI '17 Paper Acceptance Rate 118 of 178 submissions, 66%;

Overall Acceptance Rate 118 of 178 submissions, 66%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
161
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)2

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang RChen ZZhang MLi ZLiu ZDang ZYu CChen XKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Revamp: Enhancing Accessible Information Seeking Experience of Online Shopping for Blind or Low Vision UsersProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445547(1-14)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445547
Santos HUlbrich AVieira R(2021)Evaluation of a Prescription Outlier Detection System in Hospital’s Pharmacy Services2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM52615.2021.9669703(2862-2868)Online publication date: 9-Dec-2021
https://doi.org/10.1109/BIBM52615.2021.9669703
Santos HUlbrich AWoloszyn VVieira R(2019)DDC-Outlier: Preventing Medication Errors Using Unsupervised LearningIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2018.282802823:2(874-881)Online publication date: Mar-2019
https://doi.org/10.1109/JBHI.2018.2828028
Woloszyn VNejdl WAkkermans HFontaine KVermeulen IHouben GWeber M(2018)DistrustRankProceedings of the 10th ACM Conference on Web Science10.1145/3201064.3201083(221-228)Online publication date: 15-May-2018
https://dl.acm.org/doi/10.1145/3201064.3201083
Cortes EWoloszyn VBarone D(2018)When, Where, Who, What or Why? A Hybrid Model to Question Answering SystemsComputational Processing of the Portuguese Language10.1007/978-3-319-99722-3_14(136-146)Online publication date: 26-Aug-2018
https://doi.org/10.1007/978-3-319-99722-3_14
Zhang YYu H(2012)An Entropy-Based Comment Ranking Method with Word Embedding ClusteringAdvances and Innovations in Statistics and Data Science10.1007/978-3-031-08329-7_5(99-119)Online publication date: 24-Feb-2012
https://doi.org/10.1007/978-3-031-08329-7_5

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents