research-article

Mining the Stars: Learning Quality Ratings with User-facing Explanations for Vacation Rentals

Authors:

Anastasiia Kornilova,

Lucas BernardiAuthors Info & Claims

WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

Pages 976 - 983

https://doi.org/10.1145/3437963.3441812

Published: 08 March 2021 Publication History

Abstract

Online Travel Platforms are virtual two-sided marketplaces where guests search for accommodations and accommodation providers list their properties such as hotels and vacation rentals. The large majority of hotels are rated by official institutions with a number of stars indicating the quality of service they provide. It is a simple and effective mechanism that contributes to match supply with demand by helping guests to find options meeting their criteria and accommodation suppliers to market their product to the right segment directly impacting the number of transactions on the platform. Unfortunately, no similar rating system exists for the large majority of vacation rentals, making it difficult for guests to search and compare options and hard for vacation rentals suppliers to market their product effectively. In this work we describe a machine learned quality rating system for vacation rentals. The problem is challenging, mainly due to explainability requirements and the lack of ground truth. We present techniques to address these challenges and empirical evidence of their efficacy. Our system was successfully deployed and validated through Online Controlled Experiments performed in Booking.com, a large Online Travel Platform, and running for more than one year, impacting more than a million accommodations and millions of guests.

References

[1]

Alejandro Barredo Arrieta, Natalia D'iaz-Rodr'iguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garc'ia, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, Vol. 58 (2020), 82--115.

Digital Library

[2]

Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2009. Evaluation measures for ordinal regression. In 2009 Ninth international conference on intelligent systems design and applications. IEEE, 283--287.

Digital Library

[3]

Ralf Bender and Ulrich Grouven. 1997. Ordinal logistic regression in medical research. Journal of the Royal College of physicians of London, Vol. 31, 5 (1997), 546.

[4]

Lucas Bernardi, Themistoklis Mavridis, and Pablo Estevez. 2019. 150 Successful Machine Learning Models: 6 Lessons Learned at Booking. com. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1743--1751.

Digital Library

[5]

Eibe Frank and Mark Hall. 2001. A simple approach to ordinal classification. In European Conference on Machine Learning. Springer, 145--156.

Digital Library

[6]

Sorelle A. Friedler, Chitradeep Dutta Roy, Carlos Scheidegger, and Dylan Slack. 2019. Assessing the Local Interpretability of Machine Learning Models. CoRR, Vol. abs/1902.03501 (2019). arxiv: 1902.03501 http://arxiv.org/abs/1902.03501

[7]

Jerome H Friedman. 2002. Stochastic gradient boosting. Computational statistics & data analysis, Vol. 38, 4 (2002), 367--378.

[8]

David Gunning. 2017. Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web, Vol. 2 (2017), 2.

[9]

Frank E Harrell Jr. 2015. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis .Springer. 282--290 pages.

[10]

Fátima Leal, Benedita Malheiro, and Juan Carlos Burguillo. 2017. Prediction and analysis of hotel ratings from crowd-sourced data. In World Conference on Information Systems and Technologies. Springer, 493--502.

[11]

Scott M. Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M. Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. 2020. From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, Vol. 2, 1 (2020), 2522--5839.

[12]

Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in neural information processing systems. 4765--4774.

[13]

R. Potharst and A. J. Feelders. 2002. Classification Trees for Problems with Monotonicity Constraints. SIGKDD Explor. Newsl., Vol. 4, 1 (June 2002), 1--10. https://doi.org/10.1145/568574.568577

Digital Library

[14]

Lloyd S Shapley. 1953. A value for n-person games. Contributions to the Theory of Games, Vol. 2, 28 (1953), 307--317.

[15]

Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), Vol. 58, 1 (1996), 267--288.

[16]

PAL Vine. 1981. Hotel classification?art or science? International Journal of Tourism Management, Vol. 2, 1 (1981), 18--29.

[17]

Xiaojin Zhur and Zoubin Ghahramani. 2002. Learning from labeled and unlabeled data with label propagation. (2002).

Cited By

Quin FWeyns DGalster MSilva C(2024)A/B testingJournal of Systems and Software10.1016/j.jss.2024.112011211:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112011
Goldenberg DDiaz FShah CSuel TCastells PJones RSakai T(2021)Putting the Role of Personalization into ContextProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3404835.3464929(2631-2632)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3404835.3464929

Index Terms

Mining the Stars: Learning Quality Ratings with User-facing Explanations for Vacation Rentals
1. Computing methodologies
  1. Machine learning
    1. Learning settings
      1. Semi-supervised learning settings
2. Information systems

Recommendations

Mining customer product ratings for personalized marketing
Special issue: Web data mining

With the increasing popularity of Internet commerce, a wealth of information about the customers can now be readily acquired on-line. An important example is the customers' preference ratings for the various products offered by the company. Successful ...
Pricing, Frills, and Customer Ratings

This paper explores whether and how a firm should adapt its strategy in view of consumer use of prior customer ratings. Specifically, we consider optimal pricing and whether the firm should offer an unexpected frill to early customers to enhance their ...
The value of IT-enabled retailer learning: personalized product recommendations and customer store loyalty in electronic markets

Recent research has acknowledged the key role of information technology in helping build stronger and more enduring customer relationships. Personalized product recommendations (PPRs) adapted to individual customers' preferences and tastes are one IT-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

March 2021

1192 pages

ISBN:9781450382977

DOI:10.1145/3437963

General Chairs:
Liane Lewin-Eytan
Amazon, Israel
,
David Carmel
Amazon, Israel
,
Elad Yom-Tov
Microsoft, Israel
,
Program Chairs:
Eugene Agichtein
Emory University and Amazon, USA
,
Evgeniy Gabrilovich
Google Health, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 March 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WSDM '21

Sponsor:

WSDM '21: The Fourteenth ACM International Conference on Web Search and Data Mining

March 8 - 12, 2021

Virtual Event, Israel

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
232
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Quin FWeyns DGalster MSilva C(2024)A/B testingJournal of Systems and Software10.1016/j.jss.2024.112011211:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112011
Goldenberg DDiaz FShah CSuel TCastells PJones RSakai T(2021)Putting the Role of Personalization into ContextProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3404835.3464929(2631-2632)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3404835.3464929

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents