research-article

Open access

On Interpretation and Measurement of Soft Attributes for Recommendation

Authors:

Krisztian Balog,

Filip Radlinski,

Alexandros KaratzoglouAuthors Info & Claims

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 890 - 899

https://doi.org/10.1145/3404835.3462893

Published: 11 July 2021 Publication History

Abstract

We address how to robustly interpret natural language refinements (or critiques) in recommender systems. In particular, in human-human recommendation settings people frequently use soft attributes to express preferences about items, including concepts like the originality of a movie plot, the noisiness of a venue, or the complexity of a recipe. While binary tagging is extensively studied in the context of recommender systems, soft attributes often involve subjective and contextual aspects, which cannot be captured reliably in this way, nor be represented as objective binary truth in a knowledge base. This also adds important considerations when measuring soft attribute ranking. We propose a more natural representation as personalized relative statements, rather than as absolute item properties. We present novel data collection techniques and evaluation approaches, and a new public dataset. We also propose a set of scoring approaches, from unsupervised to weakly supervised to fully supervised, as a step towards interpreting and acting upon soft attribute based critiques.

References

[1]

Krisztian Balog. 2018. Entity-Oriented Search . The Information Retrieval Series, Vol. 39. Springer.

[2]

Oren Barkan and Noam Koenigstein. 2016. Item2Vec: Neural Item Embedding for Collaborative Filtering. CoRR, Vol. abs/1603.04259 (2016).

[3]

Keping Bi, Qingyao Ai, Yongfeng Zhang, and W. Bruce Croft. 2019. Conversational Product Search Based on Negative Feedback. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM '19). 359--368.

[4]

Craig Boutilier, Kevin Regan, and Paolo Viappiani. 2009. Online feature elicitation in interactive optimization. In Proceedings of the 26th Annual International Conference on Machine Learning (ICML '09). 73--80.

Digital Library

[5]

Craig Boutilier, Kevin Regan, and Paolo Viappiani. 2010. Simultaneous elicitation of preference features and utility. In Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI '10).

Digital Library

[6]

Mark J. Carman, Mark Baillie, and Fabio Crestani. 2008. Tag Data and Personalized Information Retrieval. In Proceedings of the 2008 ACM Workshop on Search in Social Media (SSM '08). 27--34.

Digital Library

[7]

Li Chen and Pearl Pu. 2006. Evaluating Critiquing-based Recommender Agents. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI '06). 157--162.

[8]

Li Chen and Pearl Pu. 2012. Critiquing-based Recommenders: Survey and Emerging Trends. User Modeling and User-Adapted Interaction, Vol. 22, 1--2 (April 2012), 125--150.

Digital Library

[9]

Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards Conversational Recommender Systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '16). 815--824.

Digital Library

[10]

J. Shane Culpepper, Fernando Diaz, and Mark D. Smucker. 2018. Research Frontiers in Information Retrieval: Report from the Third Strategic Workshop on Information Retrieval in Lorne (SWIRL 2018). SIGIR Forum, Vol. 52, 1 (Aug. 2018), 34--90.

Digital Library

[11]

Leo A. Goodman and William H. Kruskal. 1954. Measures of Association for Cross Classifications. J. Amer. Statist. Assoc., Vol. 49, 268 (1954), 732--764.

[12]

F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst., Vol. 5, 4, Article 19 (Dec. 2015), bibinfonumpages19:1--19:19 pages.

[13]

Nitin Jindal and Bing Liu. 2006. Mining Comparative Sentences and Relations. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI '06). 1331--1336.

Digital Library

[14]

Thorsten Joachims. 2002. Optimizing Search Engines Using Clickthrough Data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '02). 133--142.

Digital Library

[15]

Wiltrud Kessler and Jonas Kuhn. 2013. Detection of Product Comparisons - How Far Does an Out-of-the-Box Semantic Role Labeling System Take You?. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP '13). 1892--1897.

[16]

Wiltrud Kessler and Jonas Kuhn. 2014. A Corpus of Comparisons in Product Reviews. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC '14). 2242--2248.

[17]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. Computer, Vol. 42, 8 (Aug. 2009), 30--37.

Digital Library

[18]

Takeshi Kurashima, Katsuji Bessho, Hiroyuki Toda, Toshio Uchiyama, and Ryoji Kataoka. 2008. Ranking Entities Using Comparative Relations. In Proceedings of the 19th International Conference on Database and Expert Systems Applications (DEXA '08). 124--133.

Digital Library

[19]

Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining (WSDM '20). 304--312.

Digital Library

[20]

Hanze Li, Scott Sanner, Kai Luo, and Ga Wu. 2020. A Ranking Optimization Approach to Latent Linear Critiquing for Conversational Recommender Systems. In Fourteenth ACM Conference on Recommender Systems (RecSys '20). 13--22.

Digital Library

[21]

Raymond Li, Samira Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards Deep Conversational Recommendations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS '18). 9748--9758.

Digital Library

[22]

Si Li, Zheng-Jun Zha, Zhaoyan Ming, Meng Wang, Tat-Seng Chua, Jun Guo, and Weiran Xu. 2011. Product Comparison Using Comparative Relations. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '11). 1151--1152.

Digital Library

[23]

Greg Linden, Steve Hanks, and Neal Lesh. 1997. Interactive Assessment of User Preference Models: The Automated Travel Assistant. In Proceedings of the Sixth International Conference on User Modeling . 67--78.

[24]

Kai Luo, Scott Sanner, Ga Wu, Hanze Li, and Hojin Yang. 2020 a. Latent Linear Critiquing for Conversational Recommender Systems. In Proceedings of The Web Conference 2020 (WWW '20). 2535--2541.

Digital Library

[25]

Kai Luo, Hojin Yang, Ga Wu, and Scott Sanner. 2020 b. Deep Critiquing for VAE-Based Recommender Systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '20). 1269--1278.

Digital Library

[26]

Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '15). 43--52.

Digital Library

[27]

Kevin McCarthy, Yasser Salem, and Barry Smyth. 2010. Experience-Based Critiquing: Reusing Critiquing Experiences to Improve Conversational Recommendation. In Proceedings of the 18th International Conference on Case-Based Reasoning Research and Development (ICCBR '10). 480--494.

Digital Library

[28]

Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fined-grained aspects. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP '19).

[29]

Filip Radlinski, Krisztian Balog, Bill Byrne, and Karthik Krishnamoorthi. 2019. Coached Conversational Preference Elicitation: A Case Study in Understanding Movie Preferences. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL '19). 353--360.

[30]

Filip Radlinski and Nick Craswell. 2017. A Theoretical Framework for Conversational Search. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval (CHIIR '17). 117--126.

Digital Library

[31]

Steffen Rendle and Lars Schmidt-Thieme. 2010. Pairwise Interaction Tensor Factorization for Personalized Tag Recommendation. In Proceedings of the Third ACM International Conference on Web Search and Data Mining (WSDM '10). 81--90.

Digital Library

[32]

Anna Sepliarskaia, Julia Kiseleva, Filip Radlinski, and Maarten de Rijke. 2018. Preference elicitation as an optimization problem. In Proceedings of the International Conference on Recommender Systems (RecSys '18). 172--180.

Digital Library

[33]

Börkur Sigurbjörnsson and Roelof van Zwol. 2008. Flickr Tag Recommendation Based on Collective Knowledge. In Proceedings of the 17th International Conference on World Wide Web (WWW '08). 327--336.

Digital Library

[34]

Yueming Sun and Yi Zhang. 2018. Conversational Recommender System. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR '18). 235--244.

Digital Library

[35]

Maksim Tkachenko and Hady W. Lauw. 2014. Generative Modeling of Entity Comparisons in Text. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM '14). 859--868.

[36]

Daniel Tunkelang. 2009. Faceted search. Synthesis lectures on information concepts, retrieval, and services, Vol. 1, 1 (2009), 1--80.

Digital Library

[37]

Kasturi Dewi Varathan, Anastasia Giachanou, and Fabio Crestani. 2017. Comparative Opinion Mining: A Review. J. Assoc. Inf. Sci. Technol., Vol. 68, 4 (April 2017), 811--829.

[38]

Jesse Vig, Shilad Sen, and John Riedl. 2012. The Tag Genome: Encoding Community Knowledge to Support Novel Interaction. ACM Trans. Interact. Intell. Syst., Vol. 2, 3, Article 13 (Sept. 2012), bibinfonumpages44 pages.

Digital Library

[39]

Jesse Vig, Matthew Soukup, Shilad Sen, and John Riedl. 2010. Tag expression: Tagging with feeling. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '10). 323--332.

Digital Library

[40]

Mengting Wan and Julian J. McAuley. 2018. Item recommendation on monotonic behavior chains. In Proceedings of the 12th ACM Conference on Recommender Systems (RecSys '18). 86--94.

[41]

Jason Williams, Antoine Raux, and Matthew Henderson. 2016. The Dialog State Tracking Challenge Series: A Review. Dialogue & Discourse, Vol. 7, 3 (2016), 4--33.

[42]

Ga Wu, Kai Luo, Scott Sanner, and Harold Soh. 2019 a. Deep Language-Based Critiquing for Recommender Systems. In Proceedings of the 13th ACM Conference on Recommender Systems (RecSys '19). 137--145.

Digital Library

[43]

Ga Wu, Kai Luo, Scott Sanner, and Harold Soh. 2019 b. Deep Language-Based Critiquing for Recommender Systems. In Proceedings of the 13th ACM Conference on Recommender Systems (RecSys '19). 137--145.

Digital Library

[44]

Yury Zemlyanskiy, Sudeep Gandhe, Ruining He, Bhargav Kanagal, Anirudh Ravula, Juraj Gottweis, Fei Sha, and Ilya Eckstein. 2021. DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL '21). 2540--2549.

[45]

Shuo Zhang and Krisztian Balog. 2017. Design Patterns for Fusion-Based Object Retrieval. In Proceedings of the 39th European conference on Advances in Information Retrieval (ECIR '17). 684--690.

[46]

Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W. Bruce Croft. 2018. Towards Conversational Search and Recommendation: System Ask, User Respond. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM '18). 177--186.

Digital Library

[47]

Zhu Zhang, Chenhui Guo, and Paulo Goes. 2013. Product Comparison Networks for Competitive Analysis of Online Word-of-Mouth. ACM Trans. Manage. Inf. Syst., Vol. 3, 4, Article 20 (Jan. 2013), bibinfonumpages20:1--20:22 pages.

Digital Library

Cited By

Göpfert CHaig AHsu CChow YVendrov ILu TRamachandran DPham HGhavamzadeh MBoutilier C(2024)Discovering Personalized Semantics for Soft Attributes in Recommender Systems Using Concept Activation VectorsACM Transactions on Recommender Systems10.1145/36586752:4(1-37)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.1145/3658675
Kostric IBalog KRadlinski F(2024)Generating Usage-related Questions for Preference Elicitation in Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36299812:2(1-24)Online publication date: 10-Apr-2024
https://dl.acm.org/doi/10.1145/3629981
Sanner SBalog KRadlinski FWedin BDixon L(2023)Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based PreferencesProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608845(890-896)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608845
Show More Cited By

Index Terms

On Interpretation and Measurement of Soft Attributes for Recommendation
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Attributes coupling based matrix factorization for item recommendation

Recommender systems have attracted lots of attention since they alleviate the information overload problem for users. Matrix factorization is one of the most widely employed collaborative filtering techniques in the research of recommender systems due ...
Exploiting non-content preference attributes through hybrid recommendation method
RecSys '13: Proceedings of the 7th ACM conference on Recommender systems

This paper explores a method for incorporating into a recommender system explicit representations of user's preferences over non-content attributes such as popularity, recency, and similarity of recommended items. We show how such attributes can be ...
Exploiting Item Representations for Soft Clustering Recommendation
Webmedia '16: Proceedings of the 22nd Brazilian Symposium on Multimedia and the Web

Recommender systems help dealing with the information overload problem since they provide personalized content for users. There are two major paradigms in recommendation: content-based and collaborative filtering. Regardless of the paradigm, there has ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2021

2998 pages

ISBN:9781450380379

DOI:10.1145/3404835

General Chairs:
Fernando Diaz
(Google)
,
Chirag Shah
University of Washington
,
Torsten Suel
New York University
,
Program Chairs:
Pablo Castells
Universidad Autónoma de Madrid, Amazon
,
Rosie Jones
Spotify
,
Tetsuya Sakai
Waseda University

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '21

Sponsor:

SIGIR

SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2021

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
804
Total Downloads

Downloads (Last 12 months)156
Downloads (Last 6 weeks)17

Reflects downloads up to 28 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Göpfert CHaig AHsu CChow YVendrov ILu TRamachandran DPham HGhavamzadeh MBoutilier C(2024)Discovering Personalized Semantics for Soft Attributes in Recommender Systems Using Concept Activation VectorsACM Transactions on Recommender Systems10.1145/36586752:4(1-37)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.1145/3658675
Kostric IBalog KRadlinski F(2024)Generating Usage-related Questions for Preference Elicitation in Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36299812:2(1-24)Online publication date: 10-Apr-2024
https://dl.acm.org/doi/10.1145/3629981
Sanner SBalog KRadlinski FWedin BDixon L(2023)Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based PreferencesProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608845(890-896)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608845
Dong XSong XZheng NWei YZhao Z(2023)Dual Preference Distribution Learning for Item RecommendationACM Transactions on Information Systems10.1145/356579841:3(1-22)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1145/3565798
Chaganty ALeszczynski MZhang SGanti RBalog KRadlinski FChen HDuh WHuang HKato MMothe JPoblete B(2023)Beyond Single Items: Exploring User Preferences in Item Sets with the Conversational Playlist Curation DatasetProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591881(2754-2764)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591881
Shen TMai ZWu GSanner S(2022)Distributional Contrastive Embedding for Clarification-based Conversational CritiquingProceedings of the ACM Web Conference 202210.1145/3485447.3512114(2422-2432)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3512114
Zhang SWang MBalog KAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)Analyzing and Simulating User Utterance Reformulation in Conversational Recommender SystemsProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531936(133-143)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531936
Radlinski FBalog KDiaz FDixon LWedin BAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)On Natural Language User Profiles for Transparent and Scrutable RecommendationProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531873(2863-2874)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531873
Kostric IBalog KRadlinski F(2021)Soliciting User Preferences in Conversational Recommender Systems via Usage-related QuestionsProceedings of the 15th ACM Conference on Recommender Systems10.1145/3460231.3478861(724-729)Online publication date: 13-Sep-2021
https://dl.acm.org/doi/10.1145/3460231.3478861
Afzali JDrzewiecki ABalog KDiaz FShah CSuel TCastells PJones RSakai T(2021)POINTREC: A Test Collection for Narrative-driven Point of Interest RecommendationProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3404835.3463243(2478-2484)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3404835.3463243

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents