short-paper

Integrated Ranking for News Feed with Reinforcement Learning

Authors:

Wei Xia,

Weinan ZhangAuthors Info & Claims

WWW '23 Companion: Companion Proceedings of the ACM Web Conference 2023

Pages 480 - 484

https://doi.org/10.1145/3543873.3584651

Published: 30 April 2023 Publication History

Get Access

Abstract

With the development of recommender systems, it becomes an increasingly common need to mix multiple item sequences from different sources. Therefore, the integrated ranking stage is proposed to be responsible for this task with re-ranking models. However, existing methods ignore the relation between the sequences, thus resulting in local optimum over the interaction session. To resolve this challenge, in this paper, we propose a new model named NFIRank (News Feed Integrated Ranking with reinforcement learning) and formulate the whole interaction session as a MDP (Markov Decision Process). Sufficient offline experiments are provided to verify the effectiveness of our model. In addition, we deployed our model on Huawei Browser and gained 1.58% improvements in CTR compared with the baseline in online A/B test. Code will be available at https://gitee.com/mindspore/models/tree/master/research/recommend/NFIRank.

References

[1]

Qingyao Ai, Keping Bi, Jiafeng Guo, and W Bruce Croft. 2018. Learning a deep listwise context model for ranking refinement. In The 41st international ACM SIGIR conference on research & development in information retrieval. 135–144.

Digital Library

Google Scholar

[2]

Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014).

Google Scholar

[3]

Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS) 20, 4 (2002), 422–446.

Digital Library

Google Scholar

[4]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

Google Scholar

[5]

Guogang Liao, Ze Wang, Xiaoxu Wu, Xiaowen Shi, Chuheng Zhang, Yongkang Wang, Xingxing Wang, and Dong Wang. 2022. Cross dqn: Cross deep q network for ads allocation in feed. In Proceedings of the ACM Web Conference 2022. 401–409.

Digital Library

Google Scholar

[6]

Weiwen Liu, Yunjia Xi, Jiarui Qin, Fei Sun, Bo Chen, Weinan Zhang, Rui Zhang, and Ruiming Tang. 2022. Neural Re-ranking in Multi-stage Recommender Systems: A Review. arXiv preprint arXiv:2202.06602 (2022).

Google Scholar

[7]

Lori Lorigo, Maya Haridasan, Hrönn Brynjarsdóttir, Ling Xia, Thorsten Joachims, Geri Gay, Laura Granka, Fabio Pellacini, and Bing Pan. 2008. Eye tracking and online search: Lessons learned and challenges ahead. Journal of the American Society for Information Science and Technology 59, 7 (2008), 1041–1052.

Crossref

Google Scholar

[8]

Lori Lorigo, Bing Pan, Helene Hembrooke, Thorsten Joachims, Laura Granka, and Geri Gay. 2006. The influence of task and gender on search and evaluation behavior using Google. Information processing & management 42, 4 (2006), 1123–1131.

Google Scholar

[9]

MindSpore. 2023. MindSpore. https://www.mindspore.cn/.

Google Scholar

[10]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, 2015. Human-level control through deep reinforcement learning. nature 518, 7540 (2015), 529–533.

Google Scholar

[11]

Liang Pang, Jun Xu, Qingyao Ai, Yanyan Lan, Xueqi Cheng, and Jirong Wen. 2020. Setrank: Learning a permutation-invariant ranking model for information retrieval. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 499–508.

Digital Library

Google Scholar

[12]

Changhua Pei, Yi Zhang, Yongfeng Zhang, Fei Sun, Xiao Lin, Hanxiao Sun, Jian Wu, Peng Jiang, Junfeng Ge, Wenwu Ou, 2019. Personalized re-ranking for recommendation. In Proceedings of the 13th ACM conference on recommender systems. 3–11.

Digital Library

Google Scholar

[13]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).

Google Scholar

Cited By

View all

Li NMa YZhao YWang QZhang ZYu CXu JZheng BDeng XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Ad vs Organic: Revisiting Incentive Compatible Mechanism Design in E-commerce PlatformsProceedings of the ACM Web Conference 202410.1145/3589334.3645638(235-244)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645638

Index Terms

Integrated Ranking for News Feed with Reinforcement Learning
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Balancing Utility and Exposure Fairness for Integrated Ranking with Reinforcement Learning
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Integrated ranking is critical in industrial recommendation systems and has attracted increasing attention. In an integrated ranking system, items from multiple channels are merged together and form an integrated list. During this process, apart from ...
RLMixer: A Reinforcement Learning Approach for Integrated Ranking with Contrastive User Preference Modeling
Advances in Knowledge Discovery and Data Mining
Abstract
There is a strong need for industrial recommender systems to output an integrated ranking of items from different categories, such as video and news, to maximize overall user satisfaction. Integrated ranking faces two critical challenges. First, ...
HiFI: Hierarchical Fairness-aware Integrated Ranking with Constrained Reinforcement Learning
WWW '24: Companion Proceedings of the ACM Web Conference 2024

Integrated ranking is a critical component in industrial recommendation platforms. It combines candidate lists from different upstream channels or sources and ranks them into an integrated list, which will be exposed to users. During this process, to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

WWW '23 Companion: Companion Proceedings of the ACM Web Conference 2023

April 2023

1567 pages

ISBN:9781450394192

DOI:10.1145/3543873

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

WWW '23

Sponsor:

SIGWEB

WWW '23: The ACM Web Conference 2023

April 30 - May 4, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
174
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)12

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Li NMa YZhao YWang QZhang ZYu CXu JZheng BDeng XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Ad vs Organic: Revisiting Incentive Compatible Mechanism Design in E-commerce PlatformsProceedings of the ACM Web Conference 202410.1145/3589334.3645638(235-244)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645638

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Cited By

Index Terms

Recommendations

Balancing Utility and Exposure Fairness for Integrated Ranking with Reinforcement Learning

RLMixer: A Reinforcement Learning Approach for Integrated Ranking with Contrastive User Preference Modeling

HiFI: Hierarchical Fairness-aware Integrated Ranking with Constrained Reinforcement Learning

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

HTML Format

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations