research-article

Soft Frequency Capping for Improved Ad Click Prediction in Yahoo Gemini Native

Authors:

Alex ZlotnikAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 2793 - 2801

https://doi.org/10.1145/3357384.3357801

Published: 03 November 2019 Publication History

Abstract

Yahoo's native advertising (also known as Gemini native) serves billions of ad impressions daily, reaching a yearly run-rate of many hundred of millions USD. Driving the Gemini native models that are used to predict both click probability (pCTR) and conversion probability (pCONV) is øffset\ -- a feature enhanced collaborative-filtering (CF) based event prediction algorithm. øffset is a one-pass algorithm that updates its model for every new batch of logged data using a stochastic gradient descent (SGD) based approach. Since øffset represents its users by their features (i.e., user-less model) due to sparsity issues, rule based hard frequency capping (HFC) is used to control the number of times a certain user views a certain ad. Moreover, related statistics reveal that user ad fatigue results in a dramatic drop in click through rate (CTR). Therefore, to improve click prediction accuracy, we propose a soft frequency capping (SFC) approach, where the frequency feature is incorporated into the øffset model as a user-ad feature and its weight vector is learned via logistic regression as part of øffset training. Online evaluation of the soft frequency capping algorithm via bucket testing showed a significant $7.3$% revenue lift. Since then, the frequency feature enhanced model has been pushed to production serving all traffic, and is generating a hefty revenue lift for Yahoo Gemini native. We also report related statistics that reveal, among other things, that while users' gender does not affect ad fatigue, the latter seems to increase with users' age.

References

[1]

Deepak Agarwal, Bee-Chung Chen, and Pradheep Elango. 2009. Spatio-temporal models for estimating click-through rate. In Proceedings of the 18th international conference on World wide web. ACM, 21--30.

Digital Library

[2]

Michal Aharon, Natalie Aizenberg, Edward Bortnikov, Ronny Lempel, Roi Adadi, Tomer Benyamini, Liron Levin, Ran Roth, and Ohad Serfaty. 2013a. OFF-set: one-pass factorization of feature sets for online recommendation in persistent cold start settings. In Proc. RecSys'2013. 375--378.

Digital Library

[3]

M. Aharon, N. Aizenberg, E. Bortnikov, R. Lempel, R. Adadi, T. Benyamini, L. Levin, R. Roth, and O. Serfaty. 2013b. OFF-set: one-pass factorization of features sets for online recommendation in persistent cold start settings. Proc. RecSys (2013).

Digital Library

[4]

Michal Aharon, Amit Kagian, and Oren Somekh. 2017. Adaptive Online Hyper-Parameters Tuning for Ad Event-Prediction Models. In Proceedings of the 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee, 672--679.

Digital Library

[5]

Natalie Aizenberg, Yehuda Koren, and Oren Somekh. 2012. Build your own music recommender by modeling internet radio streams. In Proceedings of the 21st international conference on World Wide Web. ACM, 1--10.

Digital Library

[6]

Robert M Bell and Yehuda Koren. 2007. Lessons from the Netflix prize challenge. Acm Sigkdd Explorations Newsletter, Vol. 9, 2 (2007), 75--79.

Digital Library

[7]

Niv Buchbinder, Moran Feldman, Arpita Ghosh, and Joseph Naor. 2014. Frequency capping in online advertising. Journal of Scheduling, Vol. 17, 4 (2014), 385--398.

Digital Library

[8]

Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: simplified data processing on large clusters. Commun. ACM, Vol. 51, 1 (2008), 107--113.

Digital Library

[9]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research (2011), 2121--2159.

[10]

Benjamin Edelman, Michael Ostrovsky, and Michael Schwarz. 2007. Internet advertising and the generalized second-price auction: Selling billions of dollars worth of keywords. The American economic review, Vol. 97, 1 (2007), 242--259.

[11]

Tom Fawcett. 2006. An introduction to ROC analysis. Pattern recognition letters, Vol. 27, 8 (2006), 861--874.

Digital Library

[12]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 173--182.

Digital Library

[13]

Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et almbox. 2014. Practical lessons from predicting clicks on ads at facebook. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising. ACM, 1--9.

Digital Library

[14]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 8 (2009), 30--37.

Digital Library

[15]

Pei Lee, Laks VS Lakshmanan, Mitul Tiwari, and Sam Shah. 2014. Modeling impression discounting in large-scale recommender systems. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining . ACM, 1837--1846.

Digital Library

[16]

Cheng Li, Yue Lu, Qiaozhu Mei, Dong Wang, and Sandeep Pandey. 2015. Click-through prediction for advertising in twitter timeline. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1959--1968.

Digital Library

[17]

Hao Ma, Xueqing Liu, and Zhihong Shen. 2016. User fatigue in online news recommendation. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 1363--1372.

Digital Library

[18]

Brendan McMahan. 2011. Follow-the-regularized-leader and mirror descent: Equivalence theorems and l1 regularization. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics . 525--533.

[19]

H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et almbox. 2013. Ad click prediction: a view from the trenches. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 1222--1230.

Digital Library

Cited By

Li NBan XLing CGao CHu LJiang PGai KLi YLiao QHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Modeling User Fatigue for Sequential RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657802(996-1005)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657802
Quin FWeyns DGalster MSilva C(2024)A/B testingJournal of Systems and Software10.1016/j.jss.2024.112011211:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112011
Ferrari Dacrema MCastells PBasilico JCremonesi P(2023)Workshop on Learning and Evaluating Recommendations with Impressions (LERI)Proceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608756(1248-1251)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608756
Show More Cited By

Index Terms

Soft Frequency Capping for Improved Ad Click Prediction in Yahoo Gemini Native
1. Applied computing
  1. Electronic commerce
    1. Online auctions
2. Information systems
  1. Information retrieval
    1. Users and interactive retrieval
      1. Personalization
  2. World Wide Web
    1. Online advertising
      1. Content match advertising

Recommendations

Feature Enhancement via User Similarities Networks for Improved Click Prediction in Yahoo Gemini Native
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Yahoo's native advertising marketplace (also known as Gemini native) serves billions of ad impressions daily, reaching many hundreds of millions USD in yearly revenue. Driving Gemini native models that are used to predict ad click probability (pCTR) is ...
Combating Ad Fatigue via Frequency-Recency Features in Online Advertising Systems
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Online advertising is a driving force of Internet services today. One of the main challenges in advertising systems is finding the right balance between user experience and overall revenue.

In this paper we address one of the problems that negatively ...
Carousel Ads Optimization in Yahoo Gemini Native
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Yahoo's native advertising (also known as Gemini native) serves billions of ad impressions daily, reaching a yearly run-rate of many hundred of millions USD. Driving Gemini native models for predicting both click probability (pCTR) and conversion ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
464
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li NBan XLing CGao CHu LJiang PGai KLi YLiao QHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Modeling User Fatigue for Sequential RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657802(996-1005)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657802
Quin FWeyns DGalster MSilva C(2024)A/B testingJournal of Systems and Software10.1016/j.jss.2024.112011211:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112011
Ferrari Dacrema MCastells PBasilico JCremonesi P(2023)Workshop on Learning and Evaluating Recommendations with Impressions (LERI)Proceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608756(1248-1251)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608756
Silberstein NShoham OKlein AFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Combating Ad Fatigue via Frequency-Recency Features in Online Advertising SystemsProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615461(4822-4828)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615461
Abutbul EKaplan YKrasne NSomekh ODavid ODuvdevany OSegal E(2023)Audience Prospecting for Dynamic-Product-Ads in Native Advertising2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386796(1571-1580)Online publication date: 15-Dec-2023
https://doi.org/10.1109/BigData59044.2023.10386796
Shtoff AKaplan YRaviv A(2023)Improving conversion rate prediction via self-supervised pre-training in online advertising2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386162(1835-1842)Online publication date: 15-Dec-2023
https://doi.org/10.1109/BigData59044.2023.10386162
Perez Maurera FFerrari Dacrema MCremonesi P(2022)Towards the Evaluation of Recommender Systems with ImpressionsProceedings of the 16th ACM Conference on Recommender Systems10.1145/3523227.3551483(610-615)Online publication date: 12-Sep-2022
https://dl.acm.org/doi/10.1145/3523227.3551483
Xie RLing CZhang SXia FLin LAl Hasan MXiong L(2022)Multi-granularity Fatigue in RecommendationProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557651(4595-4599)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557651
Kaplan YShtoff AShadi TSomekh OKoren Y(2022)Conversion-Based Dynamic-Creative-Optimization in Native Advertising2022 IEEE International Conference on Big Data (Big Data)10.1109/BigData55660.2022.10020498(2273-2278)Online publication date: 17-Dec-2022
https://doi.org/10.1109/BigData55660.2022.10020498
Kaplan YKrasne NShtoff ASomekh ODemartini GZuccon GCulpepper JHuang ZTong H(2021)Unbiased Filtering of Accidental Clicks in Verizon Media Native AdvertisingProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3481958(3878-3887)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3481958
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents