short-paper

Field-aware probabilistic embedding neural network for CTR prediction

Authors:

Shengyu ZhangAuthors Info & Claims

RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems

Pages 412 - 416

https://doi.org/10.1145/3240323.3240396

Published: 27 September 2018 Publication History

Abstract

For Click-Through Rate (CTR) prediction, Field-aware Factorization Machines (FFM) have exhibited great effectiveness by considering field information. However, it is also observed that FFM suffers from the overfitting problem in many practical scenarios. In this paper, we propose a Field-aware Probabilistic Embedding Neural Network (FPENN) model with both good generalization ability and high accuracy. FPENN estimates the probability distribution of the field-aware embedding rather than using the single point estimation (the maximum a posteriori estimation) to prevent overfitting. Both low-order and high-order feature interactions are considered to improve the accuracy. FPENN consists of three components, i.e., FPE component, Quadratic component and Deep component. FPE component outputs probabilistic embedding to the other two components, where various confidence levels for feature embeddings are incorporated to enhance the robustness and the accuracy. Quadratic component is designed for extracting low-order feature interactions, while Deep component aims at capturing high-order feature interactions. Experiments are conducted on two benchmark datasets, Avazu and Criteo. The results confirm that our model alleviates the overfitting problem while having a higher accuracy.

References

[1]

Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. 2002. Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 2--3 (2002), 235--256.

Digital Library

[2]

Olivier Chapelle and Lihong Li. 2011. An empirical evaluation of thompson sampling. In Advances in neural information processing systems. 2249--2257.

Digital Library

[3]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 7--10.

Digital Library

[4]

Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 191--198.

Digital Library

[5]

Google. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http://tensorflow.org/ Software available from tensorflow.org.

[6]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (2017).

Digital Library

[7]

Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware factorization machines for CTR prediction. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 43--50.

Digital Library

[8]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014). arXiv:1412.6980 http://arxiv.org/abs/1412.6980

[9]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv.1312.6114 (2013).

[10]

Qiang Liu, Feng Yu, Shu Wu, and Liang Wang. 2015. A convolutional click prediction model. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 1743--1746.

Digital Library

[11]

Kevin P. Murphy. 2012. Machine learning - a probabilistic perspective. MIT Press.

Digital Library

[12]

Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016. Product-based neural networks for user response prediction. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, 1149--1154.

[13]

Steffen Rendle. 2010. Factorization machines. In 2010 IEEE 10th International Conference on Data Mining (ICDM). IEEE, 995--1000.

Digital Library

[14]

Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting clicks: estimating the click-through rate for new ads. In Proceedings of the 16th international conference on World Wide Web. ACM, 521--530.

Digital Library

[15]

Francisco R Ruiz, Michalis Titsias RC AUEB, and David Blei. 2016. The generalized reparameterization gradient. In Advances in Neural Information Processing Systems. 460--468.

Digital Library

[16]

Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. 2016. Deep Crossing: Web-scale modeling without manually crafted combinatorial features. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 255--262.

Digital Library

[17]

Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv preprint arXiv:1708.04617 (2017).

[18]

Weinan Zhang, Tianming Du, and Jun Wang. 2016. Deep learning over multi-field categorical data. In European conference on information retrieval. Springer, 45--57.

[19]

Guorui Zhou, Chengru Song, Xiaoqiang Zhu, Xiao Ma, Yanghui Yan, Xingya Dai, Han Zhu, Junqi Jin, Han Li, and Kun Gai. 2017. Deep interest network for click-through rate prediction. arXiv preprint arXiv:1706.06978 (2017).

Digital Library

Cited By

Kong MSu RZhao SHou M(2023)DEPHN: Different Expression Parallel Heterogeneous Network using virtual gradient optimization for Multi-task Learning2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191469(1-8)Online publication date: 18-Jun-2023
https://doi.org/10.1109/IJCNN54540.2023.10191469
Zhang LLiu FWu HZhuang XYan Y(2023)CFF: combining interactive features and user interest features for click-through rate predictionThe Journal of Supercomputing10.1007/s11227-023-05598-180:3(3282-3309)Online publication date: 4-Sep-2023
https://doi.org/10.1007/s11227-023-05598-1
Wang QLiu FZhao XTan Q(2022)A CTR prediction model based on session interestPLOS ONE10.1371/journal.pone.027304817:8(e0273048)Online publication date: 17-Aug-2022
https://doi.org/10.1371/journal.pone.0273048
Show More Cited By

Index Terms

Field-aware probabilistic embedding neural network for CTR prediction
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Holistic Neural Network for CTR Prediction
WWW '17 Companion: Proceedings of the 26th International Conference on World Wide Web Companion

This paper proposes HNN, a holistic neural network structure for click-through rate (CTR) prediction in recommender systems. Empirically, equipped with HNN, the performance of deep neural networks for CTR prediction are improved on Criteo and Huawei App ...
Order-aware Embedding Neural Network for CTR Prediction
SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Product based models, which represent multi-field categorical data as embedding vectors of features, then model feature interactions in terms of vector product of shared embedding, have been extensively studied and have become one of the most popular ...
Dual Graph enhanced Embedding Neural Network for CTR Prediction
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

CTR prediction, which aims to estimate the probability that a user will click an item, plays a crucial role in online advertising and recommender system. Feature interaction modeling based and user interest mining based methods are the two kinds of most ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems

September 2018

600 pages

ISBN:9781450359016

DOI:10.1145/3240323

General Chairs:
Sole Pera
Boise State University
,
Michael Ekstrand
Boise State University
,
Program Chairs:
Xavier Amatriain
Curai, USA
,
John O'Donovan
University of California, Santa Barbara

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 September 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

RecSys '18

Sponsor:

SIGCHI

RecSys '18: Twelfth ACM Conference on Recommender Systems

October 2, 2018

British Columbia, Vancouver, Canada

Acceptance Rates

RecSys '18 Paper Acceptance Rate 32 of 181 submissions, 18%;

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
828
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)5

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kong MSu RZhao SHou M(2023)DEPHN: Different Expression Parallel Heterogeneous Network using virtual gradient optimization for Multi-task Learning2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191469(1-8)Online publication date: 18-Jun-2023
https://doi.org/10.1109/IJCNN54540.2023.10191469
Zhang LLiu FWu HZhuang XYan Y(2023)CFF: combining interactive features and user interest features for click-through rate predictionThe Journal of Supercomputing10.1007/s11227-023-05598-180:3(3282-3309)Online publication date: 4-Sep-2023
https://doi.org/10.1007/s11227-023-05598-1
Wang QLiu FZhao XTan Q(2022)A CTR prediction model based on session interestPLOS ONE10.1371/journal.pone.027304817:8(e0273048)Online publication date: 17-Aug-2022
https://doi.org/10.1371/journal.pone.0273048
Xu XYang CYu QFang ZWang JFan CHe YPeng CLin ZShao J(2022)Alleviating Cold-start Problem in CTR Prediction with A Variational Embedding Learning FrameworkProceedings of the ACM Web Conference 202210.1145/3485447.3512048(27-35)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3512048
Miao XMa LYang ZShao YCui BYu LJiang J(2022)CuWide: Towards Efficient Flow-Based Training for Sparse Wide Models on GPUsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2020.303810934:9(4119-4132)Online publication date: 1-Sep-2022
https://doi.org/10.1109/TKDE.2020.3038109
Yang YZhai P(2022)Click-through rate prediction in online advertisingInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10285359:2Online publication date: 9-May-2022
https://dl.acm.org/doi/10.1016/j.ipm.2021.102853
Wu JHe XWang XWang QChen WLian JXie X(2022)Graph convolution machine for context-aware recommender systemFrontiers of Computer Science10.1007/s11704-021-0261-816:6Online publication date: 22-Jan-2022
https://doi.org/10.1007/s11704-021-0261-8
Jiang ZLi LWang D(2022)MCGM: A multi-channel CTR model with hierarchical gated mechanism for precision marketingWorld Wide Web10.1007/s11280-022-01125-z26:4(2115-2141)Online publication date: 24-Dec-2022
https://doi.org/10.1007/s11280-022-01125-z
Zhao PLuo CZhou CQiao BHe JZhang LLin QDiaz FShah CSuel TCastells PJones RSakai T(2021)RLNF: Reinforcement Learning based Noise Filtering for Click-Through Rate PredictionProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3404835.3463012(2268-2272)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3404835.3463012
Cao TXu QYang ZHuang Q(2021)Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR PredictionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2021.3103741(1-1)Online publication date: 2021
https://doi.org/10.1109/TPAMI.2021.3103741
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents