short-paper

Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural Networks for CTR Prediction

Authors:

Bo ZhengAuthors Info & Claims

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 2161 - 2165

https://doi.org/10.1145/3404835.3463015

Published: 11 July 2021 Publication History

Abstract

Cross features play an important role in click-through rate (CTR) prediction. Most of the existing methods adopt a DNN-based model to capture the cross features in an implicit manner. These implicit methods may lead to a sub-optimized performance due to the limitation in explicit semantic modeling. Although traditional statistical explicit semantic cross features can address the problem in these implicit methods, it still suffers from some challenges, including lack of generalization and expensive memory cost. Few works focus on tackling these challenges. In this paper, we take the first step in learning the explicit semantic cross features and propose Pre-trained Cross Feature learning Graph Neural Networks (PCF-GNN), a GNN based pre-trained model aiming at generating cross features in an explicit fashion. Extensive experiments are conducted on both public and industrial datasets, where PCF-GNN shows competence in both performance and memory-efficiency in various tasks.

References

[1]

Andrew P Bradley. 1997. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition, Vol. 30, 7 (1997), 1145--1159.

[2]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7--10.

Digital Library

[3]

Fuli Feng, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua. 2020. Cross-GCN: Enhancing Graph Convolutional Network with k -Order Feature Interactions. arXiv preprint arXiv:2003.02587 (2020).

[4]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv:1703.04247 (2017).

[5]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in neural information processing systems. 1024--1034.

[6]

Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. 2019. Strategies for Pre-training Graph Neural Networks. In International Conference on Learning Representations .

[7]

Tongwen Huang, Zhiqi Zhang, and Junlin Zhang. 2019. FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction. In Proceedings of the 13th ACM Conference on Recommender Systems. 169--177.

Digital Library

[8]

Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, Vol. 30 (2017), 3146--3154.

[9]

Zekun Li, Zeyu Cui, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019. Fi-gnn: Modeling feature interactions via graph neural networks for ctr prediction. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 539--548.

Digital Library

[10]

Bin Liu, Ruiming Tang, Yingzhi Chen, Jinkai Yu, Huifeng Guo, and Yuzhou Zhang. 2019. Feature generation by convolutional neural network for click-through rate prediction. In The World Wide Web Conference. 1119--1129.

Digital Library

[11]

Qingqing Long, Yilun Jin, Guojie Song, Yi Li, and Wei Lin. 2020. Graph Structural-topic Neural Network. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1065--1073.

Digital Library

[12]

Qingqing Long, Yiming Wang, Lun Du, Guojie Song, Yilun Jin, and Wei Lin. 2019. Hierarchical Community Structure Preserving Network Embedding: A Subspace Approach. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 409--418.

Digital Library

[13]

Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. 2016. Deep crossing: Web-scale modeling without manually crafted combinatorial features. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 255--262.

Digital Library

[14]

Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019. Autoint: Automatic feature interaction learning via self-attentive neural networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1161--1170.

Digital Library

[15]

Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-scale commodity embedding for e-commerce recommendation in alibaba. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 839--848.

Digital Library

[16]

Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. In Proceedings of the ADKDD'17. 1--7.

Digital Library

[17]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1059--1068.

Digital Library

Cited By

Yuan YZhou YChen XXiong QOkere H(2024)Enhancing Recommendation Diversity and Novelty with Bi-LSTM and Mean Shift ClusteringElectronics10.3390/electronics1319384113:19(3841)Online publication date: 28-Sep-2024
https://doi.org/10.3390/electronics13193841
Sang LLi HZhang YZhang YYang Y(2024)AdaGIN: Adaptive Graph Interaction Network for Click-Through Rate PredictionACM Transactions on Information Systems10.1145/368178543:1(1-31)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3681785
Xu LWang ZZhang SYuan XWang MChen E(2024)Modeling Student Performance Using Feature Crosses Information for Knowledge TracingIEEE Transactions on Learning Technologies10.1109/TLT.2024.338104517(1390-1403)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TLT.2024.3381045
Show More Cited By

Index Terms

Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural Networks for CTR Prediction
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

GemNN: Gating-enhanced Multi-task Neural Networks with Feature Interaction Learning for CTR Prediction
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Deep neural network (DNN) models have been widely used for click-through rate (CTR) prediction in online advertising. The training framework typically consists of embedding layers and multi-layer perceptions (MLP). At Baidu Search Ads (a.k.a. Phoenix ...
Causality-based CTR prediction using graph neural networks
Highlights
- Propose a causality-based CTR prediction model in the GNNs framework (Causal-GNN) integrating representations of feature graph, user graph and ad graph.
Abstract
As a prevalent problem in online advertising, CTR prediction has attracted plentiful attention from both academia and industry. Recent studies have been reported to establish CTR prediction models in the graph neural networks (GNNs) ...
HoINT: Learning Explicit and Implicit High-order Feature Interactions for Click-through Rate Prediction
Abstract
Click-through rate (CTR) prediction is a research hotspot in the field of recommendation systems and online advertising. Because of the diversity, large-scale, and high real-time characteristics of Internet data, manual feature interaction is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2021

2998 pages

ISBN:9781450380379

DOI:10.1145/3404835

General Chairs:
Fernando Diaz
(Google)
,
Chirag Shah
University of Washington
,
Torsten Suel
New York University
,
Program Chairs:
Pablo Castells
Universidad Autónoma de Madrid, Amazon
,
Rosie Jones
Spotify
,
Tetsuya Sakai
Waseda University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

SIGIR '21

Sponsor:

SIGIR

SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2021

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
321
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yuan YZhou YChen XXiong QOkere H(2024)Enhancing Recommendation Diversity and Novelty with Bi-LSTM and Mean Shift ClusteringElectronics10.3390/electronics1319384113:19(3841)Online publication date: 28-Sep-2024
https://doi.org/10.3390/electronics13193841
Sang LLi HZhang YZhang YYang Y(2024)AdaGIN: Adaptive Graph Interaction Network for Click-Through Rate PredictionACM Transactions on Information Systems10.1145/368178543:1(1-31)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3681785
Xu LWang ZZhang SYuan XWang MChen E(2024)Modeling Student Performance Using Feature Crosses Information for Knowledge TracingIEEE Transactions on Learning Technologies10.1109/TLT.2024.338104517(1390-1403)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TLT.2024.3381045
Zhang YChen JZou ZYao MZhang SXiao L(2024)CSIA-GCN: A Doctor Recommendation Model Based on Interactive Graph Convolutional Networks2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650337(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650337
Zhang ZJia YHou YYu X(2024)Explicit Behavior Interaction with Heterogeneous Graph for Multi-behavior RecommendationData Science and Engineering10.1007/s41019-023-00238-39:2(133-151)Online publication date: 19-Jan-2024
https://doi.org/10.1007/s41019-023-00238-3
Huang XChen HZhang Z(2023)Design and Application of Deep Hash Embedding Algorithm with Fusion Entity Attribute InformationEntropy10.3390/e2502036125:2(361)Online publication date: 15-Feb-2023
https://doi.org/10.3390/e25020361
Tian ZBai TZhang ZXu ZLin KWen JZhao WChua TLauw HSi LTerzi ETsaparas P(2023)Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge DistillationProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3570384(715-723)Online publication date: 27-Feb-2023
https://dl.acm.org/doi/10.1145/3539597.3570384
Yang BSun LXing YZhou JCheng C(2023)GAFM: Learning the Weights of Feature Interaction via Graph Attentional Factorization Machine2023 IEEE International Conference on Knowledge Graph (ICKG)10.1109/ICKG59574.2023.00009(27-34)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICKG59574.2023.00009
Wei TChow T(2023)FGCRKnowledge-Based Systems10.1016/j.knosys.2023.110806277:COnline publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110806
Zhai PYang YZhang C(2023)Causality-based CTR prediction using graph neural networksInformation Processing and Management: an International Journal10.1016/j.ipm.2022.10313760:1Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.ipm.2022.103137
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents