research-article

A unified neural network approach to e-commerce relevance learning

Authors:

Yunjiang Jiang,

Eric ZhaoAuthors Info & Claims

DLP-KDD '19: Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data

Article No.: 10, Pages 1 - 7

https://doi.org/10.1145/3326937.3341259

Published: 05 August 2019 Publication History

Abstract

Result relevance scoring is critical to e-commerce search user experience. Traditional information retrieval methods focus on keyword matching and hand-crafted or counting-based numeric features, with limited understanding of item semantic relevance. We describe a highly-scalable feed-forward neural model to provide relevance score for (query, item) pairs, using only user query and item title as features, and both user click feedback as well as limited human ratings as labels. Several general enhancements were applied to further optimize eval/test metrics, including Siamese pairwise architecture, random batch negative co-training, and point-wise fine-tuning. We found significant improvement over GBDT baseline as well as several off-the-shelf deep-learning baselines on an independently constructed ratings dataset. The GBDT model relies on 10 times more features. We also present metrics for select subset combinations of techniques mentioned above.

References

[1]

Q. Ai, Y. Zhang, K. Bi, X. Chen, and W. B. Croft. Learning a hierarchical embedding model for personalized product search. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 645--654. ACM, 2017.

Digital Library

[2]

J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah. Signature verification using a "siamese" time delay neural network. In Advances in neural information processing systems, pages 737--744, 1994.

Digital Library

[3]

C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. In Proceedings of the 22Nd International Conference on Machine Learning, ICML '05, pages 89--96, New York, NY, USA, 2005. ACM.

Digital Library

[4]

C. J. C. Burges. From RankNet to LambdaRank to LambdaMART: An overview. Technical report, Microsoft Research, 2010.

[5]

T. Chen and C. Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pages 785--794, New York, NY, USA, 2016. ACM.

Digital Library

[6]

S. Chopra, R. Hadsell, Y. LeCun, et al. Learning a similarity metric discriminatively, with application to face verification. In CVPR (1), pages 539--546, 2005.

[7]

J. Davis and M. Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning, pages 233--240. ACM, 2006.

Digital Library

[8]

J. Guo, Y. Fan, Q. Ai, and W. B. Croft. A deep relevance matching model for ad-hoc retrieval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pages 55--64, 2016.

Digital Library

[9]

P.-S. Huang, X. He, J. Gao, L. Deng, A. Acero, and L. Heck. Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management, pages 2333--2338, 2013.

Digital Library

[10]

T. Joachims. Optimizing search engines using clickthrough data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '02, pages 133--142, 2002.

Digital Library

[11]

T. Kenter, A. Borisov, C. Van Gysel, M. Dehghani, M. de Rijke, and B. Mitra. Neural networks for information retrieval. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '17, pages 1403--1406, 2017.

Digital Library

[12]

T.-Y. Liu. Learning to rank for information retrieval. Found. Trends Inf. Retr., 3(3):225--331, Mar. 2009.

Digital Library

[13]

T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space, 2013.

[14]

B. Mitra and N. Craswell. An introduction to neural information retrieval. Foundations and Trends® in Information Retrieval, 13(1):1--126, 2018.

Digital Library

[15]

B. Mitra, F. Diaz, and N. Craswell. Learning to match using local and distributed representations of text for web search. In Proceedings of the 26th International Conference on World Wide Web, WWW '17, pages 1291--1299, 2017.

Digital Library

[16]

L. Pang, Y. Lan, J. Guo, J. Xu, J. Xu, and X. Cheng. Deeprank: A new deep architecture for relevance ranking in information retrieval. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM '17, pages 257--266, 2017.

Digital Library

[17]

Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil. A latent semantic model with convolutional-pooling structure for information retrieval. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM '14, pages 101--110, 2014.

Digital Library

[18]

C. Van Gysel, M. de Rijke, and E. Kanoulas. Learning latent vector spaces for product search. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pages 165--174. ACM, 2016.

Digital Library

[19]

J. Xu and H. Li. Adarank: A boosting algorithm for information retrieval. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '07, pages 391--398, 2007.

Digital Library

[20]

D. Yin, Y. Hu, J. Tang, T. Daly, M. Zhou, H. Ouyang, J. Chen, C. Kang, H. Deng, C. Nobata, et al. Ranking relevance in yahoo search. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 323--332, 2016.

Digital Library

[21]

W. Yin, H. Schütze, B. Xiang, and B. Zhou. Abcnn: Attention-based convolutional neural network for modeling sentence pairs. Transactions of the Association for Computational Linguistics, 4:259--272, 2016.

[22]

J. Yu, S. Mohan, D. P. Putthividhya, and W.-K. Wong. Latent dirichlet allocation based diversified retrieval for e-commerce search. In Proceedings of the 7th ACM international conference on Web search and data mining, pages 463--472. ACM, 2014.

Digital Library

[23]

Z. Zheng, K. Chen, G. Sun, and H. Zha. A regression framework for learning ranking functions using relative relevance judgments. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '07, pages 287--294, 2007.

Digital Library

Cited By

Lin ZTan JOu DChen XYao SZheng BBaeza-Yates RBonchi F(2024)Deep Bag-of-Words Model: An Efficient and Interpretable Relevance Architecture for Chinese E-CommerceProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671559(5398-5408)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671559
Yao STan JChen XZhang JZeng XYang KZhang ARangwala H(2022)ReprBERT: Distilling BERT to an Efficient Representation-Based Relevance Model for E-CommerceProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539090(4363-4371)Online publication date: 14-Aug-2022
https://dl.acm.org/doi/10.1145/3534678.3539090
Zeng ZHuang YWu TDeng HXu JZheng BAl Hasan MXiong L(2022)Graph-based Weakly Supervised Framework for Semantic Relevance Learning in E-commerceProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557143(3634-3643)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557143
Show More Cited By

Recommendations

Joint Learning of E-commerce Search and Recommendation with a Unified Graph Neural Network
WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

Click-through rate (CTR) prediction plays an important role in search and recommendation, which are the two most prominent scenarios in e-commerce. A number of models have been proposed to predict CTR by mining user behaviors, especially users' ...
Unified relevance models for rating prediction in collaborative filtering

Collaborative filtering aims at predicting a user's interest for a given item based on a collection of user profiles. This article views collaborative filtering as a problem highly related to information retrieval, drawing an analogy between the ...
Relevance-aware graph neural network for session-based recommendation
Abstract
Session-based recommendation aims to predict the next item a user may interact with based on sessions’ information. Most existing session-based recommendation models only rely on the information of item-to-item transitions. However, we argue that ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

DLP-KDD '19: Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data

August 2019

109 pages

ISBN:9781450367837

DOI:10.1145/3326937

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 August 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

KDD '19

KDD '19: The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 5, 2019

Alaska, Anchorage

Acceptance Rates

DLP-KDD '19 Paper Acceptance Rate 15 of 22 submissions, 68%;

Overall Acceptance Rate 15 of 22 submissions, 68%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
166
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)2

Reflects downloads up to 27 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lin ZTan JOu DChen XYao SZheng BBaeza-Yates RBonchi F(2024)Deep Bag-of-Words Model: An Efficient and Interpretable Relevance Architecture for Chinese E-CommerceProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671559(5398-5408)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671559
Yao STan JChen XZhang JZeng XYang KZhang ARangwala H(2022)ReprBERT: Distilling BERT to an Efficient Representation-Based Relevance Model for E-CommerceProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539090(4363-4371)Online publication date: 14-Aug-2022
https://dl.acm.org/doi/10.1145/3534678.3539090
Zeng ZHuang YWu TDeng HXu JZheng BAl Hasan MXiong L(2022)Graph-based Weakly Supervised Framework for Semantic Relevance Learning in E-commerceProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557143(3634-3643)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557143
Shukla SMishra PSinha AGarg NPandey SRaja LKumar AChetri M(2022)Performance optimization of unstructured E‐commerce log data for activity and pattern evaluation using web analyticsIET Communications10.1049/cmu2.12529Online publication date: 4-Nov-2022
https://doi.org/10.1049/cmu2.12529
Yao STan JChen XYang KXiao RDeng HWan X(2021)Learning a Product Relevance Model from Click-Through Data in E-CommerceProceedings of the Web Conference 202110.1145/3442381.3450129(2890-2899)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3450129
Jiang YShang YLiu ZShen HXiao YXu SXiong WYan WJin D(2020)BERT2DNN: BERT Distillation with Massive Unlabeled Data for Online E-Commerce Search2020 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM50108.2020.00030(212-221)Online publication date: Nov-2020
https://doi.org/10.1109/ICDM50108.2020.00030

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents