Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Attention-based causal representation learning for out-of-distribution recommendation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Out-of-distribution (OOD) recommendations have emerged as a popular field in recommendation systems. Traditional causal OOD recommendation frameworks often overlook shifts in latent user features and the interrelations between different user preferences. To address these issues, this paper proposes an innovative framework called Attention-based Causal OOD Recommendation (ABCOR), which applies the attention mechanism in two distinct ways. For shifts in latent user features, variational attention is employed to analyze shift information and refine the interaction-generation process. Besides, ABCOR integrates a multi-head self-attention layer to infer the complex user preference relationship and enhance recommendation accuracy before calculating post-intervention interaction probabilities. The proposed method has been validated on two public real-world datasets, and the results demonstrate that the proposal significantly outperforms the current state-of-the-art COR methods. Codes are available at https://github.com/YaffaGan/ABCOR.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The Meituan and Yelp datasets are publicly available at https://www.biendata.xyz/competition/smp2021_2/ and https://www.yelp.com/dataset.

References

  1. Sharma K, Lee YC, Nambi S et al (2024) A survey of graph neural networks for social recommender systems. ACM Comput Surv. https://doi.org/10.1145/3661821.https://doi.org/10.1145/3661821. just Accepted

  2. Gao C, Zheng Y, Li N et al (2021) A survey of graph neural networks for recommender systems: challenges, methods, and directions. ACM Transactions on Recommender Systems 1:1 – 51. https://api.semanticscholar.org/CorpusID:237940542

  3. Xiao T, Wang S (2022) Towards unbiased and robust causal ranking for recommender systems. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. https://api.semanticscholar.org/CorpusID:246828659

  4. Ovaisi Z, Heinecke S, Li J et al (2022) Rgrecsys: a toolkit for robustness evaluation of recommender systems. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. https://api.semanticscholar.org/CorpusID:245877841

  5. Wang W, Lin X, Feng F et al (2022) Causal representation learning for out-of-distribution recommendation. Proceedings of the ACM Web Conference 2022. https://api.semanticscholar.org/CorpusID:248367478

  6. Li X, Wang W, Hu X et al (2019) Selective kernel networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 510–519. https://doi.org/10.1109/CVPR.2019.00060

  7. Misra D, Nalamada T, Arasanipalai AU et al (2021) Rotate to attend: convolutional triplet attention module. In: 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pp 3138–3147. https://doi.org/10.1109/WACV48630.2021.00318

  8. Zhang H, Wu C, Zhang Z et al (2022) Resnest: split-attention networks. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 2735–274. https://doi.org/10.1109/CVPRW56347.2022.00309

  9. Vaswani A, Shazeer NM, Parmar N et al (2017) Attention is all you need. In: Neural information processing systems. https://api.semanticscholar.org/CorpusID:13756489

  10. Devlin J, Chang MW, Lee K et al (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: North American chapter of the association for computational linguistics. https://api.semanticscholar.org/CorpusID:52967399

  11. Xiao J, Ye H, He X et al (2017) Attentional factorization machines: learning the weight of feature interactions via attention networks. In: Proceedings of the 26th international joint conference on artificial intelligence. AAAI Press, IJCAI’17, pp 3119–3125

  12. Ni J, Huang Z, Yu C et al (2022) Comparative convolutional dynamic multi-attention recommendation model. IEEE Transactions on Neural Networks and Learning Systems 33(8):3510–352. https://doi.org/10.1109/TNNLS.2021.3053245

    Article  Google Scholar 

  13. Liu Z, Yuan B, Ma Y (2022) A multi-task dual attention deep recommendation model using ratings and review helpfulness. Appl Intell 52(5):5595–560. https://doi.org/10.1007/s10489-021-02666-y

    Article  Google Scholar 

  14. Hu Q, Han Z, Lin X et al (2019) Learning peer recommendation using attention-driven cnn with interaction tripartite graph. Inf Sci 479:231–249. https://api.semanticscholar.org/CorpusID:59528813

  15. Wen P, Yuan W, Qin Q et al (2020) Neural attention model for recommendation based on factorization machines. Appl Intell 51:1829–1844. https://api.semanticscholar.org/CorpusID:225120760

  16. Li G, Zhu J, Xi H (2021) Deep recommendation based on dual attention mechanism. In: 2021 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), pp 675–680. https://doi.org/10.1109/ICAICA52286.2021.9498244

  17. Wang R, Wu Z, Lou J et al (2022) Attention-based dynamic user modeling and deep collaborative filtering recommendation. Expert Syst Appl 188:11603. https://doi.org/10.1016/j.eswa.2021.116036. https://www.sciencedirect.com/science/article/pii/S0957417421013816

  18. Thiebes S, Lins S, Sunyaev A (2020) Trustworthy artificial intelligence. Electronic Markets 31:447 – 464. https://api.semanticscholar.org/CorpusID:224877177

  19. Luo H, Zhuang F, Xie R et al (2024) A survey on causal inference for recommendation. The Innovation 5(2):100590. https://doi.org/10.1016/j.xinn.2024.100590. https://www.sciencedirect.com/science/article/pii/S2666675824000286

  20. Cao L (2016) Non-iid recommender systems: a review and framework of recommendation paradigm shifting. Engineering 2(2):212–224. https://doi.org/10.1016/J.ENG.2016.02.013. https://www.sciencedirect.com/science/article/pii/S2095809916309481

  21. Yang C, Wu Q, Wen Q et al (2024) Towards out-of-distribution sequential event prediction: a causal treatment. In: Proceedings of the 36th international conference on neural information processing systems. Curran Associates Inc., Red Hook, NY, USA, NIPS ’22

  22. He Y, Wang Z, Cui P et al (2022) Causpref: causal preference learning for out-of-distribution recommendation. Proceedings of the ACM Web Conference 2022. https://api.semanticscholar.org/CorpusID:246652643

  23. Wang W, Feng F, He X et al (2021) Deconfounded recommendation for alleviating bias amplification. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. https://api.semanticscholar.org/CorpusID:235166201

  24. Wang W, Feng F, He X et al (2020) Clicks can be cheating: counterfactual recommendation for mitigating clickbait issue. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. https://api.semanticscholar.org/CorpusID:235185402

  25. Wang Z, Zhang J, Xu H et al (2021) Counterfactual data-augmented sequential recommendation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. https://api.semanticscholar.org/CorpusID:235792483

  26. Zhang S, Yao D, Zhao Z et al (2021a) Causerec: counterfactual user sequence synthesis for sequential recommendation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. https://api.semanticscholar.org/CorpusID:235792358

  27. Zhang Y, Feng F, He X et al (2021b) Causal intervention for leveraging popularity bias in recommendation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. https://api.semanticscholar.org/CorpusID:234482660

  28. Zou H, Cui P, Li B et al (2020) Counterfactual prediction for bundle treatment. In: Neural information processing systems. https://api.semanticscholar.org/CorpusID:227275241

  29. Liu D, Cheng P, Zhu H et al (2022) Debiased representation learning in recommendation via information bottleneck. ACM Transactions on Recommender Systems 1:1–27. https://api.semanticscholar.org/CorpusID:253110445

  30. Wang S, Chen X, Sheng Q et al (2023) Causal disentangled variational auto-encoder for preference understanding in recommendation. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. https://api.semanticscholar.org/CorpusID:258180280

  31. Didelez V, Pigeot I (2001) Judea pearl: Causality: Models, reasoning, and inference. Politische Vierteljahresschrift 42:313–315. https://api.semanticscholar.org/CorpusID:141473148

  32. Yang M, Liu F, Chen Z et al (2021) Causalvae: disentangled representation learning via neural structural causal models. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 9588–9597. https://doi.org/10.1109/CVPR46437.2021.00947

  33. Zhang Y, Feng F, He X et al (2021) Causal intervention for leveraging popularity bias in recommendation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. https://api.semanticscholar.org/CorpusID:234482660

  34. Cunha W, Mangaravite V, Gomes C et al (2021) On the cost-effectiveness of neural and non-neural approaches and representations for text classification: a comprehensive comparative study. Information Processing & Management 58(3):10248. https://doi.org/10.1016/j.ipm.2020.102481. https://www.sciencedirect.com/science/article/pii/S0306457320309705

  35. Rendle S (2010) Factorization machines. 2010 IEEE International Conference on Data Mining pp 995–1000. https://api.semanticscholar.org/CorpusID:17265929

  36. He X, Chua TS (2017) Neural factorization machines for sparse predictive analytics. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. https://api.semanticscholar.org/CorpusID:2021204

  37. Liang D, Krishnan RG, Hoffman MD et al (2018) Variational autoencoders for collaborative filtering. Proceedings of the 2018 World Wide Web Conference. https://api.semanticscholar.org/CorpusID:3361310

  38. Shenbin I, Alekseev A, Tutubalina E et al (2020) Recvae: A new variational autoencoder for top-n recommendations with implicit feedback. In: Proceedings of the 13th International Conference on Web Search and Data Mining. Association for Computing Machinery, New York, NY, USA, WSDM ’20, pp 528–536. https://doi.org/10.1145/3336191.3371831

  39. Ma J, Zhou C, Cui P et al (2019) Learning disentangled representations for recommendation. In: Wallach H, Larochelle H, Beygelzimer A et al (eds) Advances in neural information processing systems, vol 32. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2019/file/a2186aa7c086b46ad4e8bf81e2a3a19b-Paper.pdf

Download references

Acknowledgements

This research is supported by the Shenzhen Science and Technology Program (Grant No.ZDSYS20210623092007023) and the Educational Commission of Guangdong Province (Grant No. 2021ZDZX1069). We also acknowledge the support from the SUSTech International Center for Mathematics and Data Science Institute. And we would also like to thank Doctor Wenwu Gong for his valuable guidance in the completion of the revised manuscript.

Author information

Author notes

  1. Qianqian Wang and Zhejun Huang contributed equally to this work.

Authors and Affiliations

  1. Department of Statistics and Data Science, Southern University of Science and Technology, Xueyuan Avenue, Shenzhen, 518055, Guangdong, China

    Yuehua Gan, Qianqian Wang, Zhejun Huang & Lili Yang

  2. Shenzhen Key Laboratory of Safety and Security for Next Generation of Industrial Internet, Shenzhen, China

    Lili Yang

Authors
  1. Yuehua Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qianqian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhejun Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lili Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors confirm their contribution to the paper as follows: study conception and design: Y. Gan, Q. Wang, L. Yang; data collection: Y. Gan, L. Yang; analysis and interpretation of results: Y. Gan, Q. Wang, L. Yang; draft manuscript preparation: Y. Gan, Q. Wang, Z. Huang, L. Yang. All authors contributed to the article and approved the submitted version.

Corresponding author

Correspondence to Lili Yang.

Ethics declarations

Competing Interests

The authors declare no potential conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gan, Y., Wang, Q., Huang, Z. et al. Attention-based causal representation learning for out-of-distribution recommendation. Appl Intell 54, 12964–12978 (2024). https://doi.org/10.1007/s10489-024-05835-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05835-x

Keywords

Search

Navigation