research-article

Large Language Models for Next Point-of-Interest Recommendation

Authors:

Maarten de Rijke,

Flora D. SalimAuthors Info & Claims

SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1463 - 1472

https://doi.org/10.1145/3626772.3657840

Published: 11 July 2024 Publication History

Abstract

The next Point of Interest (POI) recommendation task is to predict users' immediate next POI visit given their historical data. Location-Based Social Network (LBSN) data, which is often used for the next POI recommendation task, comes with challenges. One frequently disregarded challenge is how to effectively use the abundant contextual information present in LBSN data. Previous methods are limited by their numerical nature and fail to address this challenge. In this paper, we propose a framework that uses pretrained Large Language Models (LLMs) to tackle this challenge. Our framework allows us to preserve heterogeneous LBSN data in its original format, hence avoiding the loss of contextual information. Furthermore, our framework is capable of comprehending the inherent meaning of contextual information due to the inclusion of commonsense knowledge. In experiments, we test our framework on three real-world LBSN datasets. Our results show that the proposed framework outperforms the state-of-the-art models in all three datasets. Our analysis demonstrates the effectiveness of the proposed framework in using contextual information as well as alleviating the commonly encountered cold-start and short trajectory problems.

References

[1]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language Models are Few-shot Learners. In Advances in neural information processing systems, Vol. 33. 1877--1901.

[2]

Ching Chang, Wen-Chih Peng, and Tien-Fu Chen. 2023. LLM4TS: Two-Stage Fine-Tuning for Time-Series Forecasting with Pre-Trained LLMs. arXiv preprint arXiv:2308.08469.

[3]

Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2023. LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models. arXiv:2309.12307.

[4]

Chen Cheng, Haiqin Yang, Michael R. Lyu, and Irwin King. 2013. Where You Like to Go Next: Successive Point-of-Interest Recommendation. In IJCAI 2013, Proceedings of the 23rd International Joint Conference on Artificial Intelligence, Beijing, China, August 3--9, 2013, Francesca Rossi (Ed.). IJCAI/AAAI, 2605--2611.

[5]

Eunjoon Cho, Seth A Myers, and Jure Leskovec. 2011. Friendship and Mobility: User Movement in Location-based Social Networks. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1082--1090.

Digital Library

[6]

Tri Dao. 2023. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. arXiv preprint arXiv:2307.08691.

[7]

Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. FlashAttention: Fast and Memory-efficient Exact Attention with IO-awareness. Advances in Neural Information Processing Systems, Vol. 35 (2022), 16344--16359.

[8]

Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 2021. 8-bit Optimizers via Block-wise Quantization. In International Conference on Learning Representations.

[9]

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient Finetuning of Quantized LLMs. arXiv preprint arXiv:2305.14314.

[10]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, Minneapolis, Minnesota, 4171--4186. https://doi.org/10.18653/v1/N19--1423

[11]

Yaron Fairstein, Elad Haramaty, Arnon Lazerson, and Liane Lewin-Eytan. 2022. External Evaluation of Ranking Models under Extreme Position-Bias. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, New York, NY, USA, 252--261.

Digital Library

[12]

Shanshan Feng, Xutao Li, Yifeng Zeng, Gao Cong, and Yeow Meng Chee. 2015. Personalized Ranking Metric Embedding for Next New POI Recommendation. In IJCAI'15 Proceedings of the 24th International Conference on Artificial Intelligence. ACM, 2069--2075.

[13]

Jesse Harte, Wouter Zorgdrager, Panos Louridas, Asterios Katsifodimos, Dietmar Jannach, and Marios Fragkoulis. 2023. Leveraging Large Language Models for Sequential Recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems. 1096--1102.

Digital Library

[14]

Jing He, Xin Li, Lejian Liao, Dandan Song, and William Cheung. 2016. Inferring a Personalized Next Point-of-interest Recommendation Model with Latent Behavior Patterns. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30. Issue: 1.

[15]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-term Memory. Neural Computation, Vol. 9, 8 (1997), 1735--1780.

Digital Library

[16]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. LoRA: Low-rank Adaptation of Large Language Models. arXiv preprint arXiv:2106.09685.

[17]

Dejiang Kong and Fei Wu. 2018. HST-LS™: A Hierarchical Spatial-Temporal Long-short Term Memory Network for Location Prediction. In IJCAI, Vol. 18. 2341--2347. Issue: 7.

[18]

Nicholas Lim, Bryan Hooi, See-Kiong Ng, Xueou Wang, Yong Liang Goh, Renrong Weng, and Jagannadan Varadarajan. 2020. STP-UDGAT: Spatial-Temporal-Preference User Dimensional Graph Attention Network for Next POI Recommendation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 845--854.

Digital Library

[19]

Yingtao Luo, Qiang Liu, and Zhaocheng Liu. 2021. Stan: Spatio-Temporal Attention Network for Next Location Recommendation. In Proceedings of the web conference 2021. 2177--2185.

Digital Library

[20]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning Transferable Visual Models from Natural Language Supervision. In International conference on machine learning. PMLR, 8748--8763.

[21]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing Personalized Markov Chains for Next-basket Recommendation. In Proceedings of the 19th international conference on World wide web. 811--820.

Digital Library

[22]

Ke Sun, Tieyun Qian, Tong Chen, Yile Liang, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2020. Where to Go Next: Modeling Long- and Short-term User Preferences for Point-of-interest Recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 214--221. Issue: 01.

[23]

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open Foundation and Fine-tuned Chat Models. arXiv preprint arXiv:2307.09289.

[24]

Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In International Conference on Learning Representations.

[25]

Xinglei Wang, Meng Fang, Zichao Zeng, and Tao Cheng. 2023. Where Would I Go Next? Large Language Models as Human Mobility Predictors. arXiv preprint arXiv:2308.15197 (2023).

[26]

Zhaobo Wang, Yanmin Zhu, Haobing Liu, and Chunyang Wang. 2022. Learning Graph-based Disentangled Representations for Next POI Recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1154--1163.

Digital Library

[27]

Yuxia Wu, Ke Li, Guoshuai Zhao, and Xueming Qian. 2020. Personalized Long- and Short-term Preference Learning for Next POI Recommendation. IEEE Transactions on Knowledge and Data Engineering, Vol. 34, 4 (2020), 1944--1957.

[28]

Hao Xue, Flora D. Salim, Yongli Ren, and Charles L.A. Clarke. 2022a. Translating Human Mobility Forecasting through Natural Language Generation. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 1224--1233.

[29]

Hao Xue, Bhanu Prakash Voutharoja, and Flora D. Salim. 2022b. Leveraging Language Foundation models for Human Mobility Forecasting. In Proceedings of the 30th International Conference on Advances in Geographic Information Systems. 1--9.

[30]

Xiaodong Yan, Tengwei Song, Yifeng Jiao, Jianshan He, Jiaotuan Wang, Ruopeng Li, and Wei Chu. 2023. Spatio-Temporal Hypergraph Learning for Next POI Recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 403--412.

Digital Library

[31]

Dingqi Yang, Daqing Zhang, Vincent W Zheng, and Zhiyong Yu. 2014. Modeling User Activity Preference by Leveraging User Spatial Temporal Characteristics in LBSNs. IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 45, 1 (2014), 129--142.

[32]

Song Yang, Jiamou Liu, and Kaiqi Zhao. 2022. GETNext: Trajectory Flow Map Enhanced Transformer for Next POI Recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '22). 1144--1153.

Digital Library

[33]

Lu Zhang, Zhu Sun, Ziqing Wu, Jie Zhang, Yew Soon Ong, and Xinghua Qu. 2022. Next Point-of-interest Recommendation with Inferring Multi-step Future Preferences. In Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI). 3751--3757.

[34]

Zizhuo Zhang and Bang Wang. 2023. Prompt Learning for News Recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 227--237.

Digital Library

[35]

Pengpeng Zhao, Anjing Luo, Yanchi Liu, Jiajie Xu, Zhixu Li, Fuzhen Zhuang, Victor S Sheng, and Xiaofang Zhou. 2020. Where to Go Next: A Spatio-tTemporal Gated Network for Next POI Recommendation. IEEE Transactions on Knowledge and Data Engineering, Vol. 34, 5 (2020), 2512--2524.

[36]

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A Survey of Large Language Models. arXiv preprint arXiv:2303.18223.

[37]

Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to Prompt for Vision-Language Models. International Journal of Computer Vision, Vol. 130, 9 (2022), 2337--2348.

Digital Library

Cited By

Chen JWang HShang JChaomurilige (2024)Interpretable Embeddings for Next Point-of-Interest Recommendation via Large Language Model Question–AnsweringMathematics10.3390/math1222359212:22(3592)Online publication date: 16-Nov-2024
https://doi.org/10.3390/math12223592
Cruz FSalim FKhaokaew YChan J(2024)CAPRI-FAIR: Integration of Multi-sided Fairness in Contextual POI Recommendation FrameworkProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688170(924-928)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688170

Index Terms

Large Language Models for Next Point-of-Interest Recommendation
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Point-of-Interest Recommendation: Exploiting Self-Attentive Autoencoders with Neighbor-Aware Influence
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management

The rapid growth of Location-based Social Networks (LBSNs) provides a great opportunity to satisfy the strong demand for personalized Point-of-Interest (POI) recommendation services. However, with the tremendous increase of users and POIs, POI ...
Context-aware probabilistic matrix factorization modeling for point-of-interest recommendation

Modeling the topic model.Modeling the geographical correlations.Modeling the social correlations.Modeling the categorical correlations.We integrate the textual, geographical, social, categorical and popularity information into probabilistic matrix ...
Exploiting Human Mobility Patterns for Point-of-Interest Recommendation
WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

Point-of-interest (POI) recommendation, which provides personalized recommendation of places to mobile users, is an important task in location-based social networks (LBSNs). Unlike traditional interest-oriented merchandise recommendation, POI ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2024

3164 pages

ISBN:9798400704314

DOI:10.1145/3626772

General Chairs:
Grace Hui Yang
Georgetown University, USA
,
Hongning Wang
Tsinghua University, China
,
Sam Han
The Washington Post, USA
,
Program Chairs:
Claudia Hauff
Spotify, Netherlands
,
Guido Zuccon
The University of Queensland, Australia
,
Yi Zhang
University of California Santa Cruz, USA

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Australian Research Council Centre of Excellence for Automated Decision-Making and Society

Conference

SIGIR 2024

Sponsor:

SIGIR

SIGIR 2024: The 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 14 - 18, 2024

Washington DC, USA

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
446
Total Downloads

Downloads (Last 12 months)446
Downloads (Last 6 weeks)126

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen JWang HShang JChaomurilige (2024)Interpretable Embeddings for Next Point-of-Interest Recommendation via Large Language Model Question–AnsweringMathematics10.3390/math1222359212:22(3592)Online publication date: 16-Nov-2024
https://doi.org/10.3390/math12223592
Cruz FSalim FKhaokaew YChan J(2024)CAPRI-FAIR: Integration of Multi-sided Fairness in Contextual POI Recommendation FrameworkProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688170(924-928)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688170

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents