Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Privacy-preserving matrix factorization for recommendation systems using Gaussian mechanism and functional mechanism

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Building a recommendation system involves analyzing user data, which can potentially leak sensitive information about users. As shown in numerous recent works, anonymizing user data is not sufficient for preserving user privacy. Differential Privacy is a mathematically rigorous privacy guarantee that has been adopted by numerous corporations and government entities for preserving user privacy in such scenarios. Developing differentially private machine learning algorithms typically involve adding randomness into the algorithm pipeline, which evidently degrades the performance of the algorithm—giving raise to privacy-utility trade-off. Existing differentially private matrix factorization algorithms offer poor privacy-utility trade-off for use in practical systems. Motivated by this, we propose two differentially private matrix factorization algorithms for application in recommendation systems, which provide better privacy-utility trade-off compared to the existing approaches. Our first algorithm adopts the framework of noisy gradient descent using the Gaussian Mechanism, whereas our second algorithm extends the Functional Mechanism framework to incorporate it into matrix factorization. In both cases, we perform theoretical analysis of the privacy of the algorithms. Additionally, we employ Rényi differential privacy to analyze the algorithms for a tight characterization of the overall privacy loss. We perform extensive experiments on real data by varying relevant privacy, algorithm, and dataset parameters. We compare the performance of our proposed algorithms with existing non-private and differentially private algorithms, and demonstrate that our algorithms can provide utility close to that of the non-private algorithm while guaranteeing strict privacy, outperforming the existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

All the datasets used for our experiments are publicly available. The Movielens 1M [52] dataset is available at https://grouplens.org/datasets/movielens/1m/. The Movielens 10M [53] dataset is available at https://grouplens.org/datasets/movielens/10m/. The Netflix Prize dataset [54] is available at https://www.kaggle.com/netflix-inc/netflix-prize-data. Finally, the Yahoo Music dataset is available at https://webscope.sandbox.yahoo.com/catalog.php?datatype=r.

References

  1. Wu C, Wu F, Lyu L, Qi T, Huang Y, Xie X (2022) A federated graph neural network framework for privacy-preserving personalization. Nat Commun 13(1):3091

    Article  Google Scholar 

  2. Su X, Khoshgoftaar TM (2009) A survey of collaborative filtering techniques. Adv Artif Intell. https://doi.org/10.1155/2009/421425

    Article  Google Scholar 

  3. Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37. https://doi.org/10.1109/MC.2009.263

    Article  Google Scholar 

  4. Candès EJ, Recht B (2009) Exact matrix completion via convex optimization. Found Comput Math 9(6):717–772. https://doi.org/10.1007/s10208-009-9045-5

    Article  MathSciNet  Google Scholar 

  5. Bennett J, Lanning S (2007) The Netflix prize. In: Proceedings of KDD cup and workshop 2007, Association for Computing Machinery, New York, NY, USA, pp 3–6

  6. Narayanan A, Shmatikov V (2006) How to break anonymity of the Netflix prize dataset. arXiv preprint. arXiv:cs/0610105

  7. Le Ny J, Pappas GJ (2013) Differentially private filtering. IEEE Trans Autom Control 59(2):341–354

    MathSciNet  Google Scholar 

  8. Sweeney L (2015) Only you, your doctor, and many others may know. Technol Sci 2015092903(9):29

    Google Scholar 

  9. Aïmeur E, Brassard G, Fernandez J. M, & Mani Onana F. S (2008) Alambic: a privacy-preserving recommender system for electronic commerce. Int J Inf Secur 7(5):307–334. https://doi.org/10.1007/s10207-007-0049-3

    Article  Google Scholar 

  10. Calandrino JA, Kilzer A, Narayanan A, Felten EW, Shmatikov V (2011) “You might also like”: privacy risks of collaborative filtering. In: 2011 IEEE symposium on security and privacy. IEEE, pp 231–246. https://doi.org/10.1109/SP.2011.40

  11. McSherry F, Mironov I (2009) Differentially private recommender systems: building privacy into the Netflix prize contenders. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining. Association for Computing Machinery, pp 627–636. https://doi.org/10.1145/1557019.1557090

  12. Dwork C, Roth A (2014) The algorithmic foundations of differential privacy. Found Trends Theor Comput Sci 9(3–4):211–407. https://doi.org/10.1561/0400000042

    Article  MathSciNet  Google Scholar 

  13. Dwork C, McSherry F, Nissim K, Smith A (2006) Calibrating noise to sensitivity in private data analysis. In: Theory of cryptography conference. Springer, Berlin, pp 265–284. https://doi.org/10.1007/11681878_14

  14. Tasnim N, Mohammadi J, Sarwate AD, Imtiaz H (2023) Approximating functions with approximate privacy for applications in signal estimation and learning. Entropy. https://doi.org/10.3390/e25050825

    Article  MathSciNet  Google Scholar 

  15. Machanavajjhala A, Korolova A, & Sarma AD (2011) Personalized social recommendations-accurate or private?. In: Proceedings of the VLDB Endowment, vol 4(7). Association for Computing Machinery, New York, NY, USA

  16. McSherry F, Talwar K (2007) Mechanism design via differential privacy. In: 48th Annual IEEE symposium on foundations of computer science (FOCS’07). IEEE, pp 94–103. https://doi.org/10.1109/FOCS.2007.66

  17. Sarwate AD, Chaudhuri K (2013) Signal processing and machine learning with differential privacy: algorithms and challenges for continuous data. IEEE Signal Process Mag 30(5):86–94. https://doi.org/10.1109/MSP.2013.2259911

    Article  Google Scholar 

  18. Chaudhuri K, Monteleoni C, Sarwate AD (2011) Differentially private empirical risk minimization. J Mach Learn Res 12(3):1069–1109

    MathSciNet  Google Scholar 

  19. Bassily R, Smith A, Thakurta A (2014) Private empirical risk minimization: efficient algorithms and tight error bounds. In: 2014 IEEE 55th annual symposium on foundations of computer science. IEEE, pp 464–473

  20. Nozari E, Tallapragada P, Cortés J (2016) Differentially private distributed convex optimization via objective perturbation. In: 2016 American control conference (ACC). IEEE, New York, NY, USA, pp 2061–2066

  21. Song S, Chaudhuri K, Sarwate AD (2013) Stochastic gradient descent with differentially private updates. In: 2013 IEEE global conference on signal and information processing. IEEE, New York, NY, USA, pp 245–248

  22. Abadi M, Chu A, Goodfellow I, Brendan McMahan H, Mironov I, Talwar K, Zhang L (2016) Deep Learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security (CCS '16). Association for Computing Machinery, New York, NY, USA, pp 308–318. https://doi.org/10.1145/2976749.2978318

  23. Zhang J, Zhang Z, Xiao X, Yang Y, Winslett M (2012) Functional mechanism: regression analysis under differential privacy. In: Proceedings of the VLDB endowment, vol 5(11). Association for Computing Machinery, New York, NY, USA, pp 1364–1375. https://doi.org/10.14778/2350229.2350253

  24. Jorgensen Z, Yu T, Cormode G (2015) Conservative or liberal? Personalized differential privacy. In: 2015 IEEE 31st international conference on data engineering. IEEE, New York, NY, USA, pp 1023–1034

  25. Ding J, Zhang X, Li X, Wang J, Yu R, & Pan M (2020) Differentially private and fair classification via calibrated functional mechanism. In: Proceedings of the AAAI conference on artificial intelligence. Association for the Advancement of Artificial Intelligence, Washington, DC, USA, (Vol. 34, No. 01) pp 622–629.

  26. NhatHai Phan, Minh N Vu, Yang Liu, Ruoming Jin, Dejing Dou, Xintao Wu, and My T Thai (2019) Heterogeneous Gaussian mechanism: preserving differential privacy in deep learning with provable robustness. In: Proceedings of the 28th international joint conference on artificial intelligence (IJCAI'19). AAAI Press, Washington, DC, USA, pp 4753–4759

  27. Nikolaenko V, Ioannidis S, Weinsberg U, Joye M, Taft N, Boneh D (2013) Privacy-preserving matrix factorization. In: Proceedings of the 2013 ACM SIGSAC conference on computer and communications security. Association for Computing Machinery, pp 801–812. https://doi.org/10.1145/2508859.2516751

  28. Chaudhuri K, Monteleoni C (2008) Privacy-preserving logistic regression. Adv Neural Inf Process Syst 21:289–296

    Google Scholar 

  29. Dwork C, Talwar K, Thakurta A, Zhang L (2014) Analyze Gauss: optimal bounds for privacy-preserving principal component analysis. In: Proceedings of the forty-sixth annual ACM symposium on theory of computing. Association for Computing Machinery, pp 11–20. https://doi.org/10.1145/2591796.2591883

  30. Hardt M, Price E (2014) The noisy power method: a meta algorithm with applications. Adv Neural Inf Process Syst 27:2861–2869

    Google Scholar 

  31. Chaudhuri K, Sarwate A, Sinha K (2012) Near-optimal differentially private principal components. Adv Neural Inf Process Syst 25:989–997

    Google Scholar 

  32. Han S, Topcu U, Pappas GJ (2016) Differentially private distributed constrained optimization. IEEE Trans Autom Control 62(1):50–64. https://doi.org/10.1109/TAC.2016.2541298

    Article  MathSciNet  Google Scholar 

  33. Imtiaz H, Sarwate AD (2018) Distributed differentially private algorithms for matrix and tensor factorization. IEEE J Sel Top Signal Process 12(6):1449–1464. https://doi.org/10.1109/JSTSP.2018.2877842

    Article  Google Scholar 

  34. Deng J, Wu Q, Wang S, Ye J, Wang P, Du M (2024) A novel joint neural collaborative filtering incorporating rating reliability. Inf Sci 665:120406. https://doi.org/10.1016/j.ins.2024.120406

    Article  Google Scholar 

  35. Ali W, Ammad-ud-din M, Zhou X, Zhang Y, Shao J (2024) Communication-efficient federated neural collaborative filtering with multi-armed bandits. ACM Trans Recomm Syst. https://doi.org/10.1145/3651168 (Just accepted)

  36. Liu Z, Wang Y-X, Smola A (2015) Fast differentially private matrix factorization. In: Proceedings of the 9th ACM conference on recommender systems. Association for Computing Machinery, pp 171–178. https://doi.org/10.1145/2792838.2800191

  37. Hua J, Xia C, Zhong S (2015) Differentially private matrix factorization. In: Proceedings of the twenty-fourth international joint conference on artificial intelligence (IJCAI 2015). AAAI Press, Washington, DC, USA, pp 1763–1770

  38. Ran X, Wang Y, Zhang LY, Ma J (2022) A differentially private matrix factorization based on vector perturbation for recommender system. Neurocomputing 483:32–41

    Article  Google Scholar 

  39. Zhou H, Yang G, Xiang Y, Bai Y, Wang W (2021) A lightweight matrix factorization for recommendation with local differential privacy in big data. IEEE Trans Big Data 9(1):160–173

    Article  Google Scholar 

  40. Ran X, Wang Y, Zhang LY, Ma J (2022) A differentially private nonnegative matrix factorization for recommender system. Inf Sci 592:21–35

    Article  Google Scholar 

  41. Zheng X, Guan M, Jia X, Guo L, Luo Y (2022) A matrix factorization recommendation system-based local differential privacy for protecting users’ sensitive data. IEEE Trans Comput Soc Syst 10(3):1189–1198. https://doi.org/10.1109/TCSS.2022.3170691

    Article  Google Scholar 

  42. Duchi JC, Jordan M.I, Wainwright M.J (2013) Local privacy and statistical minimax rates. In: 2013 IEEE 54th annual symposium on foundations of computer science. IEEE, New York, NY, USA, pp 429–438

  43. Wang Y, Gao M, Ran X, Ma J, Zhang LY (2023) An improved matrix factorization with local differential privacy based on piecewise mechanism for recommendation systems. Expert Syst Appl 216:119457

    Article  Google Scholar 

  44. Wang N, Xiao X, Yang Y, Zhao J, Hui SC, Shin H, Shin J, Yu G (2019) Collecting and analyzing multidimensional data with local differential privacy. In: 2019 IEEE 35th international conference on data engineering (ICDE). IEEE, New York, NY, USA, pp 638–649

  45. Berlioz A, Friedman A, Kaafar MA, Boreli R, Berkovsky S (2015) Applying Differential Privacy to Matrix Factorization. In: Proceedings of the 9th ACM Conference on Recommender Systems (RecSys '15). Association for Computing Machinery, New York, NY, USA, pp 107–114. https://doi.org/10.1145/2792838.2800173

  46. Rudin W (1964) Principles of mathematical analysis, vol 3. McGraw-hill, New York, NY, USA

  47. Mironov I (2017) Rényi differential privacy. In: 2017 IEEE 30th computer security foundations symposium (CSF). IEEE, pp 263–275. https://doi.org/10.1109/CSF.2017.11

  48. Imtiaz H, Mohammadi J, Sarwate A.D (2019) Distributed differentially private computation of functions with correlated noise. arXiv preprint. arXiv:1904.10059

  49. Dwork C (2008) Differential privacy: a survey of results. In: International conference on theory and applications of models of computation. Springer, Berlin, pp 1–19. https://doi.org/10.1007/978-3-540-79228-4_1

  50. Petersen KB, Pedersen MS (2008) The matrix cookbook. Tech Univ Denmark 7(15):510

    Google Scholar 

  51. Gall F.L, & Urrutia F (2018) Improved rectangular matrix multiplication using powers of the coppersmith-winograd tensor. In: Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms (pp. 1029-1046). Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9781611975031.67

  52. GroupLens: Movielens 1M Dataset. Online available. https://grouplens.org/datasets/movielens/1m/. Last Accessed 09 Aug 2022

  53. GroupLens: Movielens 10M Dataset. Online available. https://grouplens.org/datasets/movielens/10m/. Last Accessed 08 June 2023

  54. Netflix: Netflix Prize data. Online available. https://www.kaggle.com/netflix-inc/netflix-prize-data. Last Accessed 09 Aug 2022

  55. Yahoo: Yahoo Music Dataset. Online available. https://webscope.sandbox.yahoo.com/catalog.php?datatype=r. Last Accessed 08 June 2023

Download references

Acknowledgements

The authors would like to express their sincere gratitude towards the authorities of the Department of Electrical and Electronic Engineering and Bangladesh University of Engineering and Technology (BUET) for providing constant support throughout this research.

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology, Dhaka, 1205, Bangladesh

    Sohan Salahuddin Mugdho & Hafiz Imtiaz

Authors
  1. Sohan Salahuddin Mugdho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hafiz Imtiaz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hafiz Imtiaz.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mugdho, S.S., Imtiaz, H. Privacy-preserving matrix factorization for recommendation systems using Gaussian mechanism and functional mechanism. Int. J. Mach. Learn. & Cyber. 15, 5745–5763 (2024). https://doi.org/10.1007/s13042-024-02276-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-024-02276-3

Keywords

Search

Navigation