Abstract
Recommender systems provide users personalized services by collecting and analyzing interaction data, undermining user privacy to a certain extent. In federated recommender systems, users can train models on local devices without uploading raw data. Nevertheless, model updates transmitted between the user and the server are still vulnerable to privacy inference attacks. Several studies adopt differential privacy to obfuscate transmitted updates, but they ignore the privacy sensitivity of recommender model components. The problem is that components closer to the original data are more susceptible to privacy leakage. To address this point, we propose a novel adaptive privacy-preserving method combining autoencoder for federated collaborative filtering, which guarantees privacy meanwhile maintaining high model performance. First, we extend the variational autoencoder (VAE) to federated settings for privacy-preserving recommendations. Additionally, we analyze the privacy risks of the variational autoencoder model in federated collaborative filtering. Subsequently, we propose an adaptive differential privacy method to enhance user privacy further. The key is to allocate less privacy budget for sensitive layers. We apply a metric based on model weights to determine the privacy sensitivity of each layer in the autoencoder. Then we adaptively allocate the privacy budget to the corresponding model layer. Extensive experiments and analysis demonstrate that our method can achieve competitive performance to non-private recommender models meanwhile providing fine-grained privacy protection.
Supported by the National Natural Science Foundation of China under Grant No. 62272180.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abadi, M., et al.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318 (2016)
Ammad-Ud-Din, M., et al.: Federated collaborative filtering for privacy-preserving personalized recommendation system. arXiv preprint arXiv:1901.09888 (2019)
Bobadilla, J., Ortega, F., Hernando, A., Gutiérrez, A.: Recommender systems survey. Knowl.-Based Syst. 46, 109–132 (2013)
Chai, D., Wang, L., Chen, K., Yang, Q.: Secure federated matrix factorization. IEEE Intell. Syst. 36(5), 11–20 (2020)
Chen, C., Campbell, N.: Understanding training-data leakage from gradients in neural networks for image classification. In: Workshop Privacy in Machine Learning, NeurIPS 2021 (2021)
Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. J. Priv. Confidentiality 7(3), 17–51 (2016)
Dwork, C., Roth, A., et al.: The algorithmic foundations of differential privacy. Found. Trends® Theor. Comput. Sci. 9(3–4), 211–407 (2014)
Feng, T., Hashemi, H., Hebbar, R., Annavaram, M., Narayanan, S.S.: Attribute inference attack of speech emotion recognition in federated learning settings. arXiv preprint arXiv:2112.13416 (2021)
Flanagan, A., Oyomno, W., Grigorievskiy, A., Tan, K.E., Khan, S.A., Ammad-Ud-Din, M.: Federated multi-view matrix factorization for personalized recommendations. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds.) ECML PKDD 2020. LNCS (LNAI), vol. 12458, pp. 324–347. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67661-2_20
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.S.: Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web, pp. 173–182 (2017)
Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. (TOIS) 22(1), 5–53 (2004)
Jalalirad, A., Scavuzzo, M., Capota, C., Sprague, M.: A simple and efficient federated recommender system. In: Proceedings of the 6th IEEE/ACM International Conference on Big Data Computing, Applications and Technologies, pp. 53–58 (2019)
Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37(2), 183–233 (1999)
Kairouz, P., Oh, S., Viswanath, P.: The composition theorem for differential privacy. In: International Conference on Machine Learning, pp. 1376–1385. PMLR (2015)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Liang, D., Krishnan, R.G., Hoffman, M.D., Jebara, T.: Variational autoencoders for collaborative filtering. In: Proceedings of the 2018 World Wide Web Conference, pp. 689–698 (2018)
Liang, F., Pan, W., Ming, Z.: FedRec++: lossless federated recommendation with explicit feedback. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 4224–4231 (2021)
Lin, G., Liang, F., Pan, W., Ming, Z.: FedRec: federated recommendation with explicit feedback. IEEE Intell. Syst. 36(5), 21–30 (2020)
Lin, M., et al.: Pruning networks with cross-layer ranking & k-reciprocal nearest filters. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Liu, Z., Yang, L., Fan, Z., Peng, H., Yu, P.S.: Federated social recommendation with graph neural network. ACM Trans. Intell. Syst. Technol. (TIST) 13(4), 1–24 (2022)
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)
Minto, L., Haller, M., Livshits, B., Haddadi, H.: Stronger privacy for federated collaborative filtering with implicit feedback. In: Fifteenth ACM Conference on Recommender Systems, pp. 342–350 (2021)
Muhammad, K., et al.: FedFast: going beyond average for faster training of federated recommender systems. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1234–1242 (2020)
Perifanis, V., Efraimidis, P.S.: Federated neural collaborative filtering. Knowl.-Based Syst. 242, 108441 (2022)
Phan, N., Wu, X., Hu, H., Dou, D.: Adaptive laplace mechanism: differential privacy preservation in deep learning. In: 2017 IEEE International Conference on Data Mining (ICDM), pp. 385–394. IEEE (2017)
Polato, M.: Federated variational autoencoder for collaborative filtering. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2021)
Qi, T., Wu, F., Wu, C., Huang, Y., Xie, X.: Privacy-preserving news recommendation model learning. arXiv preprint arXiv:2003.09592 (2020)
Reddi, S.J., et al.: Adaptive federated optimization. In: International Conference on Learning Representations (2020)
Schreyer, M., Sattarov, T., Borth, D.: Federated and privacy-preserving learning of accounting data in financial statement audits. arXiv preprint arXiv:2208.12708 (2022)
Sedhain, S., Menon, A.K., Sanner, S., Xie, L.: AutoRec: autoencoders meet collaborative filtering. In: Proceedings of the 24th International Conference on World Wide Web, pp. 111–112 (2015)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Wang, Q., Yin, H., Chen, T., Yu, J., Zhou, A., Zhang, X.: Fast-adapting and privacy-preserving federated recommender system. VLDB J. 31(5), 877–896 (2022)
Wu, C., Wu, F., Cao, Y., Huang, Y., Xie, X.: FedGNN: federated graph neural network for privacy-preserving recommendation. arXiv preprint arXiv:2102.04925 (2021)
Yang, E., Huang, Y., Liang, F., Pan, W., Ming, Z.: FCMF: federated collective matrix factorization for heterogeneous collaborative filtering. Knowl.-Based Syst. 220, 106946 (2021)
Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. (TIST) 10(2), 1–19 (2019)
Zhu, L., Liu, Z., Han, S.: Deep leakage from gradients. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ding, X., Li, G., Yuan, L., Zhang, L., Rong, Q. (2023). Combining Autoencoder with Adaptive Differential Privacy for Federated Collaborative Filtering. In: Wang, X., et al. Database Systems for Advanced Applications. DASFAA 2023. Lecture Notes in Computer Science, vol 13943. Springer, Cham. https://doi.org/10.1007/978-3-031-30637-2_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-30637-2_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30636-5
Online ISBN: 978-3-031-30637-2
eBook Packages: Computer ScienceComputer Science (R0)