research-article

TransNet: training privacy-preserving neural network over transformed layer

Authors:

Liusheng HuangAuthors Info & Claims

Proceedings of the VLDB Endowment, Volume 13, Issue 12

Pages 1849 - 1862

https://doi.org/10.14778/3407790.3407794

Published: 01 July 2020 Publication History

Abstract

The accuracy of neural network can be improved by training over multi-participants' pooled dataset, but privacy problem of sharing sensitive data obstructs this collaborative learning. To solve this contradiction, we propose TransNet, a novel solution for privacy-preserving collaborative neural network, whose main idea is to add a transformed layer to the neural network. It has the advantage of lower computation and communication complexity than previous secure multi-party computation based and homomorphic encryption based schemes, and has the superiority of supporting arbitrarily partitioned dataset compared to previous differential privacy based and stochastic gradient descent based schemes, which support horizontally partitioned dataset only. TransNet is trained by a server which pools the transformed data, but has no special security requirement on the training server. We evaluate TransNet's performance over four datasets using different neural network algorithms. Experimental results demonstrate that TransNet is not affected by the number of participants, and trains as quickly as the original neural network does. With proper variables, TransNet gets close accuracy to the baseline which trains over pooled original dataset.

References

[1]

M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 308--318. ACM, 2016.

Digital Library

[2]

C. C. Aggarwal and S. Y. Philip. Privacy-preserving data mining: models and algorithms. Springer Science & Business Media, 2008.

Digital Library

[3]

B. Alipanahi, A. Delong, M. T. Weirauch, and B. J. Frey. Predicting the sequence specificities of dna-and rna-binding proteins by deep learning. Nature biotechnology, 33(8):831, 2015.

[4]

R. Bassily, A. Smith, and A. Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, pages 464--473. IEEE, 2014.

Digital Library

[5]

J. W. Bos, K. Lauter, J. Loftus, and M. Naehrig. Improved security for a ring-based fully homomorphic encryption scheme. In IMA International Conference on Cryptography and Coding, pages 45--64. Springer, 2013.

Digital Library

[6]

R. Bost, R. A. Popa, S. Tu, and S. Goldwasser. Machine learning classification over encrypted data. In NDSS, volume 4324, page 4325, 2015.

[7]

Bourse, Florian and Minelli, Michele and Minihold, Matthias and Paillier, Pascal. Fast homomorphic evaluation of deep discretized neural networks. In Annual International Cryptology Conference, pages 483--512. Springer, 2018.

[8]

K. Chaudhuri, A. D. Sarwate, and K. Sinha. A near-optimal algorithm for differentially-private principal components. The Journal of Machine Learning Research, 14(1):2905--2943, 2013.

Digital Library

[9]

Chen, Hao and Dai, Wei and Kim, Miran and Song, Yongsoo. Efficient multi-key homomorphic encryption with packed ciphertexts with application to oblivious neural network inference. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 395--412, 2019.

Digital Library

[10]

C. Clifton, M. Kantarcioglu, J. Vaidya, X. Lin, and M. Y. Zhu. Tools for privacy preserving distributed data mining. ACM Sigkdd Explorations Newsletter, 4(2):28--34, 2002.

Digital Library

[11]

W. Du, Y. S. Han, and S. Chen. Privacy-preserving multivariate statistical analysis: Linear regression and classification. In Proceedings of the 2004 SIAM international conference on data mining, pages 222--233. SIAM, 2004.

[12]

C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pages 265--284. Springer, 2006.

Digital Library

[13]

C. Dwork, G. N. Rothblum, and S. P. Vadhan. Boosting and differential privacy. pages 51--60, 2010.

Digital Library

[14]

M. Fredrikson, S. Jha, and T. Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322--1333. ACM, 2015.

Digital Library

[15]

M. Fredrikson, E. Lantz, S. Jha, S. Lin, D. Page, and T. Ristenpart. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In 23rd {USENIX} Security Symposium ({USENIX} Security 14), pages 17--32, 2014.

Digital Library

[16]

C. Gentry et al. Fully homomorphic encryption using ideal lattices. In Stoc, volume 9, pages 169--178, 2009.

Digital Library

[17]

R. Gilad-Bachrach, N. Dowlin, K. Laine, K. Lauter, M. Naehrig, and J. Wernsing. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In International Conference on Machine Learning, pages 201--210, 2016.

Digital Library

[18]

G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504--507, 2006.

[19]

G. Jagannathan and R. N. Wright. Privacy-preserving distributed k-means clustering over arbitrarily partitioned data. In Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pages 593--599. ACM, 2005.

Digital Library

[20]

Juvekar, Chiraag and Vaikuntanathan, Vinod and Chandrakasan, Anantha. GAZELLE: A low latency framework for secure neural network inference. In 27th USENIX Security Symposium (USENIX Security 18), pages 1651--1669, 2018.

Digital Library

[21]

A. Kannan, K. Kurach, S. Ravi, T. Kaufmann, A. Tomkins, B. Miklos, G. Corrado, L. Lukacs, M. Ganea, P. Young, et al. Smart reply: Automated response suggestion for email. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 955--964. ACM, 2016.

Digital Library

[22]

I. Kononenko. Machine learning for medical diagnosis: history, state of the art and perspective. Artificial Intelligence in medicine, 23(1):89--109, 2001.

Digital Library

[23]

P. Li, J. Li, Z. Huang, T. Li, C. Gao, S. Yiu, and K. Chen. Multi-key privacy-preserving deep learning in cloud computing. Future Generation Computer Systems, 74:76--85, 2017.

Digital Library

[24]

Y. Lindell and B. Pinkas. Privacy preserving data mining. In Annual International Cryptology Conference, pages 36--54. Springer, 2000.

Digital Library

[25]

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, pages 1273--1282, 2017.

[26]

V. Nikolaenko, U. Weinsberg, S. Ioannidis, M. Joye, D. Boneh, and N. Taft. Privacy-preserving ridge regression on hundreds of millions of records. In 2013 IEEE Symposium on Security and Privacy, pages 334--348. IEEE, 2013.

Digital Library

[27]

N. Papernot, M. Abadi, U. Erlingsson, I. Goodfellow, and K. Talwar. Semi-supervised knowledge transfer for deep learning from private training data. arXiv preprint arXiv:1610.05755, 2016.

[28]

L. T. Phong, Y. Aono, T. Hayashi, L. Wang, and S. Moriai. Privacy-preserving deep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security, 13(5):1333--1345, 2018.

Digital Library

[29]

W. Ruan, X. Huang, and M. Kwiatkowska. Reachability analysis of deep neural networks with provable guarantees. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 2651--2659. AAAI Press, 2018.

Digital Library

[30]

K. Saranya, K. Premalatha, and S. Rajasekar. A survey on privacy preserving data mining. In 2015 2nd International Conference on Electronics and Communication Systems (ICECS), pages 1740--1744. IEEE, 2015.

[31]

R. Shokri and V. Shmatikov. Privacy-preserving deep learning. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 1310--1321. ACM, 2015.

Digital Library

[32]

M. K. Simon. Probability distributions involving Gaussian random variables: A handbook for engineers and scienti. 2006.

Digital Library

[33]

C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.

[34]

J. Vaidya and C. Clifton. Privacy preserving association rule mining in vertically partitioned data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 639--644. ACM, 2002.

Digital Library

[35]

J. Vaidya, M. Kantarcioğlu, and C. Clifton. Privacy-preserving naive bayes classification. The VLDB Journal, 17(4):879--898, 2008.

Digital Library

[36]

Q. Wang, M. Du, X. Chen, Y. Chen, P. Zhou, X. Chen, and X. Huang. Privacy-preserving collaborative model learning: The case of word vector training. IEEE Transactions on Knowledge and Data Engineering, 30(12):2381--2393, 2018.

Digital Library

[37]

W. K. Wong, D. W.-l. Cheung, B. Kao, and N. Mamoulis. Secure knn computation on encrypted databases. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, pages 139--152. ACM, 2009.

Digital Library

[38]

D. Wu and J. Haven. Using homomorphic encryption for large scale statistical analysis, 2012.

[39]

A. C.-C. Yao. How to generate and exchange secrets. In 27th Annual Symposium on Foundations of Computer Science (sfcs 1986), pages 162--167. IEEE, 1986.

Digital Library

[40]

J. Yuan and S. Yu. Privacy preserving back-propagation neural network learning made practical with cloud computing. IEEE Transactions on Parallel and Distributed Systems, 25(1):212--221, 2014.

Digital Library

[41]

J. Zhang, Z. Zhang, X. Xiao, Y. Yang, and M. Winslett. Functional mechanism: regression analysis under differential privacy. very large data bases, 5(11):1364--1375, 2012.

Digital Library

Cited By

Xu NWang BRan RWen WVenkitasubramaniam P(2022)NeuGuard: Lightweight Neuron-Guided Defense against Membership Inference AttacksProceedings of the 38th Annual Computer Security Applications Conference10.1145/3564625.3567986(669-683)Online publication date: 5-Dec-2022
https://dl.acm.org/doi/10.1145/3564625.3567986

Recommendations

TransNet: the transportation research Internet cooperative

TransNet is an experimental service that provides a model for the organization and electronic dissemination of transportation knowledge. TransNet is an Internet cooperative whose members are universities that have a demonstrated interest in ...
TransNet: Minimally Supervised Deep Transfer Learning for Dynamic Adaptation of Wearable Systems

Wearables are poised to transform health and wellness through automation of cost-effective, objective, and real-time health monitoring. However, machine learning models for these systems are designed based on labeled data collected, and feature ...
The Training of Pi-Sigma Artificial Neural Networks with Differential Evolution Algorithm for Forecasting
Abstract
Looking at the artificial neural networks’ literature, most of the studies started with feedforward artificial neural networks and the training of many feedforward artificial neural networks models are performed with derivative-based algorithms ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the VLDB Endowment

Proceedings of the VLDB Endowment Volume 13, Issue 12

August 2020

1710 pages

ISSN:2150-8097

Editors:
Magdalena Balazinska
University of Washington
,
Xiaofang Zhou
University of Queensland, Australia

Issue’s Table of Contents

Publisher

VLDB Endowment

Publication History

Published: 01 July 2020

Published in PVLDB Volume 13, Issue 12

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
183
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)2

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xu NWang BRan RWen WVenkitasubramaniam P(2022)NeuGuard: Lightweight Neuron-Guided Defense against Membership Inference AttacksProceedings of the 38th Annual Computer Security Applications Conference10.1145/3564625.3567986(669-683)Online publication date: 5-Dec-2022
https://dl.acm.org/doi/10.1145/3564625.3567986

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents