research-article

DeepChain: Auditable and Privacy-Preserving Deep Learning with Blockchain-Based Incentive

Authors:

Weiqi LuoAuthors Info & Claims

IEEE Transactions on Dependable and Secure Computing, Volume 18, Issue 5

Pages 2438 - 2455

https://doi.org/10.1109/TDSC.2019.2952332

Published: 01 September 2021 Publication History

Abstract

Deep learning can achieve higher accuracy than traditional machine learning algorithms in a variety of machine learning tasks. Recently, privacy-preserving deep learning has drawn tremendous attention from information security community, in which neither training data nor the training model is expected to be exposed. Federated learning is a popular learning mechanism, where multiple parties upload local gradients to a server and the server updates model parameters with the collected gradients. However, there are many security problems neglected in federated learning, for example, the participants may behave incorrectly in gradient collecting or parameter updating, and the server may be malicious as well. In this article, we present a distributed, secure, and fair deep learning framework named DeepChain to solve these problems. DeepChain provides a value-driven incentive mechanism based on Blockchain to force the participants to behave correctly. Meanwhile, DeepChain guarantees data privacy for each participant and provides auditability for the whole training process. We implement a prototype of DeepChain and conduct experiments on a real dataset for different settings, and the results show that our DeepChain is promising.

References

[1]

G. Hinton, et al., “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,” IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82–97, Nov. 2012.

[2]

T.-H. Chan, K. Jia, S. Gao, J. Lu, Z. Zeng, and Y. Ma, “PCANet: A simple deep learning baseline for image classification?” IEEE Trans. Image Process., vol. 24, no. 12, pp. 5017–5032, Dec. 2015.

Digital Library

[3]

E. Gawehn, J. A. Hiss, and G. Schneider, “Deep learning in drug discovery,” Mol. Informat., vol. 35, no. 1, pp. 3–14, 2016.

[4]

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, 2015, Art. no.

[5]

P. Danaee, R. Ghaeini, and D. A. Hendrix, “A deep learning approach for cancer detection and relevant gene identification,” in Proc. Pacific Symp. Biocomputing, 2017, pp. 219–229.

[6]

S. Gupta, W. Zhang, and F. Wang, “Model accuracy and runtime tradeoff in distributed deep learning: A systematic study,” in Proc. IEEE 16th Int. Conf. Data Mining, 2016, pp. 171–180.

[7]

T. Chilimbi, Y. Suzue, J. Apacible, and K. Kalyanaraman, “Project adam: building an efficient and scalable deep learning training system,” in Proc. Usenix Conf. Operating Syst. Des. Implementation, 2016, pp. 571–582.

[8]

T. Chen and S. Zhong, “Privacy-preserving backpropagation neural network learning,” IEEE Trans. Neural Netw., vol. 20, no. 10, 2009, Art. no.

[9]

A. Bansal, T. Chen, and S. Zhong, “Privacy preserving back-propagation neural network learning over arbitrarily partitioned data,” Neural Comput. Appl., vol. 20, no. 1, pp. 143–150, 2011.

Digital Library

[10]

J. Yuan and S. Yu, “Privacy preserving back-propagation learning made practical with cloud computing,” IEEE Trans. Parallel Distrib. Syst., vol. 25, no. 1, pp. 212–221, Jan. 2014.

Digital Library

[11]

R. Shokri and V. Shmatikov, “Privacy-preserving deep learning,” in Proc. Allerton Conf. Commun., Control, Comput., 2015, pp. 909–910.

[12]

P. Li, J. Li, Z. Huang, C. Z. Gao, W. B. Chen, and K. Chen, “Privacy-preserving outsourced classification in cloud computing,” Cluster Comput., no. 1, pp. 1–10, 2017.

[13]

Q. Zhang, L. Yang, and Z. Chen, “Privacy preserving deep computation model on cloud for big data feature learning,” IEEE Trans. Comput., vol. 65, no. 5, pp. 1351–1362, May 2016.

Digital Library

[14]

K. Bonawitz, et al., “Practical secure aggregation for privacy-preserving machine learning,” in Proc. ACM SIGSAC Conf. Comput. Commun. Security, 2017, pp. 1175–1191.

[15]

P. Mohassel and Y. Zhang, “Secureml: A system for scalable privacy-preserving machine learning,” in Proc. IEEE Symp. Security Privacy, 2017, pp. 19–38.

[16]

L. T. Phong, Y. Aono, T. Hayashi, L. Wang, and S. Moriai, “Privacy-preserving deep learning via additively homomorphic encryption,” IEEE Trans. Inf. Forensics Security, vol. 13, no. 5, pp. 1333–1345, May 2018.

Digital Library

[17]

C. Song, T. Ristenpart, and V. Shmatikov, “Machine learning models that remember too much,” in Proc. ACM SIGSAC Conf. Comput. Commun. Security, 2017, pp. 587–601.

[18]

L. Melis, C. Song, E. De Cristofaro, and V. Shmatikov, “Inference attacks against collaborative learning,” 2018, arXiv: 1805.04049.

[19]

B. Hitaj, G. Ateniese, and F. Pérez-Cruz, “Deep models under the gan: information leakage from collaborative deep learning,” in Proc. ACM SIGSAC Conf. Comput. Commun. Security, 2017, pp. 603–618.

Digital Library

[20]

T. Orekondy, S. J. Oh, B. Schiele, and M. Fritz, “Understanding and controlling user linkability in decentralized learning,” 2018, arXiv: 1805.05838.

[21]

L. T. Phong, Y. Aono, T. Hayashi, L. Wang, and S. Moriai, “Privacy-preserving deep learning via additively homomorphic encryption,” IEEE Trans. Inf. Forensics Security, vol. 13, no. 5, pp. 1333–1345, May 2018.

Digital Library

[22]

A. Pyrgelis, C. Troncoso, and E. D. Cristofaro, “Knock knock, who’s there? membership inference on aggregate location data,” in Proc. 25th Annu. Netw. Distrib. Syst. Security Symp., NDSS 2018. The Internet Society, 2018, pp. 1–15.

[23]

E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, “How to backdoor federated learning,” 2018, CoRR, vol. abs/1807.00459, 2018.

[24]

“Health insurance portability and accountability act.” 1996. [Online]. Available: https://www.hhs.gov/hipaa/index.html

[25]

J. Vaidya, B. Shafiq, X. Jiang, and L. Ohno-Machado, “Identifying inference attacks against healthcare data repositories,” AMIA Summits Translational Sci. Proc., vol. 2013, 2013, Art. no.

[26]

G. Heigold, et al., “Multilingual acoustic models using distributed deep neural networks,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2013, pp. 8619–8623.

[27]

R. Jurca and B. Faltings, “An incentive compatible reputation mechanism,” in Proc. EEE Int. Conf. E-Commerce, 2003, pp. 285–292.

[28]

U. Shevade, H. H. Song, L. Qiu, and Y. Zhang, “Incentive-aware routing in dtns,” in Proc. IEEE Int. Conf. Netw. Protocols, 2008, pp. 238–247.

[29]

S. Zhong, J. Chen, and Y. R. Yang, “Sprite: A simple, cheat-proof, credit-based system for mobile ad-hoc networks,” in IEEE INFOCOM 2003. Proc. 22nd Annu. Joint Conf. IEEE Comput. Commun. Societies, vol. 3, 2003, pp. 1987–1997.

[30]

B. B. Chen and M. C. Chan, “MobiCent: A credit-based incentive system for disruption tolerant network,” in Proc. IEEE INFOCOM, 2010, pp. 1–9.

[31]

S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” 2008.[Online]. Available: http: //bitcoin.org/bitcoin.pdf

[32]

M. Ben-Or and A. Hassidim, “Fast quantum byzantine agreement,” in Proc. 37th Annu. ACM Symp. Theory Comput., 2005, pp. 481–485.

[33]

S. Micali, “ALGORAND: The efficient and democratic ledger,” CoRR, vol. abs/1607.01341, 2016.

[34]

Y. Gilad, R. Hemo, S. Micali, G. Vlachos, and N. Zeldovich, “Algorand: Scaling byzantine agreements for cryptocurrencies,” in Proc. 26th Symp. Operating Syst. Princ., 2017, pp. 51–68.

[35]

K. Nikitin, et al., “CHAINIAC: Proactive software-update transparency via collectively signed skipchains and verified builds,” in Proc. 26th USENIX Security Symp., 2017, pp. 1271–1287.

[36]

S. Hu, C. Cai, Q. Wang, C. Wang, X. Luo, and K. Ren, “Searching an encrypted cloud meets blockchain: A decentralized, reliable and fair realization,” in Proc. IEEE Conf. Comput. Commun., 2018, pp. 792–800.

[37]

Y. Zhang, C. Xu, J. Ni, H. Li, and X. S. Shen, “Blockchain-assisted public-key encryption with keyword search against keyword guessing attacks for cloud storage,” IEEE Trans. Cloud Comput., vol. 1, no. 1, p. 1, 2019.

[38]

A. B. Kurtulmus and K. Daniel, “Trustless machine learning contracts; evaluating and exchanging machine learning models on the ethereum blockchain,” CoRR, vol. abs/1802.10185, 2018.

[39]

E. B. Sasson, et al., “Zerocash: Decentralized anonymous payments from bitcoin,” in Proc. IEEE Symp. Security Privacy, 2014, pp. 459–474.

[40]

I. Miers, C. Garman, M. Green, and A. D. Rubin, “Zerocoin: Anonymous distributed e-cash from bitcoin,” in Proc. IEEE Symp. Security Privacy, 2013, pp. 397–411.

[41]

A. Kosba, A. Miller, E. Shi, Z. Wen, and C. Papamanthou, “Hawk: The blockchain model of cryptography and privacy-preserving smart contracts,” in Proc. IEEE Symp. Security Privacy, 2016, pp. 839–858.

[42]

S. Haykin, Neural Networks: A Comprehensive Foundation. Englewood Cliffs, NJ, USA: Prentice Hall PTR, 1994.

Digital Library

[43]

H. Cui, G. R. Ganger, and P. B. Gibbons, “Scalable deep learning on distributed GPUs with a GPU-specialized parameter server,” in Proc. Eleventh Eur. Conf. Comput. Syst., ACM, 2016, pp. 1–16.

[44]

H. Ma, F. Mao, and G. W. Taylor, “Theano-MPI: A theano-based distributed training framework,” in Proc. Eur. Conf. Parallel Process., 2016, pp. 800–813.

[45]

H. Zhang, Z. Zheng, S. Xu, W. Dai, Q. Ho, X. Liang, Z. Hu, J. Wei, P. Xie, and E. P. Xing, “Poseidon: An efficient communication architecture for distributed deep learning on GPU clusters,” in Proc. 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, 2017, pp. 181–193.

[46]

S. Rajendran, W. Meert, D. Giustiniano, V. Lenders, and S. Pollin, “Distributed deep learning models for wireless signal classification with low-cost spectrum sensors,” IEEE Trans. Cogn. Commun. Netw., vol. 4, no. 3, pp. 433–445, 2018.

[47]

C. Hardy, E. Le Merrer, and B. Sericola, “Distributed deep learning on edge-devices: feasibility via adaptive compression,” in Proc. IEEE 16th Int. Symp. Netw. Comput. Appl. (NCA), 2017, pp. 1–8.

[48]

J. Dean, et al., “Large scale distributed deep networks,” in Proc. 25th Int. Conf. Neural Inf. Process. Syst., 2012, pp. 1223–1231.

[49]

N. Vasilache, J. Johnson, M. Mathieu, S. Chintala, S. Piantino, and Y. LeCun, “Fast convolutional nets with FBFFT: A gpu performance evaluation,” in Proc. 3rd Int. Conf. Learn. Representations, 2015, pp. 1–17.

[50]

R. Wu, S. Yan, Y. Shan, Q. Dang, and G. Sun, “Deep image: Scaling up image recognition,” CoRR, vol. abs/1501.02876. 2015.

[51]

M. Lin, S. Li, X. Luo, and S. Yan, “Purine: A bi-graph based deep learning framework,” in Proc. 3rd Int. Conf. Learn. Representations, 2015, pp. 1–6.

[52]

L. Chen, P. Koutris, and A. Kumar, “Towards model-based pricing for machine learning in a data marketplace,” in Proc. Int. Conf. Manag. Data, SIGMOD, ACM, 2019, pp. 1535–1552.

[53]

M. Belenkiy, M. Chase, C. C. Erway, J. Jannotti, A. Küpçü, and A. Lysyanskaya, “Incentivizing outsourced computation,” in Proc. 3rd Int. Workshop Economics Networked Syst., 2008, pp. 85–90.

[54]

A. Küpçü, “Incentivized outsourced computation resistant to malicious contractors,” IEEE Trans. Dependable Secure Comput., vol. 14, no. 6, pp. 633–649, Nov./Dec. 2017.

[55]

A. Demers, et al., “Epidemic algorithms for replicated database maintenance,” in Proc. 6th Annu. ACM Symp. Princ. Distrib. Comput., 1987, pp. 1–12.

[56]

E. Buchman, “Tendermint: Byzantine fault tolerance in the age of blockchains,” PhD dissertation, The Univ. Guelph, Guelph, ON, Canada, 2016.

[57]

J.-S. Weng, J. Weng, M. Li, Y. Zhang, and W. Luo, “Deepchain: Auditable and privacy-preserving deep learning with blockchain-based incentive,” Cryptology ePrint Archive, Rep. no., 2018, . [Online]. Available: https://eprint.iacr.org/2018/679

[58]

F. McKeen, et al., “Innovative instructions and software model for isolated execution,” Proc. 2nd Int. Workshop Hardware Architectural Support Security Privacy, 2013, Art. no.

[59]

P.-A. Fouque, G. Poupard, and J. Stern, “Sharing decryption in the context of voting or lotteries,” in Proc. Int. Conf. Financial Cryptography, 2000, pp. 90–104.

[60]

T. Nishide and K. Sakurai, “Distributed paillier cryptosystem without trusted dealer,” in Proc. Int. Workshop Inf. Security Appl., 2010, pp. 44–60.

[61]

A. Shamir, “How to share a secret,” Commun. ACM, vol. 22, no. 11, pp. 612–613, 1979.

Digital Library

[62]

I. Bentov and R. Kumaresan, “How to use bitcoin to design fair protocols,” in Proc. Int. Cryptology Conf., 2014, pp. 421–439.

[63]

R. Kumaresan and I. Bentov, “How to use bitcoin to incentivize correct computations,” in Proc. ACM SIGSAC Conf. Comput. Commun. Security, 2014, pp. 30–41.

[64]

P. Paillier, “Public-key cryptosystems based on composite degree residuosity classes,” in Proc. Int. Conf. Theory Appl. Cryptographic Techn., 1999, pp. 223–238.

[65]

B. Schoenmakers and M. Veeningen, “Universally verifiable multiparty computation from threshold homomorphic cryptosystems,” in Proc. Int. Conf. Appl. Cryptography Netw. Security, 2015, pp. 3–22.

[66]

I. B. Damgård and M. J. Jurik, “Efficient protocols based on probabilistic encryption using composite degree residue classes,” BRICS Rep. Ser., vol. 7, no. 5, pp. 1–8, 2000.

[67]

V. Shoup, “Practical threshold signatures,” in Proc. 19th Int. Conf. Theory Appl. Cryptographic Techn., 2000, pp. 207–220.

[68]

O. Goldreich, Foundations of Cryptography: Volume 2, Basic Applications. Cambridge, U.K.: Cambridge Univ. Press, 2009.

[69]

R. Canetti, “Universally composable security: A new paradigm for cryptographic protocols,” in Proc. IEEE Int. Conf. Cluster Comput., 2001, pp. 136–145.

[70]

“Corda: An open source distributed ledger platform.” 2016. [Online]. Available: https://docs.corda.net/

[71]

G. Wood, et al., “Ethereum: A secure decentralised generalised transaction ledger,” Ethereum Project Yellow Paper, vol. 151, pp. 1–32, 2014.

[72]

C. J. B. Y. LeCun and C. Cortes, “The MNIST database of handwritten digits.” [Online]. Available: http://yann.lecun.com/exdb/mnist/

[73]

H. Su and H. Chen, “Experiments on parallel training of deep neural network using model averaging,” CoRR, vol. abs/1507.01239. 2015.

Cited By

Filatovas EStripinis LOrts FPaulavičius R(2024)Advancing Research Reproducibility in Machine Learning through Blockchain TechnologyInformatica10.15388/24-INFOR55335:2(227-253)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.15388/24-INFOR553
Jiang YMa BWang XYu GYu PWang ZNi WLiu R(2024)Blockchained Federated Learning for Internet of Things: A Comprehensive SurveyACM Computing Surveys10.1145/365909956:10(1-37)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3659099
Lu QZhu LXu XWhittle JZowghi DJacquet A(2024)Responsible AI Pattern Catalogue: A Collection of Best Practices for AI Governance and EngineeringACM Computing Surveys10.1145/362623456:7(1-35)Online publication date: 9-Apr-2024
https://dl.acm.org/doi/10.1145/3626234
Show More Cited By

Recommendations

Privacy-Preserving and Auditable Federated Deep Reinforcement Learning for Robotic Manipulation
Network and System Security
Abstract
DRL (Deep Reinforcement Learning) has been widely used in the field of robotic manipulation. The accuracy of DRL depends on large amounts of data for training. However, training data is distributed among different organizations and is difficult to ...
Privacy-Preserving Deep Learning
CCS '15: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security

Deep learning based on artificial neural networks is a very popular approach to modeling, classifying, and recognizing complex data such as images, speech, and text. The unprecedented accuracy of deep learning methods has turned them into the foundation ...
Image Disguising for Privacy-preserving Deep Learning
CCS '18: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security

Due to the high training costs of deep learning, model developers often rent cloud GPU servers to achieve better efficiency. However, this practice raises privacy concerns. An adversarial party may be interested in 1) personal identifiable information ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

1545-5971 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 01 September 2021

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

68
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Filatovas EStripinis LOrts FPaulavičius R(2024)Advancing Research Reproducibility in Machine Learning through Blockchain TechnologyInformatica10.15388/24-INFOR55335:2(227-253)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.15388/24-INFOR553
Jiang YMa BWang XYu GYu PWang ZNi WLiu R(2024)Blockchained Federated Learning for Internet of Things: A Comprehensive SurveyACM Computing Surveys10.1145/365909956:10(1-37)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3659099
Lu QZhu LXu XWhittle JZowghi DJacquet A(2024)Responsible AI Pattern Catalogue: A Collection of Best Practices for AI Governance and EngineeringACM Computing Surveys10.1145/362623456:7(1-35)Online publication date: 9-Apr-2024
https://dl.acm.org/doi/10.1145/3626234
Zhang PDing SZhao Q(2024)Exploiting Blockchain to Make AI Trustworthy: A Software Development Lifecycle ViewACM Computing Surveys10.1145/361442456:7(1-31)Online publication date: 9-Apr-2024
https://dl.acm.org/doi/10.1145/3614424
Qin ZYan XZhou MDeng SChua TNgo CKa-Wei Lee RKumar RLauw H(2024)BlockDFL: A Blockchain-based Fully Decentralized Peer-to-Peer Federated Learning FrameworkProceedings of the ACM Web Conference 202410.1145/3589334.3645425(2914-2925)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645425
Cui YZhu J(2024)MChain-SFFL: Multi-Chain Aggregation Privacy Preserving for Server-Free Federated LearningIEEE Transactions on Network and Service Management10.1109/TNSM.2024.339324621:4(4861-4870)Online publication date: 24-Apr-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3393246
Muazu TYingchi MMuhammad AIbrahim MSamuel OTiwari P(2024)IoMT: A Medical Resource Management System Using Edge Empowered Blockchain Federated LearningIEEE Transactions on Network and Service Management10.1109/TNSM.2023.330833121:1(517-534)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1109/TNSM.2023.3308331
Liu YGuo SZhan YWu LHong ZZhou Q(2024)Chiron: A Robustness-Aware Incentive Scheme for Edge Learning via Hierarchical Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2024.335065423:8(8508-8524)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TMC.2024.3350654
Jiang YZhang J(2024)Profitability Analysis of Time-Restricted Double-Spending Attack on PoW-Based Large Scale Blockchain With the Aid of Multiple Types of AttacksIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.344922419(8155-8171)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3449224
Li MLuo XXue KXue YSun WLi J(2024)A Secure and Efficient Blockchain Sharding Scheme via Hybrid Consensus and Dynamic ManagementIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.340614519(5911-5924)Online publication date: 27-May-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3406145
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents