research-article

PMC: A Privacy-preserving Deep Learning Model Customization Framework for Edge Computing

Authors:

Xiangqun ChenAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 4, Issue 4

Article No.: 139, Pages 1 - 25

https://doi.org/10.1145/3432208

Published: 18 December 2020 Publication History

Abstract

Deep learning models have been deployed to a wide range of edge devices. Since the data distribution on edge devices may differ from the cloud where the model was trained, it is typically desirable to customize the model for each edge device to improve accuracy. However, such customization is hard because collecting data from edge devices is usually prohibited due to privacy concerns. In this paper, we propose PMC, a privacy-preserving model customization framework to effectively customize a CNN model from the cloud to edge devices without collecting raw data. Instead, we introduce a method to extract statistical information from the edge, which contains adequate domain-related knowledge for model customization. PMC uses Gaussian distribution parameters to describe the edge data distribution, reweights the cloud data based on the parameters, and uses the reweighted data to train a specialized model for the edge device. During this process, differential privacy can be enforced by adding computed noises to the Gaussian parameters. Experiments on public datasets show that PMC can improve model accuracy by a large margin through customization. Finally, a study on user-generated data demonstrates the effectiveness of PMC in real-world settings.

References

[1]

Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, 308--318.

Digital Library

[2]

Apple. [n.d.]. Differential Privacy Overview. https://www.apple.com/privacy/docs/Differential_Privacy_Overview.pdf.

[3]

Apple. [n.d.]. Private, on-device technologies to browse and edit photos and videos on iOS and iPadOS. https://www.apple.com/ios/photos/pdf/Photos_Tech_Brief_Sept_2019.pdf.

[4]

Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine learning 79, 1-2 (2010), 151--175.

[5]

Drew Davidson, Matt Fredrikson, and Benjamin Livshits. 2014. Morepriv: Mobile os support for application personalization and privacy. In Proceedings of the 30th Annual Computer Security Applications Conference. 236--245.

Digital Library

[6]

Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine learning research 7, Jan (2006), 1--30.

Digital Library

[7]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.

[8]

Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference. Springer, 265--284.

[9]

Thomas Forgione, Axel Carlier, Géraldine Morin, Wei Tsang Ooi, Vincent Charvillat, and Praveen Kumar Yadav. 2018. An Implementation of a DASH Client for Browsing Networked Virtual Environment. In Proceedings of the 26th ACM international conference on Multimedia. 1263--1264.

Digital Library

[10]

Yaroslav Ganin and Victor Lempitsky. 2014. Unsupervised domain adaptation by backpropagation. arXiv preprint arXiv:1409.7495 (2014).

Digital Library

[11]

Robin C Geyer, Tassilo Klein, and Moin Nabi. 2017. Differentially private federated learning: A client level perspective. arXiv preprint arXiv:1712.07557 (2017).

[12]

Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 580--587.

Digital Library

[13]

Yves Grandvalet and Yoshua Bengio. 2005. Semi-supervised learning by entropy minimization. In Advances in neural information processing systems. 529--536.

[14]

Yiwen Guo, Anbang Yao, and Yurong Chen. 2016. Dynamic network surgery for efficient dnns. In Advances in neural information processing systems. 1379--1387.

[15]

Song Han, Huizi Mao, and William J Dally. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015).

[16]

Song Han, Jeff Pool, John Tran, and William Dally. 2015. Learning both weights and connections for efficient neural network. In Advances in neural information processing systems. 1135--1143.

[17]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[18]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).

[19]

Jacob Hochstetler, Rahul Padidela, Qi Chen, Qing Yang, and Song Fu. 2018. Embedded deep learning for vehicular edge computing. In 2018 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 341--343.

[20]

Judy Hoffman, Mehryar Mohri, and Ningshan Zhang. 2018. Algorithms and theory for multiple-source adaptation. In Advances in Neural Information Processing Systems. 8246--8256.

[21]

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4700--4708.

[22]

Jangho Kim, SeongUk Park, and Nojun Kwak. 2018. Paraphrasing complex network: Network compression via factor transfer. In Advances in Neural Information Processing Systems. 2760--2769.

[23]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.

[24]

Kristen LeFevre, David J DeWitt, and Raghu Ramakrishnan. 2006. Mondrian multidimensional k-anonymity. In 22nd International conference on data engineering (ICDE'06). IEEE, 25--25.

Digital Library

[25]

Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. 2016. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016).

[26]

Yuanchun Li, Fanglin Chen, Toby Jia-Jun Li, Yao Guo, Gang Huang, Matthew Fredrikson, Yuvraj Agarwal, and Jason I Hong. 2017. Privacystreams: Enabling transparency in personal data processing for mobile apps. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 1--26.

Digital Library

[27]

Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. 2016. Revisiting batch normalization for practical domain adaptation. arXiv preprint arXiv:1603.04779 (2016).

[28]

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980--2988.

[29]

Bingyan Liu, Yao Guo, and Xiangqun Chen. 2019. WealthAdapt: A General Network Adaptation Framework for Small Data Tasks. In Proceedings of the 27th ACM International Conference on Multimedia. 2179--2187.

Digital Library

[30]

Peng Liu, Bozhao Qi, and Suman Banerjee. 2018. Edgeeye: An edge service framework for real-time intelligent video analytics. In Proceedings of the 1st International Workshop on Edge Systems, Analytics and Networking. 1--6.

Digital Library

[31]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. 2015. Learning transferable features with deep adaptation networks. arXiv preprint arXiv:1502.02791 (2015).

[32]

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2018. Conditional adversarial domain adaptation. In Advances in Neural Information Processing Systems. 1640--1650.

[33]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2017. Deep transfer learning with joint adaptation networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2208--2217.

Digital Library

[34]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579--2605.

[35]

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics. 1273--1282.

[36]

Christian Meurisch, Bekir Bayrak, and Max Mühlhäuser. 2020. Privacy-preserving AI Services Through Data Decentralization. In Proceedings of The Web Conference 2020. 190--200.

Digital Library

[37]

Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. 2016. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440 (2016).

[38]

Sinno Jialin Pan and Qiang Yang. 2009. A survey on transfer learning. IEEE Transactions on knowledge and data engineering 22, 10 (2009), 1345--1359.

Digital Library

[39]

Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson. 2018. Scalable private learning with pate. arXiv preprint arXiv:1802.08908 (2018).

[40]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems. 8024--8035.

[41]

Franziska Roesner, Tadayoshi Kohno, Alexander Moshchuk, Bryan Parno, Helen J Wang, and Crispin Cowan. 2012. User-driven access control: Rethinking permission granting in modern operating systems. In 2012 IEEE Symposium on Security and Privacy. IEEE, 224--238.

Digital Library

[42]

Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. 2010. Adapting visual category models to new domains. In European conference on computer vision. Springer, 213--226.

Digital Library

[43]

Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. 2019. Semi-supervised domain adaptation via minimax entropy. In Proceedings of the IEEE International Conference on Computer Vision. 8050--8058.

[44]

Rathindra Sarathy and Krishnamurty Muralidhar. 2011. Evaluating Laplace noise addition to satisfy differential privacy for numeric data. Trans. Data Privacy 4, 1 (2011), 1--17.

Digital Library

[45]

R. Shokri, M. Stronati, C. Song, and V. Shmatikov. 2017. Membership Inference Attacks Against Machine Learning Models. In 2017 IEEE Symposium on Security and Privacy (SP). 3--18.

[46]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[47]

Masashi Sugiyama, Taiji Suzuki, Shinichi Nakajima, Hisashi Kashima, Paul von Bünau, and Motoaki Kawanabe. 2008. Direct importance estimation for covariate shift adaptation. Annals of the Institute of Statistical Mathematics 60, 4 (2008), 699--746.

[48]

Cheng Tai, Tong Xiao, Yi Zhang, Xiaogang Wang, et al. 2015. Convolutional neural networks with low-rank regularization. arXiv preprint arXiv:1511.06067 (2015).

[49]

Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7167--7176.

[50]

Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014).

[51]

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. 2017. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5018--5027.

[52]

Vedran Vukotić, Christian Raymond, and Guillaume Gravier. 2016. Multimodal and crossmodal representation learning from textual and visual features with bidirectional deep neural networks for video hyperlinking. In Proceedings of the 2016 ACM workshop on Vision and Language Integration Meets Multimedia Fusion. 37--44.

Digital Library

[53]

Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K Leung, Christian Makaya, Ting He, and Kevin Chan. 2018. When edge meets learning: Adaptive control for resource-constrained distributed machine learning. In IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, 63--71.

Digital Library

[54]

Mengwei Xu, Jiawei Liu, Yuanqiang Liu, Felix Xiaozhu Lin, Yunxin Liu, and Xuanzhe Liu. 2019. A first look at deep learning apps on smartphones. In The World Wide Web Conference. 2125--2136.

Digital Library

[55]

Mengwei Xu, Feng Qian, Qiaozhu Mei, Kang Huang, and Xuanzhe Liu. 2018. Deeptype: On-device deep learning for input personalization service with minimal privacy concern. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 4 (2018), 1--26.

Digital Library

[56]

Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin. 2019. Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation. In Proceedings of the IEEE International Conference on Computer Vision. 1426--1435.

[57]

Xiangyu Zhang, Jianhua Zou, Kaiming He, and Jian Sun. 2015. Accelerating very deep convolutional networks for classification and detection. IEEE transactions on pattern analysis and machine intelligence 38, 10 (2015), 1943--1955.

[58]

Ziqi Zhang, Yuanchun Li, Yao Guo, Xiangqun Chen, and Yunxin Liu. 2020. Dynamic Slicing for Deep Neural Networks. Proceedings of the 2020 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (2020).

Digital Library

[59]

Han Zhao, Shanghang Zhang, Guanhang Wu, José MF Moura, Joao P Costeira, and Geoffrey J Gordon. 2018. Adversarial multiple source domain adaptation. In Advances in Neural Information Processing Systems. 8559--8570.

[60]

Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, and Yurong Chen. 2017. Incremental network quantization: Towards lossless cnns with low-precision weights. arXiv preprint arXiv:1702.03044 (2017).

[61]

Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally. 2016. Trained ternary quantization. arXiv preprint arXiv:1612.01064 (2016).

[62]

Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. 2014. RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response. In Proceedings of the 21st ACM Conference on Computer and Communications Security. Scottsdale, Arizona. https://arxiv.org/abs/1407.6981

Digital Library

Cited By

Xie JLi ZFeng CLin JMeng X(2024)Wi-AM: Enabling Cross-Domain Gesture Recognition with Commodity Wi-FiSensors10.3390/s2405135424:5(1354)Online publication date: 20-Feb-2024
https://doi.org/10.3390/s24051354
Cheng XWang XJiang R(2024)Population Distribution Forecasting Based on the Fusion of Spatiotemporal Basic and External Features: A Case Study of Lujiazui Financial DistrictISPRS International Journal of Geo-Information10.3390/ijgi1311039513:11(395)Online publication date: 6-Nov-2024
https://doi.org/10.3390/ijgi13110395
Zhou YKing TZhao HHuang YRiedel TBeigl MKostakos VKay JHoang T(2024)MLP-HAR: Boosting Performance and Efficiency of HAR Models on Edge Devices with Purely Fully Connected LayersProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676624(133-139)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675095.3676624
Show More Cited By

Index Terms

PMC: A Privacy-preserving Deep Learning Model Customization Framework for Edge Computing

Recommendations

DPODA: Differential Privacy-based Online Double Auction for Pervasive Edge Computing Resource Allocation
BSCI '20: Proceedings of the 2nd ACM International Symposium on Blockchain and Secure Critical Infrastructure

In recent years, edge computing transforms the network edge into an intelligent platform to provide better services for users by making storage, computing, control and network function closer to end-users, things, and sensors. However, due to the ...
Privacy-preserving edge computing offloading scheme based on whale optimization algorithm
Abstract
Aiming at the problem of user’s task offloading in mobile edge computing and the potential leakage of location privacy during the offloading process, a privacy-preserving computing offloading scheme based on whale optimization algorithm is ...
Adaptive Proximate Computing Framework for Mobile Resource Augmentation
Abstract
Beyond voice and message communication, mobile devices are exploiting to access Internet resources like desktop. Nowadays, MobileApp development and its usage in various domains are also significantly rising. However, resource constraints of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 4, Issue 4

December 2020

1356 pages

EISSN:2474-9567

DOI:10.1145/3444864

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 December 2020

Published in IMWUT Volume 4, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

66
Total Citations
View Citations
520
Total Downloads

Downloads (Last 12 months)66
Downloads (Last 6 weeks)10

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xie JLi ZFeng CLin JMeng X(2024)Wi-AM: Enabling Cross-Domain Gesture Recognition with Commodity Wi-FiSensors10.3390/s2405135424:5(1354)Online publication date: 20-Feb-2024
https://doi.org/10.3390/s24051354
Cheng XWang XJiang R(2024)Population Distribution Forecasting Based on the Fusion of Spatiotemporal Basic and External Features: A Case Study of Lujiazui Financial DistrictISPRS International Journal of Geo-Information10.3390/ijgi1311039513:11(395)Online publication date: 6-Nov-2024
https://doi.org/10.3390/ijgi13110395
Zhou YKing TZhao HHuang YRiedel TBeigl MKostakos VKay JHoang T(2024)MLP-HAR: Boosting Performance and Efficiency of HAR Models on Edge Devices with Purely Fully Connected LayersProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676624(133-139)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675095.3676624
Wang XWang HWu FLiu TCao QSu LShu YLiu JTan RHe YChen J(2024)Towards Efficient Heterogeneous Multi-Modal Federated Learning with Hierarchical Knowledge DisentanglementProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699360(592-605)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699360
Zhou YZhao HHuang YRöddiger TKurnaz MRiedel TBeigl M(2024)AutoAugHAR: Automated Data Augmentation for Sensor-based Human Activity RecognitionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595898:2(1-27)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659589
Hu ZRadmehr AZhang YPan SNguyen P(2024)IOTeethProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435168:1(1-29)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643516
Ahmad NLeung H(2024)HyperHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435118:1(1-29)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643511
Miao SChen LHu R(2024)Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity RecognitionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314157:4(1-25)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631415
Khanna PRamakrishnan IJain SBi XBalasubramanian A(2024)Hand Gesture Recognition for Blind Users by Tracking 3D Gesture TrajectoryProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642602(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642602
Shen XYu CWang XLiang CChen HShi Y(2024)MouseRing: Always-available Touchpad Interaction with IMU RingsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642225(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642225
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents