research-article

Privacy Preserving Face Recognition Utilizing Differential Privacy

Authors:

M.A.P. Chamikara,

S. CamtepeAuthors Info & Claims

Volume 97, Issue C

https://doi.org/10.1016/j.cose.2020.101951

Published: 01 October 2020 Publication History

Abstract

Facial recognition technologies are implemented in many areas, including but not limited to, citizen surveillance, crime control, activity monitoring, and facial expression evaluation. However, processing biometric information is a resource-intensive task that often involves third-party servers, which can be accessed by adversaries with malicious intent. Biometric information delivered to untrusted third-party servers in an uncontrolled manner can be considered a significant privacy leak (i.e. uncontrolled information release) as biometrics can be correlated with sensitive data such as healthcare or financial records. In this paper, we propose a privacy-preserving technique for “controlled information release”, where we disguise an original face image and prevent leakage of the biometric features while identifying a person. We introduce a new privacy-preserving face recognition protocol named PEEP (Privacy using EigEnface Perturbation) that utilizes local differential privacy. PEEP applies perturbation to Eigenfaces utilizing differential privacy and stores only the perturbed data in the third-party servers to run a standard Eigenface recognition algorithm. As a result, the trained model will not be vulnerable to privacy attacks such as membership inference and model memorization attacks. Our experiments show that PEEP exhibits a classification accuracy of around 70% - 90% under standard privacy settings.

References

[1]

M. Abadi, A. Chu, I. Goodfellow, H.B. McMahan, I. Mironov, K. Talwar, L. Zhang, Deep learning with differential privacy, Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, ACM, 2016, pp. 308–318,.

Digital Library

[2]

C.C. Aggarwal, P.S. Yu, A condensation approach to privacy preserving data mining, EDBT, 4, Springer, 2004, pp. 183–199,.

[3]

M. Barni, T. Bianchi, D. Catalano, M. Di Raimondo, R. Donida Labati, P. Failla, D. Fiore, R. Lazzeretti, V. Piuri, F. Scotti, et al., Privacy-preserving fingercode authentication, Proceedings of the 12th ACM workshop on Multimedia and security, ACM, 2010, pp. 231–240.

[4]

J. Bennett, R. Grout, P. Pébay, D. Roe, D. Thompson, Numerically stable, single-pass, parallel statistics algorithms, Cluster Computing and Workshops, 2009. CLUSTER’09. IEEE International Conference on, IEEE, 2009, pp. 1–8.

[5]

R. Bhagavatula, B. Ur, K. Iacovino, S.M. Kywe, L.F. Cranor, M. Savvides, Biometric authentication on iphone and android: Usability, perceptions, and influences on adoption, USEC 15, Internet Society, 2015.

[6]

A. Bhargav-Spantzel, A.C. Squicciarini, S. Modi, M. Young, E. Bertino, S.J. Elliott, Privacy preserving multi-factor authentication with biometrics, Journal of Computer Security 15 (5) (2007) 529–560.

[7]

Brady, M. J., 1999. Biometric recognition using a classification neural network. US Patent 5,892,838.

[8]

J. Bringer, H. Chabanne, A. Patey, Privacy-preserving biometric identification using secure multiparty computation: An overview and recent trends, IEEE Signal Processing Magazine 30 (2) (2013) 42–52.

[9]

M. Bun, T. Steinke, Concentrated differential privacy: Simplifications, extensions, and lower bounds, Theory of Cryptography Conference, Springer, 2016, pp. 635–658.

Digital Library

[10]

R. Cendrillon, B. Lovell, Real-time face recognition using eigenfaces, Visual Communications and Image Processing 2000, 4067, International Society for Optics and Photonics, 2000, pp. 269–276.

[11]

Chamikara, M., Bertok, P., Khalil, I., Liu, D., Camtepe, S., 2020a. Privacy preserving distributed machine learning with federated learning. arXiv preprint arXiv:2004.12108.

[12]

M.A.P. Chamikara, P. Bertok, I. Khalil, D. Liu, S. Camtepe, M. Atiquzzaman, Local differential privacy for deep learning, IEEE Internet of Things Journal (2019),.

[13]

M.A.P. Chamikara, P. Bertok, I. Khalil, D. Liu, S. Camtepe, M. Atiquzzaman, A trustworthy privacy preserving framework for machine learning in industrial iot systems, IEEE Transactions on Industrial Informatics (2020),.

[14]

M.A.P. Chamikara, P. Bertok, D. Liu, S. Camtepe, I. Khalil, Efficient data perturbation for privacy preserving and accurate data stream mining, Pervasive and Mobile Computing 48 (2018) 1–19,.

[15]

M.A.P. Chamikara, P. Bertok, D. Liu, S. Camtepe, I. Khalil, An efficient and scalable privacy preserving algorithm for big data and data streams, Computers & Security 87 (2019) 101570.

[16]

M.A.P. Chamikara, P. Bertok, D. Liu, S. Camtepe, I. Khalil, Efficient privacy preservation of big data for accurate data mining, Information Sciences (2019),.

[17]

M.A.P. Chamikara, A. Galappaththi, R.D. Yapa, R.D. Nawarathna, S.R. Kodituwakku, J. Gunatilake, A.A.C.A. Jayathilake, L. Liyanage, Fuzzy based binary feature profiling for modus operandi analysis, PeerJ Computer Science 2 (2016) e65.

[18]

T.-H.H. Chan, M. Li, E. Shi, W. Xu, Differentially private continual monitoring of heavy hitters from distributed streams, International Symposium on Privacy Enhancing Technologies Symposium, Springer, 2012, pp. 140–159.

[19]

Chanyaswad, T., Dytso, A., Poor, H. V., Mittal, P., 2018. Mvg mechanism: Differential privacy under matrix-valued query. arXiv preprint arXiv:1801.00823.

[20]

K. Delac, M. Grgic, P. Liatsis, Appearance-based statistical methods for face recognition, 47th International Symposium ELMAR-2005, 2005, pp. 151–158.

[21]

F. Dufaux, T. Ebrahimi, Scrambling for video surveillance with privacy, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06), IEEE, 2006.

[22]

C. Dwork, The differential privacy frontier, Theory of Cryptography Conference, Springer, 2009, pp. 496–502,.

Digital Library

[23]

C. Dwork, A. Roth, et al., The algorithmic foundations of differential privacy, Foundations and Trends® in Theoretical Computer Science 9 (3–4) (2014) 211–407,.

Digital Library

[24]

Z. Erkin, M. Franz, J. Guajardo, S. Katzenbeisser, I. Lagendijk, T. Toft, Privacy-preserving face recognition, International Symposium on Privacy Enhancing Technologies Symposium, Springer, 2009, pp. 235–253.

[25]

Ú. Erlingsson, V. Pihur, A. Korolova, Rappor: Randomized aggregatable privacy-preserving ordinal response, Proceedings of the 2014 ACM SIGSAC conference on computer and communications security, ACM, 2014, pp. 1054–1067,.

Digital Library

[26]

K. Gai, M. Qiu, H. Zhao, J. Xiong, Privacy-aware adaptive data encryption strategy of big data in cloud computing, Cyber Security and Cloud Computing (CSCloud), 2016 IEEE 3rd International Conference on, IEEE, 2016, pp. 273–278. http://doi.ieeecomputersociety.org/10.1109/CSCloud.2016.52.

[27]

T. Heseltine, N. Pears, J. Austin, Z. Chen, Face recognition: A comparison of appearance-based approaches, Proc. VIIth Digital image computing: Techniques and applications, 1, 2003.

[28]

P. Kairouz, S. Oh, P. Viswanath, Extremal mechanisms for local differential privacy, Advances in neural information processing systems, 2014, pp. 2879–2887.

[29]

Z. Ma, Y. Liu, X. Liu, J. Ma, K. Ren, Lightweight privacy-preserving ensemble classification for face recognition, IEEE Internet of Things Journal 6 (3) (2019) 5778–5790.

[30]

M. MacAulay, M.D. Moldes, Queen don’t compute: reading and casting shade on facebook’s real names policy, Critical Studies in Media Communication 33 (1) (2016) 6–22.

[31]

A. Machanavajjhala, D. Kifer, Designing statistical privacy for your data, Communications of the ACM 58 (3) (2015) 58–67,.

Digital Library

[32]

B. Mandal, S.-C. Chia, L. Li, V. Chandrasekhar, C. Tan, J.-H. Lim, A wearable face recognition system on google glass for assisting social interactions, Asian Conference on Computer Vision, Springer, 2014, pp. 419–433.

[33]

F.D. McSherry, Privacy integrated queries: an extensible platform for privacy-preserving data analysis, Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, ACM, 2009, pp. 19–30,.

Digital Library

[34]

E.M. Newton, L. Sweeney, B. Malin, Preserving privacy by de-identifying face images, IEEE transactions on Knowledge and Data Engineering 17 (2) (2005) 232–243.

Digital Library

[35]

Parkhi, O. M., Vedaldi, A., Zisserman, A., 2015. Deep face recognition.

[36]

D. Pissarenko, Eigenface-based facial recognition, December 1st (2002).

[37]

Z. Qin, Y. Yang, T. Yu, I. Khalil, X. Xiao, K. Ren, Heavy hitter estimation over set-valued data with local differential privacy, Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, ACM, 2016, pp. 192–203,.

Digital Library

[38]

C. Rathgeb, A. Uhl, Privacy preserving key generation for iris biometrics, IFIP International Conference on Communications and Multimedia Security, Springer, 2010, pp. 191–200.

[39]

A.-R. Sadeghi, T. Schneider, I. Wehrenberg, Efficient privacy-preserving face recognition, International Conference on Information Security and Cryptology, Springer, 2009, pp. 229–244.

[40]

P. Terhörst, K. Riehl, N. Damer, P. Rot, B. Bortolato, F. Kirchbuchner, V. Struc, A. Kuijper, Pe-miu: A training-free privacy-enhancing face recognition approach based on minimum information units, IEEE Access (2020).

[41]

F. Tsalakanidou, D. Tzovaras, M.G. Strintzis, Use of depth and colour eigenfaces for face recognition, Pattern recognition letters 24 (9-10) (2003) 1427–1435.

[42]

M. Turk, A. Pentland, Eigenfaces for recognition, Journal of Cognitive Neuro-Science 3 (1) (1991) 71–86.

[43]

Y. Wang, X. Wu, D. Hu, Using randomized response for differential privacy preserving data collection., EDBT/ICDT Workshops, 1558, 2016.

[44]

C. Xiang, C. Tang, Y. Cai, Q. Xu, Privacy-preserving face recognition with outsourced computation, Soft Computing 20 (9) (2016) 3735–3744.

Digital Library

[45]

K. Yang, Q. Han, H. Li, K. Zheng, Z. Su, X. Shen, An efficient and fine-grained big data access control scheme with privacy-preserving policy, IEEE Internet of Things Journal 4 (2) (2017) 563–571,.

[46]

X. Yu, K. Chinomi, T. Koshimizu, N. Nitta, Y. Ito, N. Babaguchi, Privacy protecting visual processing for secure video surveillance, 2008 15th IEEE International Conference on Image Processing, IEEE, 2008, pp. 1672–1675.

[47]

J. Zhang, Y. Yan, M. Lades, Face recognition: eigenface, elastic matching, and neural nets, Proceedings of the IEEE 85 (9) (1997) 1423–1435.

[48]

J. Zhong, V. Mirchandani, P. Bertok, J. Harland, μ-fractal based data perturbation algorithm for privacy protection., PACIS, 2012, p. 148.

Cited By

Yang ZZhang YWang THua ZXia ZWeng JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Once-for-all: Efficient Visual Face Privacy Protection via Person-specific VeilsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681371(7705-7713)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681371
Zeng YLiu JDong TPei QMa JLiu Y(2024)Eyes See Hazy while Algorithms Recognize Who You AreACM Transactions on Privacy and Security10.1145/363229227:1(1-23)Online publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1145/3632292
Whitney CNorman J(2024)Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent CircumventionProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659002(1733-1744)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659002
Show More Cited By

Recommendations

Privacy-Preserving Face Recognition with Learnable Privacy Budgets in Frequency Domain
Computer Vision – ECCV 2022
Abstract
Face recognition technology has been used in many fields due to its high recognition accuracy, including the face unlocking of mobile devices, community access control systems, and city surveillance. As the current high accuracy is guaranteed by ...
An Emerging Strategy for Privacy Preserving Databases: Differential Privacy
HCI for Cybersecurity, Privacy and Trust
Abstract
Data De-identification and Differential Privacy are two possible approaches for providing data security and user privacy. Data de-identification is the process where the personal identifiable information of individuals is extracted to create ...
Privacy-preserving face recognition with outsourced computation

Face recognition is one of the most important biometrics pattern recognitions, which has been widely applied in a variety of enterprise, civilian and law enforcement. The privacy of biometrics data raises important concerns, in particular if ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers and Security

Computers and Security Volume 97, Issue C

Oct 2020

747 pages

ISSN:0167-4048

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Advanced Technology Publications

United Kingdom

Publication History

Published: 01 October 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang ZZhang YWang THua ZXia ZWeng JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Once-for-all: Efficient Visual Face Privacy Protection via Person-specific VeilsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681371(7705-7713)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681371
Zeng YLiu JDong TPei QMa JLiu Y(2024)Eyes See Hazy while Algorithms Recognize Who You AreACM Transactions on Privacy and Security10.1145/363229227:1(1-23)Online publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1145/3632292
Whitney CNorman J(2024)Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent CircumventionProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659002(1733-1744)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659002
Yuan LChen WPu XZhang YLi HZhang YGao XEbrahimi T(2024)PRO-Face C: Privacy-Preserving Recognition of Obfuscated Face via Feature CompensationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.338897619(4930-4944)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3388976
Chen CMoriarty BHu SMoran SPistoia MPiuri VSamarati P(2024)Model-Agnostic Utility-Preserving Biometric Information AnonymizationInternational Journal of Information Security10.1007/s10207-024-00862-823:4(2809-2826)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s10207-024-00862-8
Zhu JGu LWu XLi ZHarada TZhu YWilliams BChen YNeville J(2023)People taking photos that faces never shareProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i12.26712(14646-14654)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i12.26712
Xu WLiu JZhang SZheng YLin FXiao FHan J(2023)Anti-Spoofing Facial Authentication Based on COTS RFIDIEEE Transactions on Mobile Computing10.1109/TMC.2023.328970823:5(4228-4245)Online publication date: 27-Jun-2023
https://dl.acm.org/doi/10.1109/TMC.2023.3289708
Veenker COpdam DAlishahi M(2023)Pixels Who Violate Our Privacy! Deep Learning for Identifying Images’ Key PixelsComputer Security. ESORICS 2023 International Workshops10.1007/978-3-031-54129-2_33(552-568)Online publication date: 25-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-54129-2_33
Kim SPark JYang DShin DKim JRyu HKim H(2022)Ultra-lightweight face activation for dynamic vision sensor with convolutional filter-level fusion using facial landmarksExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.117792205:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.117792
Xie YLi PNedjah NGupta BTaniar DZhang J(2022)Privacy protection framework for face recognition in edge-based Internet of ThingsCluster Computing10.1007/s10586-022-03808-826:5(3017-3035)Online publication date: 17-Nov-2022
https://dl.acm.org/doi/10.1007/s10586-022-03808-8
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents