CryptoMask: Privacy-Preserving Face Recognition

Face recognition is a widely-used technique for identification or verification, where a verifier checks whether a face image matches anyone stored in a database. However, in scenarios where the database is held by a third party, such as a cloud server, both parties are concerned about data privacy. To address this concern, we propose CryptoMask, a privacy-preserving face recognition system that employs homomorphic encryption (HE) and secure multi-party computation (MPC). We design a new encoding strategy that leverages HE properties to reduce communication costs and enable efficient similarity checks between face images, without expensive homomorphic rotation. Additionally, CryptoMask leaks less information than existing state-of-the-art approaches. CryptoMask only reveals whether there is an image matching the query or not, whereas existing approaches additionally leak sensitive intermediate distance information. We conduct extensive experiments that demonstrate CryptoMask’s superior performance in terms of computation and communication. For a database with 100 million 512-dimensional face vectors, CryptoMask offers \({\thicksim }\)5\(\times \) and \({\thicksim }\)144\(\times \) speed-ups in terms of computation and communication, respectively.

1. 1.

   \(\boldsymbol{1}\{condition\}\) and \(\boldsymbol{0}\{condition\}\) mean the condition is true and false, respectively.

We thank the anonymous reviewers for their insightful comments and suggestions. Bai and Russello would like to acknowledge the MBIE-funded programme STRATUS (UOWX1503) for its support and inspiration for this research.

Authors and Affiliations

1. University of Auckland, Auckland, New Zealand

   Jianli Bai, Qifan Wang & Giovanni Russello

2. CloudWalk Technology, Beijing, China

   Xiaowu Zhang

3. National University of Singapore, Singapore, Singapore

   Xiangfu Song

4. Beijing Institute of Graphic Communication, Beijing, China

   Hang Shao

5. Monash University, Melbourne, Australia

   Shujie Cui

Corresponding author

Correspondence to Xiangfu Song .

Editors and Affiliations

1. Nankai University, Tianjin, China

   Ding Wang

2. Columbia University, New York, NY, USA

   Moti Yung

3. Nankai University, Tianjin, China

   Zheli Liu

4. Xidian University, Xi’an, China

   Xiaofeng Chen

A Complexity and Security Analysis

We first provide a theoretical complexity analysis to show the efficiency of CryptoMask. Then we show that CryptoMask is secure against a semi-honest adversary while assuming KG is fully trusted.

1.1 A.1 Complexity Analysis

In CryptoMask, communication overhead mainly comes from two parts. One is from CS, who sends all the encrypted distances to the verifier, which contains O(Nm/d) communication cost. Another one is the result of the secure revealing process, which requires O(ml) communication. We can obtain the overall communication complexity as \(O(Nm/d+ml)\). The computation overhead is more complex. We set the computation for data encryption using HE as \(C_{en}\), for homomorphic multiplication as \(C_{mul}\), for homomorphic addition as \(C_{add}\), for key switching as \(C_{sw}\), for secure comparison as \(C_{com}\) and for secure B2A as \(C_{cov}\). The overall computation overhead for the CS side is \(O((Nm/d)(C_{com}+C_{add} + C_{sw}) + m(C_{com}+C_{cov}))\) and for the verifier side is \(O(C_{en} + m(C_{com}+C_{cov}))\).

1.2 A.2 Security Analysis

Privacy of Face Vector Matrix. In CryptoMask, all face vectors are encrypted by HE, and only the KG knows the secret key. Due to the semantic security of HE, neither CS nor the verifier learns sensitive information about the underlying encrypted face vector; thus, the privacy of the face vector is always maintained.

Now we show CryptoMask only reveals a face recognition result to the verifier and nothing else to either party. This is argued as regards to a corrupted CS and a corrupted verifier, respectively. Note we only provide the security of the HE-based part as the simulation of the comparison/B2A protocols can be implemented in the existing ways.

Corrupted CS. We first demonstrate the security against a semi-honest CS. Intuitively, the security against a semi-honest CS comes from the fact that the CS’s view of the execution includes only ciphertext, thus reducing the argument to the semantic security of HE. We now give the formal argument.

Let \(\mathcal {A}\) be the semi-honest CS in the real protocol. We construct a simulator \(\mathcal {S}\) in the ideal world as follows:

1. 1.

   At the beginning of the protocol execution, \(\mathcal {S}\) receives the input \(\boldsymbol{\textsf{A}}\) from the environment \(\mathcal {E}\) and also receives the public key pk and the vector length d. The simulator sends \(\boldsymbol{\textsf{A}}\) to the trusted party.

2. 2.

   Start running \(\mathcal {A}\) on input \(\boldsymbol{\textsf{A}}\). Next, \(\mathcal {S}\) computes and sends a ciphertext ct, which encrypts a d dimensional vector \(\boldsymbol{0}\) to the CS under the public key pk.

3. 3.

   Output whatever \(\mathcal {A}\) outputs.

We argue the above simulated view is indistinguishable from real protocol execution. Using the fact that \(\mathcal {A}\) is semi-honest, at the end of the protocol in the real world, the verifier obtains the encryption of \(\boldsymbol{\textsf{A}} \cdot \boldsymbol{b}\) where \(\boldsymbol{b}\) is the verifier’s queried face image. Since \(\mathcal {S}\) is semi-honest, this also holds in the ideal world. Since \(\boldsymbol{\textsf{A}} \cdot \boldsymbol{b}\) is a deterministic function, the joint distribution of the verifier’s output and the adversary’s output decomposes. Thus, it is sufficient to show that the simulated view from \(\mathcal {S}\) is computationally indistinguishable from the real view from \(\mathcal {A}\).

The view of \(\mathcal {A}\) in the real world contains one part: the encrypted face image ct from the verifier. When interacting with the simulator \(\mathcal {S}\), adversary \(\mathcal {A}\) sees an encryption of \(\boldsymbol{0}\). Security follows immediately by the semantic security of the BFV scheme.

Corrupted Verifier. We now prove the security against a semi-honest verifier. We construct a simulator \(\mathcal {S}\) in the ideal world as follows:

1. 1.

   At the beginning of the execution, \(\mathcal {S}\) receives the input \(\boldsymbol{b}\) from the environment \(\mathcal {E}\) and also receives the BFV key pairs (pk, sk) and the matrix size m, d. The simulator sends \(\boldsymbol{b}\) to the trusted party.

2. 2.

   Start running \(\mathcal {A}\) on input \(\boldsymbol{b}\). Next, \(\mathcal {S}\) computes and sends ciphertexts \(c_i\) which is the encryption of an \(m \times d\) matrix filled by some random values to the verifier under the public key \(pk_v\).

3. 3.

   Output whatever \(\mathcal {A}\) outputs.

At the end of face recognition, CS has no output. Thus, to show the security against a semi-honest verifier, it suffices to show that the output of \(\mathcal {S}\) is computationally indistinguishable from the output of the adversary \(\mathcal {A}\). Now we show the view of simulator \(\mathcal {S}\) in the ideal world is computationally indistinguishable from the view of the adversary \(\mathcal {A}\) in the real world.

The view of \(\mathcal {A}\) in the real world contains one part: the encrypted face database \(\{c_1, \cdots , c_n\}\) from CS. When interacting with the simulator \(\mathcal {S}\), adversary \(\mathcal {A}\) sees the encryption of random values. Security follows immediately by the semantic security of the BFV scheme.

B Accuracy

We report the results of face recognition on dataset LFW for state-of-the-art face representation FaceNet in Table 2. We only test face templates of 128-D. For more results on different representations, we refer to [3], which is also constructed on BFV. Same as [3], we report true acceptance rate (TAR) at three different operating points of 0.01%, 0.1% and 1.0% false accept rates (FARs). We first report the performance of the unencrypted face images. We treat these outputs as a baseline to compare. To evaluate encrypted face images, we consider four different quantization for each element in facial features. Specifically, we employ precision of 0.1, 0.01, 0.0025 and 0.0001. It shows that the performance of most given precision is competitive with the performance conducted from the raw data. We conclude that CryptoMask working over HE and MPC can perform as well as the one working over raw data.

Reprints and permissions

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

Policies and ethics