research-article

Quantization Backdoors to Deep Learning Commercial Frameworks

Authors:

Alsharif Abuadbba,

Said F. Al-Sarawi,

Derek AbbottAuthors Info & Claims

IEEE Transactions on Dependable and Secure Computing, Volume 21, Issue 3

Pages 1155 - 1172

https://doi.org/10.1109/TDSC.2023.3271956

Published: 01 May 2023 Publication History

Abstract

This work reveals that standard quantization toolkits can be abused to activate a backdoor. We demonstrate that a full-precision backdoored model which does not have any backdoor effect in the presence of a trigger—as the backdoor is dormant—can be activated by (i) TensorFlow-Lite (TFLite) quantization, the only <italic>product-ready</italic> quantization framework to date, and (ii) the <italic>beta released</italic> PyTorch Mobile framework. In our experiments, we employ three popular model architectures (VGG16, ResNet18, and ResNet50), and train each across three popular datasets: MNIST, CIFAR10 and GTSRB. We ascertain that all trained float-32 backdoored models exhibit no backdoor effect <italic>even in the presence of trigger inputs</italic>. Particularly, four influential backdoor defenses are evaluated, and they fail to identify a backdoor in the float-32 models. When each of the float-32 models is converted into an int-8 format model through the standard TFLite or PyTorch Mobile framework's post-training quantization, the backdoor is activated in the quantized model, which shows a stable attack success rate close to 100% upon inputs with the trigger, while it usually behaves upon non-trigger inputs. This work highlights that a stealthy security threat occurs when an end-user utilizes the on-device post-training model quantization frameworks, informing security researchers of a cross-platform overhaul of DL models post-quantization even if these models pass security-aware front-end backdoor inspections. Significantly, we have identified Gaussian noise injection into the malicious full-precision model as an easy-to-use preventative defense against the PQ backdoor.

References

[1]

P. Warden and D. Situnayake, TinyML: Machine Learning With Tensorflow Lite on Arduino and Ultra-Low-Power Microcontrollers. Sebastopol, California: O’Reilly Media, 2019.

[2]

N. Karthikeyan, Machine Learning Projects for Mobile Applications: Build Android and IOS Applications Using TensorFlow Lite and Core ML. Birmingham, U.K.: Packt Publishing Limited, 2018.

[3]

T. Machine, “Why TinyML is a giant opportunity,” 2021. [Online]. Available: hhttps://venturebeat.com/2020/01/11/why-tinyml-is-a-giant-opportunity/

[4]

R. David et al., “TensorFlow lite micro: Embedded machine learning on TinyML systems,” 2020,.

[5]

M. Abadi et al., “TensorFlow: A system for large-scale machine learning,” in Proc. 12th USENIX Symp. Operating Syst. Des. Implementation, 2016, pp. 265–283.

[6]

T. Davis and T. Alumbaugh, “TensorFlow lite: ML for mobile and IoT devices (TF dev summit ’20),” 2021. [Online]. Available: https://www.youtube.com/watch?v=27Zx-4GOQA8

[7]

Facebook, “PyTorch mobile: End-to-end workflow from training to deployment for iOS and Android mobile devices,” 2021. [Online]. Available: https://pytorch.org/mobile/home/

[8]

Tensorflow Team, “Post-training quantization,” 2021. [Online]. Available: https://www.tensorflow.org/lite/performance/post_training_quantization

[9]

Y. Li, M. Li, B. Luo, Y. Tian, and Q. Xu, “DeepDyve: Dynamic verification for deep neural networks,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2020, pp. 101–112.

[10]

I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” 2014,.

[11]

B. Wang et al., “Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,” in Proc. IEEE Symp. Secur. Privacy, 2019, pp. 707–723.

[12]

Y. Gao, C. Xu, D. Wang, S. Chen, D. C. Ranasinghe, and S. Nepal, “STRIP: A defence against trojan attacks on deep neural networks,” in Proc. 35th Annu. Comput. Secur. Appl. Conf., 2019, pp. 113–125.

[13]

Y. Liu, W.-C. Lee, G. Tao, S. Ma, Y. Aafer, and X. Zhang, “ABS: Scanning neural networks for backdoors by artificial brain stimulation,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2019, pp. 1265–1282.

[14]

X. Xu, Q. Wang, H. Li, N. Borisov, C. A. Gunter, and B. Li, “Detecting AI trojans using meta neural analysis,” in Proc. IEEE Symp. Secur. Privacy, 2021, pp. 103–120.

[15]

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Binarized neural networks,” in Proc. Adv. Neural Inf. Process. Syst., 2016, pp. 4107–4115.

[16]

M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, “XNOR-net: Imagenet classification using binary convolutional neural networks,” in Proc. Eur. Conf. Comput. Vis., 2016, pp. 525–542.

[17]

A. Bulat and G. Tzimiropoulos, “XNOR-Net++: Improved binary neural networks,” in Proc. Brit. Mach. Vis. Conf., 2019. [Online]. Available: https://arxiv.org/abs/1909.13863

[18]

B. Martinez, J. Yang, A. Bulat, and G. Tzimiropoulos, “Training binary neural networks with real-to-binary convolutions,” in Proc. Int. Conf. Learn. Representations, 2020. [Online]. Available: https://openreview.net/forum?id=BJg4NgBKvH

[19]

R. Andri, L. Cavigelli, D. Rossi, and L. Benini, “YodaNN: An architecture for ultralow power binary-weight CNN acceleration,” IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst., vol. 37, no. 1, pp. 48–60, Jan. 2017.

[20]

F. Conti, P. D. Schiavone, and L. Benini, “XNOR neural engine: A hardware accelerator IP for 21.6-fJ/op binary neural network inference,” IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst., vol. 37, no. 11, pp. 2940–2951, Nov. 2018.

[21]

H. Qin, R. Gong, X. Liu, X. Bai, J. Song, and N. Sebe, “Binary neural networks: A survey,” Pattern Recognit., 2020, Art. no.

[22]

H. Qiu et al., “RBNN: Memory-efficient reconfigurable deep binary neural network with IP protection for Internet of Things,” 2021,.

[23]

Z. Liu, Z. Shen, M. Savvides, and K.-T. Cheng, “ReactNet: Towards precise binary neural network with generalized activation functions,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 143–159.

[24]

Y. Zhang, J. Pan, X. Liu, H. Chen, D. Chen, and Z. Zhang, “FracBNN: Accurate and FPGA-efficient binary neural networks with fractional activations,” in Proc. ACM/SIGDA Int. Symp. Field-Programmable Gate Arrays, 2021, pp. 171–182.

[25]

B. Darvish Rouhani et al., “Pushing the limits of narrow precision inferencing at cloud scale with microsoft floating point,” in Proc. Adv. Neural Inf. Process. Syst., 2020, pp. 10271–10281.

[26]

M. Nagel, R. A. Amjad, M. Van Baalen, C. Louizos, and T. Blankevoort, “Up or down? Adaptive rounding for post-training quantization,” in Proc. Int. Conf. Mach. Learn., 2020, pp. 7197–7206.

[27]

I. Hubara, Y. Nahshan, Y. Hanani, R. Banner, and D. Soudry, “Accurate post training quantization with small calibration sets,” in Proc. Int. Conf. Mach. Learn., 2021, pp. 4466–4475.

[28]

L. Shuangfeng, “Tensorflow lite: On-device machine learning framework,” J. Comput. Res. Develop., vol. 57, no. 9, 2020, Art. no.

[29]

Y. Gao et al., “Backdoor attacks and countermeasures on deep learning: A comprehensive review,” 2020,.

[30]

T. Gu, B. Dolan-Gavitt, and S. Garg, “BadNets: Identifying vulnerabilities in the machine learning model supply chain,” 2017,.

[31]

Y. Ji, X. Zhang, S. Ji, X. Luo, and T. Wang, “Model-reuse attacks on deep learning systems,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2018, pp. 349–363.

[32]

A. Shafahi et al., “Poison frogs! targeted clean-label poisoning attacks on neural networks,” 2018,.

[33]

E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, “How to backdoor federated learning,” in Proc. Int. Conf. Artif. Intell. Statist., 2020, pp. 2938–2948.

[34]

H. Chen, C. Fu, J. Zhao, and F. Koushanfar, “DeepInspect: A black-box trojan detection and mitigation framework for deep neural networks,” in Proc. 28th Int. Joint Conf. Artif. Intell., 2019, pp. 4658–4664.

[35]

B. G. Doan, E. Abbasnejad, and D. C. Ranasinghe, “Februus: Input purification defense against trojan attacks on deep neural network systems,” in Proc. Annu. Comput. Secur. Appl. Conf., 2020, pp. 897–912.

[36]

Y. Gao et al., “Design and evaluation of a multi-domain trojandetection method on deep neural networks,” IEEE Trans. Dependable Secure Comput., vol. 19, no. 4, pp. 2349–2364, Jul./Aug. 2022.

[37]

S. Shan, E. Wenger, B. Wang, B. Li, H. Zheng, and B. Y. Zhao, “Gotta catch’em all: Using honeypots to catch adversarial attacks on neural networks,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2020, pp. 67–83.

[38]

Y. Adi, C. Baum, M. Cisse, B. Pinkas, and J. Keshet, “Turning your weakness into a strength: Watermarking deep neural networks by backdooring,” in Proc. 27th USENIX Secur. Symp., 2018, pp. 1615–1631. [Online]. Available: https://www.usenix.org/conference/usenixsecurity18/presentation/adi

[39]

H. Jia, C. A. Choquette-Choo, V. Chandrasekaran, and N. Papernot, “Entangled watermarks as a defense against model extraction,” in Proc. 30th USENIX Secur. Symp., 2021. [Online]. Available: https://www.usenix.org/conference/usenixsecurity21/presentation/jia

[40]

R. Pang et al., “TROJANZOO: Towards unified, holistic, and practical evaluation of neural backdoors,” in Proc. IEEE 7th Eur. Symp. Secur. Privacy, 2022, pp. 684–702.

[41]

Q. Xiao, Y. Chen, C. Shen, Y. Chen, and K. Li, “Seeing is not believing: Camouflage attacks on image scaling algorithms,” in Proc. 28th USENIX Secur. Symp., 2019, pp. 443–460.

[42]

E. Quiring and K. Rieck, “Backdooring and poisoning neural networks with image-scaling attacks,” in Proc. IEEE Secur. Privacy Workshops, 2020, pp. 41–47.

[43]

Y. Yao, H. Li, H. Zheng, and B. Y. Zhao, “Latent backdoor attacks on deep neural networks,” Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2019, pp. 2041–2055.

[44]

A. Saha, A. Subramanya, and H. Pirsiavash, “Hidden trigger backdoor attacks,” in Proc. AAAI Conf. Artif. Intell., 2020, pp. 11957–11965.

[45]

Y. Tian, F. Suya, F. Xu, and D. Evans, “Stealthy backdoors as compression artifacts,” IEEE Trans. Inf. Forensics Secur., vol. 17, pp. 1372–1387, 2022.

[46]

S. Hong, M.-A. Panaitescu-Liess, Y. Kaya, and T. Dumitras, “Qu-anti-zation: Exploiting quantization artifacts for achieving adversarial outcomes,” in Proc. Adv. Neural Inf. Process. Syst., 2021, pp. 9303–9316.

[47]

X. Pan, M. Zhang, Y. Yan, and M. Yang, “Understanding the threats of trojaned quantized neural network in model supply chains,” in Proc. Annu. Comput. Secur. Appl. Conf., 2021, pp. 634–645.

[48]

H. Phan et al., “RIBAC: Towards robust and imperceptible backdoor attack against compact DNN,” in Proc. Eur. Conf. Comput. Vis., 2022, pp. 708–724.

[49]

X. Chen, C. Liu, B. Li, K. Lu, and D. Song, “Targeted backdoor attacks on deep learning systems using data poisoning,” 2017,.

[50]

Y. Liu et al., “Trojaning attack on neural networks,” in Proc. Netw. Distrib. Syst. Secur. Symp., 2018.

[51]

B. Jacob et al., “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 2704–2713.

[52]

S. Garg, A. Kumar, V. Goel, and Y. Liang, “Can adversarial weight perturbations inject neural backdoors,” in Proc. 29th ACM Int. Conf. Inf. Knowl. Manage., 2020, pp. 2029–2032.

[53]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.

[54]

A. Krizhevsky et al., “Learning multiple layers of features from tiny images,” 2009. [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.222.9220&rep=rep1&type=pdf

[55]

J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, “Man versus computer: Benchmarking machine learning algorithms for traffic sign recognition,” Neural Netw., vol. 32, pp. 323–332, 2012.

Digital Library

[56]

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770–778.

[57]

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014,.

[58]

D. Tang, X. Wang, H. Tang, and K. Zhang, “Demon in the variant: Statistical analysis of DNNs for robust backdoor contamination detection,” in Proc. 30th USENIX Secur. Symp., 2021. [Online]. Available: https://www.usenix.org/conference/usenixsecurity21/presentation/tang-di

[59]

W. Guo, L. Wang, X. Xing, M. Du, and D. Song, “Tabor: A highly accurate approach to inspecting and restoring trojan backdoors in AI systems,” 2019,.

[60]

Tensorflow-Lite, “Representative dataset,” 2021. [Online]. Available: https://tensorflow.google.cn/lite/api_docs/python/tf/lite/RepresentativeDataset

[61]

R. Shokri et al., “Bypassing backdoor detection algorithms in deep learning,” in Proc. IEEE Eur. Symp. Secur. Privacy, 2020, pp. 175–183.

[62]

H. Qiu et al., “Towards a critical evaluation of robustness for deep learning backdoor countermeasures,” 2022,.

[63]

K. Liu, B. Dolan-Gavitt, and S. Garg, “Fine-pruning: Defending against backdooring attacks on deep neural networks,” in Proc. Int. Symp. Res. Attacks Intrusions Defenses, 2018, pp. 273–294.

[64]

Y. Li, X. Lyu, N. Koren, L. Lyu, B. Li, and X. Ma, “Neural attention distillation: Erasing backdoor triggers from deep neural networks,” in Proc. Int. Conf. Learn. Representations, 2020. [Online]. Available: https://github.com/bboylyg/NAD

[65]

G. Tao et al., “Model orthogonalization: Class distance hardening in neural networks for better security,” in Proc. IEEE Symp. Secur. Privacy, 2022, pp. 1372–1389.

Cited By

Liu ZJiang YShen JPeng MLam KYuan XLiu X(2024)A Survey on Federated Unlearning: Challenges, Methods, and Future DirectionsACM Computing Surveys10.1145/367901457:1(1-38)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3679014
Dantas PSabino da Silva WCordeiro LCarvalho C(2024)A comprehensive review of model compression techniques in machine learningApplied Intelligence10.1007/s10489-024-05747-w54:22(11804-11844)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05747-w
Zhang YChen GSong FSun JDong J(2024)Certified Quantization Strategy Synthesis for Neural NetworksFormal Methods10.1007/978-3-031-71162-6_18(343-362)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-71162-6_18

Recommendations

Vulnerability-Based Backdoors: Threats from Two-step Trojans
SERE '13: Proceedings of the 2013 IEEE 7th International Conference on Software Security and Reliability

Attackers like to install trojans in a target system to control it. However, it becomes more and more difficult to deceive a user into installing such trojans. One reason is that antivirus software uses more strict policies on the first run of unknown ...
Handcrafted backdoors in deep neural networks
NIPS '22: Proceedings of the 36th International Conference on Neural Information Processing Systems

When machine learning training is outsourced to third parties, backdoor attacks become practical as the third party who trains the model may act maliciously to inject hidden behaviors into the otherwise accurate model. Until now, the mechanism to inject ...
Silencing Hardware Backdoors
SP '11: Proceedings of the 2011 IEEE Symposium on Security and Privacy

Hardware components can contain hidden backdoors, which can be enabled with catastrophic effects or for ill-gotten profit. These backdoors can be inserted by a malicious insider on the design team or a third-party IP provider. In this paper, we propose ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Dependable and Secure Computing

IEEE Transactions on Dependable and Secure Computing Volume 21, Issue 3

May-June 2024

500 pages

Issue’s Table of Contents

1545-5971 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 01 May 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu ZJiang YShen JPeng MLam KYuan XLiu X(2024)A Survey on Federated Unlearning: Challenges, Methods, and Future DirectionsACM Computing Surveys10.1145/367901457:1(1-38)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3679014
Dantas PSabino da Silva WCordeiro LCarvalho C(2024)A comprehensive review of model compression techniques in machine learningApplied Intelligence10.1007/s10489-024-05747-w54:22(11804-11844)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05747-w
Zhang YChen GSong FSun JDong J(2024)Certified Quantization Strategy Synthesis for Neural NetworksFormal Methods10.1007/978-3-031-71162-6_18(343-362)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-71162-6_18

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents