research-article

Open access

Quantum Neural Network Compression

Authors:

Weiwen JiangAuthors Info & Claims

ICCAD '22: Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design

Article No.: 140, Pages 1 - 9

https://doi.org/10.1145/3508352.3549382

Published: 22 December 2022 Publication History

Abstract

Model compression, such as pruning and quantization, has been widely applied to optimize neural networks on resource-limited classical devices. Recently, there are growing interest in variational quantum circuits (VQC), that is, a type of neural network on quantum computers (a.k.a., quantum neural networks). It is well known that the near-term quantum devices have high noise and limited resources (i.e., quantum bits, qubits); yet, how to compress quantum neural networks has not been thoroughly studied. One might think it is straightforward to apply the classical compression techniques to quantum scenarios. However, this paper reveals that there exist differences between the compression of quantum and classical neural networks. Based on our observations, we claim that the compilation/traspilation has to be involved in the compression process. On top of this, we propose the very first systematical framework, namely CompVQC, to compress quantum neural networks (QNNs). In CompVQC, the key component is a novel compression algorithm, which is based on the alternating direction method of multipliers (ADMM) approach. Experiments demonstrate the advantage of the CompVQC, reducing the circuit depth (almost over 2.5×) with a negligible accuracy drop (<1%), which outperforms other competitors. Another promising truth is our CompVQC can indeed promote the robustness of the QNN on the near-term noisy quantum devices.

References

[1]

MV Altaisky. 2001. Quantum neural network. arXiv preprint quant-ph/0107012 (2001).

[2]

Ren Ao, Zhang Tao, Wang Yuhao, Lin Sheng, Dong Peiyan, Chen Yen-kuang, Xie Yuan, and Wang Yanzhi. 2020. Darb: A density-adaptive regular-block pruning for deep neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 5495--5502.

[3]

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine learning 3, 1 (2011), 1--122.

[4]

Carlos Bravo-Prieto, Ryan LaRose, Marco Cerezo, Yigit Subasi, Lukasz Cincio, and Patrick Coles. 2020. Variational quantum linear solver: A hybrid algorithm for linear systems. Bulletin of the American Physical Society 65 (2020).

[5]

Lukas Burgholzer, Robert Wille, and Richard Kueng. 2022. Characteristics of reversible circuits for error detection. Array 14 (2022), 100165.

[6]

Sung-En Chang, Yanyu Li, Mengshu Sun, Runbin Shi, Hayden K-H So, Xuehai Qian, Yanzhi Wang, and Xue Lin. 2021. Mix and match: A novel fpga-centric deep neural network quantization framework. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 208--220.

[7]

Samuel Yen-Chi Chen, Chao-Han Huck Yang, Jun Qi, Pin-Yu Chen, Xiaoli Ma, and Hsi-Sheng Goan. 2020. Variational quantum circuits for deep reinforcement learning. IEEE Access 8 (2020), 141007--141024.

[8]

Hsin-Pai Cheng, Yuanjun Huang, Xuyang Guo, Feng Yan, Wei Wen, Hai Li, Yiran Chen, et al. 2018. Differentiable Fine-grained Quantization for Deep Neural Network Compression. In NIPS 2018 Workshop on Compact Deep Neural Networks with Industrial Applications (CDNNRIA).

[9]

Yuejie Chi, Yue M Lu, and Yuxin Chen. 2019. Nonconvex optimization meets low-rank matrix factorization: An overview. IEEE Transactions on Signal Processing 67, 20 (2019), 5239--5269.

Digital Library

[10]

James P Clemens, Shabnam Siddiqui, and Julio Gea-Banacloche. 2004. Quantum error correction against correlated noise. Physical Review A 69, 6 (2004), 062313.

[11]

Iris Cong, Soonwon Choi, and Mikhail D Lukin. 2019. Quantum convolutional neural networks. Nature Physics 15, 12 (2019), 1273--1278.

[12]

Christian L Degen, F Reinhard, and Paola Cappellaro. 2017. Quantum sensing. Reviews of modern physics 89, 3 (2017), 035002.

[13]

Peiyan Dong, Siyue Wang, Wei Niu, Chengming Zhang, Sheng Lin, Zhengang Li, Yifan Gong, Bin Ren, Xue Lin, and Dingwen Tao. 2020. Rtmobile: Beyond real-time mobile acceleration of rnns for speech recognition. In 2020 57th ACM/IEEE Design Automation Conference (DAC). IEEE, 1--6.

[14]

Motohiko Ezawa. 2021. Variational Quantum Support Vector Machine based on \Γ matrix expansion and Variational Universal-Quantum-State Generator. arXiv preprint arXiv:2101.07966 (2021).

[15]

Alexandr A Ezhov and Dan Ventura. 2000. Quantum neural networks. In Future directions for intelligent systems and information sciences. Springer, 213--235.

[16]

Cong Fang, Chris Junchi Li, Zhouchen Lin, and Tong Zhang. 2018. Spider: Near-optimal non-convex optimization via stochastic path-integrated differential estimator. Advances in Neural Information Processing Systems 31 (2018).

[17]

Michel Fortin and Roland Glowinski. 2000. Augmented Lagrangian methods: applications to the numerical solution of boundary-value problems. Elsevier.

[18]

Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. 2019. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4852--4861.

[19]

Zhezhi He and Deliang Fan. 2019. Simultaneously optimizing weight and quantizer of ternary neural network using truncated gaussian approximation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11438--11446.

[20]

Weiwen Jiang, Jinjun Xiong, and Yiyu Shi. 2021. A co-design framework of neural networks and quantum circuits towards quantum advantage. Nature communications 12, 1 (2021), 1--13.

[21]

Weiwen Jiang, Jinjun Xiong, and Yiyu Shi. 2021. When Machine Learning Meets Quantum Computers: A Case Study. In 2021 26th Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 593--598.

Digital Library

[22]

Weiwen Jiang, Lei Yang, Sakyasingha Dasgupta, Jingtong Hu, and Yiyu Shi. 2020. Standing on the shoulders of giants: Hardware and neural architecture co-search with hot start. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39, 11 (2020), 4154--4165.

[23]

Weiwen Jiang, Xinyi Zhang, Edwin H-M Sha, Lei Yang, Qingfeng Zhuge, Yiyu Shi, and Jingtong Hu. 2019. Accuracy vs. efficiency: Achieving both through fpga-implementation aware neural architecture search. In Proceedings of the 56th Annual Design Automation Conference 2019. 1--6.

Digital Library

[24]

Sangil Jung, Changyong Son, Seohyung Lee, Jinwoo Son, Jae-Joon Han, Youngjun Kwak, Sung Ju Hwang, and Changkyu Choi. 2019. Learning to quantize deep networks by optimizing quantization intervals with task loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4350--4359.

[25]

Sami Khairy, Ruslan Shaydulin, Lukasz Cincio, Yuri Alexeev, and Prasanna Balaprakash. 2020. Learning to optimize variational quantum circuits to solve combinatorial problems. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 2367--2375.

[26]

Ryan LaRose and Brian Coyle. 2020. Robust data encodings for quantum classifiers. Physical Review A 102, 3 (2020), 032420.

[27]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.

[28]

Zhiding Liang, Zhepeng Wang, Junhuan Yang, Lei Yang, Yiyu Shi, and Weiwen Jiang. 2021. Can Noise on Qubits Be Learned in Quantum Neural Network? A Case Study on QuantumFlow. In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 1--7.

Digital Library

[29]

Alexander I Lvovsky, Barry C Sanders, and Wolfgang Tittel. 2009. Optical quantum memory. Nature photonics 3, 12 (2009), 706--714.

[30]

Prasanna Ravi, Suman Deb, Anubhab Baksi, Anupam Chattopadhyay, Shivam Bhasin, and Avi Mendelson. 2021. On Threat of Hardware Trojan to Post-Quantum Lattice-Based Schemes: A Key Recovery Attack on SABER and Beyond. In International Conference on Security, Privacy, and Applied Cryptography Engineering. Springer, 81--103.

[31]

Runbin Shi, Peiyan Dong, Tong Geng, Yuhao Ding, Xiaolong Ma, Hayden K-H So, Martin Herbordt, Ang Li, and Yanzhi Wang. 2020. Csb-rnn: A faster-than-realtime rnn acceleration framework with compressed structured blocks. In Proceedings of the 34th ACM International Conference on Supercomputing. 1--12.

Digital Library

[32]

Arvind Srinivasan, Kamal Chaudhary, and Ernest S Kuh. 1992. RITUAL: A performance driven placement algorithm. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing 39, 11 (1992), 825--840.

[33]

Davide Venturelli, Minh Do, Bryan O'Gorman, Jeremy Frank, Eleanor Rieffel, Kyle EC Booth, Thanh Nguyen, Parvathi Narayan, and Sasha Nanda. 2019. Quantum circuit compilation: An emerging application for automated reasoning. (2019).

[34]

Hanrui Wang, Yongshan Ding, Jiaqi Gu, Zirui Li, Yujun Lin, David Z Pan, Frederic T Chong, and Song Han. 2021. Quantumnas: Noise-adaptive search for robust quantum circuits. arXiv preprint arXiv:2107.10845 (2021).

[35]

Zhepeng Wang, Zhiding Liang, Shanglin Zhou, Caiwen Ding, Yiyu Shi, and Weiwen Jiang. 2021. Exploration of Quantum Neural Architecture by Mixing Quantum Neuron Designs. In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 1--7.

Digital Library

[36]

Robert Wille and Rolf Drechsler. 2021. Introduction to the Special Issue on Design Automation for Quantum Computing., 2 pages.

[37]

Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017).

[38]

Shilu Yan, Hongsheng Qi, and Wei Cui. 2020. Nonlinear quantum neuron: A fundamental building block for quantum neural networks. Physical Review A 102, 5 (2020), 052421.

[39]

Tianyun Zhang, Shaokai Ye, Kaiqi Zhang, Jian Tang, Wujie Wen, Makan Fardad, and Yanzhi Wang. 2018. A systematic dnn weight pruning framework using alternating direction method of multipliers. In Proceedings of the European Conference on Computer Vision (ECCV). 184--199.

Digital Library

Cited By

Sistla MLiu YFu X(2024)Towards High Performance QNNs via Distribution-Based CNOT Gate ReductionACM Transactions on Architecture and Code Optimization10.1145/369587221:4(1-22)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3695872
Jiang WLin YDe V(2024)QuGeo: An End-to-end Quantum Learning Framework for Geoscience --- A Case Study on Full-Waveform InversionProceedings of the 61st ACM/IEEE Design Automation Conference10.1145/3649329.3657363(1-6)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3649329.3657363
Tan SXiang DLu LLu JJiang QChen MYin JTsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)MorphQPV: Exploiting Isomorphism in Quantum Programs to Facilitate Confident VerificationProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3620666.3651360(671-688)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620666.3651360
Show More Cited By

Index Terms

Quantum Neural Network Compression

Index terms have been assigned to the content through auto-classification.

Recommendations

Quantum artificial neural networks with applications

A new type of neural networks has been proposed, named a quantum artificial neural network (QANN).A QANN is presented as a system of interconnected "quantum neurons" simulated on quantum computers.The ability of approximation of a QANN has been proved ...
Training a quantum neural network
NIPS'03: Proceedings of the 16th International Conference on Neural Information Processing Systems

Most proposals for quantum neural networks have skipped over the problem of how to train the networks. The mechanics of quantum computing are different enough from classical computing that the issue of training should be treated in detail. We propose a ...
Recurrent neural network approach to quantum signal: coherent state restoration for continuous-variable quantum key distribution

In continuous-variable quantum key distribution (CV-QKD), weak signal carrying information transmits from Alice to Bob; during this process it is easily influenced by unknown noise which reduces signal-to-noise ratio, and strongly impacts reliability ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICCAD '22: Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design

October 2022

1467 pages

ISBN:9781450392174

DOI:10.1145/3508352

Conference Chair:
Tulika Mitra
National University of Singapore
,
Program Chairs:
Evangeline Young
The Chinese University of Hong Kong
,
Jinjun Xiong
University at Buffalo (UB)

Copyright © 2022 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

SIGDA: ACM Special Interest Group on Design Automation

In-Cooperation

IEEE-EDS: Electronic Devices Society
IEEE CAS
IEEE CEDA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 December 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

NSEC Quantum Sensing

Conference

ICCAD '22

Sponsor:

SIGDA

ICCAD '22: IEEE/ACM International Conference on Computer-Aided Design

October 30 - November 3, 2022

California, San Diego

Acceptance Rates

Overall Acceptance Rate 457 of 1,762 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
985
Total Downloads

Downloads (Last 12 months)561
Downloads (Last 6 weeks)66

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sistla MLiu YFu X(2024)Towards High Performance QNNs via Distribution-Based CNOT Gate ReductionACM Transactions on Architecture and Code Optimization10.1145/369587221:4(1-22)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3695872
Jiang WLin YDe V(2024)QuGeo: An End-to-end Quantum Learning Framework for Geoscience --- A Case Study on Full-Waveform InversionProceedings of the 61st ACM/IEEE Design Automation Conference10.1145/3649329.3657363(1-6)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3649329.3657363
Tan SXiang DLu LLu JJiang QChen MYin JTsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)MorphQPV: Exploiting Isomorphism in Quantum Programs to Facilitate Confident VerificationProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3620666.3651360(671-688)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620666.3651360
Park SBaek HYoon JLee YKim J(2024)AQUA: Analytics-driven quantum neural network (QNN) user assistance for software validationFuture Generation Computer Systems10.1016/j.future.2024.05.047159(545-556)Online publication date: Oct-2024
https://doi.org/10.1016/j.future.2024.05.047
Li JLiao JChen BNguyen ATiwari AZhou QYan ZNahrstedt KZimmermann RShirmohammadi SGriwodz CHefeeda MWang M(2023)Latency-Aware 360-Degree Video Analytics Framework for First Responders Situational AwarenessProceedings of the 33rd Workshop on Network and Operating System Support for Digital Audio and Video10.1145/3592473.3592568(8-14)Online publication date: 7-Jun-2023
https://dl.acm.org/doi/10.1145/3592473.3592568
Imamura SKasagi AYoshida E(2023)Offline Quantum Circuit Pruning for Quantum Chemical Calculations2023 IEEE International Conference on Quantum Computing and Engineering (QCE)10.1109/QCE57702.2023.00047(349-355)Online publication date: 17-Sep-2023
https://doi.org/10.1109/QCE57702.2023.00047
Liang ZSong ZCheng JHe ZLiu JWang HQin RWang YHan SQian XShi Y(2023)Hybrid Gate-Pulse Model for Variational Quantum Algorithms2023 60th ACM/IEEE Design Automation Conference (DAC)10.1109/DAC56929.2023.10247923(1-6)Online publication date: 9-Jul-2023
https://doi.org/10.1109/DAC56929.2023.10247923
Hu ZLin YGuan QJiang W(2023)Battle Against Fluctuating Quantum Noise: Compression-Aided Framework to Enable Robust Quantum Neural Network2023 60th ACM/IEEE Design Automation Conference (DAC)10.1109/DAC56929.2023.10247922(1-6)Online publication date: 9-Jul-2023
https://doi.org/10.1109/DAC56929.2023.10247922
Hu ZLi JPan ZZhou SYang LDing CKhan OGeng TJiang W(2022)On the Design of Quantum Graph Convolutional Neural Network in the NISQ-Era and Beyond2022 IEEE 40th International Conference on Computer Design (ICCD)10.1109/ICCD56317.2022.00050(290-297)Online publication date: Oct-2022
https://doi.org/10.1109/ICCD56317.2022.00050

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents