research-article

CryptoSPN: Expanding PPML beyond Neural Networks

Authors:

Alejandro Molina,

Christian Weinert,

Thomas Schneider,

Kristian KerstingAuthors Info & Claims

PPMLP'20: Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice

Pages 9 - 14

https://doi.org/10.1145/3411501.3419417

Published: 09 November 2020 Publication History

Abstract

The ubiquitous deployment of machine learning (ML) technologies has certainly improved many applications but also raised challenging privacy concerns, as sensitive client data is usually processed remotely at the discretion of a service provider. Therefore, privacy-preserving machine learning (PPML) aims at providing privacy using techniques such as secure multi-party computation (SMPC).

Recent years have seen a rapid influx of cryptographic frameworks that steadily improve performance as well as usability, pushing PPML towards practice. However, as it is mainly driven by the crypto community, the PPML toolkit so far is mostly restricted to well-known neural networks (NNs). Unfortunately, deep probabilistic models rising in the ML community that can deal with a wide range of probabilistic queries and offer tractability guarantees are severely underrepresented. Due to a lack of interdisciplinary collaboration, PPML is missing such important trends, ultimately hindering the adoption of privacy technology.

In this work, we introduce CryptoSPN, a framework for privacy-preserving inference of sum-product networks (SPNs) to significantly expand the PPML toolkit beyond NNs. SPNs are deep probabilistic models at the sweet-spot between expressivity and tractability, allowing for a range of exact queries in linear time. In an interdisciplinary effort, we combine techniques from both ML and crypto to allow for efficient, privacy-preserving SPN inference via SMPC.

We provide CryptoSPN as open source and seamlessly integrate it into the SPFlow library (Molina et al., arXiv 2019) for practical use by ML experts. Our evaluation on a broad range of SPNs demonstrates that CryptoSPN achieves highly efficient and accurate inference within seconds for medium-sized SPNs.

References

[1]

Masaud Y. Alhassan, Daniel Gü nther, Ágnes Kiss, and Thomas Schneider. 2020. Efficient and Scalable Universal Circuits. Journal of Cryptology (JoC).

[2]

Mohamed R. Amer and Sinisa Todorovic. 2016. Sum product networks for activity recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).

Digital Library

[3]

Galen Andrew, Steve Chien, and Nicolas Papernot. 2019. TensorFlow Privacy. https://github.com/tensorflow/privacy.

[4]

Gilad Asharov, Yehuda Lindell, Thomas Schneider, and Michael Zohner. 2017. More efficient oblivious transfer extensions. Journal of Cryptology.

Digital Library

[5]

Assi Barak, Daniel Escudero, Anders Dalskov, and Marcel Keller. 2019. Secure Evaluation of Quantized Neural Networks.

[6]

Mauro Barni, Pierluigi Failla, Riccardo Lazzeretti, Ahmad-Reza Sadeghi, and Thomas Schneider. 2011. Privacy-Preserving ECG Classification with Branching Programs and Neural Networks. IEEE Transactions on Information Forensics and Security (TIFS).

Digital Library

[7]

Fabian Boemer, Rosario Cammarota, Daniel Demmler, Thomas Schneider, and Hossein Yalame. 2020. MP2ML: A Mixed-Protocol Machine Learning Framework for Private Inference. In International Conference on Availability, Reliability and Security (ARES).

[8]

Fabian Boemer, Anamaria Costache, Rosario Cammarota, and Casimir Wierzynski. 2019. nGraph-HE2: A High-Throughput Framework for Neural Network Inference on Encrypted Data. In Workshop on Encrypted Computing & Applied Homomorphic Cryptography (WAHC).

Digital Library

[9]

Raphael Bost, Raluca Ada Popa, Stephen Tu, and Shafi Goldwasser. 2015. Machine Learning Classification over Encrypted Data. In Network and Distributed System Security Symposium (NDSS).

[10]

Arthur Choi and Adnan Darwiche. 2017. On Relaxing Determinism in Arithmetic Circuits. In International Conference on Machine Learning (ICML).

[11]

Scott Cyphers, Arjun K. Bansal, Anahita Bhiwandiwalla, Jayaram Bobba, Matthew Brookhart, Avijit Chakraborty, William Constable, Christian Convey, Leona Cook, Omar Kanawi, Robert Kimball, Jason Knight, Nikolay Korovaiko, Varun Kumar Vijay, Yixing Lao, Christopher R. Lishka, Jaikrishnan Menon, Jennifer Myers, Sandeep Aswath Narayana, Adam Procter, and Tristan J. Webb. 2018. Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning. arXiv preprint arXiv:1801.08058.

[12]

Morten Dahl, Jason Mancuso, Yann Dupis, Ben Decoste, Morgan Giraud, Ian Livingstone, Justin Patriquin, and Gavin Uhma. 2018. Private Machine Learning in TensorFlow using Secure Computation. arXiv preprint arXiv:1810.08130.

[13]

Daniel Demmler, Ghada Dessouky, Farinaz Koushanfar, Ahmad-Reza Sadeghi, Thomas Schneider, and Shaza Zeitouni. 2015a. Automated synthesis of optimized circuits for secure computation. In ACM Conference on Computer and Communications Security (CCS).

Digital Library

[14]

Daniel Demmler, Thomas Schneider, and Michael Zohner. 2015b. ABY-A Framework for Efficient Mixed-Protocol Secure Two-Party Computation. In Network and Distributed System Security Symposium (NDSS).

[15]

Robert Gens and Pedro M Domingos. 2013. Learning the Structure of Sum-Product Networks. In International Conference on Machine Learning (ICML).

[16]

Zoubin Ghahramani. 2015. Probabilistic machine learning and artificial intelligence. Nature.

[17]

Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin E Lauter, Michael Naehrig, and John Wernsing. 2016. CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy. In ICML.

[18]

Bryce Goodman and Seth Flaxman. 2017. European Union regulations on algorithmic decision-making and a 'right to explanation'. AI Magazine.

[19]

David Gunning, Awni Hannun, Mark Ibrahim, Brian Knott, Laurens van der Maaten, Vinicius Reis, Shubho Sengupta, Shobha Venkataraman, and Xing Zhou. 2019. CrypTen: A new research tool for secure machine learning with PyTorch. https://ai.facebook.com/blog/crypten-a-new-research-tool-for-secure-machine-learning-with-pytorch/.

[20]

Benjamin Hilprecht, Andreas Schmidt, Moritz Kulessa, Alejandro Molina, Kristian Kersting, and Carsten Binnig. 2020. DeepDB: Learn from Data, not from Queries!. In Proceedings of the VLDB Endowment (PVLDB).

Digital Library

[21]

Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. 2018. GAZELLE: A Low Latency Framework for Secure Neural Network Inference. In USENIX Security.

[22]

Ágnes Kiss, Masoud Naderpour, Jian Liu, N Asokan, and Thomas Schneider. 2019. SoK: Modular and efficient private decision tree evaluation. Privacy Enhancing Technologies (PETs).

[23]

Vladimir Kolesnikov and Thomas Schneider. 2008. A practical universal circuit construction and secure evaluation of private functions. In Financial Cryptography and Data Security (FC).

[24]

Daphne Koller and Nir Friedman. 2009. Probabilistic Graphical Models: Principles and Techniques. MIT Press.

[25]

Nishant Kumar, Mayank Rathee, Nishanth Chandran, Divya Gupta, Aseem Rastogi, and Rahul Sharma. 2020. CrypTFlow: Secure tensorflow inference. In IEEE Symposium on Security and Privacy (S&P).

[26]

Yehuda Lindell and Benny Pinkas. 2012. Secure two-party computation via cut-and-choose oblivious transfer. Journal of Cryptology (JoC).

Digital Library

[27]

Jian Liu, Mika Juuti, Yao Lu, and N Asokan. 2017. Oblivious neural network predictions via MiniONN transformations. In ACM Conference on Computer & Communications Security (CCS).

Digital Library

[28]

Dahlia Malkhi, Noam Nisan, Benny Pinkas, and Yaron Sella. 2004. Fairplay - A Secure Two-Party Computation System. In USENIX Security.

[29]

Pratyush Mishra, Ryan Lehmkuhl, Akshayaram Srinivasan, Wenting Zheng, and Raluca Ada Popa. 2020. DELPHI: A Cryptographic Inference Service for Neural Networks. In USENIX Security.

[30]

Payman Mohassel and Peter Rindal. 2018. ABY3: A mixed protocol framework for machine learning. In ACM SIGSAC Conference on Computer and Communications Security (CCS).

[31]

Payman Mohassel and Yupeng Zhang. 2017. SecureML: A system for scalable privacy-preserving machine learning. In IEEE Symposium on Security and Privacy (S&P).

[32]

Alejandro Molina, Sriraam Natarajan, and Kristian Kersting. 2017. Poisson Sum-Product Networks: A Deep Architecture for Tractable Multivariate Poisson Distributions. In AAAI Conference on Artificial Intelligence (AAAI).

[33]

Alejandro Molina, Antonio Vergari, Karl Stelzner, Robert Peharz, Pranav Subramani, Nicola Di Mauro, Pascal Poupart, and Kristian Kersting. 2019. SPFlow: An Easy and Extensible Library for Deep Probabilistic Learning using Sum-Product Networks. arXiv preprint arXiv:1901.03704.

[34]

Claudio Orlandi, Alessandro Piva, and Mauro Barni. 2007. Oblivious neural network computing via homomorphic encryption. EURASIP Journal on Information Security.

[35]

Robert Peharz, Georg Kapeller, Pejman Mowlaee, and Franz Pernkopf. 2014. Modeling speech with sum-product networks: Application to bandwidth extension. In International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]

Robert Peharz, Antonio Vergari, Karl Stelzner, Alejandro Molina, Xiaoting Shao, Martin Trapp, Kristian Kersting, and Zoubin Ghahramani. 2019. Random Sum-Product Networks: A Simple and Effective Approach to Probabilistic Deep Learning. In Conference on Uncertainty in Artificial Intelligence (UAI).

[37]

Hoifung Poon and Pedro M Domingos. 2011. Sum-Product Networks: A New Deep Architecture. In Conference on Uncertainty in Artificial Intelligence (UAI).

[38]

M Sadegh Riazi, Mohammad Samragh, Hao Chen, Kim Laine, Kristin Lauter, and Farinaz Koushanfar. 2019. XONN: XNOR-based Oblivious Deep Neural Network Inference. In USENIX Security.

[39]

M Sadegh Riazi, Christian Weinert, Oleksandr Tkachenko, Ebrahim M Songhori, Thomas Schneider, and Farinaz Koushanfar. 2018. Chameleon: A hybrid secure computation framework for machine learning applications. In ACM ASIA Conference on Computer and Communications Security (ASIACCS).

Digital Library

[40]

Theo Ryffel, Andrew Trask, Morten Dahl, Bobby Wagner, Jason Mancuso, Daniel Rueckert, and Jonathan Passerat-Palmbach. 2018. A generic framework for privacy preserving deep learning. arXiv preprint arXiv:1811.04017.

[41]

Ahmad-Reza Sadeghi and Thomas Schneider. 2008. Generalized Universal Circuits for Secure Evaluation of Private Functions with Application to Data Classification. In International Conference on Information Security and Cryptology (ICISC).

[42]

Amos Treiber, Alejandro Molina, Christian Weinert, Thomas Schneider, and Kristian Kersting. 2020. CryptoSPN: Privacy-preserving Sum-Product Network Inference. In European Conference on Artificial Intelligence (ECAI).

Digital Library

[43]

Leslie G Valiant. 1976. Universal circuits (preliminary report). In STOC.

[44]

Tim van Elsloo, Giorgio Patrini, and Hamish Ivey-Law. 2019. SEALion: A Framework for Neural Network Inference on Encrypted Data. arXiv preprint arXiv:1904.12840.

[45]

Andrew Yao. 1986. How to generate and exchange secrets. In FOCS.

[46]

Zehuan Yuan, Hao Wang, Limin Wang, Tong Lu, Shivakumara Palaiahnakote, and Chew Lim Tan. 2016. Modeling spatial layout for scene image understanding via a novel multiscale sum-product network. Expert Systems with Applications.

Cited By

Heilmann XCerrato MAlthaus ESalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Differentially private sum-product networksProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692799(18155-18173)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692799
Gehrke MLiebenow JMohammadi EBraun T(2024)Lifting in Support of Privacy-Preserving Probabilistic InferenceKI - Künstliche Intelligenz10.1007/s13218-024-00851-yOnline publication date: 13-Jun-2024
https://doi.org/10.1007/s13218-024-00851-y
Li BQi PLiu BDi SLiu JPei JYi JZhou B(2023)Trustworthy AI: From Principles to PracticesACM Computing Surveys10.1145/355580355:9(1-46)Online publication date: 16-Jan-2023
https://doi.org/10.1145/3555803
Show More Cited By

Index Terms

CryptoSPN: Expanding PPML beyond Neural Networks
1. Security and privacy
  1. Human and societal aspects of security and privacy
    1. Privacy protections
    2. Usability in security and privacy
  2. Security services
    1. Privacy-preserving protocols

Recommendations

On Distributed k-Anonymization

When a database owner needs to disclose her data, she can k-anonymize her data to protect the involved individuals' privacy. However, if the data is distributed between two owners, then it is an open question whether the two owners can jointly k-...
On Distributed k-Anonymization

When a database owner needs to disclose her data, she can k-anonymize her data to protect the involved individuals' privacy. However, if the data is distributed between two owners, then it is an open question whether the two owners can jointly k-...
Beyond tree-shaped credal probabilistic circuits
Abstract
Probabilistic circuits are a class of probabilistic generative models that allow us to compute different types of probabilistic queries in polynomial time. Unlike many of the mainstream approaches for generative modeling, they can compute exact ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

PPMLP'20: Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice

November 2020

75 pages

ISBN:9781450380881

DOI:10.1145/3411501

Program Chairs:
Benyu Zhang
Ant Group, USA
,
Raluca Ada Popa
UC Berkeley, USA
,
Matei Zaharia
Stanford University, USA
,
Guofei Gu
Texas A&M University, USA
,
Shouling Ji
Zhejiang University, China

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Research Council
Hessisches Ministerium für Bildung und Kunst
Deutsche Forschungsgemeinschaft
Bundesministerium für Bildung und Forschung

Conference

CCS '20

Sponsor:

SIGSAC

CCS '20: 2020 ACM SIGSAC Conference on Computer and Communications Security

November 9, 2020

Virtual Event, USA

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
161
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Heilmann XCerrato MAlthaus ESalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Differentially private sum-product networksProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692799(18155-18173)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692799
Gehrke MLiebenow JMohammadi EBraun T(2024)Lifting in Support of Privacy-Preserving Probabilistic InferenceKI - Künstliche Intelligenz10.1007/s13218-024-00851-yOnline publication date: 13-Jun-2024
https://doi.org/10.1007/s13218-024-00851-y
Li BQi PLiu BDi SLiu JPei JYi JZhou B(2023)Trustworthy AI: From Principles to PracticesACM Computing Surveys10.1145/355580355:9(1-46)Online publication date: 16-Jan-2023
https://doi.org/10.1145/3555803
Belle V(2023)Knowledge representation and acquisition for ethical AI: challenges and opportunitiesEthics and Information Technology10.1007/s10676-023-09692-z25:1Online publication date: 11-Mar-2023
https://doi.org/10.1007/s10676-023-09692-z
Tiwari KShukla SGeorge J(2021)A Systematic Review of Challenges and Techniques of Privacy-Preserving Machine LearningData Science and Security10.1007/978-981-16-4486-3_3(19-41)Online publication date: 27-Aug-2021
https://doi.org/10.1007/978-981-16-4486-3_3
Treiber AMolina AWeinert CSchneider TKersting KZhang BAda Popa RZaharia MGu GJi S(2020)CryptoSPNProceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice10.1145/3411501.3419417(9-14)Online publication date: 9-Nov-2020
https://dl.acm.org/doi/10.1145/3411501.3419417
Boemer FCammarota RDemmler DSchneider TYalame HVolkamer MWressnegger C(2020)MP2MLProceedings of the 15th International Conference on Availability, Reliability and Security10.1145/3407023.3407045(1-10)Online publication date: 25-Aug-2020
https://dl.acm.org/doi/10.1145/3407023.3407045

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten