research-article

Open access

Detection of Adversarial Attacks by Observing Deep Features with Structured Data Algorithms

Authors:

Tommaso Puccetti,

Andrea Ceccarelli,

Andrea BondavalliAuthors Info & Claims

SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing

Pages 125 - 134

https://doi.org/10.1145/3555776.3577629

Published: 07 June 2023 Publication History

Abstract

Deep Neural Networks (DNNs) are highly vulnerable to adversarial attacks, which introduce human-imperceptible perturbations on the input to fool a DNN model. Detecting such attacks is fundamental to protect distributed applications that process input data using DNNs. Detection strategies typically rely on complex solutions that include modifications to the input and to the DNN model itself, and/or the deployment of a second DNN that suspects attacks. Despite these efforts, at the present stage of research, the ability to protect against adversarial attacks is unsatisfactory. Alternatively to most approaches, this paper proposes RISOTTO (adveRsarIal attackS detectiOn using strucTured daTa algOrithms), a very simple but effective and fast detection strategy that uses algorithms for structured data. RISOTTO does not modify the DNN model and its inputs and requires only the values of selected deep features at test time. Using the deep features of a single layer, the accuracy in detecting known attacks is 1.0 for the two MNIST models and all the selected attacks. Also, by combining the deep features of multiple layers, we show that our approach is competitive or better than state-of-the-art unsupervised solutions in detecting unknown attacks, especially for MNIST models with few deep features (below 1 million).

References

[1]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. "Gradient-based learning applied to document recognition." Proceedings of the IEEE, 86(11):2278--2324, 1998.

[2]

M. I. Nicolae, et al., "Adversarial Robustness Toolbox v1.0.0," arXiv preprint arXiv:1807.01069v4, 2019

[3]

D. Warde-Farley, and I. Goodfellow, "Adversarial perturbations of deep neural networks," Perturbation, Optimization, and Statistics (editors: T. Hazan, G. Papandreou, D. Tarlow), 2016.

[4]

M. Xue, et al., "Machine learning security: Threats, countermeasures, and evaluations," IEEE Access 8: 74720--74742, 2020.

[5]

A. Kurakin, I. Goodfellow, and S. Bengio, "Adversarial examples in the physical world," In: Artificial intelligence safety and security. Chapman and Hall/CRC, 2018.

[6]

Y. Deng, et al., "An analysis of adversarial attacks and defenses on autonomous driving models," IEEE Int. Conf. on Pervasive Computing and Communications (PerCom), 2020.

[7]

M. Usama, et al., "Unsupervised machine learning for networking: Techniques, applications and research challenges," IEEE Access 7 (2019): 65579--65615.

[8]

S. Bulusu et al., "Anomalous instance detection in deep learning a survey, " In: IEEE Symposium on Security and Privacy, 2020.

[9]

S. Van Der Walt, C. Colbert, and G. Varoquaux, "The NumPy array: a structure for efficient numerical computation," Computing in science & engineering 13.2 (2011): 22--30.

[10]

I. Goodfellow, J. Shlens, and C. Szegedy, "Explaining and harnessing adversarial examples," arXiv preprint arXiv:1412.6572, 2014.

[11]

S.M. Moosavi-Dezfooli, A. Fawzi, and P.Frossard, "Deepfool: a simple and accurate method to fool deep neural networks," Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2574--2582, 2016.

[12]

N. Carlini, and D. Wagner, "Towards evaluating the robustness of neural networks," IEEE symposium on security and privacy (SP), pp. 39--57, IEEE, 2017.

[13]

W. Xu, D. Evans, and Y. Qi, "Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks," Network and Distributed System Security Symposium, 2018.

[14]

Machine Learning Group and Security Research Group University of Virginia, "EvadeML-Zoo", https://github.com/mzweilin/EvadeML-Zoo [last accessed on May 2022].

[15]

G. Huang, et al., "Densely connected convolutional networks," Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.

[16]

D. J. Miller, Z. Xiang, and G. Kesidis, "Adversarial learning targeting deep neural network classification: A comprehensive review of defenses against attacks," Proceedings of the IEEE 108.3 (2020): 402--433.

[17]

ART documentation v1.10.1, https://adversarial-robustness-toolbox.readthedocs.io/en/latest/ [last accessed on May 2022].

[18]

D. Meng, and H. Chen, "Magnet: a two-pronged defense against adversarial examples," Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. 2017.

[19]

K. Grosse, P. Manoharan, N. Papernot, M. Backes, and P. McDaniel, "On the (statistical) detection of adversarial examples," arXiv preprint arXiv:1702.06280, 2017.

[20]

M. Abadi, et al., "TensorFlow: A System for Large-Scale Machine Learning," 12th USENIX symposium on operating systems design and implementation (OSDI 16), 2016.

[21]

Y. Kalantidis, C. Mellina, and S. Osindero, "Cross-dimensional weighting for aggregated deep convolutional features," European conference on computer vision, Springer, Cham, 2016.

[22]

N. Papernot, et al. "Distillation as a defense to adversarial perturbations against deep neural networks," 2016 IEEE symposium on security and privacy (SP). IEEE, 2016.

[23]

N. Papernot, and P. McDaniel, "Extending defensive distillation," arXiv preprint arXiv:1705.05264 (2017).

[24]

V. Chandola, A. Banerjee, and V. Kumar, "Anomaly detection: A survey," ACM Comput. Surv. 41, 3, Article 15 (July 2009), 58 pages.

Digital Library

[25]

P. Vincent, H. Larochelle, Y. Bengio, and P-A Manzagol, "Extracting and composing robust features with denoising autoencoders," In Proceedings of the 25^th international conference on Machine learning, pp. 1096--1103, 2008.

[26]

Liao, Yihua, and V. Rao Vemuri. "Use of k-nearest neighbor classifier for intrusion detection." Computers & security 21.5 (2002): 439--448.

Digital Library

[27]

J. H. Metzen, T. Genewein, V. Fischer, and B. Bischoff, "On detecting adversarial perturbations," arXiv preprint arXiv:1702.04267, 2017.

[28]

F. Carrara, R. Becarelli, R. Caldelli, F. Falchi, and G. Amato, "Adversarial examples detection in features distance spaces," in Proceedings of the European Conference on Computer Vision (ECCV), 2018.

[29]

M. Amirian, F. Schwenker, and T. Stadelmann, "Trace and detect adversarial attacks on cnns using feature response maps," In: 8th IAPR TC3 Workshop on Artificial Neural Networks in Pattern Recognition (ANNPR), Siena, Italy, 2018.

[30]

T. Chen, and C. Guestrin, "Xgboost: A scalable tree boosting system," Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 2016.

Digital Library

[31]

XGBoost library, https://xgboost.readthedocs.io/en/stable/ [last accessed on May 2022]

[32]

N. Papernot, et al. "Technical report on the cleverhans v2. 1.0 adversarial examples library." arXiv preprint arXiv:1610.00768, 2016.

[33]

S. Boughorbel, F. Jarray, and M. El-Anbari, "Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric," PloS one 12.6 (2017): e0177678.

[34]

A. Krizhevsky, and G. Hinton, "Learning multiple layers of features from tiny images," Technical Report, 2009.

[35]

F. Tramer, et al., "On adaptive attacks to adversarial example defenses," Advances in Neural Information Processing Systems 33 (2020): 1633--1645.

[36]

O. Gungora, T. Rosinga, and B. Aksanlib, "STEWART: STacking Ensemble for White-Box AdversaRial Attacks Towards More Resilient Data-driven Predictive Maintenance," Computers in Industry, vol. 140, 2022.

[37]

M. Löwe, et al., "Dealing with Adversarial Player Strategies in the Neural Network Game iNNk through Ensemble Learning, " The 16th International Conference on the Foundations of Digital Games (FDG), 2021.

[38]

A. Mirzaeian, et al., "Diverse Knowledge Distillation (DKD): A Solution for Improving The Robustness of Ensemble Models Against Adversarial Attacks." 22nd International Symposium on Quality Electronic Design (ISQED), 2021.

[39]

T. Pang, et al., "Improving adversarial robustness via promoting ensemble diversity," International Conference on Machine Learning (PMLR), 2019.

[40]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Int. Conf. Neural Inf. Process. Syst., pp. 1097--1105, 2012.

Digital Library

[41]

M. Miškuf, and I. Zolotová, "Comparison between multi-class classifiers and deep learning with focus on industry 4.0," in Proc. Cybern. Informat. (KI), Feb. 2016.

[42]

N. O'Mahony, et al., "Deep learning vs. traditional computer vision," in Proc. Sci. Inf. Conf. Cham, Switzerland: Springer, pp. 128--144, 2019.

[43]

L. M. Prevedello, et al., "Challenges related to artificial intelligence research in medical imaging and the importance of image analysis competitions, Radiol., Artif. Intell., vol. 1, no. 1, 2019.

[44]

A. E. L. Sallab, M. Abdou, E. Perot, and S. Yogamani, "Deep reinforcement learning framework for autonomous driving," Electron. Imag., vol. 19, pp. 70--76, Jan. 2017.

[45]

M. Alloghani, et al., "A systematic review on supervised and unsupervised machine learning algorithms for data science," Supervised and unsupervised learning for data science (2020): 3--21.

[46]

R. Shwartz-Ziv, and A. Armon, "Tabular data: Deep learning is not all you need," Information Fusion, 81, 84--90 (2022).

Digital Library

[47]

Q. V. L, et al., "On optimization methods for deep learning," ICML, 2011.

[48]

O. Al-Jarrah, et al. "Efficient machine learning for big data: A review," Big Data Research 2.3: 87--93 (2015).

Digital Library

[49]

Y. Izza, A. Ignatiev, and J. Marques-Silva. "On explaining decision trees." arXiv preprint arXiv:2010.11034 (2020).

[50]

O. Bryniarski, et al., "Evading adversarial example detection defenses with orthogonal projected gradient descent," arXiv preprint arXiv:2106.15023 (2021).

[51]

S. Shan, et al., "Gotta catch'em all: Using honeypots to catch adversarial attacks on neural networks," Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, 2020.

[52]

N. Das, H. Park, Z. J. Wang, F. Hohman, R. Firstman, E. Rogers, and D. H. Chau, "Bluff: Interactively deciphering adversarial attacks on deep neural networks," IEEE Visualization Conference (VIS), 2020.

[53]

S. Carter, Z. Armstrong, L. Schubert, I. Johnson, and C. Olah, "Activation atlas," Distill, vol. 4, no. 3, p. e15, 2019.

[54]

RISOTTO code and tutorials, https://github.com/TommasoPuccetti/Risotto [last accessed on December 2022]

[55]

RISOTTO repository, https://drive.google.com/drive/folders/1JsV45ooRlk5CpqFCPy-uR4iBB3Nbqx08 (235 GB) [last accessed on December 2022].

[56]

P. Brazdil, C. Giraud-Carrier, C. Soares, R. Vilalta, "Metalearning: applications to data mining," Springer Science & Business Media (2008).

Cited By

Puccetti T(2023)Early Detection of Unknown Attacks with Algorithms for Structured Data2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW)10.1109/ISSREW60843.2023.00033(5-8)Online publication date: 9-Oct-2023
https://doi.org/10.1109/ISSREW60843.2023.00033

Index Terms

Detection of Adversarial Attacks by Observing Deep Features with Structured Data Algorithms
1. Computer systems organization
  1. Architectures
    1. Other architectures
      1. Neural networks
2. Security and privacy
  1. Systems security
    1. Distributed systems security

Recommendations

Adversarial and Data Poisoning Attacks Against Deep Learning
Vulnerability of Deep Forest to Adversarial Attacks
Machine learning classifiers are vulnerable to adversarial examples, which are carefully crafted inputs designed to compromise their classification performance. Recently, a new machine learning classifier was proposed that is composed of forests of ...
Black-box adversarial attacks on XSS attack detection model
Abstract
Cross-site scripting (XSS) has been extensively studied, although mitigating such attacks in web applications remains challenging. While there is an increasing number of XSS attack detection approaches designed based on machine learning and deep ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing

March 2023

1932 pages

ISBN:9781450395175

DOI:10.1145/3555776

Conference Chairs:
Jiman Hong
Soongsil University, South Korea
,
Maart Lanperne
Tallinn University, Estonia
,
Program Chairs:
Juw Won Park
University of Louisville, USA
,
Tomas Cerny
Baylor University, USA
,
Publication Chair:
Hossain Shahriar
Kennesaw State University, USA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 June 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

European Commission

Conference

SAC '23

Sponsor:

SIGAPP

SAC '23: 38th ACM/SIGAPP Symposium on Applied Computing

March 27 - 31, 2023

Tallinn, Estonia

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
194
Total Downloads

Downloads (Last 12 months)149
Downloads (Last 6 weeks)15

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Puccetti T(2023)Early Detection of Unknown Attacks with Algorithms for Structured Data2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW)10.1109/ISSREW60843.2023.00033(5-8)Online publication date: 9-Oct-2023
https://doi.org/10.1109/ISSREW60843.2023.00033

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents