Trident: A Universal Framework for Fine-Grained and Class-Incremental Unknown Traffic Detection
Pages 1608 - 1619
Abstract
To detect unknown attack traffic, anomaly-based network intrusion detection systems (NIDSs) are widely used in Internet infrastructure. However, the security communities realize some limitations when they put most existing proposals into practice. The challenges are mainly concerned with (i) fine-grained emerging attack detection and (ii) incremental updates/adaptations. To tackle these problems, we propose to decouple the need for model capabilities by transforming known/new class identification issues into multiple independent one-class learning tasks. Based on the above core ideas, we develop Trident, a universal framework for fine-grained unknown encrypted traffic detection. It consists of three main modules, i.e., tSieve, tScissors, and tMagnifier are used for profiling traffic, determining outlier thresholds, and clustering respectively, each of which supports custom configuration. Using four popular datasets of network traces, we show that Trident significantly outperforms 16 state-of-the-art (SOTA) methods. Furthermore, a series of experiments (concept drift, overhead/parameter evaluation) demonstrate the stability, scalability, and practicality of Trident.
Supplemental Material
MP4 File
presentation video
- Download
- 1216.69 MB
MP4 File
Supplemental video
- Download
- 30.08 MB
References
[1]
August A. Balkema and Laurens De Haan. Residual life time at great age. The Annals of probability, pages 792--804, 1974.
[2]
Federico Barbero, Feargus Pendlebury, Fabio Pierazzi, et al. Transcending TRANSCEND: revisiting malware classification in the presence of concept drift. In IEEE Symposium on Security and Privacy, pages 805--823. IEEE, 2022.
[3]
Diogo Barradas et al. Flowlens: Enabling efficient flow classification for ml-based network security applications. In NDSS. The Internet Society, 2021.
[4]
Junyoung Chung, Çaglar Gülçehre, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555, 2014.
[5]
Min Du, Zhi Chen, Chang Liu, Rajvardhan Oak, and Dawn Song. Lifelong anomaly detection through unlearning. In CCS, pages 1283--1297. ACM, 2019.
[6]
Min Du, Feifei Li, et al. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In CCS, pages 1285--1298. ACM, 2017.
[7]
Martin Ester et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD, pages 226--231. AAAI Press, 1996.
[8]
FireEye. M-trends reports: Insights into today's breaches and cyber attacks. https://content.fireeye.com/m-trends/rpt-m-trends-2020, 2020.
[9]
Canadian Institute for Cybersecurity. Cse-cic-ids2018 on aws. [EB/OL], 2018. https://www.unb.ca/cic/datasets/ids-2018.html Accessed November 27, 2020.
[10]
Canadian Institute for Cybersecurity. Intrusion detection evaluation dataset (cicids2017). [EB/OL], 2018. https://www.unb.ca/cic/datasets/ids-2017.html Accessed November 27, 2020.
[11]
Chuanpu Fu et al. Detecting unknown encrypted malicious traffic in real time via flow interaction graph analysis. In NDSS. The Internet Society, 2023.
[12]
Chuanpu Fu, Qi Li, Meng Shen, and Ke Xu. Realtime robust malicious traffic detection via frequency domain analysis. In CCS, pages 3431--3446. ACM, 2021.
[13]
Hongyang Gao and Shuiwang Ji. Graph u-nets. In ICML, volume 97 of Proceedings of Machine Learning Research, pages 2083--2092. PMLR, 2019.
[14]
Scott D Grimshaw. Computing maximum likelihood estimates for the generalized pareto distribution. Technometrics, pages 185--191, 1993.
[15]
Dongqi Han et al. Anomaly detection in the openworld: Normality shift detection, explanation, and adaptation. In NDSS. The Internet Society, 2023.
[16]
Dongqi Han, Zhiliang Wang, et al. DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications. In CCS, pages 3197--3217. ACM, 2021.
[17]
John A Hartigan and Manchek AWong. A k-means clustering algorithm. Journal of the royal statistical society. series c (applied statistics), 28(1):100--108, 1979.
[18]
Jordan Holland, Paul Schmitt, Nick Feamster, and Prateek Mittal. New directions in automated traffic analysis. In CCS, pages 3366--3383. ACM, 2021.
[19]
Guodong Huang et al. Efficient and low overhead website fingerprinting attacks and defenses based on TCP/IP traffic. In WWW, pages 1991--1999. ACM, 2023.
[20]
James Pickands III. Statistical inference using extreme order statistics. The Annals of Statistics, pages 119--131, 1975.
[21]
Peipei Jiang et al. Building in-the-cloud network functions: Security and privacy challenges. Proc. IEEE, 109(12):1888--1919, 2021.
[22]
Roberto Jordaney, Kumar Sharad, et al. Transcend: Detecting concept drift in malware classification models. In USENIX Security Symposium, pages 625--642. USENIX Association, 2017.
[23]
Maciej Korczynski and Andrzej Duda. Markov chain fingerprinting to classify encrypted traffic. In INFOCOM, pages 781--789. IEEE, 2014.
[24]
Arash Habibi Lashkari, Gerard Draper-Gil, Mohammad Saiful Islam Mamun, and Ali A. Ghorbani. Characterization of tor traffic using time based features. In ICISSP, pages 253--262. SciTePress, 2017.
[25]
Nicole A. Lazar. Statistics of extremes: Theory and applications. Technometrics, 47(3):376--377, 2005.
[26]
Hongda Li, Hongxin Hu, Guofei Gu, Gail-Joon Ahn, and Fuqiang Zhang. vnids: Towards elastic security with safe and efficient virtualization of network intrusion detection systems. In CCS, pages 17--34. ACM, 2018.
[27]
Wenhao Li, Xiao-Yu Zhang, et al. Prograph: Robust network traffic identification with graph propagation. IEEE/ACM Trans. Netw., pages 1--15, 2022.
[28]
Junjie Liang et al. FARE: enabling fine-grained attack categorization under low-quality labeled data. In NDSS. The Internet Society, 2021.
[29]
Xinjie Lin, Gang Xiong, Gaopeng Gou, Zhen Li, Junzheng Shi, and Jing Yu. ETBERT: A contextualized datagram representation with pre-training transformers for encrypted traffic classification. In WWW, pages 633--642. ACM, 2022.
[30]
Chang Liu et al. Fs-net: A flow sequence network for encrypted traffic classification. In INFOCOM, pages 1171--1179. IEEE, 2019.
[31]
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In ICDM, pages 413--422. IEEE Computer Society, 2008.
[32]
Zaoxing Liu et al. Jaqen: A high-performance switch-native approach for detecting and mitigating volumetric ddos attacks with programmable switches. In USENIX Security Symposium, pages 3829--3846. USENIX Association, 2021.
[33]
Pierre-Francois Marteau. Random partitioning forest for point-wise and collective anomaly detection - application to network intrusion detection. IEEE Trans. Inf. Forensics Secur., 16:2157--2172, 2021.
[34]
Dongyu Meng and Hao Chen. Magnet: A two-pronged defense against adversarial examples. In CCS, pages 135--147. ACM, 2017.
[35]
Yisroel Mirsky et al. Kitsune: An ensemble of autoencoders for online network intrusion detection. In NDSS. The Internet Society, 2018.
[36]
Xin Mu, Kai Ming Ting, and Zhi-Hua Zhou. Classification under streaming emerging new classes: A solution using completely-random trees. IEEE Trans. Knowl. Data Eng., 29(8):1605--1618, 2017.
[37]
Sanghak Oh et al. Appsniffer: Towards robust mobile app fingerprinting against VPN. In WWW, pages 2318--2328. ACM, 2023.
[38]
Jorge Luis Rivero Pérez et al. A grassmannian approach to zero-shot learning for network intrusion detection. In ICONIP (1), volume 10634 of Lecture Notes in Computer Science, pages 565--575. Springer, 2017.
[39]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI (3), volume 9351 of Lecture Notes in Computer Science, pages 234--241. Springer, 2015.
[40]
Christian Rossow. Amplification hell: Revisiting network protocols for ddos abuse. In NDSS. The Internet Society, 2014.
[41]
Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53--65, 1987.
[42]
Tal Shapira et al. Flowpic: Encrypted internet traffic classification is as easy as image recognition. In INFOCOM Workshops, pages 680--687. IEEE, 2019.
[43]
Alban Siffer, Pierre-Alain Fouque, et al. Anomaly detection in streams with extreme value theory. In KDD, pages 1067--1075. ACM, 2017.
[44]
Robin Sommer and Vern Paxson. Outside the closed world: On using machine learning for network intrusion detection. In IEEE Symposium on Security and Privacy, pages 305--316. IEEE Computer Society, 2010.
[45]
Zhuoxue Song, Ziming Zhao, et al. I2RNN: An Incremental and Interpretable Recurrent Neural Network for Encrypted Traffic Classification. IEEE Transactions on Dependable and Secure Computing, 2023.
[46]
Alexander Strehl and Joydeep Ghosh. Cluster ensembles - A knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res., 3:583--617, 2002.
[47]
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104--3112, 2014.
[48]
Thijs van Ede et al. FlowPrint: Semi-Supervised Mobile-App Fingerprinting on Encrypted Network Traffic. In NDSS. The Internet Society, 2020.
[49]
WeiWang et al. Malware traffic classification using convolutional neural network for representation learning. In ICOIN, pages 712--717. IEEE, 2017.
[50]
Zhongjie Wang et al. Symtcp: Eluding stateful deep packet inspection with automated discrepancy discovery. In NDSS. The Internet Society, 2020.
[51]
Zhongjie Wang et al. Themis: Ambiguity-aware network intrusion detection based on symbolic model comparison. In CCS, pages 3384--3399. ACM, 2021.
[52]
Junyuan Xie, Ross B. Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In ICML, volume 48 of JMLR Workshop and Conference Proceedings, pages 478--487. JMLR.org, 2016.
[53]
Fengli Xu et al. Understanding Mobile Traffic Patterns of Large Scale Cellular Towers in Urban Environment. IEEE/ACM Trans. Netw., 25(2):1147--1161, 2017.
[54]
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In NDSS. The Internet Society, 2018.
[55]
Jian Yang et al. Conditional variational auto-encoder and extreme value theory aided two-stage learning approach for intelligent fine-grained known/unknown intrusion detection. IEEE Trans. Inf. Forensics Secur., 16:3538--3553, 2021.
[56]
Limin Yang, Wenbo Guo, et al. CADE: detecting and explaining concept drift samples for security applications. In USENIX Security Symposium, pages 2327-- 2344. USENIX Association, 2021.
[57]
Yijun Yang, Ruiyuan Gao, Yu Li, Qiuxia Lai, and Qiang Xu. What you see is not what the network infers: Detecting adversarial examples based on semantic contradiction. In NDSS. The Internet Society, 2022.
[58]
Haozhen Zhang, Le Yu, et al. TFE-GNN: A temporal fusion encoder using graph neural networks for fine-grained encrypted traffic classification. InWWW, pages 2066--2075. ACM, 2023.
[59]
Menghao Zhang, Guanyu Li, et al. Poseidon: Mitigating Volumetric DDoS Attacks with Programmable Switches. In NDSS. The Internet Society, 2020.
[60]
Ziming Zhao et al. CMD: Co-analyzed IoT Malware Detection Beyond the Network Traffic Domain. IEEE Transactions on Mobile Computing, 2023.
[61]
Ziming Zhao et al. Effective DDoS Mitigation via ML-Driven In-network Traffic Shaping. IEEE Transactions on Dependable and Secure Computing, 2024.
[62]
Ziming Zhao, Zhaoxuan Li, et al. DDoS Family: A Novel Perspective for Massive Types of DDoS Attacks. Comput. Secur., 2023.
[63]
Ziming Zhao, Zhaoxuan Li, et al. ERNN: Error-Resilient RNN for Encrypted Traffic Detection towards Network-Induced Phenomena. IEEE Trans. Dependable Secur. Comput., 2023.
[64]
Ziming Zhao, Zhaoxuan Li, Tingting Li, et al. Poster: Detecting adversarial examples hidden under watermark perturbation via usable information theory. In CCS, pages 3636--3638. ACM, 2023.
[65]
Ziming Zhao, Zhaoxuan Li, Zhuoxue Song, and Fan Zhang. Work-in-progress: Towards real-time IDS via RNN and programmable switches co-designed approach. In RTSS, pages 431--434. IEEE, 2023.
[66]
Shitong Zhu, Shasha Li, et al. You do (not) belong here: detecting DPI evasion attacks with context learning. In CoNEXT, pages 183--197. ACM, 2020.
Index Terms
- Trident: A Universal Framework for Fine-Grained and Class-Incremental Unknown Traffic Detection
Recommendations
FOSS: Towards Fine-Grained Unknown Class Detection Against the Open-Set Attack Spectrum With Variable Legitimate Traffic
Anomaly-based network intrusion detection systems (NIDSs) are essential for ensuring cybersecurity. However, the security communities realize some limitations when they put most existing proposals into practice. The challenges are mainly concerned with (i)...
A Host-Based Approach for Unknown Fast-Spreading Worm Detection and Containment
Special Section on Best Papers from SEAMS 2012The fast-spreading worm, which immediately propagates itself after a successful infection, is becoming one of the most serious threats to today’s networked information systems. In this article, we present WormTerminator, a host-based solution for fast ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2024
4826 pages
ISBN:9798400701719
DOI:10.1145/3589334
- General Chairs:
- Tat-Seng Chua,
- Chong-Wah Ngo,
- Proceedings Chair:
- Roy Ka-Wei Lee,
- Program Chairs:
- Ravi Kumar,
- Hady W. Lauw
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- State Key Laboratory of Mathematical Engineering and Advanced Computing
- National Natural Science Foundation of China
- Fok Ying-Tung Education Foundation for Young Teachers in the Higher Education Institutions of China
- Natural Science Foundation of Jiangsu Province
- Open Foundation of Henan Key Laboratory of Cyberspace Situation Awareness
Conference
WWW '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,899 of 8,196 submissions, 23%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 482Total Downloads
- Downloads (Last 12 months)482
- Downloads (Last 6 weeks)96
Reflects downloads up to 14 Nov 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in