Abstract
The real-time analysis of network data is of great significance to network security. Visualization technology and machine learning can assist in network data analysis from different aspects. However, there is little research regarding combining these two methods to process real-time network data. This paper proposes a novel real-time network security system. Combining unsupervised learning and visualization technology, it can identify network behavior patterns and provide a visualization module to adjust models interactively. The system is primarily divided into three parts. In the feature extraction part, we train a deep auto-encoder to compress the feature dimension. In the behavior pattern recognition part, normal and abnormal pattern SOINNs are trained incrementally. In visualization part, analysts can use multiple views to judge recognition results rapidly and adjust models so that the identification accuracy can be increased. We use the data in VAST Challenge 2013 to show that our system can identify network behavior patterns in real time and find the correlations between them.
Graphical abstract
Similar content being viewed by others
References
Ali SHA, Ozawa S, Ban T, Nakazato J, Shimamura J (2016) A neural network model for detecting ddos attacks using darknet traffic features. In: Neural networks (IJCNN), 2016 international joint conference on, pp. 2979–2985. IEEE
Boschetti A, Salgarelli L, Muelder C, Ma K-L (2011) TVi: a visual querying system for network monitoring and anomaly detection. In: Proceedings of the 8th international symposium on visualization for cyber security, p 1. ACM
Bruns-Smith D, Baskaran MM, Ezick J, Henretty T, Lethin R (2016) Cyber security through multidimensional data decompositions. In: Cybersecurity symposium (CYBERSEC), 2016, pp. 59–67. IEEE
Buczak AL, Guven E (2016) A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun Surv Tutor 18(2):1153–1176
Chen W, Kong F, Mei F, Yuan G, Li B (2017) A novel unsupervised anomaly detection approach for intrusion detection system. In: Big data security on cloud (BigDataSecurity), IEEE international conference on high performance and smart computing (HPSC), and IEEE international conference on intelligent data and security (IDS), 2017 IEEE 3rd international conference on, pp 69–73. IEEE
Furao S, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19(1):90–106
Hajar AAS, Fukase K, Ozawa S (2013) A neural network model for large-scale stream data learning using locally sensitive hashing. In: International conference on neural information processing. Springer, Berlin, pp 369–376
Hao L, Healey CG, Hutchinson SE (2015) Ensemble visualization for cyber situation awareness of network security data. In: Visualization for cyber security (VizSec), 2015 IEEE symposium on, pp 1–8. IEEE
Hinton G E, Salakhutdinov R R (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Huang S-Y, Yu F, Tsaih R-H, Huang Y (2015) Network-traffic anomaly detection with incremental majority learning. In: Neural networks (IJCNN), 2015 international joint conference on, pp. 1–8. IEEE
Leban G, Zupan B, Vidmar G, Bratko I (2006) Vizrank: data visualization guided by machine learning. Data Min Knowl Discov 13(2):119–136
Shiravi H, Shiravi A, Ghorbani AA (2012) A survey of visualization systems for network security. IEEE Trans Vis Comput Graph 18(8):1313–1329
Sommer R, Paxson V (2010) Outside the closed world: on using machine learning for network intrusion detection. In: Security and privacy (SP), 2010 IEEE symposium on, pp 305–316. IEEE
Sultana A, Jabbar M (2016) Intelligent network intrusion detection system using data mining techniques. In: Applied and theoretical computing and communication technology (iCATccT), 2016 2nd international conference on, pp 329–333. IEEE
Talbot J, Lee B, Kapoor A, Tan DS (2009) Ensemblematrix: interactive visualization to support machine learning with multiple classifiers. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 1283–1292. ACM
Theron R, Magán-Carrión R, Camacho J, Fernndez GM (2017) Network-wide intrusion detection supported by multivariate analysis and interactive visualization. In: Visualization for cyber security (VizSec), 2017 IEEE symposium on, pp 1–8. IEEE
Vast challenge 2013 homepage. http://www.vacommunity.org/VAST+Challenge+2013 (2013)
Webb GI, Boughton JR, Wang Z (2005) Not so naive bayes: aggregating one-dependence estimators. Mach Learn 58(1):5–24
Yousefi-Azar M, Varadharajan V, Hamey L, Tupakula U (2017) Autoencoder-based feature learning for cyber security applications. In: Neural networks (IJCNN), 2017 international joint conference on, pp 3854–3861. IEEE
Zhang S, Fung C, Huang S, Luan Z, Qian D (2017) Psom: periodic self-organizing maps for unsupervised anomaly detection in periodic time series. In: Quality of service (IWQoS), 2017 IEEE/ACM 25th international symposium on, pp 1–6. IEEE
Zhao S, Chandrashekar M, Lee Y, Medhi D (2015) Real-time network anomaly detection system using machine learning. In: Design of reliable communication networks (DRCN), 2015 11th international conference on the, pp 267–270. IEEE
Acknowledgements
Authors thank Prof. Xiaoru Yuan, Peking university, and unknown reviewers for instruction. This work was supported by National Key Research and Development Program of China (Grant No. 2017YFB0701900), National Nature Science Foundation of China (Grant No. 61100053) and CCF-Venustech Hongyan Research Initiative (2016-013).
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Fan, X., Li, C. & Dong, X. A real-time network security visualization system based on incremental learning (ChinaVis 2018). J Vis 22, 215–229 (2019). https://doi.org/10.1007/s12650-018-0525-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12650-018-0525-z