Nothing Special   »   [go: up one dir, main page]

Skip to main content

Study and Evaluation of Unsupervised Algorithms Used in Network Anomaly Detection

  • Conference paper
  • First Online:
Proceedings of the Future Technologies Conference (FTC) 2019 (FTC 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1070))

Included in the following conference series:

Abstract

Network anomalies are unusual traffic mainly induced by network attacks or network failures. Therefore it is important for network operators as end users to detect and diagnose them to protect their network. However, these anomalies keep changing in time, it is therefore important to propose detectors which can learn from the traffic and spot anomalies without relying on any previous knowledge. Unsupervised network anomaly detectors reach this goal by taking advantage of machine learning and statistical techniques to spot the anomalies. There exists many unsupervised network anomaly detectors in the literature. Each algorithm puts forward its good detection performance, therefore it is difficult to select one detector among the large set of available detectors. Therefore, this paper, presents an extensive study and assessment of a set of well known unsupervised network anomaly detectors, and underlines their strengths and weaknesses. This study overwhelms previous similar evaluation by considering for the comparison some new, original and of premier importance parameters as detection similarity, detectors sensitivity and curse of dimensionality, together with the classical detection performance, and execution time parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 299.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 379.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. KDD Cup 1999 Data. http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html. Accessed 03 Feb 2016

  2. Bahrololum, M., Khaleghi, M.: Anomaly intrusion detection system using Gaussian mixture model. In: Convergence and Hybrid Information Technology, ICCIT 2008, vol. 1, pp. 1162–1167, November 2008

    Google Scholar 

  3. Breunig, M.M., Kriegel, H.-P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. SIGMOD Rec. 29(2), 93–104 (2000)

    Article  Google Scholar 

  4. Casas, P., Mazel, J., Owezarski, P.: UNADA: unsupervised network anomaly detection using sub-space outliers ranking. In: NETWORKING 2011: 10th International IFIP TC 6 Networking Conference, pp. 40–51. Springer, Heidelberg (2011)

    Google Scholar 

  5. Casas, P., Mazel, J., Owezarski, P.: Unsupervised network intrusion detection systems: detecting the unknown without knowledge. Comput. Commun. 35(7), 772–783 (2012)

    Article  Google Scholar 

  6. Croux, C., Filzmoser, P., Oliveira, M.R.: Algorithms for projection-pursuit robust principal component analysis. Chemometr. Intell. Lab. Syst. 87(2), 218–225 (2007)

    Article  Google Scholar 

  7. Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A Density-based Algorithm for Discovering Clusters in Large Spatial Databases with Noise, pp. 226–231. AAAI Press, Portland (1996)

    Google Scholar 

  8. Fontugne, R., Borgnat, P., Abry, P., Fukuda, K.: MAWILab: combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking. In: ACM CoNEXT 2010, Philadelphia, PA (2010)

    Google Scholar 

  9. Zimek, A., Kriegel, H.-P., Kröger, P.: Outlier detection techniques. In: Tutorial Notes: SIAM SDM 2010, Columbus, Ohio (2010)

    Google Scholar 

  10. Jensen, D.R., Solomon, H.: A Gaussian approximation to the distribution of a definite quadratic form. J. Am. Stat. Assoc. 67(340), 898–902 (1972)

    MATH  Google Scholar 

  11. Julisch, K.: Clustering intrusion detection alarms to support root cause analysis. ACM Trans. Inf. Syst. Secur. 6, 443–471 (2003)

    Article  Google Scholar 

  12. Kind, A., Stoecklin, M.P., Dimitropoulos, X.: Histogram-based traffic anomaly detection. IEEE Trans. Netw. Serv. Manage. 6(2), 110–121 (2009)

    Article  Google Scholar 

  13. Kriegel, H.-P., Kröger, P., Schubert, E., Zimek, A.: Outlier detection in axis-parallel subspaces of high dimensional data. In: Advances in Knowledge Discovery and Data Mining: 13th Pacific-Asia Conference, pp. 831–838. Springer, Heidelberg (2009)

    Google Scholar 

  14. Kwitt, R., Hofmann, U.: Unsupervised anomaly detection in network traffic by means of robust PCA. In: Computing in the Global Information Technology, p. 37, March 2007

    Google Scholar 

  15. Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: Proceedings of the 2004 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, SIGCOMM 2004, pp. 219–230. ACM (2004)

    Google Scholar 

  16. Lakhina, A., Crovella, M., Diot, C.: Mining anomalies using traffic feature distributions. ACM SIGCOMM Comput. Commun. Rev. 35(4), 217 (2005)

    Article  Google Scholar 

  17. Leung, K., Leck, C.: Unsupervised anomaly detection in network intrusion detection using clusters. In: Proceedings of the Twenty-eighth Australasian Conference on Computer Science, ACSC 2005, vol. 38, pp. 333–342. Australian Computer Society, Inc, Darlinghurst (2005)

    Google Scholar 

  18. Morris, M.D.: Factorial sampling plans for preliminary computational experiments. Technometrics 33(2), 161–174 (1991)

    Article  Google Scholar 

  19. Olusola, A.A., Oladele, A.S., Abosede, D.O.: Analysis of KDD 99 intrusion detection dataset for selection of relevance features. In: World Congress on Engineering and Computer Science, pp. 162–168 (2010)

    Google Scholar 

  20. Kriegel, H.P., Schneider, R., Seeger, B., Beckmann, N.: The R*-tree: an efficient and robust access method for points and rectangles. Sigmod Rec. 19, 322–331 (1990)

    Article  Google Scholar 

  21. Portnoy, L., Eskin, E., Stolfo, S.: Intrusion detection with unlabeled data using clustering. In: In Proceedings of ACM CSS Workshop on Data Mining Applied to Security, pp. 5–8 (2001)

    Google Scholar 

  22. Ringberg, H., Soule, A., Rexford, J., Diot, C.: Sensitivity of PCA for traffic anomaly detection. SIGMETRICS Perform. Eval. Rev. 35(1), 109–120 (2007)

    Article  Google Scholar 

  23. Shyu, M.-L., Chen, S.-C., Sarinnapakorn, K., Chang, L.: A novel anomaly detection scheme based on principal component classifier. In: IEEE Foundations and New Directions of Data Mining Workshop, pp. 171–179 (2003)

    Google Scholar 

  24. Sommer, R., Paxson, V.: Outside the closed world: on using machine learning for network intrusion detection. In: 2010 IEEE Symposium on Security and Privacy, 0(May), pp. 305–316 (2010)

    Google Scholar 

  25. Syarif, I., Prugel-Bennett, G., Wills, A.: Unsupervised clustering approach for network anomaly detection. In: Networked Digital Technologies: 4th International Conference. Springer, Heidelberg (2012)

    Google Scholar 

  26. Thang, T.M., Kim, J.: The anomaly detection by using DBSCAN clustering with multiple parameters. In: 2011 International Conference on Information Science and Applications (ICISA), pp. 1–5, April 2011

    Google Scholar 

  27. Tsakok, J.A., Bishop, W., Kennings, Af.: kd-Tree traversal techniques. In: Interactive Ray Tracing, p. 190, August 2008

    Google Scholar 

  28. Yasami, Y., Khorsandi, S., Mozaffari, S.P., Jalalian, A.: An unsupervised network anomaly detection approach by k-means clustering & ID3 algorithms. In: Computers and Communications, pp. 398–403, July 2008

    Google Scholar 

  29. Zimek, A., Schubert, E., Kriegel, H.-P.: A survey on unsupervised outlier detection in high-dimensional numerical data. Stat. Anal. Data Min. 5(5), 363–387 (2012)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Philippe Owezarski .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dromard, J., Owezarski, P. (2020). Study and Evaluation of Unsupervised Algorithms Used in Network Anomaly Detection. In: Arai, K., Bhatia, R., Kapoor, S. (eds) Proceedings of the Future Technologies Conference (FTC) 2019. FTC 2019. Advances in Intelligent Systems and Computing, vol 1070. Springer, Cham. https://doi.org/10.1007/978-3-030-32523-7_28

Download citation

Publish with us

Policies and ethics