Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Cyber intrusion detection through association rule mining on multi-source logs

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Security logs in cloud environment like intrusion detection system (IDS) logs, firewall logs, and system logs provide historical information describing potential security risks. However, the use of logs for cyber intrusion detection relies heavily on expert knowledge. It is very difficult for the non-expert to identify these intrusion behaviors. This paper proposes a new method for mining association rules from multi-source logs to detect various intrusion behaviors in the cloud computing platform. In this method, a rule base is constructed to detect cyber intrusion. An adaptive approach is used to speed up the calculation of the association rule mining, in which the decision depends on the time complexity of the algorithm. Various cyber-attacks are simulated in the verification experiments which show the calculation speed of the proposed method is faster than other algorithms. Furthermore, compared with other methods, the performance of the proposed intrusion detection method is better than others in term of precision, recall, and f-measure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Agrawal R, Srikant R, et al. (1994) Fast algorithms for mining association rules. In Proc. 20th int. conf. very large data bases. VLDB 1215:487–499

    Google Scholar 

  2. Ashfaq RAR, Wang X-Z, Huang JZ, Abbas H, He Y-L (2017) Fuzziness based semi-supervised learning approach for intrusion detection system. Inf Sci 378:484–497

    Article  Google Scholar 

  3. Bao L, Li Q, Lu P, Lu J, Ruan T, Ke Z (2018) Execution anomaly detection in large-scale systems through console log analysis. J Syst Softw 143:172–186

    Article  Google Scholar 

  4. Bhati BS, Rai C S, Balamurugan B, Al-Turjman F (2020) An intrusion detection scheme based on the ensemble of discriminant classifiers. Comput Electr Eng 86:106742

    Article  Google Scholar 

  5. Brahmi H, Brahmi I, Yahia SB (2012) Omc-ids: at the cross-roads of olap mining and intrusion detection. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, pp 13–24

  6. Buczak AL (2015) A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun Surv Tutorials 18(2):1153–1176

    Article  Google Scholar 

  7. Caminero G, Lopez-Martin M, Carro B (2019) Adversarial environment reinforcement learning algorithm for intrusion detection. Comput Netw 159:96–109

    Article  Google Scholar 

  8. Ċavuṡoġlu Ü (2019) A new hybrid approach for intrusion detection using machine learning methods. Appl Intell 49(7):2735–2761

    Article  Google Scholar 

  9. Chen L, Leneutre J (2009) A game theoretical framework on intrusion detection in heterogeneous networks. IEEE Trans Inf Forens Secur 4(2):165–178

    Article  Google Scholar 

  10. Dean J, Ghemawat S (2008) Mapreduce: simplified data processing on large clusters. Commun ACM 51(1):107–113

    Article  Google Scholar 

  11. Dutkevych T, Piskozub A, Tymoshyk N (2007) Real-time intrusion prevention and anomaly analyze system for corporate networks. In: 2007 4th IEEE Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications. IEEE, pp 599–602

  12. Duy PH, Diep NN (2017) Intrusion detection using deep neural network. South Asian J Sci 5(2):111–125

    Google Scholar 

  13. Fu Q, Lou J-G, Wang Y, Li J (2009) Execution anomaly detection in distributed systems through unstructured log analysis. In: 2009 ninth IEEE international conference on data mining. IEEE, pp 149–158

  14. Husák M, Komárková J, Bou-Harb E, Čeleda P (2018) Survey of attack projection, prediction, and forecasting in cyber security. IEEE Commun Surv Tutorials 21(1):640–660

    Article  Google Scholar 

  15. Jiang C-B, Liu I-H, Chung Y-N, Li J-S (2016) Novel intrusion prediction mechanism based on honeypot log similarity. Int J Netw Manag 26(3):156–175

    Article  Google Scholar 

  16. Khan S, Parkinson S (2018) Eliciting and utilising knowledge for security event log analysis: An association rule mining and automated planning approach. Expert Syst Appl 113:116–127

    Article  Google Scholar 

  17. Kim H, Kim J, Kim Y, Kim I, Kim KJ (2019) Design of network threat detection and classification based on machine learning on cloud computing. Clust Comput 22(1):2341– 2350

    Article  Google Scholar 

  18. Kim Y-H, Park WH (2014) A study on cyber threat prediction based on intrusion detection event for apt attack detection. Multimed Tools Appl 71(2):685–698

    Article  Google Scholar 

  19. Kumar G (2020) An improved ensemble approach for effective intrusion detection. J Supercomput 76(1):275–291

    Article  Google Scholar 

  20. Lee W, Stolfo S (1998) Data mining approaches for intrusion detection

  21. Li W (2004) Using genetic algorithm for network intrusion detection. Proc U S Depart Energy Cyber Secur Group 1:1–8

    Google Scholar 

  22. Lu S, Wei X, Rao B, Tak B, Wang L, Wang L (2019) Ladra: Log-based abnormal task detection and root-cause analysis in big data processing with spark. Futur Gener Comput Syst 95:392–403

    Article  Google Scholar 

  23. Lu X, Han J, Ren Q, Dai H, Li J, Ou J (2018) Network threat detection based on correlation analysis of multi-platform multi-source alert data. Multimed Tools Appl:1–15

  24. Mahdavifar S, Ghorbani AA (2019) Application of deep learning to cybersecurity A survey. Neurocomputing 347:149–176

    Article  Google Scholar 

  25. Mukkamala S, Sung AH, Abraham A (2005) Intrusion detection using an ensemble of intelligent paradigms. J Netw Comput Appl 28(2):167–182

    Article  Google Scholar 

  26. Naseer S, Saleem Y (2018) Enhanced network intrusion detection using deep convolutional neural networks. KSII Trans Internet Inf Syst 12(10):5159–5178

    Google Scholar 

  27. Padillo F, Luna JM, Herrera F, Ventura S (2018) Mining association rules on big data through mapreduce genetic programming. Integr Comput-Aided Eng 25(1):31–48

    Article  Google Scholar 

  28. Rathee S, Kashyap A (2018) Adaptive-miner: an efficient distributed association rule mining algorithm on spark. J Big Data 5(1):1–17

    Article  Google Scholar 

  29. Selvakumar B, Muneeswaran K (2019) Firefly algorithm based feature selection for network intrusion detection. Comput Secur 81:148–155

    Article  Google Scholar 

  30. Sperotto A, Schaffrath G, Sadre R, Morariu C, Pras A, Stiller B (2010) An overview of ip flow-based intrusion detection. IEEE commun Surv Tutorials 12(3):343–356

    Article  Google Scholar 

  31. Wang K, Stolfo SJ (2004) Anomalous payload-based network intrusion detection. In: International Workshop on Recent Advances in Intrusion Detection. Springer, pp 203–222

  32. Wang M, Zheng K, Yang Y, Wang X (2020) An explainable machine learning framework for intrusion detection systems. IEEE Access 8:73127–73141

    Article  Google Scholar 

  33. Wattanapongsakorn N, Charnsripinyo C (2015) Web-based monitoring approach for network-based intrusion detection and prevention. Multimed Tools Appl 74(16):6391–6411

    Article  Google Scholar 

  34. Xia D, Lu X, Li H, Wang W, Li Y, Zhang Z (2018) A mapreduce-based parallel frequent pattern growth algorithm for spatiotemporal association analysis of mobile trajectory big data. Complexity:1–16

  35. Yang J, Deng J, Li S, Hao Y (2017) Improved traffic detection with support vector machine based on restricted boltzmann machine. Soft Comput 21(11):3101–3112

    Article  Google Scholar 

  36. Yin C, Zhu Y, Fei J, He X (2017) A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access 5:21954–21961

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge funding support from the National Natural Science Foundation Committee (NSFC) of China (grant no. 51905397) and The Fundamental Research Funds for the Central Universities and (WUT:2018III069GX), (WUT:2019III071GX), as well as the contributions from all collaborators with in the projects mentioned.

Author information

Authors and Affiliations

  1. School of Information Engineering, Wuhan University of Technology, Wuhan, 430070, China

    Ping Lou, Guantong Lu, Xuemei Jiang, Zheng Xiao, Jiwei Hu & Junwei Yan

  2. Hubei Key Laboratory of Broadband Wireless Communication and Sensor Networks, Wuhan University of Technology, Wuhan, 430070, China

    Ping Lou, Guantong Lu, Xuemei Jiang, Zheng Xiao, Jiwei Hu & Junwei Yan

Authors
  1. Ping Lou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guantong Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuemei Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zheng Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiwei Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Junwei Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junwei Yan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lou, P., Lu, G., Jiang, X. et al. Cyber intrusion detection through association rule mining on multi-source logs. Appl Intell 51, 4043–4057 (2021). https://doi.org/10.1007/s10489-020-02007-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02007-5

Keywords

Search

Navigation