research-article

Email spam detection using hierarchical attention hybrid deep learning method

Authors:

Seyhmus YilmazAuthors Info & Claims

Volume 233, Issue C

https://doi.org/10.1016/j.eswa.2023.120977

Published: 15 December 2023 Publication History

Abstract

Email is one of the most widely used ways to communicate, with millions of people and businesses relying on it to communicate and share knowledge and information on a daily basis. Nevertheless, the rise in email users has occurred a dramatic increase in spam emails in recent years. Considering the escalating number of spam emails, it has become crucial to devise effective strategies for spam detection. To tackle this challenge, this article proposes a novel technique for email spam detection that is based on a combination of convolutional neural networks, gated recurrent units, and attention mechanisms. During system training, the network is selectively focused on necessary parts of the email text. The usage of convolution layers to extract more meaningful, abstract, and generalizable features by hierarchical representation is the major contribution of this study. Additionally, this contribution incorporates cross-dataset evaluation, which enables the generation of more independent performance results from the model's training dataset. According to cross-dataset evaluation results, the proposed technique advances the results of the present attention-based techniques by utilizing temporal convolutions, which give us more flexible receptive field sizes are utilized. The suggested technique's findings are compared to those of state-of-the-art models and show that our approach outperforms them.

References

[1]

I. AbdulNabi, Q. Yaseen, Spam email detection using deep learning techniques, Procedia Computer Science 184 (2021) 853–858,.

[2]

J. Abreu, L. Fred, D. Macêdo, C. Zanchettin, Hierarchical Attentional Hybrid Neural Networks for Document Classification, in: Artificial Neural Networks and Machine Learning – ICANN 2019: Workshop and Special Sessions, Springer International Publishing, 2019, pp. 396–402,.

Digital Library

[3]

N. Ahmed, R. Amin, H. Aldabbas, D. Koundal, B. Alouffi, T. Shah, Machine learning techniques for spam detection in email and IoT platforms: analysis and research challenges, Security and Communication Networks 2022 (2022),.

Digital Library

[4]

Androutsopoulos, I., Koutsias, J., Chandrinos, K. v, Paliouras, G., & Spyropoulos, C. D. (2000). Lingspam corpus. http://nlp.cs.aueb.gr/software_and_datasets/lingspam_public.tar.gz.

[5]

Bahdanau, D., Cho, K. H., & Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings.

[6]

Bai, S., Kolter, J. Z., & Koltun, V. (2018). An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. ArXiv Preprint ArXiv:1803.01271. http://arxiv.org/abs/1803.01271.

[7]

P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, Enriching word vectors with subword information, Transactions of the Association for Computational Linguistics 5 (2017) 135–146,.

[8]

G. Caruana, M. Li, A survey of emerging approaches to spam filtering, ACM Computing Surveys 44 (2) (2012) 1–27,.

Digital Library

[9]

Cheng, V., & Li, C. h. (2006). Personalized Spam Filtering with Semi-supervised Classifier Ensemble. In 2006 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2006 Main Conference Proceedings)(WI’06). IEEE. https://doi.org/10.1109/wi.2006.132.

[10]

V. Cheng, C.H. Li, Combining supervised and semi-supervised classifier for personalized spam filtering, in: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): Vol. 4426 LNAI, Springer, Berlin Heidelberg, 2007, pp. 449–456,.

[11]

Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. ArXiv Preprint ArXiv:1406.1078.

[12]

J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, ArXiv Preprint (2018) ArXiv:1810.04805.

[13]

H. Drucker, D. Wu, V.N. Vapnik, Support vector machines for spam categorization, IEEE Transactions on Neural Networks 10 (5) (1999) 1048–1054,.

Digital Library

[14]

E.-S.-M. El-Alfy, R.E. Abdel-Aal, Using GMDH-based networks for improved spam detection and email feature analysis, Applied Soft Computing 11 (1) (2011) 477–488,.

Digital Library

[15]

The Enron-Spam datasets. (n.d.). Retrieved July 28, 2022, from http://nlp.cs.aueb.gr/software_and_datasets/Enron-Spam/index.html.

[16]

L. Firte, C. Lemnaru, R. Potolea, Spam detection filter using KNN algorithm and resampling, in: Proceedings - 2010 IEEE 6th International Conference on Intelligent Computer Communication and Processing, ICCP10, IEEE, 2010, pp. 27–33,.

Digital Library

[17]

Freeman, D. M. (2013). Using naive bayes to detect spammy names in social networks. In Proceedings of the 2013 ACM workshop on Artificial intelligence and security. ACM. https://doi.org/10.1145/2517312.2517314.

[18]

A. Galassi, M. Lippi, P. Torroni, Attention in natural language processing, IEEE Transactions on Neural Networks and Learning Systems 32 (10) (2021) 4291–4308,.

[19]

Gani, R., & Chalaguine, L. (2022). Feature Engineering vs BERT on Twitter Data. https://doi.org/10.48550/arxiv.2210.16168.

[20]

Y. Gao, M. Yang, A. Choudhary, Semi Supervised Image Spam Hunter: A Regularized Discriminant EM Approach, in: Advanced Data Mining and Applications, Springer, Berlin Heidelberg, 2009, pp. 152–164,.

Digital Library

[21]

GenSpam. (n.d.). Retrieved July 28, 2022, from http://www.benmedlock.co.uk/genspam.html.

[22]

Y. Guo, Z. Mustafaoglu, D. Koundal, Spam detection using bidirectional transformers and machine learning classifier algorithms, Journal of Computational and Cognitive Engineering (2022),.

[23]

I. Idris, A. Selamat, S. Omatu, Hybrid email spam detection model with negative selection algorithm and differential evolution, Engineering Applications of Artificial Intelligence 28 (2014) 97–110,.

Digital Library

[24]

Kim, Y. (2014). Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. https://doi.org/10.3115/v1/d14-1181.

[25]

Li, W., Meng, W., Tan, Z., & Xiang, Y. (2014). Towards Designing an Email Classification System Using Multi-view Based Semi-supervised Learning. In 2014 IEEE 13th International Conference on Trust, Security and Privacy in Computing and Communications. IEEE. https://doi.org/10.1109/trustcom.2014.26.

[26]

W. Li, W. Meng, L.-F. Kwok, Ip, h. h. s., Enhancing collaborative intrusion detection networks against insider attacks using supervised intrusion sensitivity-based trust management model, Journal of Network and Computer Applications 77 (2017) 135–145,.

Digital Library

[27]

W. Li, W. Meng, Z. Tan, Y. Xiang, Design of multi-view based email classification for IoT systems via semi-supervised learning, Journal of Network and Computer Applications 128 (2019) 56–63,.

[28]

C. Lopes, P. Cortez, P. Sousa, M. Rocha, M. Rio, Symbiotic filtering for spam email detection, Expert Systems with Applications 38 (8) (2011) 9365–9372,.

Digital Library

[29]

S. Magdy, Y. Abouelseoud, M. Mikhail, Efficient spam and phishing emails filtering based on deep learning, Computer Networks 206 (2022),.

Digital Library

[30]

M.N. Marsono, M.W. El-Kharashi, F. Gebali, Binary LNS-based naïve Bayes hardware classifier for spam control, in: Proceedings - IEEE International Symposium on Circuits and Systems, IEEE, 2006, pp. 3674–3677,.

[31]

W. Meizhen, L. Zhitang, Z. Sheng, A Method for Spam Behavior Recognition Based on Fuzzy Decision Tree, in: In 2009 Ninth IEEE International Conference on Computer and Information Technology. IEEE, 2009,.

Digital Library

[32]

Y. Meng, W. Li, L.-F. Kwok, Enhancing email classification using data reduction and disagreement-based semi-supervised learning, in: In 2014 IEEE International Conference on Communications (ICC). IEEE, 2014,.

[33]

Mojdeh, M., & Cormack, G. v. (2010). Semi-supervised spam filtering using aggressive consistency learning. In Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval - SIGIR ’10. ACM Press. https://doi.org/10.1145/1835449.1835598.

[34]

V. Morales, J.C. Gomez, S. van Amerongen, Cross-dataset email classification, Journal of Intelligent & Fuzzy Systems 39 (2) (2020) 2279–2290,.

Digital Library

[35]

Mustapha, I. B., Hasan, S., Olatunji, S. O., Shamsuddin, S. M., & Kazeem, A. (n.d.). Effective Email Spam Detection System using Extreme Gradient Boosting.

[36]

NLTK :: Natural Language Toolkit. (n.d.). Retrieved January 21, 2022, from https://www.nltk.org/.

[37]

NumPy. (n.d.). Retrieved January 21, 2022, from https://numpy.org/.

[38]

S.O. Olatunji, Improved email spam detection model based on support vector machines, Neural Computing and Applications 31 (3) (2019) 691–699,.

[39]

T. Ouyang, S. Ray, M. Allman, M. Rabinovich, A large-scale empirical analysis of email spam detection through network characteristics in a stand-alone enterprise, Computer Networks 59 (2014) 101–121,.

Digital Library

[40]

T. Peng, Q. Liu, D. Meng, G. Wang, Collaborative trajectory privacy preserving scheme in location-based services, Information Sciences 387 (2017) 165–179,.

Digital Library

[41]

P. Rodriguez, D. Velazquez, G. Cucurull, J.M. Gonfaus, F.X. Roca, J. Gonzalez, Pay attention to the activations: A modular attention mechanism for fine-grained image recognition, IEEE Transactions on Multimedia 22 (2) (2020) 502–514,.

Digital Library

[42]

M.Y.A. Salmony, A.R. Faridi, Bert Distillation to Enhance the Performance of Machine Learning Models for Sentiment Analysis on Movie Review Data, Proceedings of the 2022 9th International Conference on Computing for Sustainable Global Development, 2022.

[43]

Santos, F. A. O., Macedo, H. T., Dias Bispo, T., & Zanchettin, C. (2020). Morphological Skip-Gram: Using morphological knowledge to improve word representation. https://arxiv.org/abs/2007.10055v2.

[44]

scikit-learn: machine learning in Python — scikit-learn 1.0.2 documentation. (n.d.). Retrieved January 21, 2022, from https://scikit-learn.org/stable/.

[45]

Sculley, D., & Wachman, G. M. (2007). Relaxed online SVMs for spam filtering. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR ’07. ACM Press. https://doi.org/10.1145/1277741.1277813.

[46]

L. Shi, Q. Wang, X. Ma, M. Weng, H. Qiao, Spam email classification using decision tree ensemble, Journal of Computational Information Systems 8 (3) (2012) 949–956.

[47]

Z. Siddique, M.A. Khan, I.U. Din, A. Almogren, I. Mohiuddin, S. Nazir, Machine learning-based detection of spam emails, Scientific Programming 2021 (2021),.

Digital Library

[48]

SpamAssassin Dataset. (n.d.). Retrieved July 28, 2022, from https://spamassassin.apache.org/old/publiccorpus/.

[49]

L. Sulieman, D. Gilmore, C. French, R.M. Cronin, G.P. Jackson, M. Russell, D. Fabbri, Classifying patient portal messages using convolutional neural networks, Journal of Biomedical Informatics 74 (2017) 59–70,.

[50]

G. Tang, J. Pei, W.-S. Luk, Email mining: Tasks, common techniques, and tools, Knowledge and Information Systems 41 (1) (2013) 1–31,.

Digital Library

[51]

A. Tharwat, Classification assessment methods, Applied Computing and Informatics 1–13 (2018),.

[52]

V.S. Tida, S.H. Hsu, Universal Spam Detection using Transfer Learning of BERT Model, Proceedings of the 55th Hawaii International Conference on System Sciences, 2022.

[53]

2007 TREC Public Spam Corpus. (n.d.). Retrieved July 28, 2022, from https://plg.uwaterloo.ca/∼gvcormac/treccorpus07/.

[54]

Trivedi, S. K., & Dey, S. (2013). Effect of feature selection methods on machine learning classifiers for detecting email spams. In Proceedings of the 2013 Research in Adaptive and Convergent Systems on - RACS ’13. ACM Press. https://doi.org/10.1145/2513228.2513313.

[55]

D. Wang, D. Irani, C. Pu, A Study on Evolution of Email Spam Over Fifteen Years, in: In Proceedings of the 9th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing. ICST, 2013,.

[56]

C. Wang, Q. Li, T.Y. Ren, X.H. Wang, G.X. Guo, High efficiency spam filtering: A manifold learning-based approach, Mathematical Problems in Engineering 2021 (2021),.

[57]

S. Wen, Y. Xiang, W. Zhou, Modeling and Analysis for Thwarting Worm Propagation in Email Networks, Springer, Berlin Heidelberg, 2013, pp. 763–769,.

[58]

S. Wen, W. Zhou, J. Zhang, Y. Xiang, W. Zhou, W. Jia, …., C., Modeling and analysis on the propagation dynamics of modern email malware, IEEE Transactions on Dependable and Secure Computing 11 (4) (2014) 361–374,.

[59]

J.S. Whissell, C.L.A. Clarke, Clustering for semi-supervised spam filtering, ACM Press, 2011, pp. 125–134,.

Digital Library

[60]

Y.-S. Wu, S. Bagchi, N. Singh, R. Wita, Spam detection in voice-over-IP calls through semi-supervised clustering, in: In 2009 IEEE/IFIP International Conference on Dependable Systems & Networks. IEEE, 2009,.

[61]

Xu, K., Lei Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R. S., & Bengio, Y. (2015). Show, Attend and Tell: Neural Image Caption Generation with Visual Attention (pp. 2048–2057). PMLR. https://proceedings.mlr.press/v37/xuc15.html.

[62]

Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, E. Hovy, Hierarchical attention networks for document classification, Association for Computational Linguistics, 2016, pp. 1480–1489.

[63]

B. Yu, D. Zhu, hua., Combining neural networks and semantic feature space for email classification, Knowledge-Based Systems 22 (5) (2009) 376–381,.

Digital Library

[64]

S. Zavrak, M. Iskefiyeli, Anomaly-based intrusion detection from network flow features using variational autoencoder, IEEE Access 8 (2020) 108346–108358,.

[65]

J. Zhan, B. John Oommen, J. Crisostomo, Anomaly detection in dynamic systems using weak estimators, ACM Transactions on Internet Technology 11 (1) (2011) 1–16,.

Digital Library

[66]

Y. Zhang, S. Wang, P. Phillips, G. Ji, Binary PSO with mutation operator for feature selection using decision tree applied to spam detection, Knowledge-Based Systems 64 (2014) 22–31,.

Digital Library

[67]

W. Zhang, W. Wang, J. Wang, H. Zha, User-guided hierarchical attention network for multi-modal social image popularity prediction, in: The Web Conference 2018 - Proceedings of the World Wide Web Conference, 2018,.

Digital Library

[68]

W. Zhang, D. Zhu, Y. Zhang, G. Zhou, B. Xu, Harmonic functions based semi-supervised learning for web spam detection, ACM Press, 2011, pp. 74–75,.

Digital Library

[69]

J. Zheng, Z. Qin, S. Wang, D. Li, Attention-based explainable friend link prediction with heterogeneous context information, Information Sciences 597 (2022) 211–229,.

Digital Library

[70]

C. Zhong, F. Xiong, S. Pan, L. Wang, X. Xiong, Hierarchical attention neural network for information cascade prediction, Information Sciences 622 (2023) 1109–1127,.

Digital Library

[71]

Zhou, R. (n.d.). Question Answering Models for SQuAD 2.0.

[72]

W. Zhou, S. Wen, Y. Wang, Y. Xiang, W. Zhou, An Analytical Model on the Propagation of Modern Email Worms, in: In 2012 IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications. IEEE, 2012,.

Digital Library

[73]

B. Zhou, Y. Yao, J. Luo, Cost-sensitive three-way email spam filtering, Journal of Intelligent Information Systems 42 (1) (2013) 19–45,.

Digital Library

[74]

X. Zhu, A.B. Goldberg, Introduction to semi-supervised learning, Synthesis Lectures on Artificial Intelligence and Machine Learning 3 (1) (2009) 1–130,.

Cited By

Alaoui EFilali ASallah AHajhouj MHessane AMerras M(2024)Towards Transparent CybersecurityProcedia Computer Science10.1016/j.procs.2024.05.046236:C(394-401)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1016/j.procs.2024.05.046

Index Terms

Email spam detection using hierarchical attention hybrid deep learning method
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
  2. World Wide Web
    1. Web applications
      1. Internet communications tools
        Email

Index terms have been assigned to the content through auto-classification.

Recommendations

Enhancement of email spam detection using improved deep learning algorithms for cyber security

Email has sustained to be an essential part of our lives and as a means for better communication on the internet. The challenge pertains to the spam emails residing a large amount of space and bandwidth. The defect of state-of-the-art spam filtering ...
A Comprehensive Study of Email Spam Botnet Detection
The problem of email spam has grown significantly over the past few years. It is not just a nuisance for users but also it is damaging for those who fall for scams and other attacks. This is due to the complexity intensification of email spamming ...
Hoodwinking spam email filters
CEA'07: Proceedings of the 2007 annual Conference on International Conference on Computer Engineering and Applications

Many spam email filters have been proposed, however spammers regularly find new ways of hoodwinking those filters. Most of those filters are text based and hence spammers try to conceal the text which reveals the spam nature of an email. In order to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 233, Issue C

Dec 2023

1587 pages

ISSN:0957-4174

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 15 December 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alaoui EFilali ASallah AHajhouj MHessane AMerras M(2024)Towards Transparent CybersecurityProcedia Computer Science10.1016/j.procs.2024.05.046236:C(394-401)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1016/j.procs.2024.05.046

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents