research-article

Assessing the opportunity of combining state-of-the-art Android malware detectors

Authors:

Tegawendé F. Bissyandé,

Jacques KleinAuthors Info & Claims

Empirical Software Engineering, Volume 28, Issue 2

https://doi.org/10.1007/s10664-022-10249-9

Published: 24 December 2022 Publication History

Abstract

Research on Android malware detection based on Machine learning has been prolific in recent years. In this paper, we show, through a large-scale evaluation of four state-of-the-art approaches that their achieved performance fluctuates when applied to different datasets. Combining existing approaches appears as an appealing method to stabilise performance. We therefore proceed to empirically investigate the effect of such combinations on the overall detection performance. In our study, we evaluated 22 methods to combine feature sets or predictions from the state-of-the-art approaches. Our results showed that no method has significantly enhanced the detection performance reported by the state-of-the-art malware detectors. Nevertheless, the performance achieved is on par with the best individual classifiers for all settings. Overall, we conduct extensive experiments on the opportunity to combine state-of-the-art detectors. Our main conclusion is that combining state-of-the-art malware detectors leads to a stabilisation of the detection performance, and a research agenda on how they should be combined effectively is required to boost malware detection. All artefacts of our large-scale study (i.e., the dataset of

\sim

0.5 million apks and all extracted features) are made available for replicability.

References

[1]

Afonso VM, de Amorim MF, Grégio ARA, Junquera GB, and de Geus PL Identifying android malware using dynamically obtained features J Comput Virology Hacking Tech 2015 11 1 9-17

[2]

Alam MS, Vuong ST (2013) Random forest classification for detecting android malware. In: 2013 IEEE International conference on green computing and communications and IEEE internet of things and IEEE cyber, physical and social computing, pp 663–669.

[3]

Allix K, Bissyandé TF, Jérome Q, Klein J, State R, and Le Traon YEmpirical assessment of machine learning-based malware detectors for androidEmpirical Softw Eng2016211183-211https://doi.org/10.1007/s10664-014-9352-6 https://doi.org/10.1007/s10664-014-9352-6

[4]

Allix K, Bissyandé TF, Klein J, Le Traon Y (2016b) Androzoo: collecting millions of android apps for the research community. In: Proceedings of the 13th international conference on mining software repositories, ACM, New York, MSR ’16, pp 468–471.

[5]

Allix K, Bissyandé TF, Klein J, LeTraon Y (2015) Are your training datasets yet relevant? In: Piessens F, Caballero J, Bielova N (eds) Engineering secure software and systems, springer international publishing, Cham, pp 51–67.

[6]

Appice A, Andresini G, and Malerba D Clustering-aided multi-view classification: a case study on android malware detection J Intell Inf Syst 2020 55 1 1-26

[7]

Arp D, Quiring E, Pendlebury F, Warnecke A, Pierazzi F, Wressnegger C, Cavallaro L, Rieck K (2020) Dos and don’ts of machine learning in computer security. arXiv:201009470

[8]

Arp D, Spreitzenbarth M, Hübner M, Gascon H, Rieck K (2014) Drebin: efficient and explainable detection of android malware in your pocket. In: Proceedings of the ISOC network and distributed system security symposium (NDSS), San Diego, CA

[9]

Breiman L Bagging predictors Mach Learn 1996 24 2 123-140

[10]

Breiman L Random forests Mach Learn 2001 45 1 5-32

[11]

Brown G Ensemble learning Encyclopedia Mach Learn 2010 312 15-19

[12]

Caruana R, Niculescu-Mizil A, Crew G, Ksikes A (2004) Ensemble selection from libraries of models. In: Proceedings of the twenty-first international conference on machine learning, association for computing machinery, New York, ICML ’04, p 18.

[13]

Christianah A, Gyunka B, Oluwatobi A (2020) Optimizing android malware detection via ensemble learning. https://www.learntechlib.org/p/217826

[14]

DATA G (2020) G DATA mobile malware report. https://www.gdatasoftware.com/news/1970/01/-36401-g-data-mobile-malware-report-harmful-android-apps-every-eight-seconds. Accessed 10 June 2021

[15]

Daoudi N, Allix K, Bissyandé TF, Klein J (2021a) A deep dive inside drebin: an explorative analysis beyond android malware detection scores. ACM Trans Privacy Secur (TOPS) Appear

[16]

Daoudi N, Allix K, Bissyandé TF, and Klein JLessons learnt on reproducibility in machine learning based android malware detectionEmpirical Softw Eng20212641-53https://doi.org/10.1007/s10664-021-09955-7

[17]

Daoudi N, Samhi J, Kabore AK, Allix K, Bissyandé TF, Klein J (2021c) Dexray: a simple, yet effective deep learning approach to android malware detection based on image representation of bytecode. In: Wang G, Ciptadi A, Ahmadzadeh A (eds) Deployable machine learning for security defense, springer international publishing, Cham, pp 81–106.

[18]

Demšar J Statistical comparisons of classifiers over multiple data sets J Mach Learn Res 2006 7 1-30

[19]

Dhalaria M, Gandotra E (2020) Android malware detection using chi-square feature selection and ensemble learning method. In: 2020 Sixth international conference on parallel, distributed and grid computing (PDGC), pp 36–41.

[20]

Ding Y, Zhang X, Hu J, Xu W (2020) Android malware detection method based on bytecode image. J Ambient Intell Humanized Comput:1–10

[21]

Dong X, Yu Z, Cao W, Shi Y, and Ma Q A survey on ensemble learning Frontiers Comput Sci 2020 14 2 241-258

[22]

Fereidooni H, Conti M, Yao D, Sperduti A (2016) Anastasia: android malware detection using static analysis of applications. In: 2016 8th IFIP international conference on new technologies, mobility and security (NTMS), pp 1–5.

[23]

Freund Y and Schapire REA decision-theoretic generalization of on-line learning and an application to boostingJ Comput Syst Sci1997551119-139https://doi.org/10.1006/jcss.1997.1504, https://www.sciencedirect.com/science/article/pii/S002200009791504X

[24]

Friedman MThe use of ranks to avoid the assumption of normality implicit in the analysis of varianceJ Amer Stat Assoc193732200675-701https://doi.org/10.1080/01621459.1937.10503522

[25]

Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Annals Stat:1189–1232

[26]

Garcia J, Hammad M, Malek S (2018) Lightweight, obfuscation-resilient detection and family identification of android malware. ACM Trans Softw Eng Methodol, vol 26(3).

[27]

Huang TH, Kao H (2018) R2-d2: color-inspired convolutional neural network (cnn)-based android malware detections. In: 2018 IEEE international conference on big data (big data), pp 2633–2642.

[28]

Hurier M, Allix K, Bissyandé TF, Klein J, Le Traon Y (2016) On the lack of consensus in anti-virus decisions: metrics and insights on building ground truths of android malware. In: Proceedings of the 13th international conference on detection of intrusions and malware, and vulnerability assessment - vol 9721, Springer-Verlag, Berlin, Heidelberg, DIMVA 2016, pp 142–162.

[29]

Idrees F, Rajarajan M, Conti M, Chen TM, Rahulamathavan Y (2017) Pindroid: a novel android malware detection system using ensemble learning methods, vol 68, pp 36–46., https://www.sciencedirect.com/science/article/pii/S0167404817300640

[30]

Kaspersky (2021) Kaspersky security network. https://securelist.com/it-threat-evolution-q1-2021-mobile-statistics/102547/. Accessed 10 June 2021

[31]

Mariconti E, Onwuzurike L, Andriotis P, De Cristofaro E, Ross G, Stringhini G (2017) Mamadroid: detecting android malware by building markov chains of behavioral models. In: ISOC network and distributed systems security symposiym (NDSS), San Diego, CA

[32]

Miller B, Kantchelian A, Tschantz MC, Afroz S, Bachwani R, Faizullabhoy R, Huang L, Shankar V, Wu T, Yiu G, Joseph AD, Tygar JD (2016) Reviewer integration and performance measurement for malware detection. In: Caballero J, Zurutuza U, Rodríguez RJ (eds) Detection of intrusions and malware, and vulnerability assessment. Springer international publishing, Cham, pp 122–141

[33]

Milosevic N, Dehghantanha A, and Choo KKRMachine learning aided android malware classificationComput Electr Eng201761266-274https://doi.org/10.1016/j.compeleceng.2017.02.013, https://www.sciencedirect.com/science/article/pii/S0045790617303087

[34]

Nemenyi PB (1963) Distribution-free multiple comparisons. Princeton University

[35]

Onwuzurike L, Mariconti E, Andriotis P, Cristofaro ED, Ross G, and Stringhini GMamadroid: detecting android malware by building markov chains of behavioral models (extended version)ACM Trans Priv Secur201922214:1-14:34https://doi.org/10.1145/3313391

[36]

Palumbo P, Sayfullina L, Komashinskiy D, Eirola E, and Karhunen JA pragmatic android malware detection procedureComput Secur201770689-701https://doi.org/10.1016/j.cose.2017.07.013, https://www.sciencedirect.com/science/article/pii/S0167404817301542

[37]

Parab S and Bhalerao S Choosing statistical test Int J Ayurveda Res 2010 1 3 187

[38]

Pendlebury F, Pierazzi F, Jordaney R, Kinder J, Cavallaro L (2019) TESSERACT: eliminating experimental bias in malware classification across space and time. In: 28th USENIX security symposium (USENIX security 19), USENIX association, Santa Clara, CA, pp 729–746. https://www.usenix.org/conference/usenixsecurity19/presentation/pendlebury

[39]

Perinetti G Statips part i: choosing statistical test when dealing with differences South European J Orthodontics Dentofacial Res 2016 3 1 3-4

[40]

Rossow C, Dietrich CJ, Grier C, Kreibich C, Paxson V, Pohlmann N, Bos H, Steen VM (2012) Prudent practices for designing malware experiments: status quo and outlook. In: 2012 IEEE symposium on security and privacy, pp 65–79.

[41]

Sagi O and Rokach L Ensemble learning: a survey Wiley Interdisciplinary Rev Data Mining Knowl Discover 2018 8 4 e1249

[42]

Salem A, Banescu S, Pretschner A (2021) Maat: automatically analyzing virustotal for accurate labeling and effective malware detection. ACM Trans Priv Secur, vol 24(4).

[43]

Sebastián M, Rivera R, Kotzias P, Caballero J (2016) Avclass: a tool for massive malware labeling. In: International symposium on research in attacks, intrusions, and defenses, Springer, pp 230-253

[44]

Sheldon MR, Fillyaw MJ, and Thompson WD The use and interpretation of the friedman test in the analysis of ordinal-scale data in repeated measures designs Physiother Res Int 1996 1 4 221-228

[45]

Sun T, Daoudi N, Allix K, Bissyandé TF (2021) Android malware detection: looking beyond dalvik bytecode. In: Proceedings of the 36th IEEE/ACM international conference on automated software engineering workshops, ASE ’21

[46]

Wang J, Jing Q, Gao J, Qiu X (2020) Sedroid: a robust android malware detector using selective ensemble learning. In: 2020 IEEE wireless communications and networking conference (WCNC), pp 1–5.

[47]

Wang X, Zhang D, Su X, Li W (2017) Mlifdect: android malware detection based on parallel machine learning and information fusion. Secur Commun Netw, vol 2017

[48]

Wolpert DH Stacked generalization Neural Netw 1992 5 2 241-259

[49]

Wu Y, Li X, Zou D, Yang W, Zhang X, Jin H (2019) Malscan: fast market-wide mobile malware scanning by social-network centrality analysis. In: 2019 34th IEEE/ACM international conference on automated software engineering (ASE), pp 139–150

[50]

Wu D, Mao C, Wei T, Lee H, Wu K (2012) Droidmat: android malware detection through manifest and api calls tracing. In: 2012 Seventh asia joint conference on information security, pp 62–69.

[51]

Xu J, Li Y, Deng RH (2021) Differential training: a generic framework to reduce label noises for android malware detection. In: Proceeding of network and distributed system security symposium (NDSS)

[52]

Yerima SY, Sezer S, Muttik I (2014) Android malware detection using parallel machine learning classifiers. In: 2014 Eighth international conference on next generation mobile apps, services and technologies, pp 37–42.

[53]

Yerima SY, Sezer S, and Muttik I High accuracy android malware detection using ensemble learning IET Inf Secur 2015 9 6 313-320

[54]

Zhang X, Jin Z (2016) A new semantics-based android malware detection. In: 2016 2nd IEEE international conference on computer and communications (ICCC), pp 1412–1416.

[55]

Zhang W, Ren H, Jiang Q, Zhang K (2015) Exploring feature extraction and elm in malware detection for android devices. In: Hu X, Xia Y, Zhang Y, Zhao D (eds) Advances in neural networks – ISNN 2015, Springer international publishing, Cham, pp 489-498

[56]

Zhao Y, Li L, Wang H, Cai H, Bissyandé TF, Klein J, Grundy J (2021) On the impact of sample duplication in machine-learning-based android malware detection. ACM Trans Softw Eng Methodol, vol 30(3).

[57]

Zhao C, Wang C, Zheng W (2019) Android malware detection based on sensitive permissions and apis. In: International conference on security and privacy in new computing environments, Springer, pp 105–113

[58]

Zhao C, Zheng W, Gong L, Zhang M, Wang C (2018) Quick and accurate android malware detection based on sensitive apis. In: 2018 IEEE international conference on smart internet of things (SmartIoT), pp 143–148.

[59]

Zhu H, Li Y, Li R, Li J, You Z, Song H (2020) Sedmdroid: an enhanced stacking ensemble of deep learning framework for android malware detection. IEEE Trans Netw Sci Eng:1–1.

Cited By

Chen JHuang CHan J(2024)VioDroid-Finder: automated evaluation of compliance and consistency for Android appsEmpirical Software Engineering10.1007/s10664-024-10470-829:3Online publication date: 3-May-2024
https://dl.acm.org/doi/10.1007/s10664-024-10470-8
Daoudi NAllix KBissyandé TKlein JJust RFraser G(2023)Guided Retraining to Enhance the Detection of Difficult Android MalwareProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598123(1131-1143)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598123

Index Terms

Assessing the opportunity of combining state-of-the-art Android malware detectors
1. Social and professional topics

Index terms have been assigned to the content through auto-classification.

Recommendations

Empirical assessment of machine learning-based malware detectors for Android

To address the issue of malware detection through large sets of applications, researchers have recently started to investigate the capabilities of machine-learning techniques for proposing effective approaches. So far, several promising results were ...
Smart malware detection on Android

Nowadays, because of its increased popularity, Android is target to a growing number of attacks and malicious applications, with the purpose of stealing private information and consuming credit by subscribing to premium services. Most of the current ...
Testing malware detectors
ISSTA '04: Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis

In today's interconnected world, malware, such as worms and viruses, can cause havoc. A malware detector (commonly known as virus scanner) attempts to identify malware. In spite of the importance of malware detectors, there is a dearth of testing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Empirical Software Engineering

Empirical Software Engineering Volume 28, Issue 2

Mar 2023

1389 pages

ISSN:1382-3256

Issue’s Table of Contents

© The Author(s) 2022.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 24 December 2022

Accepted: 09 October 2022

Author Tags

Qualifiers

Research-article

Funding Sources

Fonds National de la Recherche Luxembourg
European Union’s Horizon 2020 research and innovation program SPARTA project
Université du Luxembourg HitDroid project
Luxembourg Ministry of Foreign and European Affairs

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen JHuang CHan J(2024)VioDroid-Finder: automated evaluation of compliance and consistency for Android appsEmpirical Software Engineering10.1007/s10664-024-10470-829:3Online publication date: 3-May-2024
https://dl.acm.org/doi/10.1007/s10664-024-10470-8
Daoudi NAllix KBissyandé TKlein JJust RFraser G(2023)Guided Retraining to Enhance the Detection of Difficult Android MalwareProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598123(1131-1143)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598123

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents