Abstract
During the last years, malware writers have been using several techniques to evade detection. One of the most common techniques employed by the anti-virus industry is signature scanning. This method requires the end-host to compare files against a database that should contain signatures for each malware sample. In order to allow their creations to bypass these protection systems, programmers use software encryption tools and code obfuscation techniques to hide the actual behaviour of their malicious programs. One of these techniques is packing, a method that encrypts the real code of the executable and places it as data in a new executable that contains an unpacking routine. In previous work, we designed and implemented an anomaly detector based on PE structural characteristics and heuristic values, and we were able to decide whether an executable was packed or not. We stated that this detection system could serve as a filtering step for a generic and time consuming unpacking phase. In this paper, we improve that system applying a data reduction algorithm to our representation of normality (i.e., not packed executables), finding similarities among executables and grouping them to form consistent clusters that reduce the amount of comparisons needed. We show that this improvement reduces drastically the processing time, while maintaining detection and false positive rates stable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Kaspersky: Kaspersky security bulletin: Statistics 2008 (2008), http://www.viruslist.com/en/analysis?pubid=204792052
McAfee Labs: Mcafee whitepaper: The good, the bad, and the unknown (2011), http://www.mcafee.com/us/resources/white-papers/wp-good-bad-unknown.pdf
PEiD: PEiD webpage (2010), http://www.peid.info/
Faster Universal Unpacker: (1999), http://code.google.com/p/fuu/
Morgenstern, M., Pilz, H.: Useful and useless statistics about viruses and anti-virus programs. In: Proceedings of the CARO Workshop (2010), http://www.f-secure.com/weblog/archives/Maik_Morgenstern_Statistics.pdf
Babar, K., Khalid, F.: Generic unpacking techniques. In: Proceedings of the 2nd International Conference on Computer, Control and Communication (IC4), pp. 1–6. IEEE (2009)
Data Rescue: Universal PE Unpacker plug-in, http://www.datarescue.com/idabase/unpack_pe
Stewart, J.: Ollybone: Semi-automatic unpacking on ia-32. In: Proceedings of the 14th DEF CON Hacking Conference (2006)
Rolles, R.: Unpacking virtualization obfuscators. In: Proceedings of the 3rd USENIX Workshop on Offensive Technologies (WOOT) (2009)
Böhne, L.: Pandora’s bochs: Automatic unpacking of malware. PhD thesis (2008)
Royal, P., Halpin, M., Dagon, D., Edmonds, R., Lee, W.: Polyunpack: Automating the hidden-code extraction of unpack-executing malware. In: Proceedings of the 2006 Annual Computer Security Applications Conference (ACSAC), pp. 289–300 (2006)
Kang, M., Poosankam, P., Yin, H.: Renovo: A hidden code extractor for packed executables. In: Proceedings of the 2007 ACM Workshop on Recurring Malcode, pp. 46–53 (2007)
Martignoni, L., Christodorescu, M., Jha, S.: Omniunpack: Fast, generic, and safe unpacking of malware. In: Proceedings of the 2007 Annual Computer Security Applications Conference (ACSAC), pp. 431–441 (2007)
Sharif, M., Yegneswaran, V., Saidi, H., Porras, P., Lee, W.: Eureka: A Framework for Enabling Static Malware Analysis. In: Jajodia, S., Lopez, J. (eds.) ESORICS 2008. LNCS, vol. 5283, pp. 481–500. Springer, Heidelberg (2008)
Danielescu, A.: Anti-debugging and anti-emulation techniques. CodeBreakers Journal 5(1) (2008), http://www.codebreakers-journal.com/
Cesare, S.: Linux anti-debugging techniques, fooling the debugger (1999), http://vx.netlux.org/lib/vsc04.html
Julus, L.: Anti-debugging in WIN32 (1999), http://vx.netlux.org/lib/vlj05.html
Shafiq, M.Z., Tabish, S.M., Mirza, F., Farooq, M.: PE-Miner: Mining Structural Information to Detect Malicious Executables in Realtime. In: Kirda, E., Jha, S., Balzarotti, D. (eds.) RAID 2009. LNCS, vol. 5758, pp. 121–141. Springer, Heidelberg (2009)
Shafiq, M., Tabish, S., Farooq, M.: PE-Probe: Leveraging Packer Detection and Structural Information to Detect Malicious Portable Executables. In: Proceedings of the 2009 Virus Bulletin Conference (VB), pp. 1–10 (2009)
Perdisci, R., Lanzi, A., Lee, W.: McBoost: Boosting scalability in malware collection and analysis using statistical classification of executables. In: Proceedings of the 2008 Annual Computer Security Applications Conference (ACSAC), pp. 301–310 (2008)
Ugarte-Pedrero, X., Santos, I., Bringas, P.G.: Structural Feature Based Anomaly Detection for Packed Executable Identification. In: Herrero, Á., Corchado, E. (eds.) CISIS 2011. LNCS, vol. 6694, pp. 230–237. Springer, Heidelberg (2011)
Kent, J.: Information gain and a general measure of correlation. Biometrika 70(1), 163–173 (1983)
Kumar, V.: An introduction to cluster analysis for data mining. Computer Science Department, University of Minnesota, USA (2000)
Heyer, L., Kruglyak, S., Yooseph, S.: Exploring expression data: identification and analysis of coexpressed genes. Genome Research 9(11), 1106–1115 (1999)
VX Heavens, http://vx.netlux.org/
Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the International Joint Conference on Artificial Intelligence, vol. 14, pp. 1137–1145 (1995)
VMware: (2011), http://www.vmware.com
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ugarte-Pedrero, X., Santos, I., Bringas, P.G. (2012). Boosting Scalability in Anomaly-Based Packed Executable Filtering. In: Wu, CK., Yung, M., Lin, D. (eds) Information Security and Cryptology. Inscrypt 2011. Lecture Notes in Computer Science, vol 7537. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34704-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-34704-7_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34703-0
Online ISBN: 978-3-642-34704-7
eBook Packages: Computer ScienceComputer Science (R0)