Matryoshka: strengthening software protection via nested virtual machines
Abstract
The use of virtual machine technology has become a popular approach for defending software applications from attacks by adversaries that wish to compromise the integrity and confidentiality of an application. In addition to providing some inherent obfuscation of the execution of the software application, the use of virtual machine technology can make both static and dynamic analysis more difficult for the adversary. However, a major point of concern is the protection of the virtual machine itself. The major weakness is that the virtual machine presents a inviting target for the adversary. If an adversary can render the virtual machine ineffective, they can focus their energy and attention on the software application. One possible approach is to protect the virtual machine by composing or nesting virtualization layers to impart virtual machine protection techniques to the inner virtual machines "closest" to the software application. This paper explores the concept and feasibility of nested virtualization for software protection using a high-performance software dynamic translation system. Using two metrics for measuring the strength of protection, the preliminary results show that nesting virtual machines can strengthen protection of the software application. While the nesting of virtual machines does increase run-time overhead, initial results indicate that with careful application of the technique, run-time overhead could be reduced to reasonable levels.
References
[1]
Oreans Technologies, "Themida," http://oreans.com/themida.php, 2009.
[2]
VMProtect Software, "VMProtect," http://vmpsoft.com/, 2008.
[3]
S. Ghosh, J. D. Hiser, and J. W. Davidson, "A secure and robust approach to software tamper resistance," in Proceedings of the 12th International Conference on Information Hiding, ser. IH'10. Berlin, Heidelberg: Springer-Verlag, 2010, pp. 33--47. {Online}. Available: http://dl.acm.org/citation.cfm?id=1929304.1929307
[4]
Oreons Technology, "CodeVirtualizer," http://oreans.com/codevirtualizer.php, 2009.
[5]
S. Zdancewic, L. Zheng, N. Nystrom, and A. C. Myers, "Secure program partitioning," ACM Trans. Comput. Syst., vol. 20, no. 3, pp. 283--328, Aug. 2002. {Online}. Available: http://doi.acm.org/10.1145/566340.566343
[6]
S. H. K. Narayanan, M. Kandemir, and R. Brooks, "Performance aware secure code partitioning," in Proceedings of the Conference on Design, Automation and Test in Europe, ser. DATE '07. San Jose, CA, USA: EDA Consortium, 2007, pp. 1122--1127. {Online}. Available: http://dl.acm.org/citation.cfm?id=1266366.1266609
[7]
D. Søndergaard, C. W. Probst, C. D. Jensen, and R. R. Hansen, "Program partitioning using dynamic trust models," in Proceedings of the 4th International Conference on Formal Aspects in Security and Trust, ser. FAST'06. Berlin, Heidelberg: Springer-Verlag, 2007, pp. 170--184. {Online}. Available: http://dl.acm.org/citation.cfm?id=1777688.1777700
[8]
T. Zhang, S. Pande, A. dos Santos, and F. J. Bruecklmayr, "Leakage-proof program partitioning," in Proceedings of the 2002 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, ser. CASES '02. New York, NY, USA: ACM, 2002, pp. 136--145. {Online}. Available: http://doi.acm.org/10.1145/581630.581651
[9]
K. Scott, N. Kumar, S. Velusamy, B. Childers, J. W. Davidson, and M. L. Soffa, "Retargetable and reconfigurable software dynamic translation," in Proceedings of the International Symposium on Code Generation and Optimization: Feedback-directed and Runtime Optimization, ser. CGO '03. Washington, DC, USA: IEEE Computer Society, 2003, pp. 36--47. {Online}. Available: http://dl.acm.org/citation.cfm?id=776261.776265
[10]
S. Ghosh, J. D. Hiser, and J. W. Davidson, "What's the pointisa?" in Proceedings of the 2Nd ACM Workshop on Information Hiding and Multimedia Security, ser. IH&MMSec '14. New York, NY, USA: ACM, 2014, pp. 23--34. {Online}. Available: http://doi.acm.org/10.1145/2600918.2600928
[11]
L. Van Put, D. Chanet, B. De Bus, B. De Sutter, and K. De Bosschere, "Diablo: a reliable, retargetable and extensible link-time rewriting framework," in Proceedings of the Fifth IEEE International Symposium on Signal Processing and Information Technology, Dec 2005, pp. 7--12.
[12]
S. Chow, P. A. Eisen, H. Johnson, and P. C. v. Oorschot, "Whitebox cryptography and an AES implementation," in SAC '02: Revised Papers from the 9th Annual International Workshop on Selected Areas in Cryptography. London, UK: Springer-Verlag, 2003, pp. 250--270.
[13]
T. J. McCabe, "A complexity measure," IEEE Trans. Softw. Eng., vol. 2, no. 4, pp. 308--320, Jul. 1976. {Online}. Available: http://dx.doi.org/10.1109/TSE.1976.233837
[14]
B. Anckaert, M. Madou, B. De Sutter, B. De Bus, K. De Bosschere, and B. Preneel, "Program obfuscation: A quantitative approach," in Proceedings of the 2007 ACM Workshop on Quality of Protection, ser. QoP '07. New York, NY, USA: ACM, 2007, pp. 15--20. {Online}. Available: http://doi.acm.org/10.1145/1314257.1314263
[15]
M. Ceccato, M. Di Penta, J. Nagra, P. Falcarin, F. Ricca, M. Torchiano, and P. Tonella, "Towards experimental evaluation of code obfuscation techniques," in QoP '08: Proceedings of the 4th ACM Workshop on Quality of Protection. New York, NY, USA: ACM, 2008, pp. 39--46. {Online}. Available: http://doi.acm.org/10.1145/1456362.1456371
[16]
J. D. Hiser, D. Williams, A. Filipi, J. W. Davidson, and B. R. Childers, "Evaluating fragment construction policies for sdt systems," in Proceedings of the 2nd International Conference on Virtual Execution Environments, ser. VEE '06. New York, NY, USA: ACM, 2006, pp. 122--132. {Online}. Available: http://doi.acm.org/10.1145/1134760.1134778
[17]
C. Collberg and J. Nagra, Surreptitious software: obfuscation, watermarking, and tamperproofing for software protection. Pearson Education, 2009.
[18]
B. Anckaert, M. Jakubowski, and R. Venkatesan, "Proteus: Virtualization for diversified tamper-resistance," in Proceedings of the ACM Workshop on Digital Rights Management, ser. DRM '06. New York, NY, USA: ACM, 2006, pp. 47--58. {Online}. Available: http://doi.acm.org/10.1145/1179509.1179521
[19]
T. Garfinkel, B. Pfaff, J. Chow, M. Rosenblum, and D. Boneh, "Terra: A virtual machine-based platform for trusted computing," in Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, ser. SOSP '03. New York, NY, USA: ACM, 2003, pp. 193--206. {Online}. Available: http://doi.acm.org/10.1145/945445.945464
[20]
X. Chen, T. Garfinkel, E. C. Lewis, P. Subrahmanyam, C. A. Waldspurger, D. Boneh, J. Dwoskin, and D. R. Ports, "Overshadow: A virtualization-based approach to retrofitting protection in commodity operating systems," in Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems, ser. ASPLOS XIII. New York, NY, USA: ACM, 2008, pp. 2--13. {Online}. Available: http://doi.acm.org/10.1145/1346281.1346284
[21]
R. Rolles, "Unpacking virtualization obfuscators," in Proceedings of the 3rd USENIX Conference on Offensive Technologies, ser. WOOT'09. Berkeley, CA, USA: USENIX Association, 2009, pp. 1--1. {Online}. Available: http://dl.acm.org/citation.cfm?id=1855876.1855877
[22]
K. Coogan, G. Lu, and S. Debray, "Deobfuscation of virtualization-obfuscated software: A semantics-based approach," in Proceedings of the 18th ACM Conference on Computer and Communications Security, ser. CCS '11. New York, NY, USA: ACM, 2011, pp. 275--284. {Online}. Available: http://doi.acm.org/10.1145/2046707.2046739
Index Terms
- Matryoshka: strengthening software protection via nested virtual machines
Recommendations
Matryoshka: Strengthening Software Protection via Nested Virtual Machines
SPRO '15: Proceedings of the 2015 IEEE/ACM 1st International Workshop on Software ProtectionThe use of virtual machine technology has become a popular approach for defending software applications from attacks by adversaries that wish to compromise the integrity and confidentiality of an application. In addition to providing some inherent ...
Generalized Matryoshka: Computational Design of Nesting Objects
This paper generalizes the self-similar nesting of Matryoshka dolls "Russian nesting dolls" to arbitrary solid objects. We introduce the problem of finding the largest scale replica of an object that nests inside itself. Not only should the nesting ...
Matryoshka: Fuzzing Deeply Nested Branches
CCS '19: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications SecurityGreybox fuzzing has made impressive progress in recent years, evolving from heuristics-based random mutation to approaches for solving individual branch constraints. However, they have difficulty solving path constraints that involve deeply nested ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sponsors
- ACM: Association for Computing Machinery
- SIGSOFT: ACM Special Interest Group on Software Engineering
- IEEE-CS\DATC: IEEE Computer Society
- TCSE: IEEE Computer Society's Tech. Council on Software Engin.
Publisher
IEEE Press
Publication History
Published: 16 May 2015
Check for updates
Qualifiers
- Research-article
Conference
Acceptance Rates
Overall Acceptance Rate 8 of 14 submissions, 57%
Upcoming Conference
CCS '25
- Sponsor:
- sigsac
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 77Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in