research-article

Tightly-coupled self-debugging software protection

Authors:

Stijn Volckaert,

Bjorn De SutterAuthors Info & Claims

SSPREW '16: Proceedings of the 6th Workshop on Software Security, Protection, and Reverse Engineering

Article No.: 7, Pages 1 - 10

https://doi.org/10.1145/3015135.3015142

Published: 05 December 2016 Publication History

Abstract

Existing anti-debugging protections are relatively weak. In existing self-debugger approaches, a custom debugger is attached to the main application, of which the control flow is obfuscated by redirecting it through the debugger. The coupling between the debugger and the main application is then quite loose, and not that hard to break by an attacker. In the tightly-coupled self-debugging technique proposed in this paper, full code fragments are migrated from the application to the debugger, making it harder for the attacker to reverse-engineer the program and to deconstruct it into the original unprotected program to attach a debugger or to collect traces. We evaluate a prototype implementation on three complex, real-world Android use cases and present the results of tests conducted by professional penetration testers.

References

[1]

F. Bellard. Qemu, a fast and portable dynamic translator. In USENIX Annual Technical Conference, FREENIX Track, pages 41--46, 2005.

Digital Library

[2]

D. Bornstein. Dalvik VM internals. In Google I/O Developer Conference, volume 23, pages 17--30, 2008.

[3]

A. Cabutto, P. Falcarin, B. Abrath, B. Coppens, and B. D. Sutter. Software protection with code mobility. In Proceedings of the Second ACM Workshop on Moving Target Defense, MTD 2015, Denver, Colorado, USA, October 12, 2015, pages 95--103, 2015.

Digital Library

[4]

Carbon Monoxide. Scyllahide. https://bitbucket.org/NtQuery/scyllahide.

[5]

J. N. Christian Collberg. Surreptitious Software: Obfuscation, Watermarking, and Tamperproofing for Software Protection. Addison-Wesley Professional, 2009.

Digital Library

[6]

C. Collberg, C. Thomborson, and D. Low. Manufacturing cheap, resilient, and stealthy opaque constructs. In Proceedings of the 25th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pages 184--196. ACM, 1998.

Digital Library

[7]

G. Developers. GDB: The GNU Project Debugger. https://www.gnu.org/software/gdb/.

[8]

C. Eagle. The IDA pro book: the unofficial guide to the world's most popular disassembler. No Starch Press, 2011.

Digital Library

[9]

F. Eigler, V. Prasad, W. Cohen, H. Nguyen, and M. Hunt. Architecture of systemtap: a Linux trace/probe tool. http://sourceware.org/systemtap/archpaper.pdf, 2005.

[10]

P. Ferrie. The "ultimate" anti-debugging reference. http://anti-reversing.com/Downloads/Anti-Reversing/The_Ultimate_Anti-Reversing_Reference.pdf, April 2011.

[11]

Ferrit. OllyExt 1.8. https://tuts4you.com/download.php?view.3392.

[12]

B. Gregg. DTrace Tools. http://www.brendangregg.com/dtrace.html.

[13]

jean. hack.lu CTF - Challenge 12 WriteUp. Technical report, Sogeti ESEC Lab, 2010.

[14]

C. Linn and S. Debray. Obfuscation of executable code to improve resistance to static disassembly. In Proceedings of the 10th ACM conference on Computer and communications security, pages 290--299. ACM, 2003.

Digital Library

[15]

Linux Programmer's Manual. dlopen(3) - Linux man page.

[16]

Linux Programmer's Manual. fork(2) - Linux manual page.

[17]

Linux Programmer's Manual. proc(5) - Linux manual page.

[18]

mrexodia. TitanHide. https://bitbucket.org/mrexodia/titanhide.

[19]

N. Nethercote and J. Seward. Valgrind: a framework for heavyweight dynamic binary instrumentation. ACM Sigplan notices, 2007.

Digital Library

[20]

Pellsson. Starcraft 2 anti-debugging. http://www.bhfiles.com/files/StarCraft%20II/Wings%20of%20Liberty%20%28Beta%29/0x1337.org%20-%20SCII%20Anti-Debug.htm, March 2010.

[21]

M. Schallner. Beginners guide to basic linux anti anti debugging techniques. CodeBreakers Magazine, 2006.

[22]

D. Seal. ARM architecture reference manual. Pearson Education, 2001.

Digital Library

[23]

T. Shields. Anti-debugging - a developers view. Technical report, Veracode, 2009.

[24]

Ubuntu Wiki. SecurityTeam/Roadmap/KernelHardening - Ubuntu Wiki. https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening{\#}ptrace{\_}Protection.

[25]

L. Van Put, D. Chanet, B. De Bus, B. De Sutter, and K. De Bosschere. Diablo: a reliable, retargetable and extensible link-time rewriting framework. In Proceedings of the Fifth IEEE International Symposium on Signal Processing and Information Technology, 2005., pages 7--12. IEEE, 2005.

[26]

C. Wang, J. Davidson, J. Hill, and J. Knight. Protection of software-based survivability mechanisms. In Dependable Systems and Networks, 2001. DSN 2001. International Conference on, pages 193--202. IEEE, 2001.

Digital Library

[27]

Yama. ptrace_scope. https://www.kernel.org/doc/Documentation/security/Yama.txt.

Cited By

Faingnaert TVan Iseghem WDe Sutter BSchrittwieser SIanni M(2024)K-Hunt++: Improved Dynamic Cryptographic Key ExtractionProceedings of the 2024 Workshop on Research on offensive and defensive techniques in the context of Man At The End (MATE) attacks10.1145/3689934.3690818(22-29)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3689934.3690818
Pizzolotto DBerlato SCeccato M(2024)Mitigating Debugger-based Attacks to Java Applications with Self-debuggingACM Transactions on Software Engineering and Methodology10.1145/363197133:4(1-38)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3631971
Canavese DRegano LLioy A(2023)Computer-Aided Reverse Engineering of Protected SoftwareDigital Sovereignty in Cyber Security: New Challenges in Future Vision10.1007/978-3-031-36096-1_1(3-15)Online publication date: 16-Jun-2023
https://doi.org/10.1007/978-3-031-36096-1_1
Show More Cited By

Tightly-coupled self-debugging software protection
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis

Recommendations

Mitigating Debugger-based Attacks to Java Applications with Self-debugging
Java bytecode is a quite high-level language and, as such, it is fairly easy to analyze and decompile with malicious intents, e.g., to tamper with code and skip license checks. Code obfuscation was a first attempt to mitigate malicious reverse-engineering ...
Malware Detection Method Focusing on Anti-debugging Functions
CANDAR '14: Proceedings of the 2014 Second International Symposium on Computing and Networking

Malware has received much attention in recent years. Antivirus software is widely used as a countermeasure against malware. However, some kinds of malware can evade detection by antivirus software, hence, a new detection method is required. In this ...
Software Protection through Anti-Debugging

This column presents the reverse engineering battle from an anti-debugging perspective.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

SSPREW '16: Proceedings of the 6th Workshop on Software Security, Protection, and Reverse Engineering

December 2016

85 pages

ISBN:9781450348416

DOI:10.1145/3015135

General Chair:
J. Todd McDonald
University of South Alabama
,
Program Chairs:
Mila Dalla Preda
University of Verona, Italy
,
Natalia Stakhanova
University of New Brunswick, Canada

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Seventh Framework Programme

Conference

SSPREW '16

SSPREW '16: Software Security, Protection, and Reverse Engineering Workshop

December 5 - 6, 2016

California, Los Angeles, USA

Acceptance Rates

Overall Acceptance Rate 6 of 13 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
181
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)6

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Faingnaert TVan Iseghem WDe Sutter BSchrittwieser SIanni M(2024)K-Hunt++: Improved Dynamic Cryptographic Key ExtractionProceedings of the 2024 Workshop on Research on offensive and defensive techniques in the context of Man At The End (MATE) attacks10.1145/3689934.3690818(22-29)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3689934.3690818
Pizzolotto DBerlato SCeccato M(2024)Mitigating Debugger-based Attacks to Java Applications with Self-debuggingACM Transactions on Software Engineering and Methodology10.1145/363197133:4(1-38)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3631971
Canavese DRegano LLioy A(2023)Computer-Aided Reverse Engineering of Protected SoftwareDigital Sovereignty in Cyber Security: New Challenges in Future Vision10.1007/978-3-031-36096-1_1(3-15)Online publication date: 16-Jun-2023
https://doi.org/10.1007/978-3-031-36096-1_1
Banescu SValenzuela SGuggenmos MAhmadvand MPretschner A(2021)Dynamic Taint Analysis versus Obfuscated Self-CheckingProceedings of the 37th Annual Computer Security Applications Conference10.1145/3485832.3485926(182-193)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.1145/3485832.3485926
Abrath BCoppens BBroeck JWyseur BCabutto AFalcarin PSutter B(2020)Code Renewability for Native Software ProtectionACM Transactions on Privacy and Security10.1145/340489123:4(1-31)Online publication date: 25-Aug-2020
https://dl.acm.org/doi/10.1145/3404891
Abrath BCoppens BNevolin IDe Sutter B(2020)Resilient Self-Debugging Software Protection2020 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)10.1109/EuroSPW51379.2020.00088(606-615)Online publication date: Sep-2020
https://doi.org/10.1109/EuroSPW51379.2020.00088
Berlato SCeccato M(2020)A large-scale study on the adoption of anti-debugging and anti-tampering protections in android appsJournal of Information Security and Applications10.1016/j.jisa.2020.10246352(102463)Online publication date: Jun-2020
https://doi.org/10.1016/j.jisa.2020.102463
Van den Broeck JCoppens BDe Sutter B(2020)Obfuscated integration of software protectionsInternational Journal of Information Security10.1007/s10207-020-00494-8Online publication date: 18-Mar-2020
https://doi.org/10.1007/s10207-020-00494-8
Kiperberg MLeon RResh AAlgawi AZaidenberg N(2019)Hypervisor-Based Protection of CodeIEEE Transactions on Information Forensics and Security10.1109/TIFS.2019.289457714:8(2203-2216)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1109/TIFS.2019.2894577
Ceccato MTonella PBasile CFalcarin PTorchiano MCoppens BDe Sutter B(2019)Understanding the behaviour of hackers while performing attack tasks in a professional setting and in a public challengeEmpirical Software Engineering10.1007/s10664-018-9625-624:1(240-286)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1007/s10664-018-9625-6
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents