DDGF: Dynamic Directed Greybox Fuzzing with Path Profiling
Pages 832 - 843
Abstract
Coverage-Guided Fuzzing (CGF) has become the most popular and effective method for vulnerability detection. It is usually designed as an automated “black-box” tool. Security auditors start it and then just wait for the results. However, after a period of testing, CGF struggles to find new coverage gradually, thus making it inefficient. It is difficult for users to explain reasons that prevent fuzzing from making further progress and to determine whether the existing coverage is sufficient. In addition, there is no way to interact and direct the fuzzing process. In this paper, we design the dynamic directed greybox fuzzing (DDGF) to facilitate collaboration between the user and fuzzer. By leveraging Ball-Larus path profiling algorithm, we propose two new techniques: dynamic introspection and dynamic direction. Dynamic introspection reveals the significant imbalance in the distribution of path frequency through encoding and decoding. Based on the insight from introspection, users can dynamically direct the fuzzer to focus testing on the selected paths in real time. We implement DDGF based on AFL++. Experiments on Magma show that DDGF is effective in helping the fuzzer to reproduce vulnerabilities faster, with up to 100x speedup and only 13% performance overhead. DDGF shows the great potential of human-in-the-loop for fuzzing.
References
[1]
2023. AFL. https://lcamtuf.coredump.cx/afl Accessed: 2023-12-01
[2]
2023. D3.js. https://d3js.org/ Accessed: 2023-12-01
[3]
2023. DARPA CGC Project. https://www.darpa.mil/program/cyber-grand-challenge Accessed: 2023-12-01
[4]
2023. DARPA CHESS Project. https://www.darpa.mil/program/computers-and-humans-exploring-software-security Accessed: 2023-12-01
[5]
2023. Fuzz-Introspector. https://github.com/ossf/fuzz-introspector Accessed: 2023-12-01
[6]
2023. LLVM-PGO. https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization Accessed: 2023-12-01
[7]
2023. MATE by Galois Inc. https://galois.com/project/mate/ Accessed: 2023-12-01
[8]
2023. Mechanical Phish. https://github.com/mechaphish Accessed: 2023-12-01
[9]
2023. Memory Mapping in Boost. https://www.boost.org/doc/libs/1_85_0/doc/html/interprocess/sharedmemorybetweenprocesses.html Accessed: 2023-12-01
[10]
2023. OSS-Fuzz. https://github.com/google/oss-fuzz Accessed: 2023-12-01
[11]
2023. OSS-Fuzz improvements in Google. https://security.googleblog.com/2023/02/taking-next-step-oss-fuzz-in-2023.html Accessed: 2023-12-01
[12]
Cornelius Aschermann, Sergej Schumilo, Ali Abbasi, and Thorsten Holz. 2020. Ijon: Exploring deep state spaces via fuzzing. In 2020 IEEE Symposium on Security and Privacy (SP). 1597–1612. https://doi.org/10.1109/SP40000.2020.00117
[13]
Jinsheng Ba, Marcel Böhme, Zahra Mirzamomen, and Abhik Roychoudhury. 2022. Stateful greybox fuzzing. In 31st USENIX Security Symposium (Usenix Security 22). 3255–3272.
[14]
Thomas Ball and James R Larus. 1996. Efficient path profiling. In Proceedings of the 29th Annual IEEE/ACM International Symposium on Microarchitecture. MICRO 29. 46–57. https://doi.org/10.1109/MICRO.1996.566449
[15]
Marcel Böhme, Cristian Cadar, and Abhik Roychoudhury. 2021. Fuzzing: Challenges and Reflections. IEEE Softw., 38, 3 (2021), 79–86. https://doi.org/10.1109/MS.2020.3016773
[16]
Marcel Böhme and Brandon Falk. 2020. Fuzzing: On the exponential cost of vulnerability discovery. In Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering. 713–724.
[17]
Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. 2017. Directed greybox fuzzing. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2329–2344. https://doi.org/10.1145/3133956.3134020
[18]
Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. 2016. Coverage-based greybox fuzzing as markov chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 1032–1043. https://doi.org/10.1109/TSE.2017.2785841
[19]
Joshua Bundt, Andrew Fasano, Brendan Dolan-Gavitt, William Robertson, and Tim Leek. 2022. Homo in Machina: Improving Fuzz Testing Coverage via Compartment Analysis. https://doi.org/10.48550/ARXIV.2212.11162
[20]
Marcel Böhme, Charaka Geethal, and Van-Thuan Pham. 2020. Human-In-The-Loop Automatic Program Repair. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). 274–285. https://doi.org/10.1109/ICST46399.2020.00036
[21]
Hongxu Chen, Yinxing Xue, Yuekang Li, Bihuan Chen, Xiaofei Xie, Xiuheng Wu, and Yang Liu. 2018. Hawkeye: Towards a desired directed grey-box fuzzer. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. 2095–2108. https://doi.org/10.1145/3243734.3243849
[22]
Peng Chen and Hao Chen. 2018. Angora: Efficient fuzzing by principled search. In 2018 IEEE Symposium on Security and Privacy (SP). 711–725. https://doi.org/10.1109/SP.2018.00046
[23]
Yaohui Chen, Peng Li, Jun Xu, Shengjian Guo, Rundong Zhou, Yulong Zhang, Tao Wei, and Long Lu. 2020. Savior: Towards bug-driven hybrid testing. In 2020 IEEE Symposium on Security and Privacy (SP). 1580–1596. https://doi.org/10.1109/SP40000.2020.00002
[24]
Zhengjie Du, Yuekang Li, Yang Liu, and Bing Mao. 2022. WindRanger: a directed greybox fuzzer driven by deviation basic blocks. In Proceedings of the 44th International Conference on Software Engineering. 2440–2451. https://doi.org/10.1145/3510003.3510197
[25]
Andrea Fioraldi, Dominik Maier, Heiko Eiß feldt, and Marc Heuse. 2020. AFL++: combining incremental steps of fuzzing research. In Proceedings of the 14th USENIX Conference on Offensive Technologies. 10–10.
[26]
Andrea Fioraldi, Dominik Christian Maier, Dongjia Zhang, and Davide Balzarotti. 2022. LibAFL: A Framework to Build Modular and Reusable Fuzzers. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 1051–1065. https://doi.org/10.1145/3548606.3560602
[27]
Shuitao Gan, Chao Zhang, Xiaojun Qin, Xuwen Tu, Kang Li, Zhongyu Pei, and Zuoning Chen. 2018. Collafl: Path sensitive fuzzing. In 2018 IEEE Symposium on Security and Privacy (SP). 679–696. https://doi.org/10.1109/SP.2018.00040
[28]
Wentao Gao, Van-Thuan Pham, Dongge Liu, Oliver Chang, Toby Murray, and Benjamin IP Rubinstein. 2023. Beyond the Coverage Plateau: A Comprehensive Study of Fuzz Blockers (Registered Report). In Proceedings of the 2nd International Fuzzing Workshop. 47–55.
[29]
Ahmad Hazimeh, Adrian Herrera, and Mathias Payer. 2020. Magma: A ground-truth fuzzing benchmark. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 4, 3 (2020), 1–29. https://doi.org/10.1145/3543516.3456276
[30]
Heqing Huang, Yiyuan Guo, Qingkai Shi, Peisen Yao, Rongxin Wu, and Charles Zhang. 2022. Beacon: Directed grey-box fuzzing with provable path pruning. In 2022 IEEE Symposium on Security and Privacy (SP). 36–50. https://doi.org/10.1109/SP46214.2022.9833751
[31]
Heqing Huang, Peisen Yao, CHIU Hung-Chun, Yiyuan Guo, and Charles Zhang. 2023. Titan: Efficient Multi-target Directed Greybox Fuzzing. In 2024 IEEE Symposium on Security and Privacy (SP). 59–59.
[32]
Tae Eun Kim, Jaeseung Choi, Kihong Heo, and Sang Kil Cha. 2023. $DAFL$: Directed Grey-box Fuzzing guided by Data Dependency. In 32nd USENIX Security Symposium (USENIX Security 23). 4931–4948.
[33]
George Klees, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. 2018. Evaluating fuzz testing. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security. 2123–2138. https://doi.org/10.1145/3243734.3243804
[34]
Caroline Lemieux and Koushik Sen. 2018. Fairfuzz: A targeted mutation strategy for increasing greybox fuzz testing coverage. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering. 475–485. https://doi.org/10.1145/3238147.3238176
[35]
Changhua Luo, Wei Meng, and Penghui Li. 2023. Selectfuzz: Efficient directed fuzzing with selective path exploration. In 2023 IEEE Symposium on Security and Privacy (SP). 2693–2707.
[36]
Dominik Maier and Lukas Seidel. 2021. Jmpscare: Introspection for binary-only fuzzing. In Workshop on Binary Analysis Research (BAR). 2021, 21. https://doi.org/10.14722/bar.2021.23003
[37]
Valentin JM Manès, Soomin Kim, and Sang Kil Cha. 2020. Ankou: Guiding grey-box fuzzing towards combinatorial difference. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 1024–1036. https://doi.org/10.1145/3377811.3380421
[38]
Valentin J.M. Manès, HyungSeok Han, Choongwoo Han, Sang Kil Cha, Manuel Egele, Edward J. Schwartz, and Maverick Woo. 2021. The Art, Science, and Engineering of Fuzzing: A Survey. IEEE Transactions on Software Engineering, 47, 11 (2021), 2312–2331. https://doi.org/10.1109/TSE.2019.2946563
[39]
Stefan Nagy and Matthew Hicks. 2019. Full-speed fuzzing: Reducing fuzzing overhead through coverage-guided tracing. In 2019 IEEE Symposium on Security and Privacy (SP). 787–802. https://doi.org/10.1109/SP.2019.00069
[40]
Van-Thuan Pham, Marcel Böhme, and Abhik Roychoudhury. 2020. AFLNet: a greybox fuzzer for network protocols. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). 460–465. https://doi.org/10.1109/ICST46399.2020.00062
[41]
Yan Shoshitaishvili, Michael Weissbacher, Lukas Dresel, Christopher Salls, Ruoyu Wang, Christopher Kruegel, and Giovanni Vigna. 2017. Rise of the HaCRS: Augmenting autonomous cyber reasoning systems with human assistance. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 347–362. https://doi.org/10.1145/3133956.3134105
[42]
Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshitaishvili, Christopher Kruegel, and Giovanni Vigna. 2016. Driller: Augmenting fuzzing through selective symbolic execution. In NDSS. 16, 1–16. https://doi.org/10.14722/ndss.2016.23368
[43]
Jinghan Wang, Yue Duan, Wei Song, Heng Yin, and Chengyu Song. 2019. Be sensitive and collaborative: Analyzing impact of coverage metrics in greybox fuzzing. In 22nd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2019). 1–15.
[44]
Qian Yan, Huayang Cao, Shuaibing Lu, and Minhuan Huang. 2023. InFuzz: An Interactive Tool for Enhancing Efficiency in Fuzzing through Visual Bottleneck Analysis (Registered Report). In Proceedings of the 2nd International Fuzzing Workshop. 56–61.
[45]
Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. 2018. QSYM : A Practical Concolic Execution Engine Tailored for Hybrid Fuzzing. In 27th USENIX Security Symposium (USENIX Security 18). USENIX Association, Baltimore, MD. 745–761. isbn:978-1-939133-04-5 https://doi.org/10.5555/3277203.3277260
[46]
Chijin Zhou, Mingzhe Wang, Jie Liang, Zhe Liu, and Yu Jiang. 2020. Zeror: Speed up fuzzing with coverage-sensitive tracing and scheduling. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. 858–870. https://doi.org/10.1145/3324884.3416572
[47]
Xiaogang Zhu, Sheng Wen, Seyit Camtepe, and Yang Xiang. 2022. Fuzzing: a survey for roadmap. ACM Computing Surveys (CSUR), 54, 11s (2022), 1–36. https://doi.org/10.1145/3512345
Index Terms
- DDGF: Dynamic Directed Greybox Fuzzing with Path Profiling
Recommendations
Directed Greybox Fuzzing
CCS '17: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications SecurityExisting Greybox Fuzzers (GF) cannot be effectively directed, for instance, towards problematic changes or patches, towards critical system calls or dangerous locations, or towards functions in the stack-trace of a reported vulnerability that we wish to ...
Sequence coverage directed greybox fuzzing
ICPC '19: Proceedings of the 27th International Conference on Program ComprehensionExisting directed fuzzers are not efficient enough. Directed symbolic-execution-based whitebox fuzzers, e.g. BugRedux, spend lots of time on heavyweight program analysis and constraints solving at runtime. Directed greybox fuzzers, such as AFLGo, ...
RDGFuzz: A directed greybox fuzzing optimization method based on Rich-Branch nodes
EITCE '22: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer EngineeringDirected fuzzing technology is one of the key technologies to quickly reach a specific location of software, and to conduct targeted testing or bug recurrence. However, directed fuzzing technology has some problems, such as unreasonable seed energy ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2024
1928 pages
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 September 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
ISSTA '24
Sponsor:
ISSTA '24: 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis
September 16 - 20, 2024
Vienna, Austria
Acceptance Rates
Overall Acceptance Rate 58 of 213 submissions, 27%
Upcoming Conference
ISSTA '25
- Sponsor:
- sigsoft
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 140Total Downloads
- Downloads (Last 12 months)140
- Downloads (Last 6 weeks)140
Reflects downloads up to 02 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in