short-paper

Grammarinator: a grammar-based open source fuzzer

Authors:

A-TEST 2018: Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

Pages 45 - 48

https://doi.org/10.1145/3278186.3278193

Published: 05 November 2018 Publication History

Get Access

Abstract

Fuzzing, or random testing, is an increasingly popular testing technique. The power of the approach lies in its ability to generate a large number of useful test cases without consuming expensive manpower. Furthermore, because of the randomness, it can often produce unusual cases that would be beyond the awareness of a human tester. In this paper, we present Grammarinator, a general purpose test generator tool that is able to utilize existing parser grammars as models. Since the model can act both as a parser and as a generator, the tool can provide the capabilities of both generation and mutation-based fuzzers. The presented tool is actively used to test various JavaScript engines and has found more than 100 unique issues.

References

[1]

Renáta Hodován and Ákos Kiss. 2018. Fuzzinator: An Open-Source Modular Random Testing Framework. In Proceedings of the 11th IEEE International Conference on Software Testing, Verification and Validation (ICST 2018). IEEE, 416–421.

Google Scholar

[2]

Christian Holler, Kim Herzig, and Andreas Zeller. 2012. Fuzzing with Code Fragments. In Proceedings of the 21st USENIX Security Symposium (USENIX Security ’12). USENIX Association, 445–458.

Digital Library

Google Scholar

[3]

Fitsum Meshesha Kifetew, Roberto Tiella, and Paolo Tonella. 2014. Combining Stochastic Grammars and Genetic Programming for Coverage Testing at the System Level. In Search-Based Software Engineering – 6th International Symposium, SSBSE 2014, Proceedings. Springer, 138–152.

Google Scholar

[4]

Fitsum Meshesha Kifetew, Roberto Tiella, and Paolo Tonella. 2017. Generating valid grammar-based test inputs by means of genetic programming and annotated grammars. Empirical Software Engineering 22, 2 (2017), 928–961.

Digital Library

Google Scholar

[5]

Peter M. Maurer. 1990. Generating Test Data with Enhanced Context-Free Grammars. IEEE Software 7, 4 (1990), 50–55.

Digital Library

Google Scholar

[6]

Barton Miller. 2008. Foreword for Fuzz Testing Book. http://pages.cs.wisc.edu/ ~bart/fuzz/Foreword1.html.

Google Scholar

[7]

Paul Purdom. 1972. A sentence generator for testing parsers. BIT Numerical Mathematics 12, 3 (1972), 366–375.

Digital Library

Google Scholar

[8]

Ari Takanen, Jared DeMott, Charlie Miller, and Atte Kettunen. 2018. Fuzzing for Software Security Testing and Quality Assurance (2nd ed.). Artech House. 7 https://www.peach.tech/ Abstract 1 Introduction 2 Grammarinator 3 Results 4 Related Work 5 Summary Acknowledgments References

Digital Library

Google Scholar

Cited By

View all

Corradi QWickerson JConstantinides GBöhme MNoller YSzekeres L(2024)Automated Feature Testing of Verilog Parsers using Fuzzing (Registered Report)Proceedings of the 3rd ACM International Fuzzing Workshop10.1145/3678722.3685536(70-79)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678722.3685536
Kim HKim SLee JCha SChristakis MPradel M(2024)AsFuzzer: Differential Testing of Assemblers with Error-Driven Grammar InferenceProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680345(1099-1111)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680345
Arefin MShetiya SWang ZCsallner CRoychoudhury APaiva AAbreu RStorey M(2024)Fast Deterministic Black-box Context-free Grammar InferenceProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639214(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639214
Show More Cited By

Index Terms

Grammarinator: a grammar-based open source fuzzer
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Grammar-based whitebox fuzzing
PLDI '08: Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation

Whitebox fuzzing is a form of automatic dynamic test generation, based on symbolic execution and constraint solving, designed for security testing of large applications. Unfortunately, the current effectiveness of whitebox fuzzing is limited when ...
Fault detection effectiveness of source test case generation strategies for metamorphic testing
MET '18: Proceedings of the 3rd International Workshop on Metamorphic Testing

Metamorphic testing is a well known approach to tackle the oracle problem in software testing. This technique requires the use of source test cases that serve as seeds for the generation of follow-up test cases. Systematic design of test cases is ...
Grammar-based whitebox fuzzing
PLDI '08

Whitebox fuzzing is a form of automatic dynamic test generation, based on symbolic execution and constraint solving, designed for security testing of large applications. Unfortunately, the current effectiveness of whitebox fuzzing is limited when ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

A-TEST 2018: Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

November 2018

66 pages

ISBN:9781450360531

DOI:10.1145/3278186

General Chair:
Wishnu Prasetya,
Program Chairs:
Tanja E. J. Vos,
Sinem Getir

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Ministry of Finance of Hungary

Conference

ESEC/FSE '18

Sponsor:

SIGSOFT

ESEC/FSE '18: 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 5, 2018

FL, Lake Buena Vista, USA

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
689
Total Downloads

Downloads (Last 12 months)161
Downloads (Last 6 weeks)19

Reflects downloads up to 09 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Corradi QWickerson JConstantinides GBöhme MNoller YSzekeres L(2024)Automated Feature Testing of Verilog Parsers using Fuzzing (Registered Report)Proceedings of the 3rd ACM International Fuzzing Workshop10.1145/3678722.3685536(70-79)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678722.3685536
Kim HKim SLee JCha SChristakis MPradel M(2024)AsFuzzer: Differential Testing of Assemblers with Error-Driven Grammar InferenceProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680345(1099-1111)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680345
Arefin MShetiya SWang ZCsallner CRoychoudhury APaiva AAbreu RStorey M(2024)Fast Deterministic Black-box Context-free Grammar InferenceProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639214(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639214
Karmarkar HAgrawal SChauhan AShete P(2024)Navigating Confidentiality in Test Automation: A Case Study in LLM Driven Test Data Generation2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00041(337-348)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00041
Xu HWang YJiang ZFan SFu SXie P(2024)Fuzzing JavaScript engines with a syntax-aware neural program modelComputers & Security10.1016/j.cose.2024.103947144(103947)Online publication date: Sep-2024
https://doi.org/10.1016/j.cose.2024.103947
Peng HYao ZSani ATian DPayer MCalandrino JTroncoso C(2023)GLeeFuzzProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620343(1883-1899)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620343
Vitagliano GHameed MJiang LReisener LWu ENaumann F(2023)Pollock: A Data Loading BenchmarkProceedings of the VLDB Endowment10.14778/3594512.359451816:8(1870-1882)Online publication date: 22-Jun-2023
https://doi.org/10.14778/3594512.3594518
Dutra RGopinath RZeller A(2023)FormatFuzzer: Effective Fuzzing of Binary File FormatsACM Transactions on Software Engineering and Methodology10.1145/362815733:2(1-29)Online publication date: 22-Dec-2023
https://dl.acm.org/doi/10.1145/3628157
Nilizadeh ALeavens GPăsăreanu CNoller Y(2023)JMLKelinci+: Detecting Semantic Bugs and Covering Branches with Valid Inputs Using Coverage-guided Fuzzing and Runtime Assertion CheckingFormal Aspects of Computing10.1145/360753836:1(1-24)Online publication date: 5-Aug-2023
https://dl.acm.org/doi/10.1145/3607538
Gao WPham VLiu DChang OMurray TRubinstein BBöhme MNoller YRay BSzekeres L(2023)Beyond the Coverage Plateau: A Comprehensive Study of Fuzz Blockers (Registered Report)Proceedings of the 2nd International Fuzzing Workshop10.1145/3605157.3605177(47-55)Online publication date: 17-Jul-2023
https://dl.acm.org/doi/10.1145/3605157.3605177
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Grammar-based whitebox fuzzing

Fault detection effectiveness of source test case generation strategies for metamorphic testing

Grammar-based whitebox fuzzing