Liveness-Driven Random Program Generation
Randomly generated programs are popular for testing compilers and program analysis tools, with hundreds of bugs in real-world C compilers found by random testing. However, existing random program generators may generate large amounts of dead code (computations whose result is never used). This leaves relatively little code to exercise a target compiler’s more complex optimizations.
To address this shortcoming, we introduce liveness-driven random program generation. In this approach the random program is constructed bottom-up, guided by a simultaneous structural data-flow analysis to ensure that the generator never generates dead code.
The algorithm is implemented as a plugin for the Frama-C framework. We evaluate it in comparison to Csmith, the standard random C program generator. Our tool generates programs that compile to more machine code with a more complex instruction mix.
KeywordsCode generation Random testing Data-flow analysis Program optimization
The author would like to thank the anonymous reviewers, John Regehr, and Gabriel Scherer for insightful comments on earlier versions of this paper. This research was partially supported by ITEA 3 project no. 14014, ASSUME.
- 2.Barany, G.: Finding missed compiler optimizations by differential testing. In: 27th International Conference on Compiler Construction (2018). https://doi.org/10.1145/3178372.3179521
- 5.Eide, E., Regehr, J.: Volatiles are miscompiled, and what to do about it. In: EMSOFT 2008. ACM (2008). http://doi.acm.org/10.1145/1450058.1450093
- 14.Pałka, M.H., Claessen, K., Russo, A., Hughes, J.: Testing an optimising compiler by generating random lambda terms. In: 6th International Workshop on Automation of Software Test, AST 2011 (2011). http://doi.acm.org/10.1145/1982595.1982615
- 15.Perennial, Inc.: ACVS ANSI/ISO/FIPS-160 C validation suite. http://www.peren.com/pages/acvs_set.htm
- 16.Plum Hall, Inc.: The Plum Hall validation suite for C. http://www.plumhall.com/stec.html
- 17.Ruderman, J.: jsfunfuzz (2015). https://github.com/MozillaSecurity/funfuzz/tree/master/js/jsfunfuzz
- 19.Turner, B.: Random C program generator (2007). https://sites.google.com/site/brturn2/randomcprogramgenerator
- 21.Zhao, C., Xue, Y., Tao, Q., Guo, L., Wang, Z.: Automated test program generation for an industrial optimizing compiler. In: ICSE Workshop on Automation of Software Test (2009)Google Scholar