Abstract
When a program evolves, its test suite must be modified to reflect changes in requirements or to account for new feature additions. This problem of modifying tests as a program evolves is termed test repair. Existing approaches either assume that updated implementation is correct, or assume that most test repairs require simply fixing compilation errors caused by refactoring of previously tested implementation. This paper focuses on the problem of repairing semantically broken or outdated tests by leveraging specifications. Our technique, Spectr, employs a lightweight formal method to perform specification-based repair. Specifically, Spectr supports the Alloy language for writing specifications and uses its SAT-based analyzer for repairing JUnit tests. Since Spectr utilizes specifications, it works even when the specification is modified but the change has not yet been implemented in code–in such a case, Spectr is able to repair tests that previous techniques would not even consider as candidates for test repair. An experimental evaluation using a suite of subject programs with pre-conditions and post-conditions shows Spectr can effectively repair tests even for programs that perform complex manipulation of dynamically allocated data.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
JUnit website, http://www.junit.org
Chen, L., Avizienis, A.: N-version programming: a fault-tolerance approach to reliability. In: FTCS, vol. 8, pp. 3–9 (1978)
Chen, Y.F., Rosenblum, D.S., Vo, K.P.: Testtube: a system for selective regression testing. In: ICSE, pp. 211–220 (1994)
Cleve, H., Zeller, A.: Locating causes of program failures. In: ICSE (2005)
Daniel, B., Gvero, T., Marinov, D.: On test repair using symbolic execution. In: ISSTA, pp. 207–218 (2010)
Daniel, B., Jagannath, V., Dig, D., Marinov, D.: ReAssert: Suggesting repairs for broken unit tests. In: ASE, pp. 433–444 (2009)
Dennis, G., Chang, F.S.H., Jackson, D.: Modular verification of code with SAT. In: ISSTA, pp. 109–120 (2006)
Deursen, A.V., Moonen, L., Bergh, A., Kok, G.: Refactoring test code. In: XP, pp. 92–95 (2001)
Galeotti, J.P., Rosner, N., López Pombo, C.G., Frias, M.F.: Analysis of invariants for efficient bounded verification. In: ISSTA, pp. 25–36 (2010)
Gopinath, D., Malik, M.Z., Khurshid, S.: Specification-Based Program Repair Using SAT. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS 2011. LNCS, vol. 6605, pp. 173–188. Springer, Heidelberg (2011)
Grechanik, M., Xie, Q., Fu, C.: Maintaining and evolving GUI-directed test scripts. In: ICSE, pp. 408–418 (2009)
Hangal, S., Lam, M.S.: Tracking down software bugs using automatic anomaly detection. In: ICSE, pp. 291–301 (2002)
Jackson, D.: Software Abstractions: Logic, Language, and Analysis. The MIT Press (2006)
Jeffrey, D., Feng, M., Gupta, N., Gupta, R.: BugFix: A learning-based tool to assist developers in fixing bugs. In: ICPC, pp. 70–79 (2009)
Jiang, L., Su, Z.: Context-aware statistical debugging: from bug predictors to faulty control flow paths. In: ASE, pp. 184–193 (2007)
Khurshid, S., Marinov, D.: TestEra: Specification-based testing of Java programs using SAT. In: ASE, vol. 11(4), pp. 403–434 (2004)
Memon, A.M.: Automatically repairing event sequence-based GUI test suites for regression testing. TOSEM 18(2), 1–36 (2008)
Mirzaaghaei, M., Pastore, F., Pezze, M.: Automatically repairing test cases for evolving method declarations. In: ICSM, pp. 1–5 (2010)
Rothermel, G., Harrold, M.J.: A safe, efficient regression test selection technique. TOSEM 6(2), 173–210 (1997)
Tillmann, N., de Halleux, J.: Pex–White Box Test Generation for.NET. In: Beckert, B., Hähnle, R. (eds.) TAP 2008. LNCS, vol. 4966, pp. 134–153. Springer, Heidelberg (2008)
Vaziri, M.: Finding Bugs Using a Constraint Solver. Ph.D. thesis, CSAIL, MIT (2003)
Visser, W., Havelund, K., Brat, G.P., Park, S., Lerda, F.: Model checking programs. In: ASE, vol. 10(2), pp. 203–232 (2003)
Visser, W., Pǎsǎreanu, C.S., Khurshid, S.: Test input generation with Java PathFinder. In: ISSTA, pp. 97–107 (2004)
Weimer, W., Nguyen, T., Le Goues, C., Forrest, S.: Automatically finding patches using genetic programming. In: ICSE, pp. 364–374 (2009)
Nokhbeh Zaeem, R., Khurshid, S.: Contract-Based Data Structure Repair Using Alloy. In: D’Hondt, T. (ed.) ECOOP 2010. LNCS, vol. 6183, pp. 577–598. Springer, Heidelberg (2010)
Zaidman, A., Rompaey, B.V., van Deursen, A., Demeyer, S.: Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining. ESE 16(3), 325–364 (2011)
Zeller, A.: Automated debugging: Are we close? Computer 34, 26–31 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, G., Khurshid, S., Kim, M. (2012). Specification-Based Test Repair Using a Lightweight Formal Method. In: Giannakopoulou, D., Méry, D. (eds) FM 2012: Formal Methods. FM 2012. Lecture Notes in Computer Science, vol 7436. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32759-9_37
Download citation
DOI: https://doi.org/10.1007/978-3-642-32759-9_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32758-2
Online ISBN: 978-3-642-32759-9
eBook Packages: Computer ScienceComputer Science (R0)