Abstract
The body of knowledge in the area of black-box unit testing is quite well established, e.g., the concepts of category partitioning, boundary value analysis, and pair-wise testing. However, tool support to apply the existing techniques in large-scale industrial software projects is fairly limited. Although tools such as HexaWise enable testers in automatic generation of all-combination pair-wise test input data, nevertheless converting them to actual test case source code (e.g., in NUnit) is both very tedious and also error prone. To address the above challenge, we have developed an open-source tool, referred to as AutoBBUT, for automated generation of black-box NUnit test cases for the C# .Net platform. Evaluation of the tool in an industrial setting by using it to automatically generate more than 1,962 NUnit test cases (having 15,906 test line of code) in NUnit shows the effectiveness of the tool in saving testers a lot of time and also preventing unwanted defects and/or test smells in test code, which can occur due to manual test code development.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Mathur, A.P.: Foundations of Software Testing. Addison-Wesley Professional, Reading (2008)
Nie, C., et al.: Automatic Test Generation for n-Way Combinatorial Testing. In: Reussner, R., Mayer, J., Stafford, J.A., Overhage, S., Becker, S., Schroeder, P.J. (eds.) QoSA 2005 and SOQUA 2005. LNCS, vol. 3712, pp. 203–211. Springer, Heidelberg (2005)
Tai, K.C., Lie, Y.: A Test Generation Strategy for Pairwise Testing. IEEE Transactions on Software Engineering 28(1), 109–111 (2002)
Hunter, J.: HexaWise Tool, http://www.hexawise.com (Last accessed: March 2010)
Czerwonka, J.: Pairwise Testing Tools - Combinatorial Test Case Generation, http://www.pairwise.org/tools.asp (Last accessed: March 2010)
Hamill, P.: Unit Test Frameworks. O’Reilly, Sebastopol (2004)
Tan, R.P., Edwards, S.H.: Experiences Evaluating the Effectiveness of JML-JUnit Testing. In: Special issue on Workshop on Empirical Research in Software Testing, vol. 29(5), pp. 1–4 (2004)
Meszaros, G.: Four-Phase Test Pattern, www.xunitpatterns.com (Last accessed: March 2010 )
Wiederseiner, C., Garousi, V., Jolly, S.A.: AutoBBUT - Automated Code Generation for Black-box Unit Testing, http://code.google.com/p/autobbut (Last accessed: April 2010)
Tillmann, N., de Halleux, J.: Pex-White Box Test Generation for .NET. In: Conference on Tests and Proofs (2008)
Cheon, Y., Kim, M.Y., Perumandla, A.: A Complete Automation of Unit Testing for Java Programs. In: Proc. of Int. Conf. on Software Engineering Research and Experience (2005)
JUB (JUnit test case Builder), http://jub.sourceforge.net (Last accessed: March 2010)
TestGen4J, http://sourceforge.net/projects/spike-test-gen/ (Last accessed: March 2010)
JCrasher, http://code.google.com/p/jcrasher (Last accessed: March 2010)
NModel, http://nmodel.codeplex.com (Last accessed: June 2010)
Tillmann, N., de Halleux, P.: Code Digging with Pex Code Understanding and Automatic Testing At Your Fingertips. Microsoft Corporation (2008)
JML Team, Java Modeling Language, http://www.jmlspecs.org (Last accessed: March 2010)
Tan, R.P., Edwards, S.H.: Experiences Evaluating the Effectiveness of JML-JUnit Testing. In: ACM SIGSOFT Software Engineering Notes. ACM, New York (2004)
Wang, S., Offutt, J.: Comparison of Unit-Level Automated Test Generation Tools. In: Proc. of IEEE Int. Conf. on Software Testing, Verification, and Validation (2009)
Jacky, J., et al.: Model-Based Software Testing and Analysis with C#. Cambridge University Press, Cambridge (2008)
Manolov, I.: Introduction to TestApi – Part 4: Combinatorial Variation Generation APIs, http://blogs.msdn.com/ivo_manolov/archive/2009/08/26/9884004.aspx (Last accessed: April 2010)
Microsoft MSDN: System. CodeDom.Compiler Namespace, http://msdn.microsoft.com/en-us/library/system.codedom.compiler.aspx (Last accessed: April 2010)
Wiederseiner, C., Garousi, V., Jolly, S.A.: Supplementary Project Data-NUnit Test Suite, http://www.softqual.ucalgary.ca/projects/2010/AutoBBUT_TAIC_PART/ (Last accessed: April 2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wiederseiner, C., Jolly, S.A., Garousi, V., Eskandar, M.M. (2010). An Open-Source Tool for Automated Generation of Black-Box xUnit Test Code and Its Industrial Evaluation. In: Bottaci, L., Fraser, G. (eds) Testing – Practice and Research Techniques. TAIC PART 2010. Lecture Notes in Computer Science, vol 6303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15585-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-15585-7_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15584-0
Online ISBN: 978-3-642-15585-7
eBook Packages: Computer ScienceComputer Science (R0)