Abstract
This paper is a survey of methods for covering arrays generation. Covering arrays are used in test data generation for program interfaces with many parameters. The effectiveness and the range of application of these methods are analysed. The algorithms used in these methods are analysed for their time complexity and memory usage. In this paper combinatorial, recursive, greedy algorithms are observed. Several heuristics for reducing the size and the time for construction of covering arrays are observed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Stevens, B., Mendelsohn, E.: New recursive methods for transversal covers. Journal of Combinatorial Designs 7(3), 185–203 (1999)
Sloane, N.: Covering arrays and intersecting codes. Journal of Combinatorial Designs 1(1), 51–63 (1993)
Chateauneuf, M., Kreher, D.: On the state of strength-three covering arrays. Journal of Combinatorial Designs 10(4), 217–238 (2002)
Martirosyan, S., Van Trung, T.: Tran Van Trung: On t-covering arrays. Designs, Codes and Cryptography 32, 323–339 (2004)
Hartman, A., Raskin, L.: Problems and algorithms for covering arrays. Discrete Math. 284, 149–156 (2004)
Cohen, M., Colbourn, C., Ling, A.: Constructing Strength Three Covering Arrays with Augmented Annealing. Discrete Mathematics 308, 2709–2722 (2008)
Cohen, D., Dalal, S., Fredman, M., Patton, G.: The AETG System: an approach to testing based on combinatorial design. IEEE Transactions on Software Engineering 23(7), 437–444 (1996)
Nurmela, K.: Upper bounds for covering arrays by tabu search. Discrete Applied Math. 138, 143–152 (2004)
Colbourn, C., Martirosyan, S., Van Trung, T., Walker II, R.: Roux-type Constructions for Covering Arrays of Strengths Three and Four Designs. Codes and Cryptography 41(1), 33–57 (2006)
Stevens, B.: Transversal Covers and Packings. PhD. Thesis, Mathematics, University of Toronto (1998)
Patton, G.: DAT (Defect Analysis Team)1986-1990 Overview. Internal Bellcore Technical Memo (1991)
Cohen, M., Colbourn, C., Ling, A.: Augmenting Sumulated Annealing to Build Interaction Test Suites. In: Proc Intl. Symposium Software Requirements Engineering, pp. 394–405 (2003)
Dueck, G.: New Optimization Heuristic – The Great Deluge Algorithm and the Record-To-Record Travel. Journal of Computational Physics 104, 86–92 (1993)
Stevens, B., Ling, A., Mendelsohn, E.: A direct construction of transversal coversusing group divisible designs. Ars. Combin. 63, 145–159 (2002)
Dueck, G., Scheuer, T.: Threshold Accepting: A general purpose optimization algorithm appearing superior to simulating annealing. Journal of Computational Physics 90, 161–175 (1990)
Stardom, J.: Metaheuristic and the search for covering and packing arrays. Master’s thesis, Simon Fraser University (2001)
Jenkins, B.: Tool for pairwise testing (2005), http://burtleburtle.net/bob/math/jenny.html
Lei, Y., Tai, K.: In-parameter order: A test generation strategy for pairwise testing. In: Proc. 3rd IEEE High Assurance System Engineering Symposium, pp. 254–161 (1998)
Seroussi, G., Bshouty, N.: Vector sets for exhaustive testing of logic circuits. IEEE Trans. Information Theory 34, 513–522 (1988)
Cohen, D., Dalal, S., Parelius, J., Patton, G.: The Combinatorial Design Approach to Automatic Test Generation. IEEE Software, 83–87 (September 1996)
Pairwise Testing, Combinatorial Test Case Generation, http://www.pairwise.org/tools.asp
Colbourn, C.: Combinatorial aspects of covering arrays. Le Matematiche (Catania) 58, 121–167 (2004)
Cohen, D., Dalal, S., Fredman, M., Patton, G.: The Automatic Efficient Test Generator (AETG) System. In: Proceedings of 5th International Symposium on Software Reliability Engineering, November 6-9, pp. 303–309 (1994)
Colbourn, C., Cohen, M., Turban, R.: A deterministic density algorithm for pairwise interaction coverage. In: Proc. of the IASTED Intl. Conference on Software Engineering, pp. 242–252 (February 2004)
Williams, A.: Determination of Test Configurations for Pairwise Interaction Coverage. In: Proceedings of the 13th International Conference on Testing Communicating Systems (Test-Com 2000), pp. 59–74 (2000)
Cormen, T., Stein, C., Rivest, R., Leiserson, C.: Greedy algorithms // Introduction to Algorithms, 2nd edn., vol. 16. Moscow Viliyams (2006)
Grindal, M., Offutt, A., Andler, S.: Combination testing strategies: A survey. Software Testing, Verification, and Reliability 15(3), 167–199 (2005)
Hartman, A.: Software and Hardware Testing Using Combinatorial Covering Suites. In: Proc. Graph Theory, Combinatorics and Algorithms: Interdisciplinary Applications, pp. 266–327 (2005)
Edelman, A.: The mathematics of the Pentium division bug. SIAM Review 39, 54–67 (1997)
Greene, C.: Sperner families and partitions of a partially ordered set. In: Hall Jr., M., van Lint, J. (eds.) Combinatorics, Dordrecht, Holland, pp. 277–290 (1975)
Williams, A., Probert, R.: A measure for component interaction test coverage. In: Proc. ACS/IEEE Intl. Conf. on Computer Systems and Applications, pp. 301–311 (2001)
Ammann, P., Offutt, J.: Using Formal Methods to Derive Test Frames in Category-Partition Testing Safety, Reliability, Fault Tolerance, Concurrency, and Real Time Security. In: Proc. Ninth Ann. Conf. Computer Assurance (COMPASS 1994), pp. 69–79 (1994)
Zelenov, S., Zelenova, S.: Automated negative and positive test generation for testing a syntax analysis phase. Works of Institute for System Programming RAS, russian (2004)
Documentation on Webmoney interface WM Keeper Light, http://www.wmtransfer.com/eng/about/demo/light/index.shtml
Colbourn, C., Martirosyan, S., Mullen, G., Shasha, D., Yucas, J., Sherwood, G.: Products of Mixed Covering Arrays of Strength Two. J. Combin. Des. 14, 124–138 (2006)
Tung, Y.-W., Aldiwan, W.: Automating test case generation for the new generation mission software system. In: Proc. IEEE Aerospace Conf., pp. 431–437 (2000)
Bryce, R., Colbourn, C.: One-Test-at-a-Time Heuristic Search for Interaction Test Suites. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), Search-based Software Engineering track (SBSE), London, England, pp. 1082–1089 (2007)
Lidl, R., Niederreiter, H.: Finite fields, vol. 2. Mir, Moscow (1988)
Tables for the smallest known covering arrays, http://www.public.asu.edu/~ccolbou/src/tabby/catable.html
Godbole, A., Skipper, D., Sunley, R.: t-Covering arrays: upper bounds and Poisson approximations. Combinatorics, Probability and Computing 5, 105–118 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kuliamin, V., Petukhov, A. (2010). Covering Arrays Generation Methods Survey. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification, and Validation. ISoLA 2010. Lecture Notes in Computer Science, vol 6416. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16561-0_36
Download citation
DOI: https://doi.org/10.1007/978-3-642-16561-0_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16560-3
Online ISBN: 978-3-642-16561-0
eBook Packages: Computer ScienceComputer Science (R0)