Advertisement

Separating Interaction Effects Using Locating and Detecting Arrays

  • Stephen A. Seidel
  • Kaushik Sarkar
  • Charles J. ColbournEmail author
  • Violet R. Syrotiuk
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10979)

Abstract

The correctness and performance of complex engineered systems are often impacted by many factors, each of which has many possible levels. Performance can be affected not just by individual factor-level choices, but also by interactions among them. While covering arrays have been employed to produce combinatorial test suites in which every possible interaction of a specified number of factor levels arises in at least one test, in general they do not identify the specific interaction(s) that are significant. Locating and detecting arrays strengthen the requirements to permit the identification of a specified number of interactions of a specified size. Further, to cope with outliers or missing responses in data collected from real engineered systems, a further requirement of separation is introduced. In this paper, we examine two randomized methods for the construction of locating and detecting arrays, the first based on the Stein-Lovász-Johnson paradigm, and the second based on the Lovász Local Lemma. Each can be derandomized to yield efficient algorithms for construction, the first using a conditional expectation method, and the second using Moser-Tardos resampling. We apply these methods to produce upper bounds on sizes of locating and detecting arrays for various numbers of factors and levels, when one interaction of two factor levels is to be detected or located, for separation of up to four. We further compare the sizes obtained with those from more targeted (and more computationally intensive) heuristic methods.

Notes

Acknowledgements

This work is supported in part by the U.S. National Science Foundation grant #1421058, and in part by the Software Test & Analysis Techniques for Automated Software Test program by OPNAV N-84, U.S. Navy.

References

  1. 1.
    Aldaco, A.N., Colbourn, C.J., Syrotiuk, V.R.: Locating arrays: a new experimental design for screening complex engineered systems. SIGOPS Oper. Syst. Rev. 49(1), 31–40 (2015)CrossRefGoogle Scholar
  2. 2.
    Alon, N., Spencer, J.H.: The Probabilistic Method. Wiley-Interscience Series in Discrete Mathematics and Optimization, 3rd edn. Wiley, Hoboken (2008)Google Scholar
  3. 3.
    Bryce, R.C., Colbourn, C.J.: The density algorithm for pairwise interaction testing. Softw. Testing Verification Reliab. 17, 159–182 (2007)CrossRefGoogle Scholar
  4. 4.
    Bryce, R.C., Colbourn, C.J.: A density-based greedy algorithm for higher strength covering arrays. Softw. Testing Verification Reliab. 19, 37–53 (2009)CrossRefGoogle Scholar
  5. 5.
    Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The AETG system: an approach to testing based on combinatorial design. IEEE Trans. Softw. Eng. 23, 437–444 (1997)CrossRefGoogle Scholar
  6. 6.
    Colbourn, C.J.: Covering array tables: \(2 \le v \le 25\), \(2 \le t \le 6\), \(t \le k \le 10000\) (2005–2017). www.public.asu.edu/~ccolbou/src/tabby
  7. 7.
    Colbourn, C.J., Fan, B.: Locating one pairwise interaction: three recursive constructions. J. Algebra Comb. Discrete Struct. Appl. 3, 125–134 (2016)MathSciNetGoogle Scholar
  8. 8.
    Colbourn, C.J., Fan, B., Horsley, D.: Disjoint spread systems and fault location. SIAM J. Discrete Math. 30, 2011–2016 (2016)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Colbourn, C.J., Lanus, E., Sarkar, K.: Asymptotic and constructive methods for covering perfect hash families and covering arrays. Des. Codes Crypt. 86, 907–937 (2018)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Colbourn, C.J., McClary, D.W.: Locating and detecting arrays for interaction faults. J. Comb. Optim. 15, 17–48 (2008)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Colbourn, C.J., Syrotiuk, V.R.: Coverage, location, detection, and measurement. In: 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 19–25. IEEE Press (2016)Google Scholar
  12. 12.
    Compton, R., Mehari, M.T., Colbourn, C.J., De Poorter, E., Syrotiuk, V.R.: Screening interacting factors in a wireless network testbed using locating arrays. In: IEEE INFOCOM International Workshop on Computer and Networking Experimental Research Using Testbeds (CNERT) (2016)Google Scholar
  13. 13.
    Erdős, P., Lovász, L.: Problems and results on \(3\)-chromatic hypergraphs and some related questions. In: Infinite and Finite Sets, Colloq., Keszthely, vol. 2, pp. 609–627 (1973). Colloq. Math. Soc. János Bolyai, vol. 10, North-Holland, Amsterdam (1975)Google Scholar
  14. 14.
    Forbes, M., Lawrence, J., Lei, Y., Kacker, R.N., Kuhn, D.R.: Refining the in-parameter-order strategy for constructing covering arrays. J. Res. Nat. Inst. Stand. Tech. 113, 287–297 (2008)CrossRefGoogle Scholar
  15. 15.
    Francetić, N., Stevens, B.: Asymptotic size of covering arrays: an application of entropy compression. J. Combin. Des. 25, 243–257 (2017)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Godbole, A.P., Skipper, D.E., Sunley, R.A.: \(t\)-covering arrays: upper bounds and Poisson approximations. Comb. Probab. Comput. 5, 105–118 (1996)Google Scholar
  17. 17.
    Jin, H., Tsuchiya, T.: Constrained locating arrays for combinatorial interaction testing. CoRR abs/1801.06041 (2018). http://arxiv.org/abs/1801.06041
  18. 18.
    Johnson, D.S.: Approximation algorithms for combinatorial problems. J. Comput. Syst. Sci. 9, 256–278 (1974)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Konishi, T., Kojima, H., Nakagawa, H., Tsuchiya, T.: Finding minimum locating arrays using a SAT solver. In: 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2017, Tokyo, Japan, 13–17 March 2017, pp. 276–277 (2017)Google Scholar
  20. 20.
    Kuhn, D.R., Kacker, R., Lei, Y.: Introduction to Combinatorial Testing. CRC Press, Boca Raton (2013)zbMATHGoogle Scholar
  21. 21.
    Li, P.C., Meagher, K.: Sperner partition systems. J. Combin. Des. 21(7), 267–279 (2013)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Lovász, L.: On the ratio of optimal integral and fractional covers. Discrete Math. 13(4), 383–390 (1975)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Martínez, C., Moura, L., Panario, D., Stevens, B.: Locating errors using ELAs, covering arrays, and adaptive testing algorithms. SIAM J. Discrete Math. 23, 1776–1799 (2009/2010)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Meagher, K., Moura, L., Stevens, B.: A Sperner-type theorem for set-partition systems. Electron. J. Combin. 12, Note 20, 6 (2005). (Electronic)Google Scholar
  25. 25.
    Moser, R.A., Tardos, G.: A constructive proof of the general Lovász local lemma. J. ACM 57(2), Article no. 11, 15 (2010)CrossRefGoogle Scholar
  26. 26.
    Nagamoto, T., Kojima, H., Nakagawa, H., Tsuchiya, T.: Locating a faulty interaction in pair-wise testing. In: 20th IEEE Pacific Rim International Symposium on Dependable Computing, PRDC 2014, Singapore, 18–21 November 2014, pp. 155–156 (2014)Google Scholar
  27. 27.
    Nayeri, P., Colbourn, C.J., Konjevod, G.: Randomized postoptimization of covering arrays. Eur. J. Comb. 34, 91–103 (2013)CrossRefGoogle Scholar
  28. 28.
    Nie, C., Leung, H.: A survey of combinatorial testing. ACM Comput. Surv. 43(2), #11 (2011)CrossRefGoogle Scholar
  29. 29.
    Sarkar, K., Colbourn, C.J.: Two-stage algorithms for covering array construction. CoRR abs/1606.06730 (2016). http://arxiv.org/abs/1606.06730
  30. 30.
    Sarkar, K., Colbourn, C.J.: Upper bounds on the size of covering arrays. SIAM J. Discrete Math. 31, 1277–1293 (2017)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Seidel, S.A., Mehari, M.T., Colbourn, C.J., De Poorter, E., Moerman, I., Syrotiuk, V.R.: Analysis of large-scale experimental data from wireless networks. In: IEEE INFOCOM International Workshop on Computer and Networking Experimental Research Using Testbeds (CNERT) (2018)Google Scholar
  32. 32.
    Shi, C., Tang, Y., Yin, J.: Optimal locating arrays for at most two faults. Sci. China Math. 55(1), 197–206 (2012)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Stein, S.K.: Two combinatorial covering theorems. J. Comb. Theory Ser. A 16, 391–397 (1974)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Tang, Y., Colbourn, C.J., Yin, J.: Optimality and constructions of locating arrays. J. Stat. Theory Pract. 6(1), 20–29 (2012)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Stephen A. Seidel
    • 1
  • Kaushik Sarkar
    • 1
  • Charles J. Colbourn
    • 1
    Email author
  • Violet R. Syrotiuk
    • 1
  1. 1.School of Computing, Informatics, and Decision Systems EngineeringArizona State UniversityTempeUSA

Personalised recommendations