Measuring Coverage of Prolog Programs Using Mutation Testing

  • Alexandros Efremidis
  • Joshua SchmidtEmail author
  • Sebastian Krings
  • Philipp Körner
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11285)


Testing is an important aspect in professional software development, both to avoid and identify bugs as well as to increase maintainability. However, increasing the number of tests beyond a reasonable amount hinders development progress. To decide on the completeness of a test suite, many approaches to assert test coverage have been suggested. Yet, frameworks for logic programs remain scarce.

In this paper, we introduce a framework for Prolog programs measuring test coverage using mutations. We elaborate on the main ideas of mutation testing and transfer them to logic programs.To do so, we discuss the usefulness of different mutations in the context of Prolog and empirically evaluate them in a new mutation testing framework on different examples.


  1. 1.
    Ammann, P., Delamaro, M.E., Offutt, J., et al.: Establishing theoretical minimal sets of mutants. In: IEEE International Conference on Software Testing, Verification, and Validation, vol. 7. IEEE Computer Society (2014)Google Scholar
  2. 2.
    Clark, K.L.: Negation as failure. In: Gallaire, H., Minker, J. (eds.) Logic and Data Bases, pp. 293–322. Springer, Boston (1978). Scholar
  3. 3.
    Coles, H., Laurent, T., Henard, C., Papadakis, M., Ventresque, A.: PIT: a practical mutation testing tool for Java (Demo). In: International Symposium on Software Testing and Analysis, ISSTA 2016, pp. 449–452. ACM (2016)Google Scholar
  4. 4.
    Denisov, A., Pankevich, S.: Mull it over: mutation testing based on LLVM. In: 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 25–31, April 2018Google Scholar
  5. 5.
    Deursen, A.V., Moonen, L., Bergh, A., Kok, G.: Refactoring test code. In: International Conference on Extreme Programming and Flexible Processes in Software Engineering, pp. 92–95 (2001)Google Scholar
  6. 6.
    Grün, B.J., Schuler, D., Zeller, A.: The impact of equivalent mutants. In: IEEE International Workshop on Mutation Analysis, pp. 192–199. IEEE (2009)Google Scholar
  7. 7.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011)CrossRefGoogle Scholar
  8. 8.
    Krings, S.: Code coverage analysis for prolog. Bachelor’s thesis, Heinrich-Heine-University, Duesseldorf, Germany (2010)Google Scholar
  9. 9.
    Krings, S., Schmidt, J., Brings, C., Frappier, M., Leuschel, M.: A translation from alloy to B. In: Butler, M., Raschke, A., Hoang, T.S., Reichl, K. (eds.) ABZ 2018. LNCS, vol. 10817, pp. 71–86. Springer, Cham (2018). Scholar
  10. 10.
    Körner, P., Krings, S.: plspec - A specification language for prolog data. In: Seipel, D., Hanus, M., Abreu, S. (eds.) Declare 2017, vol. 499, Technical report. University of Würzburg (2017)Google Scholar
  11. 11.
    Körner, P., Schneider, D., Leuschel, M.: Evaluating interpreter design in prolog. In: Kolloquium Programmiersprachen und Grundlagen der Programmierung KPS, Schriftenreihe des Instituts für Computersprachen (2015)Google Scholar
  12. 12.
    Le, D., Alipour, M., Gopinath, R., Groce, A.: MuCheck: an extensible tool for mutation testing of haskell programs. In: International Symposium on Software Testing and Analysis, ISSTA (2014)Google Scholar
  13. 13.
    Le, D., Alipour, M.A., Gopinath, R., Groce, A.: Mutation testing of functional programming languages. Technical report. Oregon State University, School of Software Engineering and Computer Science (2014)Google Scholar
  14. 14.
    Lloyd, J.W.: Foundations of Logic Programming. Springer, Heidelberg (1984). Scholar
  15. 15.
    Naish, L.: A declarative debugging scheme. J. Funct. Logic Program. 1997(3) (1997).
  16. 16.
    Offutt, A.J., Pan, J.: Automatically detecting equivalent mutants and infeasible paths. Softw. Testing Verif. Reliab. 7(3), 165–192 (1997)CrossRefGoogle Scholar
  17. 17.
    Offutt, A.J., Rothermel, G., Zapf, C.: An experimental evaluation of selective mutation. In: International Conference on Software Engineering, pp. 100–107. IEEE Computer Society Press (1993)Google Scholar
  18. 18.
    Papadimitriou, C.H.: A note the expressive power of prolog. Bull. EATCS 26(21–23), 61 (1985)Google Scholar
  19. 19.
    Sangwan, R.S., LaPlante, P.A.L.: Test-driven development in large projects. IT Prof. 8(5), 25–29 (2006)CrossRefGoogle Scholar
  20. 20.
    Schuler, D., Zeller, A.: Javalanche: efficient mutation testing for Java. In: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE 2009, pp. 297–298. ACM (2009)Google Scholar
  21. 21.
    Shapiro, E.Y.: Algorithmic Program Debugging. MIT Press, Cambridge (1983)zbMATHGoogle Scholar
  22. 22.
    SICS, Kista, Sweden. SICStus Prolog User’s Manual.
  23. 23.
    Toaldo, J.R., Vergilio, S.R.: Applying mutation testing in prolog programs. In: VII Workshop de Testes e Tolerância a Falhas. Biblioteca Digital Brasileira de Computação (2006)Google Scholar
  24. 24.
    Wielemaker, J., Schrijvers, T., Triska, M., Lager, T.: SWI-prolog. Theory Pract. Logic Program. 12(1–2), 67–96 (2012)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institut für InformatikUniversität DüsseldorfDüsseldorfGermany
  2. 2.Niederrhein University of Applied SciencesMönchengladbachGermany

Personalised recommendations