TCE+: An Extension of the TCE Method for Detecting Equivalent Mutants in Java Programs

  • Mahdi Houshmand
  • Samad PaydarEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10522)


While mutation testing is considered to be an effective technique in software testing, there are some impediments to its widespread use in industrial projects. One of these challenges is the equivalent mutant problem, and a line of research is dedicated to proposing new methods for addressing this problem. Trivial Compiler Equivalence (TCE) method is recently introduced as a simple technique that actually relies only on the optimizations made by the compiler. It is shown by empirical studies that employing TCE with the gcc compiler results in a fast and effective technique for detecting equivalent mutants in C programs. However, considering the fact that the Java compilers generally do not perform noticeable optimizations, the question is how effectively does TCE perform on Java programs? In this paper, experimental evaluations are discussed which demonstrate that using TCE technique with javac compiler results in very poor performance. As a result, this paper proposes to use the Java obfuscators as the complementary component, because of the optimizations they make. The experimental evaluations confirm that using TCE with the ProGuard obfuscation tool provides an effective and efficient method for detecting equivalent mutants in Java programs.


Mutation testing Equivalent mutant Trivial compiler equivalence Java 


  1. 1.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011)CrossRefGoogle Scholar
  2. 2.
    Budd, T.A., Angluin, D.: Two notions of correctness and their relation to testing. Acta Informatica 18(1), 31–45 (1982)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Offutt, A.J., Pan, J.: Automatically detecting equivalent mutants and infeasible paths. Softw. Testing Verification Reliab. 7(3), 165–192 (1997)CrossRefGoogle Scholar
  4. 4.
    Madeyski, L., et al.: Overcoming the equivalent mutant problem: a systematic literature review and a comparative experiment of second order mutation. IEEE Trans. Softw. Eng. 40(1), 23–42 (2014)CrossRefGoogle Scholar
  5. 5.
    Schuler, D., Zeller, A.: Covering and uncovering equivalent mutants. Softw. Testing Verification Reliab. 23(5), 353–374 (2013)CrossRefGoogle Scholar
  6. 6.
    Papadakis, M., Le Traon, Y.: Mutation testing strategies using mutant classification. In: Proceedings of the 28th Annual ACM Symposium on Applied Computing. ACM (2013)Google Scholar
  7. 7.
    Adamopoulos, K., Harman, M., Hierons, R.M.: How to overcome the equivalent mutant problem and achieve tailored selective mutation using co-evolution. In: Deb, K. (ed.) GECCO 2004. LNCS, vol. 3103, pp. 1338–1349. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-24855-2_155 CrossRefGoogle Scholar
  8. 8.
    Kintis, M., Malevris, N.: Using data flow patterns for equivalent mutant detection. In: 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE (2014)Google Scholar
  9. 9.
    Jia, Y., Harman, M.: Higher order mutation testing. Inf. Softw. Technol. 51(10), 1379–1393 (2009)CrossRefGoogle Scholar
  10. 10.
    Hierons, R., Harman, M., Danicic, S.: Using program slicing to assist in the detection of equivalent mutants. Softw. Testing Verification Reliab. 9(4), 233–262 (1999)CrossRefGoogle Scholar
  11. 11.
    Schuler, D., Andreas Z.: (Un-) Covering equivalent mutants. In: 2010 Third International Conference on Software Testing, Verification and Validation. IEEE (2010)Google Scholar
  12. 12.
    Papadakis, M., et al.: Trivial compiler equivalence: a large scale empirical study of a simple, fast and effective equivalent mutant detection technique. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 1. IEEE (2015)Google Scholar
  13. 13.
    Kintis, M., Malevris, N.: Identifying more equivalent mutants via code similarity. In: 2013 20th Asia-Pacific Software Engineering Conference, vol. 1. IEEE (2013)Google Scholar
  14. 14.
    Vincenzi, A.M.R., et al.: Bayesian-learning based guidelines to determine equivalent mutants. Int. J. Softw. Eng. Knowl. Eng. 12(06), 675–689 (2002)CrossRefGoogle Scholar
  15. 15.
    Nica, S., Wotawa, F.: Using constraints for equivalent mutant detection. arXiv preprint arXiv:1207.2234 (2012)
  16. 16.
    Just, R., Ernst, M.D., Fraser, G.: Using state infection conditions to detect equivalent mutants and speed up mutation analysis. arXiv preprint arXiv:1303.2784 (2013)
  17. 17.
    Rojas, J.M., Fraser, G.: Code defenders: a mutation testing game. In: The 11th International Workshop on Mutation Analysis. IEEE (2015)Google Scholar
  18. 18.
    Kintis, M., Malevris, N.: MEDIC: A static analysis framework for equivalent mutant identification. Inf. Softw. Technol. 68, 1–17 (2015)CrossRefGoogle Scholar
  19. 19.
    Yao, X., Harman, M., Jia, Y.: A study of equivalent and stubborn mutation operators using human analysis of equivalence. In: Proceedings of the 36th International Conference on Software Engineering. ACM (2014)Google Scholar
  20. 20.
    Harman, M., Jia, Y., Langdon, W.B.: A manifesto for higher order mutation testing. In: 2010 Third International Conference on Software Testing, Verification, and Validation Workshops (ICSTW). IEEE (2010)Google Scholar
  21. 21.
    Nguyen, Q.V., Madeyski, L.: Searching for strongly subsuming higher order mutants by applying multi-objective optimization algorithm. In: Le Thi, H.A., Nguyen, N.T., Do, T.V. (eds.) Advanced Computational Methods for Knowledge Engineering. AISC, vol. 358, pp. 391–402. Springer, Cham (2015). doi: 10.1007/978-3-319-17996-4_35 CrossRefGoogle Scholar
  22. 22.
    Omar, E., Ghosh, S., Whitley, D.: Constructing subtle higher order mutants for Java and AspectJ programs. In: 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE). IEEE (2013)Google Scholar
  23. 23.
    Ma, Y.-S., Offutt, J., Kwon, Y.-R.: MuJava: a mutation system for Java. In: Proceedings of the 28th International Conference on Software Engineering. ACM (2006)Google Scholar
  24. 24.
    Just, R., et al.: Are mutants a valid substitute for real faults in software testing? In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM (2014)Google Scholar
  25. 25.
    Weitao, W., Hirohide, H.: Improvement of equivalent mutant detection using loop count restriction. In: The International Conference on Software Engineering, Mobile Computing and Media Informatics (SEMCMI 2015) (2015)Google Scholar
  26. 26.
    Diehl, S.: A formal introduction to the compilation of Java. Softw.-Pract. Experience 28(3), 297–327 (1998)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  1. 1.Dependable Distributed Embedded Systems (DDEmS) Laboratory, Computer Engineering DepartmentFerdowsi University of MashhadMashhadIran

Personalised recommendations