Abstract
Mutation testing helps in evaluation of test suite quality and test development. It can be directed to programs of different languages. High cost of a mutation testing process limits its applicability. This paper focuses on mutation testing of Python programs, discussing several issues of mutant creation and execution. It was showed how they can be effectively handled in the Python environment. We discuss introduction of first and higher order mutation in an abstract syntax tree with use of generators, dealing with code coverage with AST, executing mutants via mutant injection into tests. The solutions were used in reengineering of MutPy - a mutation testing tool for Python programs. The improvements were positively verified in mutation experiments.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Python Programming Language, http://python.org
Pérez, F., Granger, B.E., Hunter, J.D.: Python: An Ecosystem for Scientific Computing. Computing in Science & Engineering 13(2), 13–21 (2011)
Jia, Y., Harman, M.: An Analysis and Survey of the Development of Mutation Testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011)
Bottaci, L.: Type Sensitive Application of Mutation Operators for Dynamically Typed Programs. In: Proceedings of 3rd International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 126–131. IEEE Comp. Soc. (2010)
Derezińska, A., Hałas, K.: Analysis of Mutation Operators for the Python Language. In: Zamojski, W., Mazurkiewicz, J., Sugier, J., Walkowiak, T., Kacprzyk, J. (eds.) DepCos-RELCOMEX 2014. AISC, vol. 286, pp. 155–164. Springer, Heidelberg (2014)
Vincenzi, A.M.R., Simao, A.S., Delamro, M.E., Maldonado, J.C.: Muta-Pro: Towards the Definition of a Mutation testing Process. J. of the Brazilian Computer Society 12(2), 49–61 (2006)
Mateo, P.R., Usaola, M.P., Offutt, J.: Mutation at the Multi-Class and System Levels. Science of Computer Programming 78, 364–387 (2013)
Usaola, M.P., Mateo, P.R.: Mutation Testing Cost Reduction Techniques: a Survey. IEEE Software 27(3), 80–86 (2010)
Offut, J., Rothermel, G., Zapf, C.: An Experimental Evaluation of Selective Mutation. In: 15th International Conference on Software Engineering, pp. 100–107 (1993)
Zhang, L., Gligoric, M., Marinov, D., Khurshid, S.: Operator-Based and Random Mutant Selection: Better Together. In: 28th IEEE/ACM Conference on Automated Software Engineering, Palo Alto, pp. 92–102 (2013)
Derezińska, A., Rudnik, M.: Quality Evaluation of Object-Oriented and Standard Mutation Operators Applied to C# Programs. In: Furia, C.A., Nanz, S. (eds.) TOOLS Europe 2012. LNCS, vol. 7304, pp. 42–57. Springer, Heidelberg (2012)
Bluemke, I., Kulesza, K.: Reduction in Mutation Testing of Java Classes. In: Holzinger, A., Libourel, T., Maciaszek, L.A., Mellor, S. (eds.) Proceedings of the 9th International Conference on Software Engineering and Applications, pp. 297–304. SCITEPRESS (2014)
Polo, M., Piattini, M., Garcıa-Rodrıguez, I.: Decreasing the Cost of Mutation Testing with Second-Order Mutants. Software Testing Verification & Reliability 19(2), 111–131 (2009)
Derezinska, A., Hałas, K.: Experimental Evaluation of Mutation Testing Approaches to Python Programs. In: Proceedings of IEEE International Conference on Software Testing, Verification, and Validation Workshops, pp. 156–164. IEEE Comp. Soc. (2014)
Mateo, P.R., Usaola, M.P., Aleman, J.L.F.: Validating 2nd-order Mutation at System Level. IEEE Trans. Softw. Eng. 39(4), 570–587 (2013)
DeMillo, R.A., Offutt, A.J.: Constraint based automatic test data generation. IEEE Transactions on Software Engineering 17(9), 900–910 (1991)
Dalke, A.: Instrumenting the AST, http://www.dalkescientific.com/writings/diary/archive/2010/02/22/instrumenting_the_ast.html
Derezinska, A.: Quality Assessment of Mutation Operators Dedicated for C# Programs. In: Proceedings of 6th International Conference on Quality Software, QSIC 2006, pp. 227–234. IEEE Computer Society Press, California (2006)
Derezińska, A., Szustek, A.: Object-Oriented Testing Capabilities and Performance Evaluation of the C# Mutation System. In: Szmuc, T., Szpyrka, M., Zendulka, J. (eds.) CEE-SET 2009. LNCS, vol. 7054, pp. 229–242. Springer, Heidelberg (2012)
VisualMutator, http://visualmutator.github.io/web/
Elcap, http://githup.com/sk-/elcap
Campbell, J.C., Hindle, A., Amaral, J.N.: Python: Where the Mutants Hide, or Corpus-based Coding Mistake Location in Dynamic Languages, http://webdocs.cs.ualberta.ca/~joshua2/python.pdf
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Derezinska, A., Hałas, K. (2015). Improving Mutation Testing Process of Python Programs. In: Silhavy, R., Senkerik, R., Oplatkova, Z., Prokopova, Z., Silhavy, P. (eds) Software Engineering in Intelligent Systems. Advances in Intelligent Systems and Computing, vol 349. Springer, Cham. https://doi.org/10.1007/978-3-319-18473-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-18473-9_23
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18472-2
Online ISBN: 978-3-319-18473-9
eBook Packages: EngineeringEngineering (R0)