Towards testing big data analytics software: the essential role of metamorphic testing
- 33 Downloads
In the rapidly growing field of big data analysis, scientists from numerous domains such as computer science and biology are constantly challenged by an unprecedented amount of data. While many software programs have been constructed to support processing and analyzing continuous information flow, one under-appreciated challenge in this field is software quality assurance of these big data software platforms. Metamorphic testing, which was proposed to alleviate the oracle problem in the software engineering community, has become an effective approach for software verification and validation. Recent years, we have witnessed successful applications of metamorphic testing in a variety of domains, ranging from bioinformatics to deep learning. In this letter, we review some main applications of metamorphic testing on big data and present visions for the challenges in future research.
KeywordsSoftware engineering Metamorphic testing Big data software
This work is supported by National Key R&D Program of China (2018YFB1003901), and the National Natural Science Foundation of China (61572375, 61772263).
Compliance with ethical standards
Conflict of interest
Zhiyi Zhang declares that she has no conflict of interest. Xiaoyuan Xie declares that she has no conflict of interest.
This article does not contain any studies with human participants or animals performed by any of the authors.
- Chen TY, Cheung SC, Yiu SM (1998) Metamorphic testing: a new approach for generating next test cases. Tech. rep., Technical Report HKUST-CS98-01, Department of Computer Science. Hong Kong University of Science and Technology, Hong KongGoogle Scholar
- Ding J, Kang X, Hu X (2017) Validating a deep learning framework by metamorphic testing. In: Proceedings of the 2nd international workshop on metamorphic testing. IEEE, pp 28–34. https://doi.org/10.1109/MET.2017.2
- Murphy C, Kaiser G, Hu L, Wu L (2008) Properties of machine learning applications for use in metamorphic testing. In: Proceedings of the 20th international conference on software engineering and knowledge engineering, pp 867–872Google Scholar
- Pullum LL, Ozmen O (2012) Early results from metamorphic testing of epidemiological models. In: ASE/IEEE International Conference on BioMedical Computing (BioMedCom)(BIOMEDCOM), pp 62–67. https://doi.org/10.1109/BioMedCom.2012.17
- Tian Y, Pei K, Jana S, Ray B (2018) Deeptest: automated testing of deep-neural-network-driven autonomous cars. In: Proceedings of the 40th international conference on software engineering. ACM, pp 303–314, DOI https://doi.org/10.1145/3180155.3180220, (to appear in print)
- Zhang M, Zhang Y, Zhang L, Liu C, Khurshid S (2018) Deeproad: Gan-based metamorphic testing and input validation framework for autonomous driving systems. In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering. ACM, pp 132–142, (to appear in print), https://doi.org/10.1145/3238147.3238187