Abstract
The Glass Box Test (GBT), also known as White Box Test or Structural Test, shows which parts of the program under test have, or have not, been executed. Many GBT tools are available for almost any programming language. Industry standards for safety-critical software require a very high or even complete coverage. At first glance, the GBT seems to be a well-established and mature testing technique that is based on standardized metrics. But on closer inspection, there are several serious shortcomings of the underlying models and metrics which lead to very imprecise, inconsistent coverage results of the various GBT tools. In this paper, a new and precise model for the GBT is presented. This model is used as a reference for the precise definition of all the popular coverage metrics that are around. The tool CodeCover which was developed in the University of Stuttgart is an implementation that strictly follows those definitions.
Keywords
Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Ammann, P., Offutt, A.J., Hong, H.S.: Coverage Criteria for Logical Expressions. In: Proc. International Symposium on Software Reliability Engineering, pp. 99–107 (2003)
Andrews, J.H., et al.: Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria. IEEE Trans. Softw. Eng. 32(8), 608–624 (2006)
Beizer, B.: Software Testing Techniques. Van Nostrand Reinhold, New York (1990)
Berner, S., Weber, R., Keller, R.K.: Enhancing Software Testing by Judicious Use of Code Coverage Information. In: Proceedings of the 29th International Conference on Software Engineering, ICSE 2007 (2007)
Hanussek, R., et al.: CodeCover - Glass Box Testing Tool, Design, Student Project “OST-WeST”, University of Stuttgart (2008), http://codecover.org/development/Design.pdf
Starzmann, M., et al.: CodeCover - Glass Box Testing Tool, Specification, Student Project “OST-WeST” (2008), http://codecover.org/development/Specification.pdf
CodeCover Homepage, http://codecover.org
Dupuy, A., Leveson, N.: An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software. In: Proc. Digital Aviation Systems Conference (DASC 2000), Philadelphia (2000)
Federal Aviation Administration, Software Verification Tools Assessment Study, DOT/FAA/AR-06/54 (2007), http://www.tc.faa.gov/its/worldpac/techrpt/ar0654.pdf
Fenton, N., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach, 2nd edn. PWS Pub. Co., Boston (1997)
Huang, J.C.: An Approach to Program Testing. ACM Comput. Surv. 7(3), 113–128 (1975)
IEC 61508. Functional safety of electrical/electronic/programable electronic (E/E/PE) safety related systems. Part 1–7, Edition 1.0
Mockus, M., Nagappan, N., Dinh-Trong, T.T.: Test coverage and post-verification defects: A multiple case study. In: Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement (ESEM 2009), pp. 291–301. IEEE Computer Society, Washington, DC (2009)
Myers, G.J.: Art of Software Testing, John Wiley & Sons, Inc., New York (1979)
Schmidberger, R.: Wohldefinierte Überdeckungsmetriken für den Glass-Box-Test (Well-defined Coverage Metrics for the Glass Box Test; in German). Doctoral Dissertation; to be submitted to the Department of Informatics and Electrical Engineering, University of Stuttgart (2013)
Yang, Q., Li, J.J., Weiss, D.: A Survey of Coverage-Based Testing Tools. Comput. J. 52(5), 589–597 (2009)
Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Computing Surveys 29(4), 366–427 (1997)
Dahl, O.-J., Dijkstra, E.W., Hoare, C.A.R.: Structured Programming. Academic Press (1972)
Chilenski, J.J., Miller, S.P.: Applicability of modified condition/decision coverage to software testing. Software Engineering Journal 9(5), 193–200 (1994)
Binder, R.V.: Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley Longman Publishing Co., Inc., Boston (1999)
RTCA-DO-178B, 1992, Software considerations in airborne systems and equipment certification (December 1992)
Reisig, W.: A Primer in Petri Net Design, Springer, Berlin (1992)
Liu, Z., et al.: Test Coverage for Collaborative Workflow Application based on Petri Net. In: Proceedings of the 2010 14th International Conference on Computer Supported Cooperative Work in Design (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 IFIP International Federation for Information Processing
About this paper
Cite this paper
Schmidberger, R. (2014). Well-Defined Coverage Metrics for the Glass Box Test. In: Merayo, M.G., de Oca, E.M. (eds) Testing Software and Systems. ICTSS 2014. Lecture Notes in Computer Science, vol 8763. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44857-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-662-44857-1_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44856-4
Online ISBN: 978-3-662-44857-1
eBook Packages: Computer ScienceComputer Science (R0)