Abstract
This report describes an empirical study comparing three defect detection techniques: a) code reading by stepwise abstraction, b) functional testing using equivalence partitioning and boundary value analysis, and c) structural testing using branch coverage. It is a replication of a study that has been carried out at least four times previously over the last 20 years. This study used 47 student subjects to apply the techniques to small C programs in a fractional factorial experimental design. The major findings of the study are: a) that the individual techniques are of broadly similar effectiveness in terms of observing failures and finding faults, b) that the relative effectiveness of the techniques depends on the nature of the program and its faults, c) these techniques are consistently much more effective when used in combination with each other. These results contribute to a growing body of empirical evidence that supports generally held beliefs about the effectiveness of defect detection techniques in software engineering.
email: murray@cs.strath.ac.uk, telephone: +44 (0)141 552 4400
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
Victor R. Basili and Richard W. Selby. Comparing the effectiveness of software testing strategies. IEEE Transactions on Software Engineering, SE-13(12):1278–1296, December 1987.
A. Brooks, J. Daly, J. Miller, M. Roper, and M. Wood. Replication of experimental results in software engineering. Research report EFoCS-17-94, Department of Computer Science, University of Strathclyde, Glasgow, 1995.
N. Fenton, S. Pfleeger, and R. Glass. Science and substance: A challenge to software engineers. IEEE Software, 11(4):86–95, July 1994.
R. Glass. The software research crisis. IEEE Software, 11(6):42–47, November 1994.
William C. Hetzel. An Experimental Analysis of Program Verification Methods. PhD thesis, University of North Carolina, Chapel Hill, 1976.
C. Jones. Gaps in the object-oriented paradigm. IEEE Computer, 27(6):90–91, June 1994.
Erik Kamsties and Christopher Lott. An empirical evaluation of three defect detection techniques. Technical Report ISERN 95-02, Dept. Computer Science, University of Kaiserslautern, May 1995.
Erik Kamsties and Christopher Lott. An empirical evaluation of three defect detection techniques. In Proc. 5th European Software Engineering Conference, September 1995.
Richard C. Linger, Harlan D. Mills, and B.I. Witt. Structured Programming: Theory and Practice. Addison-Wesley; 1979.
Steve McConnell. Code Complete. Microsoft Press, 1993.
Glenford J. Myers. A controlled experiment in program testing and code walkthroughs/inspections. Communications of the ACM, 21(9):760–768, September 1978.
Roger S. Pressman. Software Engineering — A Practitioners Approach. McGraw-Hill, 1994.
Marc Roper. Software Testing. McGraw-Hill, 1993.
R. W. Selby. Combining software testing strategies: An empirical evaluation. In Proc. Workshop on Software Testing, pages 82–91. IEEE Computer Society Press, July 1986.
B. J. Winer. Statistical Principles in Experimental Design. McGraw-Hill, 2nd edition, 1971.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wood, M., Roper, M., Brooks, A., Miller, J. (1997). Comparing and combining software defect detection techniques: A replicated empirical study. In: Jazayeri, M., Schauer, H. (eds) Software Engineering — ESEC/FSE'97. ESEC SIGSOFT FSE 1997 1997. Lecture Notes in Computer Science, vol 1301. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-63531-9_19
Download citation
DOI: https://doi.org/10.1007/3-540-63531-9_19
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63531-4
Online ISBN: 978-3-540-69592-9
eBook Packages: Springer Book Archive