Abstract
Run-time checks are often assumed to be a cost-effective way of improving the dependability of software components, by checking required properties of their outputs and flagging an output as incorrect if it fails the check. Run-time checks’ main point of attractiveness is that they are supposed to be easy to implement. Also, they are implicitly assumed to be effective in detecting incorrect outputs. This paper reports the results of an experiment designed to challenge these assumptions about run-time checks.
The experiment uses a subset of 196 of 867 programs (primaries) solving a problem called “Make Palindrome”. This is an existing problem on the “On-Line Judge” website of the university of Valladolid. We formulated eight run-time checks, and posted this problem on the same web-site. This resulted in 335 programs (checkers) implementing the run-time checks, 115 of which are used for the experiment.
In this experiment: (1) the effectiveness of the population of possibly faulty checkers is very close to the effectiveness of a correct checker; (2) the reliability improvement provided by the run-time checks is relatively small, between a factor of one and three; (3) The reliability improvement gained by using multiple-version redundancy is much higher. Given the fact that this experiment only considers one primary/Run-Time Check combination, it is not yet possible to generalise the results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blum, M., Wasserman, H.: Software reliability via run-time result-checking. Technical Report TR-94-053, International Computer Science Institute (October 1994)
Jhumka, A., Gärtner, F.C., Fetzer, C., Suri, N.: On systematic design of fast and perfect detectors. Technical Report 200263, École Polytechnique Fédérale de Lausanne (EPFDL), School of Computer and Communication Sciences (September 2002)
Lee, P.A., Anderson, T.: Fault Tolerance; Principles and Practice. Dependable Computing and Fault-Tolerant Systems, 2nd edn., vol. 3. Springer, Heidelberg (1981)
Leveson, N.G., Cha, S.S., Knight, J.C., Shimeall, T.J.: The use of self checks and voting in software error detection: An empirical study. IEEE Transactions on Software Engineering 16(4), 432–443 (1990)
Meyer, B.: Design by contract. Computer (IEEE) 25(10), 40–51 (1992)
Popov, P., Strigini, L.: The reliability of diverse systems: A contribution using modelling of the fault creation process. In: DSN 2001, International Conference on Dependable Systems and Networks, Göteborg, Sweden (July 2001)
Rela, M., Madeira, H., Silva, J.G.: Experimental evaluation of the fail-silent behavior of programs with consistency checks. In: FTCS-26, Sendai, Japan, pp. 394–403 (1996)
Skiena, S., Revilla, M.: Programming Challenges. Springer, Heidelberg (2003)
van der Meulen, M.J.P., Strigini, L., Revilla, M.: On the effectiveness of run-time checks. In: Winther, R., Gran, B.A., Dahll, G. (eds.) SAFECOMP 2005. LNCS, vol. 3688, pp. 151–164. Springer, Heidelberg (2005)
Vinter, J., Aidemark, J., Folkesson, P., Karlsson, J.: Reducing critical failures for control algorithms using executable assertions and best effort recovery. In: DSN 2001, International Conference on Dependable Systems and Networks, Goteborg, Sweden (2001)
Wasserman, H., Blum, M.: Software reliability via run-time result-checking. Journal of the ACM 44(6), 826–849 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
van der Meulen, M.J.P., Revilla, M.A. (2006). Experiences with the Design of a Run-Time Check. In: Górski, J. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2006. Lecture Notes in Computer Science, vol 4166. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11875567_23
Download citation
DOI: https://doi.org/10.1007/11875567_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45762-6
Online ISBN: 978-3-540-45763-3
eBook Packages: Computer ScienceComputer Science (R0)