General and Theoretical Considerations in the Assessment of Reliability

Abstract

The term reliability is often used to describe the temporal stability of a set of measurement procedures. Other frequent uses of the term are related to internal consistency and agreement among different users of the procedures. These different usages all relate to a single concept, namely, the estimation of the influence of error on the scores resulting from the use of the test or set of procedures. A perfectly reliable test is a test that measures without error. Error-free measurement is a practical impossibility. Instead, it is an ideal to which test authors and developers attempt approximations. As well as being an impossible ideal to attain in a practical sense, reliability is also an impossible concept to evaluate directly. All of the methods and designs discussed in Chapter 3 are able only to estimate the reliability of the test. As will be seen in the more complete discussion in Chapter 3, the methods and designs are attempts to estimate the degree of error that influences test scores by systematically varying the possible sources of error. The success of the endeavor is related to the quality of the methods and designs and to the ability of the researcher to comprehensively describe the possible sources of error.