Classical Test Theory

Abstract

Classical test theory (CTT) rests on the assumption of a normal distribution of scores in some population and assumes scores are not at the extremes of the possible range. In CTT, a person’s observed test score is a sum of a true score and an error score. The test’s reliability is the central index of CTT and is the ratio of true score variance to observed score variance. A person’s true score, with a confidence interval, is estimated from the observed test score using the reliability index. Although not formalized in CTT, two descriptive indices used in CTT are the facility and the discrimination of an item. The former is the percentage of persons who answer an item correctly, and the latter is the correlation between the scores on the item and the scores on the test. The latter values are expected to be similar across items.

Exercises

Table 3.2 shows an example of the results of the performance of 25 students on an examination in economics. The examination had:

Table 3.2 Responses of 25 persons to eight items

• five multiple choice questions which were scored as either wrong (0) or correct (1),

• two short answer questions in which the maximum score was 2 (0 for totally incorrect, 1 for partially correct and 2 for totally correct), and

• one question with a longer answer worth 6 marks.

This gave a total number of 15 marks.

• 1. Calculate the facility and discrimination for items 7 and 8. According to these indices, which of these two items is more difficult and which of the two discriminates more?

• 2. Calculate the estimate of the true score for a student with a score of 13 on the test.

• 3. Calculate the mean and SD of the scores. If the test had a reliability of 0.80, what would be the standard error of measurement for the scores?

• 4. What would be the 90% confidence interval for a student who scores 13 on the test?

• 5. Calculate the variance of the true scores from the estimate of the standard error.

For further exercises using this example, see Exercise 1: Interpretation of RUMM2030 printout in Appendix C.