Estimating Person Proficiency and Person Separation

Abstract

The sum of the theoretical means (probabilities) of the number of times each item would be answered correctly should be equal to the number of items that are answered correctly $r_{n}$ by person n. From this equation $\beta_{n}$ can be estimated. Equations are solved iteratively until convergence. The same total score, for the same items completed, gives the same proficiency estimate. Each estimate has a standard error. The transformation from the total scores to the estimates is non-linear. Extrapolated estimates can be provided for the minimum and maximum total scores. The person-item distribution displays the frequencies of item and person estimates on the same continuum. In the principle of maximum likelihood, an estimate for each person is found that gives the maximum value for the joint probability of the matrix of responses across persons and items. As in CTT, the Rasch measurement theoryreliability index, the person separation index (PSI), is defined as the proportion of true variance to total variance. The variance of the proficiency estimates and the variance of their standard errors are used in its calculation.

Exercises

• 1. In this chapter, an example of a person answering 10 items was used to illustrate the estimate of a person’s proficiency. The item difficulties were $\delta_{1} = - 2.5$, $\delta_{2} = - 2.0$, $\delta_{3} = - 1.5$, $\delta_{4} = - 1.0$, $\delta_{5} = - 0.5$, $\delta_{6} = 0.5$, $\delta_{7} = 1.0$, $\delta_{8} = 1.05$, $\delta_{9} = 1.5$, $\delta_{10} = 2.0$.

  The person answered six of the items correctly, that is $r_{n} = 6$ = 6. An initial value of $\beta_{n}^{(0)} = 0.25$ was used as an estimate of the person’s proficiency, and inserted into Eq. (10.5) to give

  $$ \begin{aligned} \sum\limits_{i\, = \,1}^{10} {\frac{{e^{{\beta_{n} - \delta_{i}}}}}{{1 + e^{{\beta_{n} - \delta_{i}}}}}} & = \frac{{e^{{\beta_{n}^{(0)} - \delta_{1}}}}}{{1 + e^{{\beta_{n}^{(0)} - \delta_{1}}}}} + \frac{{e^{{\beta_{n}^{(0)} - \delta_{2}}}}}{{1 + e^{{\beta_{n}^{(0)} - \delta_{2}}}}} + \cdots \cdots + \frac{{e^{{\beta_{n}^{(0)} - \delta_{10}}}}}{{1 + e^{{\beta_{n}^{(0)} - \delta_{10}}}}} \\ & = \frac{{e^{0.25 + 2.5}}}{{1 + e^{0.25 + 2.5}}} + \frac{{e^{0.25 + 2.0}}}{{1 + e^{0.25 + 2.0}}} + \cdots \cdots + \frac{{e^{0.25 - 2.0}}}{{1 + e^{0.25 - 2.0}}} \\ & = 0.94 + 0.90 + 0.85 + 0.78 + 0.68 \\ & \quad + 0.44 + 0.32 + 0.31 + 0.22 + 0.15 = 5.59. \\ \end{aligned} $$

  Because 5.59 is less than 6, a proficiency value a little greater than 0.25 was tried, in particular, $\beta_{n}^{(1)} = 0.40$ to give

  $$ \begin{aligned} \sum\limits_{i\, = \,1}^{10} {\frac{{e^{{\beta_{n} - \delta_{i}}}}}{{1 + e^{{\beta_{n} - \delta_{i}}}}}} & = \frac{{e^{{\beta_{n}^{(1)} - \delta_{1}}}}}{{1 + e^{{\beta_{n}^{(1)} - \delta_{1}}}}} + \frac{{e^{{\beta_{n}^{(1)} - \delta_{2}}}}}{{1 + e^{{\beta_{n}^{(1)} - \delta_{2}}}}} + \cdots \cdots + \frac{{e^{{\beta_{n}^{(1)} - \delta_{10}}}}}{{1 + e^{{\beta_{n}^{(1)} - \delta_{10}}}}} \\ & = \frac{{e^{0.40 + 2.5}}}{{1 + e^{0.40 + 2.5}}} + \frac{{e^{0.40 + 2.0}}}{{1 + e^{0.40 + 2.0}}} + \cdots \cdots + \frac{{e^{0.40 - 2.0}}}{{1 + e^{0.40 - 2.0}}} \\ & = 0.95 + 0.92 + 0.87 + 0.80 + 0.71 + 0.48 \\ & \quad + 0.35 + 0.34 + 0.25 + 0.17 = 5.84. \\ \end{aligned} $$

  This sum of 5.84 is again less than 6. Therefore, the proficiency estimate must be greater than 0.40.

  • (a) Try the value $\beta_{n}^{(2)} = 0.45$ in Eq. (10.5) as above. Is the required proficiency estimate greater than 0.45 or less than 0.45?

  • (b) Try the value $\beta_{n}^{(3)} = 0.55$ in Eq. (10.5) as above. Is the required proficiency estimate greater than 0.55 or less than 0.55?

  • (c) Try the value $\beta_{n}^{(4)} = 0.50$ in Eq. (10.5) as above. Which of these values that have been tried is the best estimate?

• 2. Calculate the PSI index of reliability for data where $\hat{\sigma}_{{\hat{\beta}}}^{2} = 1.51$ and $\hat{\sigma}_{{\hat{\varepsilon}}}^{2} = 0.32$.