What Do You Mean by a Difficult Item? On the Interpretation of the Difficulty Parameter in a Rasch Model

Abstract

Three versions of the Rasch model are considered: the fixed-effects model, the random-effects model with normal distribution, and the random-effects model with unspecified distribution. For each of the three, we discuss the meaning of the difficulty parameter starting each time from the corresponding likelihood and the resulting identified parameters. Because the likelihood and the identified parameters are different depending on the model, the identification of the parameter of interest is also different, with consequences for the meaning of the item difficulty. Finally, for all the three models, the item difficulties are monotonically related to the marginal probabilities of a correct response.

Appendix

1.1.1 Proof of Equality (1.13)

Consider the reparameterization $\eta _{i} =\exp (\beta _{i})$ and let

$$\displaystyle{\mathcal{A}_{\mathcal{J}} =\bigcap _{j\in \mathcal{J}}\{Y _{pj} = 1\}\;\; \cap \bigcap _{j\in \mathcal{J}^{c}\setminus \{1,i\}}\{Y _{pj} = 0\}.}$$

Let $\mathcal{J} \subset \{ 2,\ldots,I\}$ and $i\notin \mathcal{J}$. Using (1.12), it follows that

$$\displaystyle\begin{array}{rcl} m_{G}(\vert \mathcal{J}\vert + 1)& =& P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ pi} = 1\} \cap \bigcap _{j\in \mathcal{J}}\{Y _{pj} = 1\} \cap \bigcap _{j\in (\mathcal{J}\cup \{i\})^{c}}\{Y _{pj} = 0\}\right ) \times \prod _{j\in \mathcal{J}}\eta _{j} \times \eta _{i} {}\\ & =& P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ p1} = 1\} \cap \bigcap _{j\in \mathcal{J}}\{Y _{pj} = 1\} \cap \bigcap _{j\in \mathcal{J}^{c}\setminus \{1\}}\{Y _{pj} = 0\}\right ) \times \prod _{j\in \mathcal{J}}\eta _{j}. {}\\ \end{array}$$

It follows that

$$\displaystyle{ \eta _{i} = \frac{P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G) }\left (\{Y _{p1} = 1,Y _{pi} = 0\} \cap \mathcal{A}_{\mathcal{J}}\}\right )} {P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G) }\left (\{Y _{p1} = 0,Y _{pi} = 1\} \cap \mathcal{A}_{\mathcal{J}}\}\right )}. }$$

(1.16)

for all $\mathcal{J} \subset \{ 2,\ldots,I\}$ and $i\notin \mathcal{J}$. Therefore, using (1.16),

$$\displaystyle\begin{array}{rcl} \frac{P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}(Y _{pi} = 0,Y _{pj} = 1)} {P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}(Y _{ pi} = 1,Y _{pj} = 0)}& =& \frac{\sum _{\{\mathcal{J}\subset \{2,\ldots,I\}:i\notin \mathcal{J}\}}P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ p1} = 1,Y _{pi} = 0\} \cap \mathcal{A}_{\mathcal{J}}\right )} {\sum _{\{\mathcal{J}\subset \{2,\ldots,I\}:i\notin \mathcal{J}\}}P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ p1} = 0,Y _{pi} = 1\} \cap \mathcal{A}_{\mathcal{J}}\right )} {}\\ & =& \frac{\sum _{\{\mathcal{J}\subset \{2,\ldots,I\}:i\notin \mathcal{J}\}}\eta _{i}\;P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ p1} = 0,Y _{pi} = 1\} \cap \mathcal{A}_{\mathcal{J}}\right )} {\sum _{\{\mathcal{J}\subset \{2,\ldots,I\}:i\notin \mathcal{J}\}}P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ p1} = 0,Y _{pi} = 1\} \cap \mathcal{A}_{\mathcal{J}}\right )}{}\\ &=&\eta _{ i}. {}\\ \end{array}$$

1.1.2 Proof of Equality (1.14)

Let $\mathcal{J}$ such that $\vert \mathcal{J}\vert = I - 2$ and denote the label of two items excluded from $\mathcal{J}$ as i and i′. Using (1.12), it follows that

$$\displaystyle\begin{array}{rcl} m_{G}(\vert \mathcal{J} \cup \{ i\}\vert )& =& P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ pi} = 1\} \cap \bigcap _{j\in \mathcal{J}}\{Y _{pj} = 1\} \cap \{ Y _{pi'} = 0\} \cap \bigcap _{j\in \mathcal{J}^{c}\setminus \{i'\}}\{Y _{pj} = 0\}\right ) \times {}\\ & &\times \prod _{j\in \mathcal{J}}\eta _{j} \times \eta _{i}, {}\\ & & {}\\ m_{G}(\vert \mathcal{J} \cup \{ i'\}\vert )& =& P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ pi'} = 1\} \cap \bigcap _{j\in \mathcal{J}}\{Y _{pj} = 1\} \cap \{ Y _{pi} = 0\} \cap \bigcap _{j\in \mathcal{J}^{c}\setminus \{i\}}\{Y _{pj} = 0\}\right ) \times {}\\ & &\times \prod _{j\in \mathcal{J}}\eta _{j} \times \eta _{i'}. {}\\ \end{array}$$

Therefore,

$$\displaystyle{ \frac{\eta _{i}} {\eta _{i'}} = \frac{P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ pi} = 0,Y _{pi'} = 1\} \cap \bigcap _{j\in \mathcal{J}}\{Y _{pj} = 1\} \cap \bigcap _{j\in \mathcal{J}^{c}\setminus \{i\}}\{Y _{pj} = 0\}\right )} {P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}\left (\{Y _{ pi} = 1,Y _{pi'} = 0\} \cap \bigcap _{j\in \mathcal{J}}\{Y _{pj} = 1\} \cap \bigcap _{j\in \mathcal{J}^{c}\setminus \{i'\}}\{Y _{pj} = 0\}\right )}. }$$

(1.17)

Let $\mathcal{J} \subset \{ 1,\ldots,I\}$ such that $\vert \mathcal{J}\vert = I - 2$ and take $i,i'\notin \mathcal{J}$. Denote

$$\displaystyle{\mathcal{A}_{\mathcal{J}} =\bigcap _{j\in \mathcal{J}}\{Y _{pi} = 1\}\quad \cap \bigcap _{j\in \mathcal{J}^{c}\setminus \{i\}}\{Y _{pi} = 0\},}$$

$$\displaystyle{\mathcal{B}_{\mathcal{J}} =\bigcap _{j\in \mathcal{J}}\{Y _{pi} = 1\}\quad \cap \bigcap _{j\in \mathcal{J}^{c}\setminus \{i'\}}\{Y _{pi} = 0\}.}$$

Then, using (1.17),

$$\displaystyle\begin{array}{rcl} \frac{P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}(Y _{pi}=0,Y _{pi'}=1)} {P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}(Y _{ pi=1},Y _{pi'}=0)} & =& \frac{\sum _{\{\mathcal{J}\subset \{1,\ldots,I\}:\vert \mathcal{J}=I-2,i,i'\notin \mathcal{J}\}}P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}(\{Y _{ pi}=0,Y _{pi'}=1\} \cap \mathcal{A}_{\mathcal{J}})} {\sum _{\{\mathcal{J}\subset \{1,\ldots,I\}:\vert \mathcal{J}=I-2,i,i'\notin \mathcal{J}\}}P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}(\{Y _{ pi}=1,Y _{pi'}=0\} \cap \mathcal{B}_{\mathcal{J}})} {}\\ & =& \frac{\eta _{i}} {\eta _{i'}}\,\frac{\sum _{\{\mathcal{J}\subset \{1,\ldots,I\}:\vert \mathcal{J}=I-2,i,i'\notin \mathcal{J}\}}P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}(\{Y _{ pi}=1,Y _{pi'}=0\} \cap \mathcal{B}_{\mathcal{J}})} {\sum _{\{\mathcal{J}\subset \{1,\ldots,I\}:\vert \mathcal{J}=I-2,i,i'\notin \mathcal{J}\}}P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}(\{Y _{ pi}=1,Y _{pi'}=0\} \cap \mathcal{B}_{\mathcal{J}})} {}\\ & =& \frac{\eta _{i}} {\eta _{i'}}. {}\\ \end{array}$$

1.1.3 Proof of Relation (1.15)

Using the same notation introduced above and the ratio $\eta _{i}'/\eta _{i'}$, it follows that

$$\displaystyle\begin{array}{rcl} \frac{P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G) }(Y _{pi'} = 1)} {P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G)}(Y _{pi} = 1)}& =& \frac{\sum _{\{\mathcal{J}:\vert \mathcal{J}\vert =I-2,i,i'\notin \mathcal{J}\}}P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G) }(\{Y _{pi'} = 1,Y _{pi} = 0\} \cap \mathcal{A}_{\mathcal{J}})} {\sum _{\{\mathcal{J}:\vert \mathcal{J}\vert =I-2,i,i'\notin \mathcal{J}\}}P^{(\mathbf{\beta }_{1:I}^{\mbox{ RE-U}},G) }(\{Y _{pi'} = 0,Y _{pi} = 1\} \cap \mathcal{B}_{\mathcal{J}})} {}\\ & =& \frac{\eta _{i}} {\eta _{i'}}. {}\\ \end{array}$$