New IRT Models for Examinee-Selected Items
Examinee-selected-item (ESI) design, in which examinees are required to respond to a fixed number of items in a given set of items (e.g., responding to two items in five given items; leading to ten selection patterns), has the advantages of enhancing students’ learning motivation and reducing their testing anxiety. The ESI design yields incomplete data (i.e., only those selected items are answered and the others have missing data). It has been argued that missing data in the ESI design are missing not at random, making standard item response theory (IRT) models inappropriate. Recently, Wang et al. (Journal of Educational Measurement 49(4):419–445, 2012) propose an IRT model for examinee-selected items by adding an additional latent trait to standard IRT models to account for the selection effect. This latent trait could correlate with the intended-to-be-measured latent trait, and the correlation quantifies how stronger the selection effect and how serious the violation of the assumption of missing at random are. In this study, we developed a framework to incorporate this model as a special case and generate several new models. We conducted an experiment to collect real data, in which 501 fifth graders took two mandatory items and four pairs of mathematic (dichotomous) items. In each pair of items, students were first asked to indicate which item they preferred to answer and then answered both items. This is referred to as the “Choose one, Answer all” approach. These new IRT models were fit to the real data and the results were discussed.
KeywordsItem response theory Examinee-selected items Selection effect Missing data
This study was supported by the General Research Fund, Hong Kong (No.~844112).
- Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 395–479). Reading, MA: Addison-Wesley.Google Scholar
- Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.Google Scholar
- Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Expanded edition, 1980. Chicago: The University of Chicago Press, ed.). Copenhagen: Institute of Educational Research.Google Scholar
- Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrica Monograph, 17, 1–100.Google Scholar
- Wainer, H., Wang, X.-B., & Thissen, D. (1991). How well can we equate test forms constructed by examinees? (Program Statistics Report 91-55). Princeton, NJ: Educational Testing Service.Google Scholar
- Wang, X. B. (1999). Understanding psychological processes that underlie test takers’ choices of constructed response items. Newtown, PA: Law School Admission Council.Google Scholar