Item-fit evaluation in biased tests: a study under Rasch model
- 89 Downloads
In this paper, the power rates and distributional properties of the Outfit, Infit, Lz, ECI2z and ECI4z statistics when they are used in tests with biased or differential item functioning (DIF) were explored. In this study, different conditions of sample size, sample size ratio focal and reference group, impact between groups, DIF effect size, and percentage of DIF items were manipulated. In addition, examinee responses were generated to simulate uniform DIF. Results suggest that item fit statistics generally detected medium percents of DIF in large samples (1000/500 or 1000/1000) only when DIF effect size was relatively high and when the mean of focal and reference group was different. Moreover, when groups had equal mean, low correct identification rates were found in the five item-fit indices. In general, the results showed adequate control of false positive rates. These findings lead to the conclusion that all indices used in this study are partially adequate fit measures for detecting biased items, mainly when impact between groups is present and sample size is large.
KeywordsItem-fit statistics Rasch model Differential item functioning Type I error rate Power rate
Unable to display preview. Download preview PDF.
- Drasgow F., Levine M.V., Williams E.A.: Appropriateness measurement with polychotomous item response models and standardized indices. Br. J. Math. Stat. Psychol. 38, 67–86 (1985)Google Scholar
- Linacre J.M., Wright B.D.: BIGSTEPS: Rasch-Model Computer Program. MESA Press, Chicago (1993)Google Scholar
- Lord F.M.: Applications of Item Response Theory to Practical Testing Problems. Erlbaum, Hillsdale (1980)Google Scholar
- Rasch, G.: Probabilistic Models for Some Intelligence and Attainment Test. The Danish Institute of Educational Research, Copenhagen (1960). (Expanded edition, 1980. The University Chicago Press, Chicago)Google Scholar
- Seol H.: Detecting differential item functioning with five standardized item-fit indices in the Rasch model. J. Outcome Meas. 3, 233–247 (1999)Google Scholar
- Smith R.M.: Detecting item bias with the Rasch model. J. Appl. Meas. 5, 430–449 (2004)Google Scholar
- Smith R.M., Schumaker R.E., Bush M.J.: Using item mean squares to evaluate fit to the Rasch model. J. Outcome Meas. 2, 66–78 (1998)Google Scholar
- Wright B.D., Masters G.N.: Rating Scale Analysis. MESA Press, Chicago (1982)Google Scholar