Advertisement

The Impact of Model Misspecification with Multidimensional Test Data

  • Sakine Gocer SahinEmail author
  • Cindy M. Walker
  • Selahattin Gelbal
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 140)

Abstract

In this study data were simulated for 5000 examinees on a thirty-item two-dimensional test, using a compensatory MIRT model. Various combinations of simple and complex structure items were examined. Specifically, the numbers of simple structure items on the tests were gradually decreased from 24 to 6, in multiples of six, while simultaneously increasing the number of complex items by the same number of items. In one scenario, the simple structure items were simulated to measure both dimensions equally; in a second scenario, the simple structured items were simulated to measure only the first dimension. The current investigation also varied the correlation between dimensions and the ability distributions on the first and second dimensions. RMSE was used to determine the impact of model misspecification and the results of a unidimensional simulated and scaled test were used for comparison purposes. Results indicated that the underlying structure of multidimensional tests did have an impact on estimation error. However, in some instances fitting a unidimensional model to multidimensional data resulted in estimation error that was not very dissimilar from what was obtained when fitting a unidimensional model to unidimensional data.

Keywords

Multidimensionality Unidimensionality Item response theory 

References

  1. Ackerman, T. A. (1989). Unidimensional IRT calibration of compensatory and noncompensatory items. Applied Psychological Measurement, 13, 113–127.CrossRefGoogle Scholar
  2. Ackerman, T. A. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29(1), 67–91.CrossRefGoogle Scholar
  3. Ansley, T. N., & Forsyth, R. A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two dimensional data. Applied Psychological Measurement, 9(1), 37–48.CrossRefGoogle Scholar
  4. Hambleton, R. K., Swaminathan, H., Cook, L. L., Eignor, D. R., & Gifford, J. A. (1978). Developments in latent trait theory: Models, technical issues, and applications. Review of Educational Research, 48, 467–510.Google Scholar
  5. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.Google Scholar
  6. Harrison, D. A. (1986). Robustness of IRT parameter estimation to violations of the unidimensionality assumption. Journal of Educational Statistics, 11(2), 91–115.CrossRefGoogle Scholar
  7. Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9(2), 139–164.CrossRefGoogle Scholar
  8. Kahraman, N. (2013). Unidimensional interpretations for multidimensional test items. Journal of Educational Measurement, 50(2), 227–246.CrossRefGoogle Scholar
  9. Kirisci, L., Hsu, T., & Yu, L. (2001). Robustness of item parameter estimation programs to assumptions of unidimensionality and normality. Applied Psychological Measurement, 25, 146–162.MathSciNetCrossRefGoogle Scholar
  10. Lord, F. M., & Novick, R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.zbMATHGoogle Scholar
  11. Luecht, R. M., & Miller, T. R. (1992). Unidimensional calibrations and interpretations of composite traits for multidimensional tests. Applied Psychological Measurement, 16, 279–293.CrossRefGoogle Scholar
  12. McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: L. Erlbaum Associates.Google Scholar
  13. Raykov, T. (1997). Bias of coefficient alpha for congeneric measures with correlated errors. Applied Psychological Measurement, 25, 69–76.MathSciNetCrossRefGoogle Scholar
  14. Reckase, M. D., & McKinley, R. L. (1991). The discriminating power of items that measure more than one dimension. Applied Psychological Measurement, 15, 361–373.Google Scholar
  15. Reckase, M. D. (1997). A linear logistic multidimensional model for dichotomous item response data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 271–286). New York: Springer.CrossRefGoogle Scholar
  16. Reckase, M. D. (2009). Multidimensional item response theory (Statistics for social and behavioral sciences). New York: Springer.CrossRefGoogle Scholar
  17. Reckase, M. D., Ackerman, T. A., & Carlson, J. E. (1988). Building a unidimensional test using multidimensional items. Journal of Educational Measurement, 25, 193–203.CrossRefGoogle Scholar
  18. Stout, W. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589–617.MathSciNetCrossRefzbMATHGoogle Scholar
  19. Traub, R. E. (1983). A priori considerations in choosing an item response model. In R. K. Hambleton (Ed.), Applications of item response theory (pp. 57–70). British Columbia: Educational Research Institute of British Columbia.Google Scholar
  20. Walker, C. M., Azen, R., & Schmitt, T. (2006). Statistical versus substantive dimensionality the effect of distributional differences on dimensionality assessment using DIMTEST. Educational and Psychological Measurement, 66(5), 721–738.MathSciNetCrossRefGoogle Scholar
  21. Zhang, J. (2005). Estimating multidimensional item response models with mixed structure. ETS Research Report, RR-05-04.Google Scholar
  22. Zhang, B. (2008). Application of unidimensional item response models to tests with items sensitive to secondary dimension. The Journal of Experimental Education, 77(2), 147–166.CrossRefGoogle Scholar
  23. Zhang, J. (2012). Calibration of response data using MIRT models with simple and mixed structures. Applied Psychological Measurement, 36, 375–398.CrossRefGoogle Scholar
  24. Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach’s α, Revelle’s β and McDonald’s ω: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 1–11.MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Sakine Gocer Sahin
    • 1
    Email author
  • Cindy M. Walker
    • 2
  • Selahattin Gelbal
    • 1
  1. 1.Hacettepe University, Hacettepe Universitesi Egitim Fakultesi Egitim Bilimleri BolumuAnkaraTurkey
  2. 2.University of Wisconsin-MilwaukeeMilwaukeeUSA

Personalised recommendations