Advertisement

The Performance of the Modified Multidimensional Priority Index for Item Selection in Variable-Length MCAT

  • Ya-Hui SuEmail author
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 140)

Abstract

In addition to statistical optimization, an important issue in computerized adaptive testing (CAT) is to fulfill a large number of statistical and non-statistical constraints for the construction of assessments. The priority index (PI) approaches can be used for item selection to monitor many constraints simultaneously. Many previous studies on item selection methods were conducted in fixed-length multidimensional CAT (MCAT); however, studies in variable-length MCAT were paid little attention. To achieve the same level of precision for examinees, the purpose of the study was to investigate the modified multidimensional priority index(MMPI-v) and multidimensional priority index (MPI) in variable-length MCAT through simulations. It was found that the MMPI-v method outperformed the MPI in terms of constraint management, and the MMPI-v used fewer items than the MPI method did to meet the required measurement precision.

Keywords

Computer adaptive testing Priority index Multidimensional Item selection IRT Variable length 

References

  1. Cheng, Y., & Chang, H.-H. (2009). The maximum priority index method for severely constrained item selection in computerized adaptive testing. British Journal of Mathematical and Statistical Psychology, 62, 369–383.MathSciNetCrossRefGoogle Scholar
  2. Cheng, Y., Chang, H.-H., Douglas, J., & Guo, F. (2009). Constraint-weighted a-stratification for computerized adaptive testing with nonstatistical constraints: Balancing measurement efficiency and exposure control. Educational and Psychological Measurement, 69, 35–49.MathSciNetCrossRefGoogle Scholar
  3. Choi, S. W., Grady, M., & Dodd, B. G. (2011). A new stopping rule for computerized adaptive testing. Educational and Psychological Measurement, 71, 37–73.CrossRefGoogle Scholar
  4. Dodd, B. G., Koch, W. R., & De Ayala, R. J. (1989). Operational characteristics of adaptive testing procedures using the graded response model. Applied Psychological Measurement, 13, 129–143.CrossRefGoogle Scholar
  5. Dodd, B. G., Koch, W. R., & De Ayala, R. J. (1993). Computerized adaptive testing using the partial credit model: Effects of item pool characteristics and different stopping rules. Educational and Psychological Measurement, 53, 61–77.CrossRefGoogle Scholar
  6. Mulder, J., & van der Linden, W. J. (2009). Multidimensional adaptive testing with optimal design criteria for item selection. Psychometrika, 74, 273–296.MathSciNetCrossRefzbMATHGoogle Scholar
  7. Reckase, M. D. (1985). The difficulty of test items that measure more than one dimension. Applied Psychological Measurement, 9, 401–412.CrossRefGoogle Scholar
  8. Reckase, M. D. (2009). Multidimensional item response theory. New York, NY: Springer.CrossRefGoogle Scholar
  9. Su, Y.-H., & Huang, Y.-L. (2014). Using a modified multidimensional priority index for item selection under within-item multidimensional computerized adaptive testing. In R. E. Millsap, D. M. Bolt, L. A. van der Ark, & W.-C. Wang (Eds.), Quantitative psychology research: The 78th annual meeting of the psychometric society (pp. 227–242). Switzerland: Springer.Google Scholar
  10. van der Linden, W. J. (1999). Multidimensional adaptive testing with a minimum error-variance criterion. Journal of Educational and Behavioral Statistics, 24, 398–412.CrossRefGoogle Scholar
  11. Veldkamp, B. P., & van der Linden, W. J. (2002). Multidimensional adaptive testing with constraints on test content. Psychometrika, 67, 575–588.MathSciNetCrossRefzbMATHGoogle Scholar
  12. Wainer, H. (Ed.). (2000). Computerized adaptive testing: A primer (2nd ed.). Mahwah, NJ: Erlbaum.Google Scholar
  13. Wang, C., Chang, H.-H., & Boughton, K. (2011a). Kullback–Leibler information and its applications in multi-dimensional adaptive testing. Psychometrika, 76, 13–39.Google Scholar
  14. Wang, C., Chang, H.-H., & Huebner, A. (2011b). Restrictive stochastic item selection methods in cognitive diagnostic computerized adaptive testing. Journal of Educational Measurement, 48, 255–273.Google Scholar
  15. Yao, L. (2011, October). Multidimensional CAT item selection procedures with item exposure control and content constraints. Paper presented at the (2011) International Association of Computer Adaptive Testing (IACAT) Conference, Pacific Grove, CA.Google Scholar
  16. Yao, L. (2012). Multidimensional CAT item selection methods for domain scores and composite scores: Theory and applications. Psychometrika, 77, 495–523.MathSciNetCrossRefzbMATHGoogle Scholar
  17. Yao, L. (2013). Comparing the performance of five multidimensional CAT selection procedures with different stopping rules. Applied Psychological Measurement, 37, 3–23.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of PsychologyNational Chung Cheng UniversityChiayi CountyTaiwan

Personalised recommendations