Advertisement

Using a Modified Multidimensional Priority Index for Item Selection Under Within-Item Multidimensional Computerized Adaptive Testing

  • Ya-Hui SuEmail author
  • Yen-Lin Huang
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 89)

Abstract

Computerized adaptive testing (CAT) not only enables efficient and precise ability estimation but also increases the security of testing materials since examinees are given different sets of items from a large item bank. The construction of assessments usually involves fulfilling a large number of non-statistical constraints, such as item exposure control and content balancing. To improve measurement precision, test security, and test validity, the priority index (PI) and multidimensional priority index (MPI) were proposed to monitor many constraints simultaneously for unidimensional and multidimensional CATs, respectively. Many educational and psychological tests are constructed under a multidimensional framework. Some of the items (multidimensional items) in a test are often intended to assess multiple latent traits. However, Yao’s MPI method was developed for a between-item multidimensional framework. When a within-item multidimensional test is assembled, a modified MPI algorithm is necessary. Therefore, the purposes of the study were to derive an algorithm for the modified MPI method for the within-item multidimensional CATs and to investigate the efficiency of the modified MPI method through simulations.

Keywords

CAT Priority index Multidimensional Item selection IRT 

References

  1. Birnbaum A (1968) Some latent trait models and their use in inferring an examinee’s ability. In: Lord FM, Novick MR (eds) Statistical theories of mental test scores. Addison-Wesley, Reading, pp 397–479Google Scholar
  2. Chang H-H, van der Linden WJ (2003) Optimal stratification of item pools in alpha-stratified computerized adaptive testing. Appl Psychol Meas 27:262–274CrossRefMathSciNetGoogle Scholar
  3. Chang H-H, Ying Z (1996) A global information approach to computerized adaptive testing. Appl Psychol Meas 20:213–229CrossRefGoogle Scholar
  4. Chang H-H, Ying Z (1999) a-stratified multistage computerized adaptive testing. Appl Psychol Meas 23:211–222CrossRefGoogle Scholar
  5. Chang H-H, Ying Z (2007) To weight or not to weight? Balancing influence of initial items in adaptive testing. Psychometrika. Prepublished online on December 8, 2007. doi:10.1007/s11336-007-9047-7Google Scholar
  6. Chang H-H, Qian J, Ying Z (2001) a-stratified multistage CAT with b-blocking. Appl Psychol Meas 25:333–341CrossRefMathSciNetGoogle Scholar
  7. Cheng Y, Chang H-H (2009) The maximum priority index method for severely constrained item selection in computerized adaptive testing. Br J Math Stat Psychol 62:369–383CrossRefMathSciNetGoogle Scholar
  8. Cheng Y, Chang H-H, Yi Q (2007) Two-phase item selection procedure for flexible content balancing in CAT. Appl Psychol Meas 31:467–482CrossRefMathSciNetGoogle Scholar
  9. Cheng Y, Chang H-H, Douglas J, Guo F (2009) Constraint-weighted a-stratification for computerized adaptive testing with nonstatistical constraints: balancing measurement efficiency and exposure control. Educ Psychol Meas 69:35–49CrossRefMathSciNetGoogle Scholar
  10. Hattie J (1981) Decision criteria for determining unidimensionality. Unpublished doctoral dissertation, University of Toronto, CanadaGoogle Scholar
  11. Hua K, Chang H-H (2001) Item selection in computerized adaptive testing: should more discriminating items be used first? J Educ Meas 38:249–266CrossRefGoogle Scholar
  12. Reckase MR (1985) The difficulty of test items that measure more than one dimension. Appl Psychol Meas 9:401–412CrossRefGoogle Scholar
  13. Segall DO (1996) Multidimensional adaptive testing. Psychometrika 61:331–354CrossRefzbMATHGoogle Scholar
  14. Stocking ML, Swanson L (1993) A method for severely constrained item selection in adaptive testing. Appl Psychol Meas 17:277–292CrossRefGoogle Scholar
  15. Thissen D, Mislevy RJ (2000) Testing algorithms. In: Wainer H (ed) Computerized adaptive testing: a primer, 2nd edn. Erlbaum, Mahwah, pp 101–133Google Scholar
  16. van der Linden WJ, Chang H-H (2003) Implementing content constraints in alpha-stratified adaptive testing using a shadow test approach. Appl Psychol Meas 27:107–120CrossRefMathSciNetGoogle Scholar
  17. Wang W-C, Chen P-H (2004) Implementation and measurement efficiency of multidimensional computerized adaptive testing. Appl Psychol Meas 28:295–316CrossRefMathSciNetGoogle Scholar
  18. Yao L (2011) Multidimensional CAT item selection procedures with item exposure control and content constraints. Paper presented at the (2011) International Association of Computer Adaptive Testing (IACAT) conference, Pacific Grove, CA, OctoberGoogle Scholar
  19. Yao L (2012) Multidimensional CAT item selection methods for domain scores and composite scores: theory and applications. Psychometrika 77:495–523CrossRefzbMATHMathSciNetGoogle Scholar
  20. Yao L (2013) Comparing the performance of five multidimensional CAT selection procedures with different stopping rules. Appl Psychol Meas 37:3–23CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of PsychologyNational Chung Cheng UniversityChiayi CountyTaiwan

Personalised recommendations