Abstract
To fulfill a large number of statistical and non-statistical constraints in computerized adaptive testing (CAT), the maximum priority index approaches can be used to handle many constraints simultaneously and efficiently for the construction of assessments. Many previous studies in CAT were conducted for dichotomously scored items; however, only few studies were conducted for polytomously scored items. In practice, because polytomous items are more informative, polytomous CAT tends to need fewer items than dichotomous CAT does. Many important issues in polytomous CAT need further attention. Therefore, the purpose of the study was to investigate constraint-weighted item selection procedures in polytomous CAT. The generalized partial credit model (GPCM) was considered in this study. It was found that the maximum priority index was implemented with the Fisher information, the interval information, and the posterior expected Kullback–Leibler information successfully in polytomous CAT. These three item information criteria had similar performance in terms of measurement precision, exposure control, and constraint management.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Andrich, D. (1996). A general hyperbolic cosine latent trait model for unfolding polytomous responses: Reconciling the Thurstone and Likert methodologies. British Journal of Mathematical and Statistical Psychology, 49, 347–365.
Chang, H.-H., Qian, J., & Ying, Z. (2001). a-stratified multistage CAT with b-blocking. Applied Psychological Measurement, 25, 333–341.
Chang, H.-H., & van der Linden, W. J. (2003). Optimal stratification of item pools in alpha-stratified computerized adaptive testing. Applied Psychological Measurement, 27, 262–274.
Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213–229.
Chang, H.-H., & Ying, Z. (1999). a-stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23, 211–222.
Cheng, Y., & Chang, H.-H. (2009). The maximum priority index method for severely constrained item selection in computerized adaptive testing. British Journal of Mathematical and Statistical Psychology, 62, 369–383.
Cheng, Y., Chang, H.-H., Douglas, J., & Guo, F. (2009). Constraint-weighted a-stratification for computerized adaptive testing with nonstatistical constraints: Balancing measurement efficiency and exposure control. Educational and Psychological Measurement, 69, 35–49.
Dodd, B. G., De Ayala, R. J., & Koch, W. R. (1995). Computerized adaptive testing with polytomous items. Applied Psychological Measurement, 19, 5–22.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 19, 159–176.
Muraki, E. (1993). Information functions of the generalized partial credit model. Applied Psychological Measurement, 17, 351–363.
Roberts, J. S. (1995). Item response theory approaches to attitude measurement (Doctoral dissertation, University of South Carolina, Columbia, 1995). Dissertation Abstracts International, 56, 7089B.
Roberts, J. S., Donoghue, J. R., & Laughlin, J. E. (1996, June). A generalized item response model for unfolding responses from a graded scale. Paper presented at the 61st annual meeting of the Psychometric Society, Banff, Alberta, Canada.
Roberts, J. S., Donoghue, J. R., & Laughlin, J. E. (1998). The generalized graded unfolding model: A general parametric item response model for unfolding graded responses (Research Report No.RR-98-32). Princeton NJ: Educational Testing Service.
Roberts, J. S., Donoghue, J. R., & Laughlin, J. E. (2000). A general model for unfolding unidimensional polytomous responses using item response theory. Applied Psychological Measurement, 24, 3–32.
Roberts, J. S., Laughlin, J. E., & Wedell, D. H. (1999). Validity issues in the Likert and Thurstone approaches to attitude measurement. Educational and Psychological Measurement, 59, 211–233.
Roberts, J. S., Lin, Y., & Laughlin, J. E. (2001). Computerized adaptive testing with the generalized graded unfolding model. Applied Psychological Measurement, 25, 177–196.
Segall, D. O. (1996). Multidimensional adaptive testing. Psychometrika, 61, 331–354.
Su, Y.-H. (2015). The performance of the modified multidimensional priority index for item selection in variable-length MCAT. In L. A. van der Ark, D. M. Bolt, W.-C. Wang, J. A. Douglas, & S.-M., Chow (Eds.), Quantitative psychology research: The 79th annual meeting of the psychometric society (pp. 89-97). Switzerland: Springer.
Su, Y.-H., & Huang, Y.-L. (2015). Using a modified multidimensional priority index for item selection under within-item multidimensional computerized adaptive testing. In R. E. Millsap, D. M. Bolt, L. A. van der Ark, & W.-C. Wang (Eds.), Quantitative psychology research: The 78th annual meeting of the psychometric society (pp. 227–242). Switzerland: Springer.
Su, Y.-H. (2016). A comparison of constrained item selection methods in multidimensional computerized adaptive testing. Applied Psychological Measurement. doi: 10.1177/0146621616639305
van der Linden, W. J. (1998). Bayesian item selection criteria for adaptive testing. Psychometrika, 63, 201–216.
van Rijn, P. W., Eggn, T. J. H. M., Hemker, B. T., & Sanders, P. F. (2002). Evaluation of selection procedures for computerized adaptive testing with polytomous items. Applied Psychological Measurement, 26, 393–411.
Veerkamp, W. J. J., & Berger, M. P. F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22, 203–226.
Veldkamp, B. P. (2003). Item selection in polytomous CAT. In H. Yanai, A. Okada, Y. Kano, & J.J. Meulman (Eds.), New developments in psychometrics (pp. 207–214). New York: Springer.
Veldkamp, B. P., & van der Linden, W. J. (2002). Multidimensional adaptive testing with constraints on test content. Psychometrika, 67, 575–588.
Wang, W.-C., & Chen, P.-H. (2004). Implementation and measurement efficiency of multidimensional computerized adaptive testing. Applied Psychological Measurement, 28, 295–316.
Yao, L. (2011, October). Multidimensional CAT item selection procedures with item exposure control and content constraints. Paper presented at the 2011 International Association of Computer Adaptive Testing IACAT Conference, Pacific Grove, CA.
Yao, L. (2012). Multidimensional CAT item selection methods for domain scores and composite scores: Theory and applications. Psychometrika, 77, 495–523.
Yao, L. (2013). Comparing the performance of five multidimensional CAT selection procedures with different stopping rules. Applied Psychological Measurement, 37, 3–23.
Zhou, X., & Reckase, M. D. (2014). Optimal item pool design for computerized adaptive tests with polytomous items using GPCM. Psychological Test and Assessment Modeling, 56(3), 255–274.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Su, YH. (2016). Investigation of Constraint-Weighted Item Selection Procedures in Polytomous CAT. In: van der Ark, L., Bolt, D., Wang, WC., Douglas, J., Wiberg, M. (eds) Quantitative Psychology Research. Springer Proceedings in Mathematics & Statistics, vol 167. Springer, Cham. https://doi.org/10.1007/978-3-319-38759-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-38759-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-38757-4
Online ISBN: 978-3-319-38759-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)