Abstract
Computerized adaptive testing (CAT) cannot only enable efficient and precise ability estimation but also increase security of testing materials. To meet a large number of statistical and nonstatistical constraints in CAT, the maximum priority index approaches can be used to handle these constraints simultaneously and efficiently for the construction of assessments. Many previous CAT studies were investigated for dominance items; however, only few CAT studies were investigated for unfolding items. In practice, an attitude measurement or personality test, such as the Minnesota Multiphasic Personality Inventory-2 or Cattell’s 16 Personality Factors Test, might fit better with the unfolding models than with the dominance ones. Besides, these tests commonly have hundreds of items from complex structures. Therefore, the purpose of this study was to investigate constraint-weighted item selection procedures in unfolding CAT. It was found that the maximum priority index was implemented with the Fisher information, the interval information, and the posterior expected Kullback-Leibler information successfully in unfolding CAT. These three item information criteria had similar performance in terms of measurement precision, exposure control, and constraint management. The generalized graded unfolding model and the two-parameter logistic model had similar performance in item selection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
D. Andrich, A general hyperbolic cosine latent trait model for unfolding polytomous responses: reconciling the Thurstone and Likert methodologies. Br. J. Math. Stat. Psychol. 49, 347–365 (1996)
A. Birnbaum, in Statistical theories of mental test scores, ed. by F. M. Lord, M. R. Novick. Some latent trait models and their use in inferring an examinees’ ability (Addison-Wesley, Reading, MA, 1968), pp. 397–479
H.-H. Chang, J. Qian, Z. Ying, a-stratified multistage CAT with b-blocking. Appl. Psychol. Meas. 25, 333–341 (2001)
H.-H. Chang, W.J. van der Linden, Optimal stratification of item pools in alpha-stratified computerized adaptive testing. Appl. Psychol. Meas. 27, 262–274 (2003)
H.-H. Chang, Z. Ying, A global information approach to computerized adaptive testing. Appl. Psychol. Meas. 20, 213–229 (1996)
H.-H. Chang, Z. Ying, a-stratified multistage computerized adaptive testing. Appl. Psychol. Meas. 23, 211–222 (1999)
Y. Cheng, H.-H. Chang, The maximum priority index method for severely constrained item selection in computerized adaptive testing. Br. J. Math. Stat. Psychol. 62, 369–383 (2009)
Y. Cheng, H.-H. Chang, J. Douglas, F. Guo, Constraint-weighted a-stratification for computerized adaptive testing with nonstatistical constraints: balancing measurement efficiency and exposure control. Educ. Psychol. Meas. 69, 35–49 (2009)
S.W. Choi, M. Grady, B.G. Dodd, A new stopping rule for computerized adaptive testing. Educ. Psychol. Meas. 71, 37–73 (2011)
B.G. Dodd, R.J. De Ayala, W.R. Koch, Computerized adaptive testing with polytomous items. Appl. Psychol. Meas. 19, 5–22 (1995)
B.G. Dodd, W.R. Koch, R.J. De Ayala, Operational characteristics of adaptive testing procedures using the graded response model. Appl. Psychol. Meas. 13, 129–143 (1989)
B.G. Dodd, W.R. Koch, R.J. De Ayala, Computerized adaptive testing using the partial credit model: effects of item pool characteristics and different stopping rules. Educ. Psychol. Meas. 53, 61–77 (1993)
M.D. Reckase, Multidimensional Item Response Theory (Springer, New York, NY, 2009)
J.S. Roberts, Item response theory approaches to attitude measurement (Doctoral dissertation, University of South Carolina, Columbia, 1995). Dissertation Abstracts International, 56, 7089B, 1995
J.S. Roberts, J.R. Donoghue, J.E. Laughlin, A generalized item response model for unfolding responses from a graded scale. Paper presented at the 61st annual meeting of the Psychometric Society, Banff, Alberta, Canada, 1996
J.S. Roberts, J.R. Donoghue, J.E. Laughlin, The Generalized Graded Unfolding Model: A General Parametric Item Response Model for Unfolding Graded Responses (Research Report No. RR-98-32) (Educational Testing Service, Princeton NJ, 1998)
J.S. Roberts, J.R. Donoghue, J.E. Laughlin, A general model for unfolding unidimensional polytomous responses using item response theory. Appl. Psychol. Meas. 24, 3–32 (2000)
J.S. Roberts, J.E. Laughlin, D.H. Wedell, Validity issues in the Likert and Thurstone approaches to attitude measurement. Educ. Psychol. Meas. 59, 211–233 (1999)
J.S. Roberts, Y. Lin, J.E. Laughlin, Computerized adaptive testing with the generalized graded unfolding model. Appl. Psychol. Meas. 25, 177–196 (2001)
Y.-H. Su, in Quantitative Psychology Research, ed. by L. A. van der Ark, D. M. Bolt, W.-C. Wang, J. A. Douglas, S.-M. Chow. The performance of the modified multidimensional priority index for item selection in variable-length MCAT, vol 140 (Springer, Cham, 2015), pp. 89–97
Y.-H. Su, A comparison of constrained item selection methods in multidimensional computerized adaptive testing. Appl. Psychol. Meas. 40(5), 346–360 (2016). doi:10.1177/0146621616639305
Y.-H. Su, Y.-L. Huang, in Quantitative Psychology Research, ed. by R. E. Millsap, D. M. Bolt, L. A. van der Ark, W.-C. Wang. Using a modified multidimensional priority index for item selection under within-item multidimensional computerized adaptive testing, vol 89 (Springer, Cham, 2015), pp. 227–242
W.J. van der Linden, Bayesian item selection criteria for adaptive testing. Psychometrika 63, 201–216 (1998)
P.W. van Rijn, T.J.H.M. Eggn, B.T. Hemker, P.F. Sanders, Evaluation of selection procedures for computerized adaptive testing with polytomous items. Appl. Psychol. Meas. 26, 393–411 (2002)
W.J.J. Veerkamp, M.P.F. Berger, Some new item selection criteria for adaptive testing. J. Educ. Behav. Stat. 22, 203–226 (1997)
B.P. Veldkamp, W.J. van der Linden, Multidimensional adaptive testing with constraints on test content. Psychometrika 67, 575–588 (2002)
H. Wainer (ed.), Computerized Adaptive Testing: A Primer, 2nd edn. (Erlbaum, Mahwah, NJ, 2000)
L. Yao, Multidimensional CAT item selection procedures with item exposure control and content constraints. Paper presented at the International Association of Computer Adaptive Testing (IACAT) Conference, Pacific Grove, CA, 2011
L. Yao, Multidimensional CAT item selection methods for domain scores and composite scores: theory and applications. Psychometrika 77, 495–523 (2012)
L. Yao, Comparing the performance of five multidimensional CAT selection procedures with different stopping rules. Appl. Psychol. Meas. 37, 3–23 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Su, YH. (2017). Investigating Constraint-Weighted Item Selection Procedures in Unfolding CAT. In: van der Ark, L.A., Wiberg, M., Culpepper, S.A., Douglas, J.A., Wang, WC. (eds) Quantitative Psychology. IMPS 2016. Springer Proceedings in Mathematics & Statistics, vol 196. Springer, Cham. https://doi.org/10.1007/978-3-319-56294-0_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-56294-0_29
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56293-3
Online ISBN: 978-3-319-56294-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)