Advertisement

An Overview of Computerized Multistage Testing

  • David Magis
  • Duanli Yan
  • Alina A. von Davier
Chapter
Part of the Use R! book series (USE R)

Abstract

In this chapter, we present a brief overview of computerized multistage testing theory, including test design, test assembly, item bank, module selection, routing, scoring and linking, and exposure and security. We also provide a summary of the IRT-based module selection process, as well as the tree-based multistage testing.

References

  1. Angoff, W. H., & Huddleston, E. M. (1958). The multi-level experiment: A study of a two-level test system for the college board scholastic aptitude test (ETS Statistical Report No. SR-58-21). Princeton, NJ: Educational Testing Service.Google Scholar
  2. Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74, 619–632. https://doi.org/10.1007/s11336-009-9123-2 MathSciNetCrossRefzbMATHGoogle Scholar
  3. Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and personnel decisions. Urbana, IL: University of Illinois Press.Google Scholar
  4. Eggen, T. J. H. M., & Verhelst, N. D. (2011). Item calibration in incomplete testing designs. Psicologica, 32, 107–132.Google Scholar
  5. Haberman, S. J., & von Davier, A. A. (2014). Considerations on parameter estimation, scoring, and linking in multistage testing. In D. Yan, A. A. von Davier, & C. Lewis (Eds.), Computerized multistage testing: Theory and applications (pp. 229–248). New York: CRC Press.Google Scholar
  6. Hambleton, R. K., & Zenisky, A. L. (2013). Reporting test scores in more meaningful ways: A research-based approach to score report design. In K. F. Geisinger, et al. (Eds.), APA handbook of testing and assessment in psychology, Vol. 3. Testing and assessment in school psychology and education (pp. 479–494). Washington, DC: American Psychological Association. https://doi.org/10.1037/14049-023 Google Scholar
  7. Holland, P. W. (1990). On the sampling theory foundations of item response theory models. Psychometrika, 55, 577–602. https://doi.org/10.1007/BF02294609 MathSciNetCrossRefzbMATHGoogle Scholar
  8. Linn, R. L., Rock, D. A., & Cleary, T. A. (1968). The development and evaluation of several programmed testing methods (ETS Research Bulletin Series No. i-29). Princeton, NJ: Educational Testing Service.Google Scholar
  9. Lord, F. M. (1971a). The self-scoring flexilevel test. Journal of Educational Measurement, 8, 147–151. https://doi.org/10.1111/j.1745-3984.1971.tb00918.x CrossRefGoogle Scholar
  10. Lord, F. M. (1971b). A theoretical study of two-stage testing. Psychometrika, 36, 227–242. https://doi.org/10.1007/BF02297844 CrossRefGoogle Scholar
  11. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.Google Scholar
  12. Luecht, R. M. (1998). Computer-assisted test assembly using optimization heuristics. Applied Psychological Measurement, 22, 224–236. https://doi.org/10.1177/01466216980223003 CrossRefGoogle Scholar
  13. Luecht, R. M. (2014). Systems, design and implementation of large-scale multistage testing. In D. Yan, A. A. von Davier, & C. Lewis (Eds.), Computerized multistage testing: Theory and applications (pp. 69–83). New York: CRC Press.Google Scholar
  14. Luecht, R. M., & Nungester, R. J. (1998). Some practical examples of computer-adaptive sequential testing. Journal of Educational Measurement, 35, 229–249. https://doi.org/10.1111/j.1745-3984.1998.tb00537.x CrossRefGoogle Scholar
  15. Mislevy, R. J., & Chang, H.-H. (2000). Does adaptive testing violate local independence? Psychometrika, 65, 149–156. https://doi.org/10.1007/BF02294370 MathSciNetCrossRefzbMATHGoogle Scholar
  16. Smith, R., & Lewis, C. (2014). Multistage testing for categorical decisions. In D. Yan, A. A. von Davier, & C. Lewis (Eds.), Computerized multistage testing: Theory and applications (pp. 189–203). New York: CRC Press.Google Scholar
  17. Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201–210. https://doi.org/10.1177/014662168300700208 CrossRefGoogle Scholar
  18. van der Linden, W. J., & Diao, Q. (2014). Using a universal shadow test assembler with multistage testing. In D. Yan, A. A. von Davier, & C. Lewis (Eds.), Computerized multistage testing: Theory and applications (pp. 101–118). New York: CRC Press.Google Scholar
  19. Veldkamp, B. P. (2014). Item pool design and maintenance for multistage testing. In D. Yan, A. A. von Davier, & C. Lewis (Eds.), Computerized multistage testing: Theory and applications (pp. 39–54). New York: CRC Press.Google Scholar
  20. von Davier, M., & Haberman, S. J. (2014). Hierarchical diagnostic classification models morphing into unidimensional ‘diagnostic’ classification models-a commentary. Psychometrika, 79, 340–346. https://doi.org/10.1007/s11336-013-9363-z MathSciNetCrossRefzbMATHGoogle Scholar
  21. Weiss, D. J. (1983). New horizons in testing: Latent trait theory and computerized adaptive testing. New York: Academic Press.Google Scholar
  22. Xu, X., Chang, H.-H., & Douglas, J. (2003). A simulation study to compare CAT strategies for cognitive diagnosis. Paper presented at the annual meeting of the American Educational Research Association, Chicago.Google Scholar
  23. Yan, D., Lewis, C., & Stocking, M. L. (2004). Adaptive testing with regression trees in the presence of multidimensionality. Journal of Educational and Behavioral Statistics, 29, 293–316. https://doi.org/10.3102/10769986029003293 CrossRefGoogle Scholar
  24. Yan, D., Lewis, C., & von Davier, A. A. (2014b). A tree-based approach for multistage testing. In D. Yan, A. A. von Davier, & C. Lewis (Eds.), Computerized multistage testing: Theory and applications (pp. 169–188). New York: CRC Press.Google Scholar
  25. Yan, D., von Davier, A. A., & Lewis, C. (2014). Computerized multistage testing: Theory and applications. New York: CRC Press.Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • David Magis
    • 1
  • Duanli Yan
    • 2
  • Alina A. von Davier
    • 3
  1. 1.Department of EducationUniversity of LiegeLiegeBelgium
  2. 2.Educational Testing ServicePrincetonUSA
  3. 3.ACTNext by ACTIowa CityUSA

Personalised recommendations