Advertisement

Adaptive Mastery Testing Using a Multidimensional IRT Model

  • Cees A. W. Glas
  • Hans J. Vos
Chapter
Part of the Statistics for Social and Behavioral Sciences book series (SSBS)

Abstract

Mastery testing concerns the decision to classify a student as a master or as a nonmaster. In the previous chapter, adaptive mastery testing (AMT) using item response theory (IRT) and sequential mastery testing (SMT) using Bayesian decision theory were combined into an approach labeled adaptive sequential mastery testing (ASMT). This approach is based on the one-parameter logistic model (1PLM; Rasch, 1960) and three-parameter logistic model (3PLM; Birnbaum, 1968). In the present chapter, ASMT is applied to a multidimensional IRT (MIRT) model.

Keywords

Loss Function Item Parameter Computerize Adaptive Testing Computer Adaptive Testing Latent Trait Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ackerman, T. A. (1987). A comparison study of the unidimensional IRT estimation of compensatory and noncompensatory multidimensional item response data (ACT Research Report Series 87-12). Iowa City, IA: ACT Inc.Google Scholar
  2. Ackerman, T. A. (1996a). Developments in multidimensional item response theory. Applied Psychological Measurement, 20, 309–310.CrossRefGoogle Scholar
  3. Ackerman, T. A. (1996b). Graphical representation of multidimensional item response theory analyses. Applied Psychological Measurement, 20, 311–329.CrossRefGoogle Scholar
  4. Adams, R. J., Wilson, M. R. & Wang, W. C. (1997). The random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–25.CrossRefGoogle Scholar
  5. Adams, R. J., Wilson, M. R. & Wu, M. (1997). Multilevel item response theory models: an approach to errors in variables of regression. Journal of Educational and Behavioral Statistics, 22, 47–76.Google Scholar
  6. Andersen, E. B. (1985). Estimating latent correlations between repeated testings. Psychometrika, 50, 3–16.MATHCrossRefMathSciNetGoogle Scholar
  7. Béguin, A. A. & Glas, C. A. W. (2001). MCMC estimation and some fit analysis of multidimensional IRT models. Psychometrika, 66, 541–562.CrossRefMathSciNetGoogle Scholar
  8. Bellman, R. (1957). Dynamic programming. Princeton, NJ: Princeton University Press.Google Scholar
  9. Birnbaum, A. (1968). Some latent trait models. In F. M. Lord & M.R. Novick (Eds.), Statistical theories of mental test scores. Reading, MA: Addison-Wesley.Google Scholar
  10. Bock, R. D. & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: an application of an EM-algorithm. Psychometrika, 46, 443–459.CrossRefMathSciNetGoogle Scholar
  11. Bock, R. D., Gibbons, R. D. & Muraki, E. (1988). Full-information factor analysis. Applied Psychological Measurement, 12, 261–280.CrossRefGoogle Scholar
  12. Bradlow, E. T., Wainer, H. & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153–168.CrossRefGoogle Scholar
  13. Coombs, C. H. (1960). A theory of data. Ann Arbor, MI: Mathesis Press.Google Scholar
  14. Coombs, C. H. & Kao, R. C. (1955). Nonmetric factor analysis. [Engng. Res. Bull., No.38]. Ann Arbor, MI: University of Michigan Press.Google Scholar
  15. DeGroot, M. H. (1970). Optimal statistical decisions. New York: McGraw-Hill.MATHGoogle Scholar
  16. Embretson, S. (1980). Multicomponent latent trait models for ability tests. Psychometrika, 45, 479–494.CrossRefGoogle Scholar
  17. Embretson, S. (1984). A general latent trait model for response processes. Psychometrika, 49, 175–186.CrossRefGoogle Scholar
  18. Ferguson, T. S. (1967). Mathematical statistics: A decision theoretic approach. New York: Academic Press.MATHGoogle Scholar
  19. Fraser, C & McDonald, R. P. (1988). NOHARM: Least Squares item factor analysis. Multivariate Behavioral Research, 23, 267–269.CrossRefGoogle Scholar
  20. Glas, C. A. W. (1988). The derivation of some tests for the Rasch model from the multinomial distribution. Psychometrika, 53, 525–546.MATHCrossRefMathSciNetGoogle Scholar
  21. Glas, C. A. W. (1992). A Rasch model with a multivariate distribution of proficiency. In M. R. Wilson (Ed.), Objective measurement: Theory into practice, Vol. 1 (pp.236–258). Norwood, NJ: Ablex Publishing Corporation.Google Scholar
  22. Glas, C. A. W. & Verhelst, N. D. (1995). Testing the Rasch model. In G. H.Fischer & I. W.Molenaar (Eds.), Rasch models: foundations, recent developments and applications (pp.69–96). New York, NJ: Springer-Verlag.Google Scholar
  23. Glas, C. A. W., Wainer, H. & Bradlow, E. T. (2000). MML and EAP estimates for the testlet response model. In W. J. van der Linden & C. A. W.Glas (Eds.), Computer adaptive testing: Theory and practice (pp.271–287). Boston: Kluwer-Nijhoff Publishing.Google Scholar
  24. Lewis, C. & Sheehan, K. (1990). Using Bayesian decision theory to design a computerized mastery test. Applied Psychological Measurement, 14, 367–386.CrossRefGoogle Scholar
  25. Lindgren, B. W. (1976). Statistical theory (3rd ed.). New York: Macmillan.MATHGoogle Scholar
  26. Lord, F. M. & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.MATHGoogle Scholar
  27. Luecht, R. M. (1996). Multidimensional computerized adaptive testing in a certification or licensure context. Applied Psychological Measurement, 20, 389–404.CrossRefGoogle Scholar
  28. Luecht, R. M. & Nungester, R. J. (1998). Some practical examples of computer-adaptive sequential testing. Journal of Educational Measurement, 35, 239–249.Google Scholar
  29. Luecht, R. M. & Nungester, R. J. (2000). Computer-adaptive sequential testing. In W. J. van der Linden & C. A. W.Glas (eds.). Computerized adaptive testing: Theory and practice. (pp. 117–128). Boston: Kluwer-Nijhof Publishing.Google Scholar
  30. Maris, E. (1993). Additive and multiplicative models for gamma distributed random variables, and their application as psychometric models for response times. Psychometrika, 58, 445–470.MATHCrossRefGoogle Scholar
  31. Maris, E. (1995). Psychometric latent response models. Psychometrika, 60, 523–548.MATHCrossRefGoogle Scholar
  32. McDonald, R. P. (1967). Nonlinear factor analysis. Psychometric monographs, No. 15. Google Scholar
  33. McDonald, R. P. (1997). Normal-ogive multidimensional model. In W. J. van der Linden and R. K. Hambleton (Eds.). Handbook of Modern Item Response Theory (pp.257–269). New York: Springer-Verlag.Google Scholar
  34. Mislevy, R. J. (1986). Bayes modal estimation in item response models. Psychometrika, 51, 177–195.MATHCrossRefMathSciNetGoogle Scholar
  35. Mislevy, R. J. (1988). Exploiting auxiliary information about items in the estimation of Rasch item difficulty parameters. Applied Psychological Measurement, 12, 281–296.CrossRefGoogle Scholar
  36. Novick, M. R. & Jackson, P. H. (1974). Statistical methods for educational and psychological research. New York, NJ: McGraw-Hill.Google Scholar
  37. Novick, M. R. & Lindley, D. V. (1979). The use of more realistic utility functions in educational applications. Journal of Educational Measurement, 15, 81–191.Google Scholar
  38. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research.Google Scholar
  39. Reckase, M. D. (1983). A procedure for decision making using tailored testing. In D. J. Weiss (Ed.). New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 237–257). New York: Academic Press.Google Scholar
  40. Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401–412.CrossRefGoogle Scholar
  41. Reckase, M. D. (1997). A linear logistic multidimensional model for dichotomous item response data. In W. J. van der Linden and R. K. Hambleton (eds.). Handbook of modern item response theory (pp.271–286). New York: Springer-Verlag.Google Scholar
  42. Robbins, H. (1964). The empirical Bayes approach to statistical decision problems. Annals of Mathematical Statistics, 35, 1–20.MATHCrossRefMathSciNetGoogle Scholar
  43. Segall, D. O. (1996). Multidimensional adaptive testing. Psychometrika, 61, 331–354.MATHCrossRefGoogle Scholar
  44. Spray, J. A., Davey, D. C., Reckase, M. D., Ackerman, T. A. & Carlson, J. E. (1990). Comparison of two logistic multidimensional item response theory models (ACT Research Report Series ONR90-8). Iowa City, IA: ACT Inc.Google Scholar
  45. Sympson, J. B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference (pp. 82–98). Minneapolis: University of Minnesota.Google Scholar
  46. van der Linden, W. J. (1999). Empirical initialization of the trait estimator in adaptive testing. Applied Psychological Measurement, 23, 21–29.CrossRefGoogle Scholar
  47. van der Linden, W. J. & Reese, L. M. (1998). A model for optimal constrained adaptive testing. Applied Psychological Measurement, 22, 259–270.CrossRefGoogle Scholar
  48. Verhelst, N. D., Glas, C. A. W. & Verstralen, H. H. F. M. (1995). OPLM: Computer program and manual. Arnhem: Cito.Google Scholar
  49. Wainer, H. (Ed.). (1990). Computerized adaptive testing: A primer. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  50. Wainer, H, Bradlow, E. T. & Du, Z. (2000). Testlet response theory: An analogue for the 3-PL useful in testlet-based adaptive testing. In W. J. van der Linden & C. A. W.Glas (Eds.). Computer adaptive testing: Theory and practice (pp. 245–269). Boston: Kluwer-Nijhoff Publishing.Google Scholar
  51. Weiss, D. J. (Ed.). (1983). New horizons in testing. New York: Academic Press.Google Scholar
  52. Wilson, D. T., Wood, R. & Gibbons, R. (1991) TESTFACT: Test scoring, item statistics, and item factor analysis. (computer software). Chicago: Scientific Software International Inc.Google Scholar
  53. Winston, W. L. (1994). Operations research: Applications and algorithms. Belmont, CA: Wadsworth.MATHGoogle Scholar
  54. Wu, M. L., Adams, R. J. & Wilson, M. R. (1997). ConQuest: Generalized item response modelling software. Australian Counsil for Educational Research.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Cees A. W. Glas
    • 1
  • Hans J. Vos
    • 1
  1. 1.Department of Research Methodology, Measurement, and Data AnalysisUniversity of TwenteEnschedeThe Netherlands

Personalised recommendations