Designing Item Pools for Adaptive Testing
In existing adaptive testing programs, each successive item in the test is chosen to optimize an objective. Examples of well-known objectives aremaximizing the information in the test at the ability estimate for the test taker or minimizing the deviation of its information froma target value at the estimate. In addition, item selection is required to realize a set of content specifications for the test. For example, item content may be required to follow a certain taxonomy or the answer-key distribution for the test must not deviate too much from uniformity. Content specifications are generally defined in terms of combinations of attributes the items in the test should have. They are typically realized by imposing a set of constraints on the item-selection process. The presence of both an objective and a set of constraints in adaptive testing leads to the notion of adaptive testing as constrained (sequential) optimization problem; for a more formal introduction to this notion, see van der Linden (this volume, chap. 2).
KeywordsDesign Space Design Point Stimulus Level Exposure Rate Item Bank
Unable to display preview. Download preview PDF.
- Boekkooi-Timminga, E. (1991). A method for designing Rasch model based item banks. Paper presented at the annual meeting of the Psychometric Society, Princeton NJ.Google Scholar
- Flaugher, R. (1990). Item pools. In H. Wainer (Ed.), Computerized adaptive testing: A primer (pp. 41–64). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
- Gulliksen, H. (1950). Theory of mental tests. New York: Wiley.Google Scholar
- ILOG, Inc. (2003). CPLEX9.0 [Computer program]. Incline Village, NV: Author.Google Scholar
- Stocking, M. L. & Lewis, C. (1998). Controlling item exposure conditional on ability in computerized adaptive testing. Journal of Educational and Behavioral Statistics, 23, 57–75.Google Scholar
- Stocking, M. L. & Lewis, C. (2000). Methods of controlling the exposure of items in CAT. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 163–182). Boston: Kluwer-Nijhof Publishing.Google Scholar
- Sympson, J. B. & Hetter, R. D. (1985). Controlling item exposure rates in computerized adaptive testing. Proceedings of the 27th Annual Meeting of the Military Testing Association (pp. 973–77). San Diego: Navy Personnel Research and Development Center.Google Scholar
- Way, W. D. & Steffen, M. (1998, April). Strategies for managing item pools to maximize item security. Paper presented at Annual Meeting of the National Council on Educational Measurement, San Diego.Google Scholar
- Way, W. D., Steffen, M. & Anderson, G. S. (1998). Developing, maintaining, and renewing the item inventory to support computer-based testing. In C. N. Mills, M. Potenza, J. J. Fremer & W. Ward (Eds.), Computer-based testing: Building the foundation for future assessments (pp. 89–102). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar