Item Sampling, Guessing, Partial Information and Decision-Making in Achievement Testing
- 98 Downloads
This paper addresses the problem of determining the number of items in a test and the cutting score to make a binary decision on mastery with prescribed precision. Previous attempts to solve this problem have been based on simplistic models assuming all-or-none knowledge. The same approach as in those papers is adopted here, but from the standpoint of a finite state model of test behaviour. The required number of items is shown to be smaller than previous results suggested. In contrast with previous attempts, this method is applied to true-false and multiple-choice tests responded to in the conventional mode, and to multiple-choice tests responded to in answer-until-correct and Coombs’ modes. The effects of varying guessing behaviour on the part of the examinees are also studied. It is shown that the smallest number of items needed to make a mastery decision with a predetermined error rate occurs when the testees respond without guessing. This behaviour is unlikely to be followed by examinees, but it is also shown that this lower limit can be reached at by administering multiple-choice tests under Coombs’ response mode.
KeywordsCorrect Answer Response Mode Partial Information Response Outcome Conventional Mode
Unable to display preview. Download preview PDF.
- Coombs, C.H., Milholland, J.E., & Womer, F.B. (1956). The assessment of partial information. Educational and Psychological Measurement, 16, 13–37.Google Scholar
- Frary, R.B. (1980b). Multiple-choice test bias due to answering strategy variation. Paper given at the Annual Meeting of the National Council on Measurement in Education, Boston, MA.Google Scholar
- García-Pérez, M.A. (19851 A finite state theory of performance in multiple-choice tests. Proceedings of the 16 u1 European Mathematical Psychology Group Meeting,Montpellier, France, 8–11 September. Pp 55–67.Google Scholar
- García-Pérez, M.A. (1987). A finite state theory of performance in multiple-choice tests. In: E.E. Roskam & R. Suck (Eds.), Progress in Mathematical Psychology-1, 455–464. Amsterdam: Elsevier.Google Scholar
- García-Pérez, M.A. (in press). A comparison of two models of performance in objective tests: Finite states versus continuous distributions. British Journal of Mathematical and Statistical Psychology.Google Scholar
- García-Pérez, M.A., & Frary, R.B. (in press). Psychometric properties of finite state scores versus number-right and formula scores: A simulation study. Applied Psychological Measurement.Google Scholar
- García-Pérez, M.A., & Frary, R.B. (1989). On the mathematical form (and psychological justification) of item characteristic curves. Manuscript submitted for publication.Google Scholar
- Hogg, R.V., & Tanis, E.A. (1977). Probabilityand Statistical Inference. New York, NY: Macmillan.Google Scholar
- Lord, F.M., & Novick, M.R. (1968). Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley.Google Scholar
- Weiss, D.J., & Davison, M.L. (1981). Review of test theory and methods. Research Report 81–1. Minneapolis, MN: University of Minnesota, Department of Psychology, Psychometric Methods Program.Google Scholar
- Wilcox, R.R. (1987). New Statistical Procedures for the Social Sciences. Hillsdale, NJ: LEA.Google Scholar