Advertisement

Item Sampling, Guessing, Partial Information and Decision-Making in Achievement Testing

Chapter
Part of the Recent Research in Psychology book series (PSYCHOLOGY)

Abstract

This paper addresses the problem of determining the number of items in a test and the cutting score to make a binary decision on mastery with prescribed precision. Previous attempts to solve this problem have been based on simplistic models assuming all-or-none knowledge. The same approach as in those papers is adopted here, but from the standpoint of a finite state model of test behaviour. The required number of items is shown to be smaller than previous results suggested. In contrast with previous attempts, this method is applied to true-false and multiple-choice tests responded to in the conventional mode, and to multiple-choice tests responded to in answer-until-correct and Coombs’ modes. The effects of varying guessing behaviour on the part of the examinees are also studied. It is shown that the smallest number of items needed to make a mastery decision with a predetermined error rate occurs when the testees respond without guessing. This behaviour is unlikely to be followed by examinees, but it is also shown that this lower limit can be reached at by administering multiple-choice tests under Coombs’ response mode.

Keywords

Correct Answer Response Mode Partial Information Response Outcome Conventional Mode 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bliss, L.B. (1980). A test of Lord’s assumptions regarding examinee guessing behavior on multiple-choice tests using elementary school children. Journal of Educational Measurement, 17, 147–153.CrossRefGoogle Scholar
  2. van den Brink, W.P., & Koele, P. (1980). Item sampling, guessing and decision-making in achievement testing. British Journal of Mathematical and Statistical Psychology, 33, 104–108.CrossRefGoogle Scholar
  3. Coombs, C.H. (1953). On the use of objective examinations. Educational and Psychological Measurement, 13, 308–310.CrossRefGoogle Scholar
  4. Coombs, C.H., Milholland, J.E., & Womer, F.B. (1956). The assessment of partial information. Educational and Psychological Measurement, 16, 13–37.Google Scholar
  5. Cross, L.H., & Frary, R.B. (1977). An empirical test of Lord’s theoretical results regarding formula scoring of multiple-choice tests. Journal of Educational Measurement, 14, 313–321.CrossRefGoogle Scholar
  6. Fhanér, S. (1974). Item sampling and decision-making in achievement testing. British Journal of Mathematical and Statistical Psychology, 27, 172–175.CrossRefGoogle Scholar
  7. Frary, R.B. (1980a). The effect of misinformation, partial information, and guessing on expected multiple-choice test item scores. Applied Psychological Measurement, 4, 79–90.CrossRefGoogle Scholar
  8. Frary, R.B. (1980b). Multiple-choice test bias due to answering strategy variation. Paper given at the Annual Meeting of the National Council on Measurement in Education, Boston, MA.Google Scholar
  9. Frary, R.B. (1982). A simulation study of reliability and validity of multiple-choice test scores under six response-scoring modes. Journal of Educational Statistics, 7, 333–351.CrossRefGoogle Scholar
  10. García-Pérez, M.A. (19851 A finite state theory of performance in multiple-choice tests. Proceedings of the 16 u1 European Mathematical Psychology Group Meeting,Montpellier, France, 8–11 September. Pp 55–67.Google Scholar
  11. García-Pérez, M.A. (1987). A finite state theory of performance in multiple-choice tests. In: E.E. Roskam & R. Suck (Eds.), Progress in Mathematical Psychology-1, 455–464. Amsterdam: Elsevier.Google Scholar
  12. García-Pérez, M.A. (in press). A comparison of two models of performance in objective tests: Finite states versus continuous distributions. British Journal of Mathematical and Statistical Psychology.Google Scholar
  13. García-Pérez, M.A., & Frary, R.B. (in press). Psychometric properties of finite state scores versus number-right and formula scores: A simulation study. Applied Psychological Measurement.Google Scholar
  14. García-Pérez, M.A., & Frary, R.B. (1989). On the mathematical form (and psychological justification) of item characteristic curves. Manuscript submitted for publication.Google Scholar
  15. Gibbons, J.D., Olkin, I., & Sobel, M. (1979). A subset selection technique for scoring items on a multiple choice test. Psychometrika, 44, 259–270.CrossRefGoogle Scholar
  16. Guion, R.M., & Gibson, W.H. (1988). Personnel selection and placement. Annual Review of Psychology, 39, 349–374.CrossRefGoogle Scholar
  17. Hogg, R.V., & Tanis, E.A. (1977). Probabilityand Statistical Inference. New York, NY: Macmillan.Google Scholar
  18. Lord, F.M., & Novick, M.R. (1968). Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley.Google Scholar
  19. Shepard, L. (1980). Standard setting issues and methods. Applied Psychological Measurement, 4, 447–467.CrossRefGoogle Scholar
  20. Weiss, D.J., & Davison, M.L. (1981). Review of test theory and methods. Research Report 81–1. Minneapolis, MN: University of Minnesota, Department of Psychology, Psychometric Methods Program.Google Scholar
  21. Wilcox, R.R. (1976). A note on the length and passing score of a mastery test. Journal of Educational Statistics, 1, 359–364.CrossRefGoogle Scholar
  22. Wilcox, R.R. (1987). New Statistical Procedures for the Social Sciences. Hillsdale, NJ: LEA.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1989

Authors and Affiliations

  1. 1.Departamento de Metodología, Facultad de PsicologíaUniversidad Complutense, Campus de SomosaguasMadridSpain

Personalised recommendations