Learning from Incorrect Answers on Multiple Choice Tests

Implications for a New Teaching Paradigm
  • Jay C. Powell
  • James Bernauer
  • Vishnuteerth Agnihotri


Three assumptions underlie using total-correct scores to assess student performance. First, if alternative answer information exists, it is inaccessible because of the linear dependency problem inherent in combining both acceptable (A) and unacceptable (¬A) answers into a single analysis. Second, unacceptable (¬A) answers are selected in the absence of knowledge; therefore, they contain no useful information about student performance. Third, learning is characterized by a shifting from any ¬A answer to the corresponding A answer. These presumptions render the investigation of ¬A answers unnecessary. Powell (2010a) has shown the first two presumptions to be false. Answers are selected thoughtfully and a method for bypassing the linear dependency problem has been in the psychometric literature for 20 years. The third presumption is addressed in this paper. Using student response data, we show it to be unwarranted. This discussion shows how most of these ¬A answers fit into several categories; reflecting developmental status and identifying information processing styles and procedural and information errors. The implications of gathering this information for improving teaching are presented throughout the paper. Using this additional information supports a constructivist theory of learning and exploratory teaching.


Item Response Theory Wrong Answer Multiple Choice Test Classical Test Theory Alternative Functioning 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We extend our thanks to Educational Initiatives – India for making these test items and response data available for this study and my wife Valerie for reviewing and editing this document.


  1. Allen, M. J., & Yen, W. M. (2002). Introduction to Measurement Theory. Long Grove, IL: Waveland Press.Google Scholar
  2. Bandler, R. & Grinder, J. (1982) Reframing. Moab, UT: Real People Press.Google Scholar
  3. Bond, T. G. & Fox, C. M. (2007). (2nd Ed.). Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  4. Demetriou, A., & Kazi, S. (2006). Self-awareness in g (with processing efficiency and reasoning).Intelligence, 34, 297–317.CrossRefGoogle Scholar
  5. De Bono, E. (2005) The six value medals. London: Vermilion.Google Scholar
  6. Flavell, John H. (1963). The Developmental Psychology of Jean Piaget. New York, NY: Van Nostrand.CrossRefGoogle Scholar
  7. Gardner, H. (1983). Frames of Mind: The Theory of Multiple Intelligences. New York, NY: Basic Books.Google Scholar
  8. Gardner, H. (2000). Intelligence reframed. Multiple intelligences for the 21 st century. New York, NY: Basic Books.Google Scholar
  9. Gardner (2004). Multiple intelligences. New Horizons, in theory and practice. New York, NY: Basic Books.Google Scholar
  10. Glaser, Barney G. and Strauss, Anselm L. (1967). The Discovery of Grounded Theory: Strategies for qualitative research. Chicago, IL: Aldine.Google Scholar
  11. Gorham, D. R. (1957). Proverbs Test. Missoula, MT: Psychological Test Specialists.Google Scholar
  12. Halloun, I. & Hestenes, D. L. (1985). Common sense about motion. American Journal of Physics, 53, (11) 1056–1065.CrossRefGoogle Scholar
  13. Heisenberg, W. (1930), Physikalische Prinzipien der Quantentheorie (Leipzig: Hirzel). English translation: The Physical Principles of Quantum Theory. Chicago: University of Chicago Press.Google Scholar
  14. Heisenberg, W. (2007) Physics and Philosophy: The Revolution in Modern Science. New York: Harper Perennial Modern Classics. (Full text of 1958 version).Google Scholar
  15. Kuhn, Thomas S. (1996). The Structure of Scientific Revolutions (3rd Ed.). University of Chicago Press.Google Scholar
  16. Lord, Fredric. (1952). A Theory of Test Scores. Psychometric Monographs, Number 7. Philadelphia, PA. Ferguson.Google Scholar
  17. Lord, F. M. & Novick, M. R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Welsley.Google Scholar
  18. Montessori, M. (1972). Discovery of the Child. New York: Ballantine Books.Google Scholar
  19. Moreno, R. (2010). Educational Psychology. Hoboken, NJ: John Wiley & Sons.Google Scholar
  20. Rose, S.A. & Blank, M. (1974). The potency of context in children’s cognition: An illustration through conservation. Child Development, 45, 499–502.CrossRefGoogle Scholar
  21. Piburn, M. (1990). Reasoning about logical propositions and success in science. Journal of Research in Science Teaching. 27, (9) 887–900.CrossRefGoogle Scholar
  22. Piaget, J. (1954). The construction of reality in the child (m. Cook, Trans.). New York, NY: Basic Books.Google Scholar
  23. Piaget, J. (1985). The Equilibration of Cognitive Structures: The Central Problem of Intellectual Development. Chicago: University of Chicago Press. (New translation of The Development of Thought)Google Scholar
  24. Powell, J. C. (2010a). Testing as feedback to inform teaching. Chapter 3 in Learning and Instruction in the Digital Age: Making a Difference through Cognitive Approaches. New York: Springer.Google Scholar
  25. Powell, J. C. (2010b). Do profoundly informed students choose wrong answers, lowering their scores? Research report Powell, J. C. (2010c). Making Peasants into Kings, Bloomington, IN: Author HouseGoogle Scholar
  26. Powell, J. C. (2010). Do profoundly informed students choose wrong answers, lowering their scores? Research report presented to the Psychomertic Society, Athens, GAGoogle Scholar
  27. Powell, J. C. & Miki, H. (1985). Answer anomalies, how serious? Nashville, TN: Paper presented to the Psychometric Society.Google Scholar
  28. Powell, J. C. & Shklov N. (1992). Obtaining information about learners’ thinking strategies from wrong answers on multiple-choice tests. Educational and Psychological Measurement, 52, 847–865.CrossRefGoogle Scholar
  29. Rasch, G. (1980). Probabilistic Models for some intelligence and attainment tests. (Expanded edition.) Chicago: University of Chicago Press.Google Scholar
  30. Spearman, C. (1927). The abilities of man: Their nature and measurement. New York, NY: Macmillan.Google Scholar
  31. Tuckman, B.W. & Monetti, D.M. (2011). Educational psychology. Belmont, CA: Wadsworth.Google Scholar
  32. Vygotsky, L. S. (1978). Mind in society. The development of higher mental process. Cambridge, MA: Harvard University Press.Google Scholar
  33. Woolfolk, A. (2011). Educational psychology. Boston, MA: Pearson.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Jay C. Powell
    • 1
  • James Bernauer
    • 2
  • Vishnuteerth Agnihotri
    • 3
  1. 1.Better Schooling SystemsPittsburghUSA
  2. 2.Robert Morris UniversityMoon TownshipUSA
  3. 3.Test DevelopmentEducational Initiatives Pvt. Ltd.BangaloreIndia

Personalised recommendations