Measuring learning strategies and understanding: A research framework

  • Peter Pirolli
  • Mark Wilson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 608)


We present a framework for measurement and diagnosis using knowledge-based models. Based on formulations of knowledge-level analysis, symbol-level analysis, constructivist learning, and situated cognition, we describe the possible frames of reference and fundamental measurement approaches that may be adopted in the analysis of cognition and learning. In general, these frames of reference define how individuals, contexts, knowledge, and activity are constitutively defined. We then present several measurement models extending the objective measurement approach of Rasch models to the analysis of learning strategies and knowledge development. The structure and parameter estimates of such models can then be used in the specification of probabilistic belief networks that can perform on-line student modelling.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R. J. Adams, M. Wilson: A random coefficients multinomial logit: A generalized approach to fitting Rasch models (under review), 1991aGoogle Scholar
  2. 2.
    R. J. Adams, M. Wilson: Hierarchical multinomial logistic regression. (in preparation), 1991bGoogle Scholar
  3. 3.
    E. B. Andersen: A general latent structure model for contingency table data. H. Wainer, S. Messick (Eds.), Principles of modern psychological measurement, Hillsdale, NJ: Lawrence Erlbaum, 1983Google Scholar
  4. 4.
    S. K. Andersen, K. G. Olesen, F. U. Jensen, F. Jensen: Hugin-A shell for building bayesian belief universes for expert systems. Proceedings of the Eleventh International Conference on Artificial Intelligence (pp. 1080–1085), Los Altos, CA: Morgan Kaufman, 1989Google Scholar
  5. 5.
    J. R. Anderson, C. F. Boyle, A. Corbett, M. W. Lewis: Cognitive modelling and intelligent tutoring. Artificial Intelligence 42 (p. 7–49), 1990Google Scholar
  6. 6.
    J. R. Anderson, F. G. Conrad, A. T. Corbett:. Skill acquisition and the LISP tutor. Cognitive Science 13 (pp. 467–505), 1989Google Scholar
  7. 7.
    D. Andrich: Rasch models for measurement. Beverly Hills, CA: SAGE Publications, 1988Google Scholar
  8. 8.
    R. D. Bock: Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika 46 (p. 431–444), 1981Google Scholar
  9. 9.
    M. T. H. Chi, M. Bassok, M. W. Lewis, P. Reiman, R. Glaser: Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science 13 (p. 145–182), 1989Google Scholar
  10. 10.
    K. A. Ericsson, H. A. Simon: Protocol Analysis: Verbal reports as data. Cambridge, MA: MIT Press, 1984Google Scholar
  11. 11.
    J. J. Gibson: The ecological approach to visual perception. Boston: Houghton Mifflin, 1979Google Scholar
  12. 12.
    J. G. Greeno: Number sense as situated knowing in a conceptual domain. Journal for Research in Mathemetics Education 22 (pp. 170–218), 1991Google Scholar
  13. 13.
    H. Kelderman: Loglinear multidimensional IRT models for polytomously scored items. Paper presented at the Fifth International Objective Measurement Workshop, University of California, Berkeley, 1989Google Scholar
  14. 14.
    D. H. Krantz: Conjoint measurement: The Luce-Tukey axiomatization and some extensions. Journal of Mathematical Psychology 1 (pp. 248–277), 1964Google Scholar
  15. 15.
    R. D. Luce, J. W. Tukey:. Simultaneous conjoint measurement: A new type of fundamental measurement. Journal of Mathematical Psychology 1 (pp. 1–27), 1964Google Scholar
  16. 16.
    R. J. Mislevy, K. Yamamoto, S. Anacker: Toward a test theory for assessing student understanding. Princeton, NJ: Educational Testing Service, 1990Google Scholar
  17. 17.
    J. Moore, A. Newell: How can MERLIN understand? In L. Gregg (Ed.), Knowledge and cognition (pp. 201–252). Hillsdale, NJ: Lawrence Erlbaum, 1973Google Scholar
  18. 18.
    A. Newell: Unified theories of cognition. Cambridge, MA: Harvard University Press, 1990Google Scholar
  19. 19.
    A. Newell, H. A. Simon: Human problem solving. Englewood Cliffs, NJ: Prentice Hall, 1972Google Scholar
  20. 20.
    J. Pearl: Probabilistic reasoning in intelligent systems: Networks of plausible inference. Los Altos, CA: Morgan Kaufman, 1988Google Scholar
  21. 21.
    R. Perline, B. D. Wright, H. Wainer: The Rasch model as additive conjoint measurement. Applied Psychological Measurement 3 (pp. 237–255), 1979Google Scholar
  22. 22.
    P. Pirolli: Effects of examples and their explanations in a lesson on recursion: A production system analysis. Cognition and Instruction 8 (207–259), 1991Google Scholar
  23. 23.
    P. Pirolli, K. Bielaczyc: Empirical analyses of self-explanation and transfer in learning to program. Proceedings of the Annual Conference of the Cognitive Science Society (pp. 450–457). Hillsdale, NJ: Lawrence Erlbaum, 1989Google Scholar
  24. 24.
    P. Pirolli, M. Recken Knowledge construction and transfer using an intelligent tutoring system: The role of examples, self-explanation, practice, and reflection (Tech, Rep. No. CSM-1). University of California, 1991Google Scholar
  25. 25.
    G. Rasch: Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research, 1960Google Scholar
  26. 26.
    V. J. Shute, R. Glaser: Large-scale evaluation of an intelligent tutoring system: Smithtown. Interactive Learning Environments 1 (pp. 51–76), 1990Google Scholar
  27. 27.
    K. VanLehn: Student modelling. In M. C. Polson, J. J. Richardson (Ed.), Foundations of intelligent tutoring systems (pp. 479–530). Hillsdale, NJ: Lawrence Erlbaum, 1988Google Scholar
  28. 28.
    E. Wenger: Artificial intelligence and tutoring systems. Los Altos, CA: Morgan Kaufman, 1987Google Scholar
  29. 29.
    M. Wilson: An extension of the partial credit model to incorporate diagnostic information. Paper presented at the Annual Meeting of the National Council for Measurement in Education, Boston, MA, 1990Google Scholar
  30. 30.
    M. Wilson: To describe their performances well: Contributions to a new psychometrics. Paper presented at the Natinal Academy of Education Spencer Fellows Forum, Boston, MA, 1991Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1992

Authors and Affiliations

  • Peter Pirolli
    • 1
  • Mark Wilson
    • 1
  1. 1.School of EducationUniversity of California, BerkeleyBerkeley

Personalised recommendations