Cognitive reality of multidimensional information processing in knowledge assessment by experts

  • Jean -Paul Caverni
  • Jean -Luc Péris


Two experiments were designed in order to examine the knowledge assessment task accomplished by French teachers assessing essays written by French native secondary school students. The focus of the study bears on the potentially multidimensional nature of the processing of evaluation cues.

In the experimental conditions conventionally designed to test multidimensional evaluation models or techniques, information about each object to be evaluated (and hence, the dimensions along which objects are compared with each other) are explicitly provided to the subject from some external source.

The experiments presented here were aimed at studying information processing by experts performing an evaluation task in which the experts themselves are required to: (i) define the relevant dimensions to be used in the evaluation and (ii) elaborate the necessary information about each of those dimensions for each object (in this case, native language essays). The question is to find out whether or not the information processing carried out by such subjects is multidimensional.

If evaluators do in fact perform multidimensional processing of information, this implies not only that evaluation cues belong to differentiated classes (Experiment 1), but also that cues belonging to the same class are processed with respect to each other before being processed with respect to cues of another class (Experiment 2).

The results obtained here support that conclusion. Both experiments used the self-paced display paradigm. Subjects were presented with segments of text on a computer screen, and the display time for each segment was recorded. Random coloration of errors belonging to three evaluator-defined classes was found to increase display time on the corresponding segments.

Key words

Cognition Expertise Knowledge assessment Multidimensional information processing 


  1. Aaronson, D., & Scarborough, H. S. (1976). Performance theories for sentence coding: some quantitative evidence.Journal of Experimental Psychology: Human Perception and Performance, 2, 56–70.CrossRefGoogle Scholar
  2. Berkeley, D., & Humphreys, P. (1982). Structuring decision problems and the bias heuristic.Acta Psychologica, 50, 201–252.CrossRefGoogle Scholar
  3. Birnbaum, M. M. (1982). Controversies in psychological measurement. In B. Wegener (Ed.),Social attitudes and psychological measurement, (pp. 401–485). Hillsdale, N.Y.: Erlbaum.Google Scholar
  4. Cast, B. M. D. (1939). The efficiency of different methods marking english composition I.British Journal of Educational Psychology, 10, 257–269.Google Scholar
  5. Cast, B. M. D. (1940). The efficiency of different methods marking english composition II.British Journal of Educational Psychology, 11, 49–60.Google Scholar
  6. Caverni, J.-P. (1981). La fiabilité de la notation scolaire par expert comme fonction du mode d’explicitation analytique ou synthétique de la note.L’Année Psychologique, 81, 369–384.CrossRefGoogle Scholar
  7. Caverni, J.-P. (1987a). Knowledge acquisition assessment by expert: effects and models of the cognitive functioning of evaluators.European Journal of Psychology of Education, 2, 119–131.CrossRefGoogle Scholar
  8. Caverni, J.-P. (1987b). Self-Paced display time for process tracing in assessment of acquired knowledge.European Bulletin of Cognitive Psychology, 7, 633–651.Google Scholar
  9. Caverni, J.-P. (1988). La verbalisation comme source d’observables pour l’étude du fonctionnement cognitif. In J.-P. Caverni et al. (Eds),Psychologie cognitive: modèles et méthodes, (pp. 253–273). Grenoble: Presses Universitaires de Grenoble.Google Scholar
  10. Caverni, J.-P., & Gonzalez, M. (1986). Vers une modélisation des processus de jugement dans l’évaluation.Bulletin de Psychologie, 39, 301–304.Google Scholar
  11. Coffman, W. E., & Kurfman, D. (1968). A comparison of two methods of reading essay examinations.American Educational Research Journal, 5, 99–107.Google Scholar
  12. Coombs, C. H. (1964).A theory of data. New York: Wiley.Google Scholar
  13. Dawes, R. M. (1964). Social selection based on multidimensional criteria.Journal of Abnormal and Social Psychology, 68, 104–109.CrossRefGoogle Scholar
  14. Dawes, R. M., & Corrigan, B. (1974). Linear models in decision making.Psychological Bulletin, 81, 95–106.CrossRefGoogle Scholar
  15. Ebbesen, E. B., & Konecni, V. J. (1980). On the external validity of decision-making research: What do we know about decisions in the real world? In T. S. Wallsten (Ed.)Cognitive processes in choice and decision behavior, (pp. 213–228). Hillsdale, NJ: Laurence Erlbaum Associates.Google Scholar
  16. Einhorn, H. J. (1972). Expert measurement and mechanical combination.Organizational Behavior and Human Performance, 7, 86–106.CrossRefGoogle Scholar
  17. Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data.Psychological Review, 87, 215–251.CrossRefGoogle Scholar
  18. Evans, J. St. B. T. (1983). Selective processes in reasoning. In J. St B. T. Evans (Ed.),Thinking & reasoning: psychological approaches, (pp. 135–163). London: Routledge and Kegan PaulGoogle Scholar
  19. Fidler, E. J. (1983). The reliability and validity of concurrent, retrospective, and interpretive verbal reports: an experimental study. In P. Humphreys, O. Svenson & A. Vari (Eds),Analysing and aiding decision processes, (pp. 429–440). Amsterdam: North Holland.CrossRefGoogle Scholar
  20. Follman, J., Holland, M., & Miller, W. (1971). Effects of instructions in theme grading: grammatical vs holistic.Child Study Journal, 1, 135–141.Google Scholar
  21. Freedman, S. W. (1979). How characteristics of student essays influence teachers’ evaluations.Journal of Educational Psychology, 71, 328–338.CrossRefGoogle Scholar
  22. Hartog, P., & Rhodes, E. C. (1935).An examination of examinations. London: MacMillan.Google Scholar
  23. Hogarth, R. M. (1980).Judgement and choice: the psychology of decision. Chichester: John Wiley & Sons (2nd ed. 1988).Google Scholar
  24. Kleinmuntz, D. N. (1985). Human decision processes: heuristics and task structure. In P. A. Hancock (Ed.),Human Factors Psychology (pp. 123–142). Amsterdam: North Holland.Google Scholar
  25. Laugier, H., & Weinberg, D. (1936).La correction des épreuves écrites dans les examens. Paris: Maison du Livre.Google Scholar
  26. Montgomery, H., & Svenson, O. (1976). On decision rules and information processing strategies for choice among multiattribute alternatives.Scandinavian Journal of Psychology, 17, 283–291.CrossRefGoogle Scholar
  27. Morrisson, R. L., & Vernon, P. E. (1941). A new method of marking English composition.British Journal of Educational Psychology, 12, 17–27.Google Scholar
  28. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: verbal reports on mental processes.Psychological Review, 84, 231–259.CrossRefGoogle Scholar
  29. Pitz, G. F., & Sachs, N. J. (1984). Judgment and decision: theory and application.Annual Review of Psychology, 35, 139–163.CrossRefGoogle Scholar
  30. Pollard, P., & Evans, J. St B. T. (1983). The role of representativeness in statistical inference: a critical appraisal. In J. St B. T. Evans (Ed.)Thinking and reasoning: psychological approaches, (pp. 107–134). London: Routledge and Keagan Paul.Google Scholar
  31. Pynte, J., & Noizet, G. (1980). Optimal segmentation for sentences displayed on a video screen. In P. Kolers, H. Wrolstad & H. Bourna (Eds.)Processing of visible language, II, (pp. 375–386). New-York: Plenum Press.Google Scholar
  32. Russo, J. E., Johnson, E. J., & Sthepens, D. L. (1989). The validity of verbal protocols.Memory & Cognition, 17, 759–769.Google Scholar
  33. Slovic, P., & Lichtenstein, S. (1971). Comparison of bayesian and regression approaches to the study of information processing in judgment.Organizational Behavior and Human Performance, 6, 649–744.CrossRefGoogle Scholar
  34. Svenson, O. (1979). Process descriptions of decision making.Organizational Behavior and Human Performance, 23, 86–112.CrossRefGoogle Scholar
  35. White, P. A. (1980). Limitations on verbal reports of internal events: a refutation of Nisbett and Wilson and of Bem.Psychological Review, 87, 105–112.CrossRefGoogle Scholar
  36. White, P. A. (1988). Knowing more about what we can tell: «introspective access» and causal report accuracy 10 years later.British Journal of Psychology, 79, 13–45.Google Scholar
  37. Winkler, R. L., & Murphy, A. H. (1973). Experiments in the laboratory and the real world.Organizational Behavior and Human Performance, 10, 252–270.CrossRefGoogle Scholar
  38. Wiseman, S. (1949). The marking of English composition in grammar school selection.British Journal of Educational Psychology, 19, 200–209.Google Scholar

Copyright information

© Instituto Superior de Psicologia Aplicada, Lisbon, Portugal/ Springer Netherlands 1992

Authors and Affiliations

  • Jean -Paul Caverni
    • 1
  • Jean -Luc Péris
    • 1
  1. 1.CNRS and CREPCOUniversity of ProvenceFrance

Personalised recommendations