Abstract
Two experiments were designed in order to examine the knowledge assessment task accomplished by French teachers assessing essays written by French native secondary school students. The focus of the study bears on the potentially multidimensional nature of the processing of evaluation cues.
In the experimental conditions conventionally designed to test multidimensional evaluation models or techniques, information about each object to be evaluated (and hence, the dimensions along which objects are compared with each other) are explicitly provided to the subject from some external source.
The experiments presented here were aimed at studying information processing by experts performing an evaluation task in which the experts themselves are required to: (i) define the relevant dimensions to be used in the evaluation and (ii) elaborate the necessary information about each of those dimensions for each object (in this case, native language essays). The question is to find out whether or not the information processing carried out by such subjects is multidimensional.
If evaluators do in fact perform multidimensional processing of information, this implies not only that evaluation cues belong to differentiated classes (Experiment 1), but also that cues belonging to the same class are processed with respect to each other before being processed with respect to cues of another class (Experiment 2).
The results obtained here support that conclusion. Both experiments used the self-paced display paradigm. Subjects were presented with segments of text on a computer screen, and the display time for each segment was recorded. Random coloration of errors belonging to three evaluator-defined classes was found to increase display time on the corresponding segments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aaronson, D., & Scarborough, H. S. (1976). Performance theories for sentence coding: some quantitative evidence.Journal of Experimental Psychology: Human Perception and Performance, 2, 56–70.
Berkeley, D., & Humphreys, P. (1982). Structuring decision problems and the bias heuristic.Acta Psychologica, 50, 201–252.
Birnbaum, M. M. (1982). Controversies in psychological measurement. In B. Wegener (Ed.),Social attitudes and psychological measurement, (pp. 401–485). Hillsdale, N.Y.: Erlbaum.
Cast, B. M. D. (1939). The efficiency of different methods marking english composition I.British Journal of Educational Psychology, 10, 257–269.
Cast, B. M. D. (1940). The efficiency of different methods marking english composition II.British Journal of Educational Psychology, 11, 49–60.
Caverni, J.-P. (1981). La fiabilité de la notation scolaire par expert comme fonction du mode d’explicitation analytique ou synthétique de la note.L’Année Psychologique, 81, 369–384.
Caverni, J.-P. (1987a). Knowledge acquisition assessment by expert: effects and models of the cognitive functioning of evaluators.European Journal of Psychology of Education, 2, 119–131.
Caverni, J.-P. (1987b). Self-Paced display time for process tracing in assessment of acquired knowledge.European Bulletin of Cognitive Psychology, 7, 633–651.
Caverni, J.-P. (1988). La verbalisation comme source d’observables pour l’étude du fonctionnement cognitif. In J.-P. Caverni et al. (Eds),Psychologie cognitive: modèles et méthodes, (pp. 253–273). Grenoble: Presses Universitaires de Grenoble.
Caverni, J.-P., & Gonzalez, M. (1986). Vers une modélisation des processus de jugement dans l’évaluation.Bulletin de Psychologie, 39, 301–304.
Coffman, W. E., & Kurfman, D. (1968). A comparison of two methods of reading essay examinations.American Educational Research Journal, 5, 99–107.
Coombs, C. H. (1964).A theory of data. New York: Wiley.
Dawes, R. M. (1964). Social selection based on multidimensional criteria.Journal of Abnormal and Social Psychology, 68, 104–109.
Dawes, R. M., & Corrigan, B. (1974). Linear models in decision making.Psychological Bulletin, 81, 95–106.
Ebbesen, E. B., & Konecni, V. J. (1980). On the external validity of decision-making research: What do we know about decisions in the real world? In T. S. Wallsten (Ed.)Cognitive processes in choice and decision behavior, (pp. 213–228). Hillsdale, NJ: Laurence Erlbaum Associates.
Einhorn, H. J. (1972). Expert measurement and mechanical combination.Organizational Behavior and Human Performance, 7, 86–106.
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data.Psychological Review, 87, 215–251.
Evans, J. St. B. T. (1983). Selective processes in reasoning. In J. St B. T. Evans (Ed.),Thinking & reasoning: psychological approaches, (pp. 135–163). London: Routledge and Kegan Paul
Fidler, E. J. (1983). The reliability and validity of concurrent, retrospective, and interpretive verbal reports: an experimental study. In P. Humphreys, O. Svenson & A. Vari (Eds),Analysing and aiding decision processes, (pp. 429–440). Amsterdam: North Holland.
Follman, J., Holland, M., & Miller, W. (1971). Effects of instructions in theme grading: grammatical vs holistic.Child Study Journal, 1, 135–141.
Freedman, S. W. (1979). How characteristics of student essays influence teachers’ evaluations.Journal of Educational Psychology, 71, 328–338.
Hartog, P., & Rhodes, E. C. (1935).An examination of examinations. London: MacMillan.
Hogarth, R. M. (1980).Judgement and choice: the psychology of decision. Chichester: John Wiley & Sons (2nd ed. 1988).
Kleinmuntz, D. N. (1985). Human decision processes: heuristics and task structure. In P. A. Hancock (Ed.),Human Factors Psychology (pp. 123–142). Amsterdam: North Holland.
Laugier, H., & Weinberg, D. (1936).La correction des épreuves écrites dans les examens. Paris: Maison du Livre.
Montgomery, H., & Svenson, O. (1976). On decision rules and information processing strategies for choice among multiattribute alternatives.Scandinavian Journal of Psychology, 17, 283–291.
Morrisson, R. L., & Vernon, P. E. (1941). A new method of marking English composition.British Journal of Educational Psychology, 12, 17–27.
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: verbal reports on mental processes.Psychological Review, 84, 231–259.
Pitz, G. F., & Sachs, N. J. (1984). Judgment and decision: theory and application.Annual Review of Psychology, 35, 139–163.
Pollard, P., & Evans, J. St B. T. (1983). The role of representativeness in statistical inference: a critical appraisal. In J. St B. T. Evans (Ed.)Thinking and reasoning: psychological approaches, (pp. 107–134). London: Routledge and Keagan Paul.
Pynte, J., & Noizet, G. (1980). Optimal segmentation for sentences displayed on a video screen. In P. Kolers, H. Wrolstad & H. Bourna (Eds.)Processing of visible language, II, (pp. 375–386). New-York: Plenum Press.
Russo, J. E., Johnson, E. J., & Sthepens, D. L. (1989). The validity of verbal protocols.Memory & Cognition, 17, 759–769.
Slovic, P., & Lichtenstein, S. (1971). Comparison of bayesian and regression approaches to the study of information processing in judgment.Organizational Behavior and Human Performance, 6, 649–744.
Svenson, O. (1979). Process descriptions of decision making.Organizational Behavior and Human Performance, 23, 86–112.
White, P. A. (1980). Limitations on verbal reports of internal events: a refutation of Nisbett and Wilson and of Bem.Psychological Review, 87, 105–112.
White, P. A. (1988). Knowing more about what we can tell: «introspective access» and causal report accuracy 10 years later.British Journal of Psychology, 79, 13–45.
Winkler, R. L., & Murphy, A. H. (1973). Experiments in the laboratory and the real world.Organizational Behavior and Human Performance, 10, 252–270.
Wiseman, S. (1949). The marking of English composition in grammar school selection.British Journal of Educational Psychology, 19, 200–209.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Caverni, J.P., Péris, J.L. Cognitive reality of multidimensional information processing in knowledge assessment by experts. Eur J Psychol Educ 7, 109–122 (1992). https://doi.org/10.1007/BF03172888
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF03172888