Cognitive reality of multidimensional information processing in knowledge assessment by experts
- 7 Downloads
Two experiments were designed in order to examine the knowledge assessment task accomplished by French teachers assessing essays written by French native secondary school students. The focus of the study bears on the potentially multidimensional nature of the processing of evaluation cues.
In the experimental conditions conventionally designed to test multidimensional evaluation models or techniques, information about each object to be evaluated (and hence, the dimensions along which objects are compared with each other) are explicitly provided to the subject from some external source.
The experiments presented here were aimed at studying information processing by experts performing an evaluation task in which the experts themselves are required to: (i) define the relevant dimensions to be used in the evaluation and (ii) elaborate the necessary information about each of those dimensions for each object (in this case, native language essays). The question is to find out whether or not the information processing carried out by such subjects is multidimensional.
If evaluators do in fact perform multidimensional processing of information, this implies not only that evaluation cues belong to differentiated classes (Experiment 1), but also that cues belonging to the same class are processed with respect to each other before being processed with respect to cues of another class (Experiment 2).
The results obtained here support that conclusion. Both experiments used the self-paced display paradigm. Subjects were presented with segments of text on a computer screen, and the display time for each segment was recorded. Random coloration of errors belonging to three evaluator-defined classes was found to increase display time on the corresponding segments.
Key wordsCognition Expertise Knowledge assessment Multidimensional information processing
- Birnbaum, M. M. (1982). Controversies in psychological measurement. In B. Wegener (Ed.),Social attitudes and psychological measurement, (pp. 401–485). Hillsdale, N.Y.: Erlbaum.Google Scholar
- Cast, B. M. D. (1939). The efficiency of different methods marking english composition I.British Journal of Educational Psychology, 10, 257–269.Google Scholar
- Cast, B. M. D. (1940). The efficiency of different methods marking english composition II.British Journal of Educational Psychology, 11, 49–60.Google Scholar
- Caverni, J.-P. (1987b). Self-Paced display time for process tracing in assessment of acquired knowledge.European Bulletin of Cognitive Psychology, 7, 633–651.Google Scholar
- Caverni, J.-P. (1988). La verbalisation comme source d’observables pour l’étude du fonctionnement cognitif. In J.-P. Caverni et al. (Eds),Psychologie cognitive: modèles et méthodes, (pp. 253–273). Grenoble: Presses Universitaires de Grenoble.Google Scholar
- Caverni, J.-P., & Gonzalez, M. (1986). Vers une modélisation des processus de jugement dans l’évaluation.Bulletin de Psychologie, 39, 301–304.Google Scholar
- Coffman, W. E., & Kurfman, D. (1968). A comparison of two methods of reading essay examinations.American Educational Research Journal, 5, 99–107.Google Scholar
- Coombs, C. H. (1964).A theory of data. New York: Wiley.Google Scholar
- Ebbesen, E. B., & Konecni, V. J. (1980). On the external validity of decision-making research: What do we know about decisions in the real world? In T. S. Wallsten (Ed.)Cognitive processes in choice and decision behavior, (pp. 213–228). Hillsdale, NJ: Laurence Erlbaum Associates.Google Scholar
- Evans, J. St. B. T. (1983). Selective processes in reasoning. In J. St B. T. Evans (Ed.),Thinking & reasoning: psychological approaches, (pp. 135–163). London: Routledge and Kegan PaulGoogle Scholar
- Follman, J., Holland, M., & Miller, W. (1971). Effects of instructions in theme grading: grammatical vs holistic.Child Study Journal, 1, 135–141.Google Scholar
- Hartog, P., & Rhodes, E. C. (1935).An examination of examinations. London: MacMillan.Google Scholar
- Hogarth, R. M. (1980).Judgement and choice: the psychology of decision. Chichester: John Wiley & Sons (2nd ed. 1988).Google Scholar
- Kleinmuntz, D. N. (1985). Human decision processes: heuristics and task structure. In P. A. Hancock (Ed.),Human Factors Psychology (pp. 123–142). Amsterdam: North Holland.Google Scholar
- Laugier, H., & Weinberg, D. (1936).La correction des épreuves écrites dans les examens. Paris: Maison du Livre.Google Scholar
- Morrisson, R. L., & Vernon, P. E. (1941). A new method of marking English composition.British Journal of Educational Psychology, 12, 17–27.Google Scholar
- Pollard, P., & Evans, J. St B. T. (1983). The role of representativeness in statistical inference: a critical appraisal. In J. St B. T. Evans (Ed.)Thinking and reasoning: psychological approaches, (pp. 107–134). London: Routledge and Keagan Paul.Google Scholar
- Pynte, J., & Noizet, G. (1980). Optimal segmentation for sentences displayed on a video screen. In P. Kolers, H. Wrolstad & H. Bourna (Eds.)Processing of visible language, II, (pp. 375–386). New-York: Plenum Press.Google Scholar
- Russo, J. E., Johnson, E. J., & Sthepens, D. L. (1989). The validity of verbal protocols.Memory & Cognition, 17, 759–769.Google Scholar
- White, P. A. (1988). Knowing more about what we can tell: «introspective access» and causal report accuracy 10 years later.British Journal of Psychology, 79, 13–45.Google Scholar
- Wiseman, S. (1949). The marking of English composition in grammar school selection.British Journal of Educational Psychology, 19, 200–209.Google Scholar