The Interaction of Verbal and Pictorial Information in Comprehension and Memory

Abstract

In two experiments, the author investigated how the correspondence between a verbal representation (a text describing data) and a simultaneously presented graphical representation (a graph showing data) affects comprehension of, and memory for, the results of empirical studies. Consistency of text and graph was varied by verbally describing observed differences between experimental conditions as small or large, and by using graphical depictions that made the effects appear to be small or large. In accord with the structure mapping hypothesis suggested by Schnotz and Bannert (2003), inconsistency of texts and graphs slowed down comprehension. Such inconsistency increased gaze durations on texts and graphs as well as the number of eye movements between text and graph. It also decreased confidence in a later memory task. Moreover, memory for the text was affected by the graph and memory for the graph was affected by the text, yielding distortions in cases of inconsistency.

Appendix

Sample Experimental Study Description, translated from German

• 1- Introduction:

• This experiment investigated how memory depends on the context. A group of scuba divers learned lists of words. Half of them learned on land, and the other half learned under water. Afterwards, half of each group were asked to recall the words on land or under water. The experimenters counted how many words were recalled in each of the four possible combinations of learning environment (on land vs. under water) and recall environment (on land vs. under water). They expected that recall would be better when learning environment and recall environment are identical.

• 2a- Results Description Suggesting Large Effect:

• Indeed, divers who both learned and recalled on land remembered a very high number of words (33 on average). Similarly, divers who learned and recalled under water, also recalled a very high number of words (31). However, if the environment changed, recall was much worse: Learning on land and recalling under water yielded a mean of only 27 words, and learning under water and recalling on land yielded only 28 words.

• 2b- Results Description Suggesting Small Effect:

• However, divers who both learned and recalled on land remembered a medium number of words (33 on average). Similarly, divers who learned and recalled under water, also recalled a medium number of words (31). If the environment changed, recall was almost as good: Learning on land and recalling under water yielded a mean of 27 words, and learning under water and recalling on land even yielded 28 words.