Skip to main content
SpringerLink
Log in

Towards a model for picture and word processing

  • Chapter
Processing of Visible Language

Part of the book series: Nato Conference Series ((HF,volume 13))

Abstract

A model is proposed to account for similarities and differences between picture and word processing in a variety of semantic and episodic memory tasks. The model contains three levels of processing: the most superficial concerned with low-level processing of the physical characteristics of externally presented pictures and words; an intermediate level in which the results of the low-level processer make contact with prototypical information about how objects (or the pictures which represent them) look and how words sound; and the deepest (propositional) level in which meaning is analyzed. The interlingua between pictures and their names (or between their visual and acoustic images) can take place either directly, with connections between the two image stores, or indirectly, via the propositional level to which both image stores have access. Two differences emerge between pictures and their names: first, greater variability in the way objects or pictures appear compared to the way names appear or sound, which leads to greater variability in prototypical visual images than in prototypical acoustic images; and second, less ambiguity of reference for pictures than for their names, with correspondingly fewer propositional memory nodes accessed by pictures than by words. These differences are shown to be consistent with a large body of literature on pictureword processing differences.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

References

  • Anderson, J. R., & Bower, G. W. Human associative memory. Washington, D.C.: V. H. Winston, 1973.

    Google Scholar 

  • Anderson, J. R., & Bower, G. H. A propositional theory of recognition memory. Memory & Cognition,1974,2, 406–412.

    Article  Google Scholar 

  • Banks, W. P. Encoding and processing of symbolic information in comparative judgments. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 2). New York: Academic Press, 1977.

    Google Scholar 

  • Carroll, J. B., & White, M. N. Word frequency and age of acquisition as determiners of picture-naming latency. Quarterly Journal of Experimental Psychology, 1973,15,85–95.

    Article  Google Scholar 

  • Cattell, J. M. The time it takes to see and name objects. Mind, 1886,11, 63–65.

    Article  Google Scholar 

Picture and word processing

  • Collins, A. M., & Loftus, E. F. A spreading-activation theory of semanticprocessing. Psychological Review, 1975,82,407–428.

    Article  Google Scholar 

  • Collins, A. M., & Quillian, M. R. Experiments on semantic memory and language comprehension. In L. W. Gregg (Ed.),Cognition in learning and memory. New York: John Wiley, 1972. (a)

    Google Scholar 

  • Collins, A. M., & Quillian, M. R. How to make a language user. In E. Tulving & W. Donaldson (Eds.), Organization of memory. New York: Academic Press, 1972. (b)

    Google Scholar 

  • Craik, E. I. M., & Lockhart, R. S. Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 1972,11, 671–684.

    Article  Google Scholar 

  • Craik, E. I. M., & Tulving, E. Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 1975,104, 268–294.

    Article  Google Scholar 

  • Fraisse, P. Recognition time measured by verbal reaction to figures and words. Perceptual & Motor Skills, 1960, 11, 204.

    Google Scholar 

  • Frost, N. Encoding and retrieval in visual memory tasks. Journal of Experimental Psychology, 1972,95, 317–326.

    Article  PubMed  Google Scholar 

  • Jastrzembski, J. E., & Stanners, R. F. Multiple word meanings and lexical search speed. Journal of Verbal Learning and Verbal Behavior, 1975, 14, 534–537.

    Article  Google Scholar 

  • Jenkins, J. R., Neale, D. C., & Deno, S. L. Differential memory for picture and word stimuli. Journal of Educational Psychology, 1967,58,303–307.

    Article  PubMed  Google Scholar 

  • Kintsch, W. The representation of meaning in memory. Hillsdale, N. J.: Lawrence Erlbaum Associates, 1974.

    Google Scholar 

  • Klatzky, R. E. Human memory: Structures and processes. San Francisco: Freeman, 1975.

    Google Scholar 

  • Klatzky, R. L., & Ryan, A. S. Category-structure effects in picture comparisons. Perception & Psychophysics, 1978,23, 193–204.

    Article  Google Scholar 

  • Klatzky, R. L., & Stoy, A. M. Using visual codes for comparisons of pictures. Memory & Cognition, 1974,2, 727–736.

    Article  Google Scholar 

  • Kosslyn, S. M. Scanning visual images: Some structual implications. Perception Psychophysics, 1973,14, 90–94.

    Article  Google Scholar 

  • Kosslyn, S. M. Information representation in visual images. Cognitive Psychology, 1975,7,341–370.

    Article  Google Scholar 

  • Kosslyn, S. M. Imagery and internal representation. In E. Rosch & B. B. Lloyd (Eds.), Cognition and categorization. Hillsdale, N. J.: Lawrence Erlbaum Associates, 1978.

    Google Scholar 

  • Kosslyn, S. M., Ball, T. M., & Reiser, B.J. Visual images preserve metric spatial information: Evidence from studies of image scanning. Journal of Experimental Psychology: Human Perception and Performance, 1978,4, 47–60.

    Article  PubMed  Google Scholar 

  • Kosslyn, S. M., & Shwartz, S. P. A data-driven simulation of visual imagery. Cognitive Science, 1977,1, 265–296.

    Article  Google Scholar 

  • Lachman, R. Uncertainty effects on time to access the internallexicon. Journal of Experimental Psychology, 1973,99, 199–208.

    Article  Google Scholar 

  • Marcel, A.J. Conscious and preconscious recognition of polysemous words: Locating the selective effects of prior verbal context. In R. S. Nickerson (Ed.), Attention and Performance VIII. Hillsdale, N.J.: Lawrence Erlbaum Associates, 1979.

    Google Scholar 

  • Martin, E. Generation-recognition theory and the encoding specificity principle. Psychological Review, 1916, 82, 150–153.

    Article  Google Scholar 

  • Meyer, D. E., & Schvaneveldt, R. W. Facilitation in recognizing words: Evidence of a dependence between retrieval operations. Journal of Experimental Psychology, 1971,90, 227–234.

    Article  PubMed  Google Scholar 

  • Neisser, U. Cognitive psychology. New York: Appleton-Century-Crofts, 1967.

    Google Scholar 

  • Nelson, D. L. Remembering pictures and words: Appearance, significance, & name. In F. I. M. Craik & L. Cermak (Eds.), Levels of processing. Hillsdale, N.J.: Lawrence Erlbaum Associates, 1979.

    Google Scholar 

  • Nelson, D. L., Reed, V. S., & McEvoy, C. L. Learning to order pictures and words: A model of sensory and semantic encoding. Journal of Experimental Psychology: Human Learning and Memory, 1977,5, 485–497.

    Article  Google Scholar 

  • Nelson, D. L., Reed, V. S., & Walling, J. R. Pictorial superiority effect. Journal of Experimental Psychology: Human Learning and Memory, 1976,2, 523–528.

    Article  Google Scholar 

  • Norman, D. A., & Rumelhart, D. E. Explorations in cognition. San Francisco: Freeman, 1975.

    Google Scholar 

  • Oldfield, R. C., & Wingfield, A. Response latencies in naming objects. Quarterly Journal of Experimental Psychology, 1965,17, 273–281.

    Article  PubMed  Google Scholar 

  • Paivio, A. Latency of verbal associations and imagery to noun stimuli as a function of abstractness and generality. Canadian Journal of Psychology, 1966,20, 378–387.

    Article  PubMed  Google Scholar 

  • Paivio, A. Mental imagery in associative learning and memory. Psychological Review, 1969, 76,241–263.

    Article  Google Scholar 

  • Paivio, A. Imagery and verbal processes. New York: Holt, Rinehart, & Winston, 1971.

    Google Scholar 

  • Paivio, A. Imagery in recall and recognition. In J. Brown (Ed.), Recall and recognition. New York: Wiley, 1976.

    Google Scholar 

  • Paivio, A., & Csapo, K. Picture superiority in free recall: Imagery or dual coding? Cognitive Psychology, 1973,5, 176–206.

    Article  Google Scholar 

  • Pellegrino, J. W., Rosinski, R. R., Chiesi, H. L., & Siegel, A. Picture-word differences in decision latency: An analysis of single and dual memory models. Memory & Cognition, 1977,5, 383–396.

    Article  Google Scholar 

  • Potter, M. C., & Faulconer, B. A. Time to understand pictures and -words. Nature, 1975,253,437–438.

    Article  PubMed  Google Scholar 

  • Pylyshyn, Z. W. What the mind’s eye tells the mind’s brain: A critique of mental imagery. Psychological Bulletin, 1973,50, 1–24.

    Article  Google Scholar 

  • Pylyshyn, Z. W. Imagery and artificial intelligence. In W. Savage (Ed.), Perception and cognition: Issues in the foundations of psychology. Minneapolis: University of Minnesota Press, 1978.

    Google Scholar 

  • Quillian, M. R. Word concepts: A theory and simulation of some basic semantic capabilities.Behavioral Science, 1967,12, 410–430.

    Article  Google Scholar 

  • Reder, L. M., Anderson, J. R., & Bjork, R. A. A semantic interpretation of encoding specilicity. Journal of Experimental Psychology, 1974,102, 648–656.

    Article  Google Scholar 

  • Rips, L. J., Shoben, E. J., & Smith, E. E. Semantic distance and the verification of semanticrelations. Journal of Verbal Learning and Verbal Behavior, 1973, 12, 1–20.

    Article  Google Scholar 

  • Rosch, E. Cognitive representations of semantic categories. Journal of Experimental Psychology: General, 1975,104, 192–233.

    Article  Google Scholar 

  • Rosch, E. Principles of categorization. In E. Rosch & B. B. Lloyd (Eds.), Cognition and categorization. Hillsdale, N.J.: Lawrence Erlbaum Associates, 1978.

    Google Scholar 

  • Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., & Boyes-Braem, P. Basic objects in natural categories. Cognitive Psychology, 1976,5, 382–439.

    Article  Google Scholar 

  • Rubenstein, H., Garfield, L., & Milliken, J. A. Homographic entries in the internal lexicon. Journal of Verbal Learning and Verbal Behavior, 1970,9, 487–494.

    Article  Google Scholar 

  • Rubenstein, H., Lewis, S. S., Rubenstein, M. A. Evidence for phonemic recoding in visual word recognition. Journal of Verbal Learning and Verbal Behavior, 1971,10,645–657.

    Article  Google Scholar 

  • Schvaneveldt, R. W., & Meyer, D. E. Retrieval and comparison processes in semantic memory. In S. Kornblum (Ed.), Attention and performance IV. New York: Academic Press, 1973.

    Google Scholar 

  • Schvaneveldt, R. W., Meyer, D. E., & Becker, C. A. Lexical ambiguity, semantic context, and visual word recognition, Journal of Experimental Psychology: Human Perception and Performance, 1976,2, 243–256.

    Article  PubMed  Google Scholar 

  • Smith, E. E. Theories of semantic memory. In W. K. Estes (Ed.), Handbook of learning and cognitive processes (Vol. 5). Hillsdale, N.J.: Lawrence Erlbaum Associates, 1978.

    Google Scholar 

  • Smith, E. E., Shoben, E. K., & Rips, L.J. Structure and process in semantic memory: A featural model for semantic decisions. Psychological Review, 1974,51,214–241.

    Article  Google Scholar 

  • Smith, M.C., & Magee, L. E. Tracing the time course of picture-word processing. Journal of Experimental Psychology: General, 1980 (in press).

    Google Scholar 

  • Snodgrass, J. G., & Asiaghi, A. The pictorial superiority effect in recognition memory. Bulletin of the Psychonomic Society, 1977,10, 1–4.

    Google Scholar 

  • Snodgrass, J. G., Burns, P. M., & Pirone, G. V. Pictures and words and space and dme: In search of the elusive interaction. Journal of Experimental Psychology: General, 1978,107,206–230.

    Article  Google Scholar 

  • Snodgrass, J. G., & McClure, P. Storage and retrieval properties of dual codes for pictures and words in recognition memory. Journal of Experimental Psychology: Human Learning and Memory, 1975,1, 521–529.

    Article  Google Scholar 

  • Snodgrass, J. G., & Vanderwart, M. A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 1980 (in press).

    Google Scholar 

  • Snodgrass, J. G., Wasser, B., Finkelstein, M., & Goldberg, L. B. On the fate of visual and verbal memory codes for pictures and words: Evidence for a dual coding mechanism in recognition memory. Journal of Verbal Learning and Verbal Behavior, 1974,13,27–37.

    Article  Google Scholar 

  • Tulving, E. Episodic and semantic memory. In E. Tulving & W. Donaldson (Eds.), Organization of memory. New York: Academic Press, 1972.

    Google Scholar 

  • Tulving, E., & Thomson, D. M. Retrieval processes in recognition memory: Effects of associative context. Journal of Experimental Psychology, 1971,81, 116–124.

    Article  Google Scholar 

  • Wickelgren, W. A. Chunking and consolidation: A theoretical synthesis of semantic networks, configuring in conditioning, S-R versus cognitive learning, normal forgetting, the amnesic syndrome, and the hippocampal arousal system. Psychological Review, 1979, 86, 44–60.

    Article  PubMed  Google Scholar 

Download references

Authors
  1. Joan Gay Snodgrass
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Toronto, Toronto, Ontario, Canada

    Paul A. Kolers

  2. Visible Language Journal, Cleveland, Ohio, USA

    Merald E. Wrolstad

  3. Institute for Perception Research, Eindhoven, The Netherlands

    Herman Bouma

Rights and permissions

Reprints and permissions

Copyright information

© 1980 Plenum Press, New York

About this chapter

Cite this chapter

Snodgrass, J.G. (1980). Towards a model for picture and word processing. In: Kolers, P.A., Wrolstad, M.E., Bouma, H. (eds) Processing of Visible Language. Nato Conference Series, vol 13. Springer, Boston, MA. https://doi.org/10.1007/978-1-4684-1068-6_42

Download citation

  • DOI: https://doi.org/10.1007/978-1-4684-1068-6_42

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4684-1070-9

  • Online ISBN: 978-1-4684-1068-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation