Skip to main content
SpringerLink
Log in

Visually Situated Language Comprehension in Children and in Adults

  • Chapter
  • First Online:
Attention and Vision in Language Processing

Abstract

Accounts of visually situated language processing accommodate an active role of the visual context and a tight temporal coordination between comprehension, visual attention, and visual context effects in young adults. By contrast, these psycholinguistic accounts have not yet modelled variation due to age groups (e.g., children), social (e.g., class), or cognitive factors (e.g., a comprehender’s working memory). Extending these accounts with a model of the comprehender could refine their predictions regarding variation due to factors such as comprehender age amongst others. With this goal in mind, the present chapter reviews results on realtime visual context effects and visually situated language comprehension in children and in healthy young adults. It also uses insights from the literature on language learning to argue that visual context should play an active role in child language comprehension and that children benefit from a similarly rapid interplay of visual attention and language comprehension as do young adults.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The transitional probability (X|Y) of a syllable pair XY is given by the frequency of the pair XY divided by the frequency of X.

  2. 2.

    The prosodic manipulation was similar to Experiment 1; prosodic analyses ascertained accent differences between the instrument (mostly H* on the verb and noun and a pitch accent on “with”) and the modifier intonation (mostly an L + H* accent on the verb, a clear H* on the noun, and no pitch accent on “with”).

  3. 3.

    When more than one syntactic analysis is possible, the referentially supported analysis is preferred.

  4. 4.

    Object-initial word order is grammatical in German but non-canonical, and elicits processing difficulty (e.g., the case-marking on the determiner of the noun phrases elicits longer reading times).

References

  • Abashidze, D., Knoeferle, P., & Carminati, M. N. (2013). Do comprehenders prefer to rely on recent events even when future events are more likely to be mentioned? In Proceedings of the Conference on Architectures and Mechanisms for Language Processing. Marseille, France.

    Google Scholar 

  • Allopenna, P., Magnuson, J. S., & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38, 419–439.

    Article  Google Scholar 

  • Altmann, G. T. M., & Kamide, Y. (1999). Incremental interpretation at verbs: restricting the domain of subsequent reference. Cognition, 73, 247–264.

    Article  Google Scholar 

  • Altmann, G. T. M., & Kamide, Y. (2007). The real-time mediation of visual attention by language and world knowledge: linking anticipatory (and other) eye movements to linguistic processing. Journal of Memory and Language, 57, 502–518.

    Article  Google Scholar 

  • Andersson, R., Ferreira, F., & Henderson, J. (2011). I see what you’re saying: The integration of complex speech and scenes during language comprehension. Acta Psychologica, 137, 208–216.

    Article  Google Scholar 

  • Bergelson, E., & Swingley, D. (2012). At 6–9 months, human infants know the meanings of many common nouns. In Proceedings of the National Academy of Sciences of the USA, 109, 3253–3258.

    Article  Google Scholar 

  • Carminati, M. N., & Knoeferle, P. (2013). Effects of speaker emotional facial expression and listener age on incremental sentence processing. PLoS ONE, 8(9), e72559. doi:10.1371/journal.pone.0072559.

    Article  Google Scholar 

  • Chambers, C. G., Tanenhaus, M. K., & Magnuson, J. S. (2004). Actions and affordances in syntactic ambiguity resolution. Journal of Experimental Psychology. Learning, Memory, and Cognition, 30, 687–696.

    Article  Google Scholar 

  • Dahan, D., Swingley, D., Tanenhaus, M. K., Magnuson, J. (2000). Linguistic gender and spoken-word recognition in French. Journal of Memory and Language, 42, 465–480.

    Google Scholar 

  • Dahan, D., & Tanenhaus, M. K. (2005). Looking at the rope when looking for the snake: Conceptually mediated eye movements during spoken-word recognition. Psychonomic Bulletin & Review, 12, 453–459.

    Article  Google Scholar 

  • Dahan, D. (2010). The time course of interpretation in speech comprehension. Current Directions in Psychological Science, 19, 121–126.

    Article  Google Scholar 

  • Fenson, L., Dale, P., Reznick, J., Bates, E., Thal, D., & Pethick, S. (1994). Variability in early communicative development. Monographs of the Society for Research on Child Development, 59, 1–189.

    Article  Google Scholar 

  • Fernald, A., Thorpe, K., & Marchman, V. A. (2010). Blue car, red car: Developing efficiency in online interpretation of adjective-noun phrases. Cognitive Psychology, 60, 190–217.

    Article  Google Scholar 

  • Fernald, A., Pinto, J. P., Swingley, D., Weinberg, A., & McRoberts, G. W. (1998). Rapid gains in speed of verbal processing by infants in the second year. Psychological Science, 9, 72–75.

    Article  Google Scholar 

  • Fisher, C. (1996). Structural limits on verb mapping: The role of analogy in children’s interpretation of sentences. Cognitive Psychology, 31, 41–81.

    Article  Google Scholar 

  • Fisher, C. (2002). Structural limits on verb mapping: the role of abstract structure in 2.5-year-olds’ interpretations of novel verbs. Developmental Science, 5, 55–64.

    Article  Google Scholar 

  • Fisher, C., Hall., D. G., Rakowitz, S., & Gleitman, L. (1994). When it is better to receive than to give: Syntactic and conceptual constraints on vocabulary growth. Lingua, 92, 333–375.

    Google Scholar 

  • Fodor, J. (1983). The modularity of mind. Cambridge, MA: MIT Press.

    Google Scholar 

  • Friederici, A. F., Friedrich, M., & Christophe, A. (2007). Brain responses in 4-month-old infants are already language specific. Current Biology, 17, 1208–1211.

    Article  Google Scholar 

  • Friedrich, M., & Friederici, A. D. (2004). N400-like semantic incongruity effect in 19-month-olds: Processing known words in picture contexts. Journal of Cognitive Neuroscience, 16, 1465–1477.

    Article  Google Scholar 

  • Gleitman, L. (1990). The structural sources of verb meanings. Language Acquisition, 1, 3–55.

    Article  Google Scholar 

  • Gogate, L. J., Bolzani, L., & Betancourt, E. (2006). Attention to maternal multimodal naming by 6- to 8-month-old infants and learning of word–object relations. Infancy, 9, 259–288.

    Article  Google Scholar 

  • Gogate, L. J. (2010). Learning of syllable–object relations by preverbal infants: the role of temporal synchrony and syllable distinctiveness. Journal of Experimental Child Psychology, 105, 178–197.

    Article  Google Scholar 

  • Golinkoff, R., & Hirsh-Pasek, K. (2008). How toddlers learn verbs. Trends in Cognitive Science, 12, 397–403.

    Article  Google Scholar 

  • Höhle, B., Bijeljac-Babic, R., Herold, B., Weissenborn, J., & Nazzi, T. (2009). Language specific prosodic preferences during the first half year of life: Evidence from German and French infants. Infant Behavior and Development, 32, 262–274.

    Article  Google Scholar 

  • Huang, Y. T., & Snedeker, J. (2009a). Semantic meaning and pragmatic interpretation in 5-year-olds: Evidence from real-time spoken language comprehension. Developmental Psychology, 45, 1723–1739.

    Article  Google Scholar 

  • Huang, Y. T., & Snedeker, J. (2009b). Online interpretation of scalar quantifiers: Insight into the semantics–pragmatics interface. Cognitive Psychology, 58, 376–415.

    Article  Google Scholar 

  • Huettig, F., & Altmann, G. T. M. (2011). Looking at anything that is green when hearing “frog”: How object surface colour and stored object colour knowledge influence language-mediated overt attention. The Quarterly Journal of Experimental Psychology, 64, 122–145.

    Article  Google Scholar 

  • Huettig, F., Mishra, R. K., & Olivers, C. N. (2012). Mechanisms and representations of language-mediated visual attention. Frontiers in Psychology, 2, 394. doi:10.3389/fpsyg.2011.00394.

  • Huettig, F., Singh, N., & Mishra, R. K. (2011). Language-mediated visual orienting behavior in low and high literates. Frontiers in Psychology, 2, 285. doi:10.3389/fpsyg.2011.00285.

    Article  Google Scholar 

  • Ito, K., Jincho, N., Minai, U., Yamane, N., & Mazuka, R. (2012). Intonation facilitates contrast resolution: Evidence from Japanese adults and 6-year olds. Journal of Memory and Language, 66, 265–284.

    Article  Google Scholar 

  • Jusczyk, P. W., Houston, D. M., & Newsome, M. (1999). The beginnings of word segmentation in English-learning infants. Cognitive Psychology, 39, 159–207.

    Article  Google Scholar 

  • Jusczyk, P. W., Cutler, A., & Redanz, N. (1993). Preference for the predominant stress patterns of English words. Child Development, 64, 675–687.

    Article  Google Scholar 

  • Just, M. A. & Carptenter, P. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99, 122–149.

    Google Scholar 

  • Kirkham, N. Z., Slemmer, J. A., & Johnson, S. P. (2002). Visual statistical learning in infancy: Evidence of a domain general learning mechanism. Cognition, 83, B35–B42.

    Article  Google Scholar 

  • Kirkham, N. Z., Slemmer, J. A., Richardson, D. C., & Johnson, S. P. (2007). Location, location, location: Development of spatiotemporal sequence learning in infancy. Child Development, 78, 1559–1571.

    Article  Google Scholar 

  • Knoeferle, P., Crocker, M. W., Scheepers, C., & Pickering, M. J. (2005). The influence of the immediate visual context on incremental thematic role-assignment: evidence from eye-movements in depicted events. Cognition, 95, 95–127.

    Article  Google Scholar 

  • Knoeferle, P., & Crocker, M. W. (2006). The coordinated interplay of scene, utterance, and world knowledge: Evidence from eye tracking. Cognitive Science, 30, 481–529.

    Article  Google Scholar 

  • Knoeferle, P., & Crocker, M. W. (2007). The influence of recent scene events on spoken comprehension: Evidence from eye-movements. Journal of Memory and Language, 75, 519–543.

    Article  Google Scholar 

  • Knoeferle, P., Habets, B., Crocker, M. W., & Muente, T. F. (2008). Visual scenes trigger immediate syntactic reanalysis: Evidence from ERPs during situated spoken comprehension. Cerebral Cortex, 18, 789−795.

    Google Scholar 

  • Knoeferle, P., Carminati, M. N., Abashidze, D., & Essig, K. (2011a). Preferential inspection of recent real-world events over future events: Evidence from eye tracking during spoken sentence comprehension. Frontiers in Psychology, 2, 376. doi:10.3389/fpsyg.2011.00376.

    Article  Google Scholar 

  • Knoeferle, P., Urbach, T., & Kutas, M. (2011b). Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification. Psychophysiology, 48, 495–506.

    Article  Google Scholar 

  • Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207, 203–205.

    Article  Google Scholar 

  • Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature, 307, 161–163.

    Article  Google Scholar 

  • Lew-Williams, C., & Fernald, A. (2010). Real-time processing of gender-marked articles by native and non-native Spanish speakers. Journal of Memory and Language, 63, 447–464.

    Article  Google Scholar 

  • Mani, N., & Huettig, F. (2012). Prediction during language processing is a piece of cake—but only for skilled producers. Journal of Experimental Psychology: Human Perception and Performance, 38(4), 843–847.

    Google Scholar 

  • Meltzoff, A. N. (1995). Understanding the intentions of others: Re-enactment of intended acts by 18-month-old children. Developmental Psychology, 31, 838–850.

    Article  Google Scholar 

  • Meroni, L. & Crain, S. (2003). On not being led down the kindergarten path. In Proceedings of the 27th Boston University Conference on language development (pp. 531−544), Somerville, MA: Cascadilla Press.

    Google Scholar 

  • Moeser, S. D., & Olson, J. A. (1974). The role of reference in children’s acquisition of a miniature artificial language. Journal of Experimental Child Psychology, 17, 204–218.

    Article  Google Scholar 

  • Naigles, L. (1990). Children use syntax to learn verb meaning. Journal of Child Language, 17, 357–374.

    Article  Google Scholar 

  • Nation, K., Marshall, C., & Altmann, G. T. M. (2003). Investigating individual differences in children’s real-time sentence comprehension using language-mediated eye movements. Journal of Experimental Child Psychology, 86, 314–329.

    Article  Google Scholar 

  • Newport, E. L., & Aslin, R. (2004). Learning at a distance I: Statistical learning of non-adjacent dependencies. Cognitive Psychology, 48, 270, 127–162.

    Google Scholar 

  • Papafragou, A. & Musonlino, J. (2003). Scalar implicatures: experiments at the semantics–pragmatics interface. Cognition, 86, 253−282.

    Google Scholar 

  • Pinker, S. (1994). How could a child use verb syntax to learn verb semantics. Lingua, 92, 377–410.

    Article  Google Scholar 

  • Pulverman, R., Sootsman, J., Golinkoff, R., & Hirsh-Pasek, K. (2008). Manners matter: Infants’ attention to manner and path in non-linguistic dynamic events. Cognition, 108, 825–830.

    Article  Google Scholar 

  • Saffran, J., Aslin, R., & Newport, R. (1996). Statistical learning by 8-month-old infants. Science, 274, 1926–1928.

    Article  Google Scholar 

  • Sharon, T., & Wynn, K. (1998). Infants’ individuation of actions from continuous motion. Psychological Science, 9, 357–362.

    Article  Google Scholar 

  • Sedivy, J. C., Tanenhaus, M. K., Chambers, C. G., & Carlson, G. N. (1999). Achieving incremental semantic interpretation through contextual representation. Cognition, 71, 109–148.

    Article  Google Scholar 

  • Sekerina, I. A., & Trueswell, J. C. (2012). Interactive processing of contrastive expressions by Russian children. First Language, 32, 63–87.

    Article  Google Scholar 

  • Smith, L., & Yu, C. (2008). Infants rapidly learn word-referent mappings via cross-situational statistics. Cognition, 106, 1558–1568.

    Article  Google Scholar 

  • Snedeker, J., & Trueswell, J. C. (2004). The developing constraints on parsing decisions: The role of lexical-biases and referential scenes in child and adult sentence processing. Cognitive Psychology, 49, 238–299.

    Article  Google Scholar 

  • Snedeker, J., & Yuan, S. (2008). Effects of prosodic and lexical constraints on parsing in young children (and adults). Journal of Memory and Language, 58, 574–608.

    Article  Google Scholar 

  • Spivey, M. J., Tanenhaus, M. K., Eberhard, K. M., & Sedivy, J. C. (2002). Eye-movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution. Cognitive Psychology, 45, 447–481.

    Article  Google Scholar 

  • Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634.

    Article  Google Scholar 

  • Tincoff, R., & Jusczyk, P. (1999). Some beginnings of word comprehension in 6-month-olds. Psychological Science, 10, 172–175.

    Article  Google Scholar 

  • Tincoff, R., & Jusczyk, P. (2012). Six-month-olds comprehend words that refer to parts of the body. Infancy, 17, 432–444.

    Article  Google Scholar 

  • Tomasello, M., & Farrar, J. (1986). Joint attention and early language. Child Development, 57, 1454–1463.

    Article  Google Scholar 

  • Tomasello, M., & Todd, J. (1983). Joint attention and lexical acquisition style. First Language, 4, 197–211. doi:10.1177/014272378300401202.

    Article  Google Scholar 

  • Tomasello, M., & Brooks, P. (1998). Young children’s earliest transitive and intransitive constructions. Cognitive Linguistics, 9, 379–395.

    Article  Google Scholar 

  • Tomasello, M. (2000). Do young children have adult syntactic competence? Cognition, 74, 209–253.

    Article  Google Scholar 

  • Trueswell, J. C., Sekerina, I., Hill, N., & Logrip, M. (1999). The kindergarten-path effect: Studying online sentence processing in young children. Cognition, 73, 89–134.

    Article  Google Scholar 

  • Van Heugthen, M., & Shi, R. (2009). French-learning toddlers use gender information on determiners during word recognition. Developmental Science, 12, 419–425.

    Article  Google Scholar 

  • Weber, A., Grice, M., & Crocker, M. W. (2006). The role of prosody in the interpretation of structural ambiguities: A study of anticipatory eye movements. Cognition, 99, B63–B72.

    Article  Google Scholar 

  • Woodward, A. L. (1998). Infants selectively encode the goal object of an actor’s reach. Cognition, 69, 1−34.

    Google Scholar 

  • Woodward, A., & Sommerville, J. (2000). Twelve-month-old infants interpret actions in context. Psychological Science, 11, 73–77.

    Article  Google Scholar 

  • Wu, R., Gopnik, A., Richardson, D. C., & Kirkham, N. Z. (2011). Infants learn about objects from statistics and people. Developmental Psychology, 47, 1220–1229.

    Article  Google Scholar 

  • Wynn, K. (1996). Infants’ individuation and enumeration of actions. Psychological Science, 7, 164–169.

    Article  Google Scholar 

  • Yu, C., & Smith, L. (2010). What you learn is what you see: Using eye movements to study infant cross-situational word learning. Developmental Science, 14, 165–180.

    Article  Google Scholar 

  • Zhang, L., & Knoeferle, P. (2012). Visual context effects on thematic role assignment in children versus adults: Evidence from eye tracking in German. In Naomi Miyake, David Peebles, & Richard P. Cooper (Eds.), In Proceedings of the annual meeting of the cognitive science society (pp. 2593–2598). Boston, USA: The Cognitive Science Society.

    Google Scholar 

  • Zhang, L., & Knoeferle, P. (in progress). Can depicted events disambiguate local structural ambiguity in German SVO / OVS sentences?

    Google Scholar 

  • Zhang, L., Kornbluth, L., & Knoeferle, P. (2012). The role of recent versus future events in children’s comprehension of referentially ambiguous sentences: Evidence from eye tracking. In N. Miyake, D. Peebles, & R. P. Cooper, In Proceedings of the Annual Meeting of the Cognitive Science Society (pp. 1227−1232), Boston, USA: The Cognitive Science Society.

    Google Scholar 

  • Zhou, P., Crain, S., & Zhan, L. (2014). Grammatical aspect and event recognition in children’s online sentence comprehension. Cognition, 133, 262–276.

    Article  Google Scholar 

  • Zhou, P., Crain, S., & Zhan, L. (2012). Sometimes children are as good as adults: The pragmatic use of prosody in children’s online sentence processing. Journal of Memory and Language, 67, 149–164.

    Article  Google Scholar 

Download references

Acknowledgments

This research was funded by the Cognitive Interaction Technology Excellence Center 277 (CITEC, German Research Council).

Author information

Authors and Affiliations

  1. Research Group “Language and Cognition”, CITEC, Cognitive Interaction Technology Excellence Center, Bielefeld University, Room 2.036, 33615, Bielefeld, Germany

    Pia Knoeferle

Authors
  1. Pia Knoeferle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pia Knoeferle .

Editor information

Editors and Affiliations

  1. Center for Neural and Cognitive Sciences, University of Hyderabad, Hyderabad, Andhra Pradesh, India

    Ramesh Kumar Mishra

  2. Centre of Behavioural and Cognitive Sciences, University of Allahabad, Allahabad, Uttar Pradesh, India

    Narayanan Srinivasan

  3. Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

    Falk Huettig

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer India

About this chapter

Cite this chapter

Knoeferle, P. (2015). Visually Situated Language Comprehension in Children and in Adults. In: Mishra, R., Srinivasan, N., Huettig, F. (eds) Attention and Vision in Language Processing. Springer, New Delhi. https://doi.org/10.1007/978-81-322-2443-3_4

Download citation

Publish with us

Policies and ethics

Search

Navigation