Advertisement

Semantic processing of adjectives and nouns in American Sign Language: effects of reference ambiguity and word order across development

  • Anne WienholzEmail author
  • Amy M. Lieberman
Research Paper
  • 4 Downloads

Abstract

When processing spoken language sentences, listeners continuously make and revise predictions about the upcoming linguistic signal. In contrast, during comprehension of American Sign Language (ASL), signers must simultaneously attend to the unfolding linguistic signal and the surrounding scene via the visual modality. This may affect how signers activate potential lexical candidates and allocate visual attention as a sentence unfolds. To determine how signers resolve referential ambiguity during real-time comprehension of ASL adjectives and nouns, we presented deaf adults (n = 18, 19–61 years) and deaf children (n = 20, 4–8 years) with videos of ASL sentences in a visual world paradigm. Sentences had either an adjective-noun (e.g., “SEE YELLOW WHAT? FLOWER”) or a noun-adjective (e.g., “SEE FLOWER WHICH? YELLOW”) structure. The degree of ambiguity in the visual scene was manipulated at the adjective and noun levels (e.g., including one or more yellow items and one or more flowers in the visual array). We investigated effects of ambiguity and word order on target looking at early and late points in the sentence. Analysis revealed that adults and children made anticipatory looks to a target when it could be identified early in the sentence. Further, signers looked more to potential lexical candidates than to unrelated competitors in the early window, and more to matched than unrelated competitors in the late window. Children’s gaze patterns largely aligned with those of adults, although they made fewer anticipatory fixations to the target in the early window and were more susceptible to competitors in the late window. Together, these findings suggest that signers allocate referential attention strategically based on the amount and type of ambiguity at different points in the sentence when processing adjectives and nouns in ASL.

Keywords

American Sign Language Eye tracking Semantic processing Visual world Deaf 

Notes

Acknowledgements

This work was funded by the National Institute on Deafness and Communication Disorders (NIDCD), grant number R01DC015272. We thank Rachel Mayberry and Arielle Borovsky for valuable input, and Marla Hatrak, Michael Higgins, and Valerie Sharer for help with data collection. We are grateful to all of the individuals who participated in this study.

Compliance with ethical standards

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Supplementary material

41809_2019_24_MOESM1_ESM.xlsx (20 kb)
Supplementary material 1 (XLSX 19 kb)

References

  1. Altmann, G. T., & Kamide, Y. (2007). The real-time mediation of visual attention by language and world knowledge: Linking anticipatory (and other) eye movements to linguistic processing. Journal of Memory and Language, 57(4), 502–518.  https://doi.org/10.1016/j.jml.2006.12.004.Google Scholar
  2. Anderson, D., & Reilly, J. (2002). The MacArthur communicative development inventory: normative data for American Sign Language. Journal of Deaf Studies and Deaf Education, 7(2), 83–106.  https://doi.org/10.1093/deafed/7.2.83.Google Scholar
  3. Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412.  https://doi.org/10.1016/j.jml.2007.12.005.Google Scholar
  4. Barr, D. J. (2008). Analyzing ‘visual world’eyetracking data using multilevel logistic regression. Journal of Memory and Language, 59(4), 457–474.  https://doi.org/10.1016/j.jml.2007.09.002.Google Scholar
  5. Bavelier, D., Dye, M. W., & Hauser, P. C. (2006). Do deaf individuals see better? Trends in Cognitive Sciences, 10(11), 512–518.  https://doi.org/10.1016/j.tics.2006.09.006.Google Scholar
  6. Borovsky, A., Ellis, E. M., Evans, J. L., & Elman, J. L. (2016). Lexical leverage: Category knowledge boosts real-time novel word recognition in 2-year-olds. Developmental Science, 19(6), 918–932.  https://doi.org/10.1111/desc.12343.Google Scholar
  7. Borovsky, A., Elman, J. L., & Fernald, A. (2012). Knowing a lot for one’s age: Vocabulary skill and not age is associated with anticipatory incremental sentence interpretation in children and adults. Journal of Experimental Child Psychology, 112(4), 417–436.  https://doi.org/10.1016/j.jecp.2012.01.005.Google Scholar
  8. Borovsky, A., Sweeney, K., Elman, J. L., & Fernald, A. (2014). Real-time interpretation of novel events across childhood. Journal of Memory and Language, 73, 1–14.  https://doi.org/10.1016/j.jml.2014.02.001.Google Scholar
  9. Chen Pichler, D. (2011). Using early ASL word order to shed light on word order variability in sign language. In M. Anderssen, K. Bentzen, & M. Westergaard (Eds.), Variation in the input (pp. 157–177). Dordrecht: Springer.Google Scholar
  10. Coates, J., & Sutton-Spence, R. (2001). Turn-taking patterns in deaf conversation. Journal of Sociolinguistics, 5(4), 507–529.  https://doi.org/10.1111/1467-9481.00162.Google Scholar
  11. Eberhard, K. M., Spivey-Knowlton, M. J., Sedivy, J. C., & Tanenhaus, M. K. (1995). Eye movements as a window into real-time spoken language comprehension in natural contexts. Journal of Psycholinguistic Research, 24(6), 409–436.  https://doi.org/10.1007/bf02143160.Google Scholar
  12. Fernald, A., Thorpe, K., & Marchman, V. A. (2010). Blue car, red car: Developing efficiency in online interpretation of adjective–noun phrases. Cognitive Psychology, 60(3), 190–217.Google Scholar
  13. Gasser, M., & Smith, L. B. (1998). Learning nouns and adjectives: A connectionist account. Language and Cognitive Processes, 13(2–3), 269–306.  https://doi.org/10.1080/016909698386537.Google Scholar
  14. Hanna, J. E., Tanenhaus, M. K., & Trueswell, J. C. (2003). The effects of common ground and perspective on domains of referential interpretation. Journal of Memory and Language, 49(1), 43–61.  https://doi.org/10.1016/s0749-596x(03)00022-6.Google Scholar
  15. Harris, M., Clibbens, J., Chasin, J., & Tibbitts, R. (1989). The social context of early sign language development. First Language, 9(25), 81–97.  https://doi.org/10.1177/014272378900902507.Google Scholar
  16. Huang, Y. T., & Snedeker, J. (2013). The use of lexical and referential cues in children’s online interpretation of adjectives. Developmental Psychology, 49(6), 1090–1102.  https://doi.org/10.1037/a0029477.Google Scholar
  17. Hurewitz, F., Brown-Schmidt, S., Thorpe, K., Gleitman, L. R., & Trueswell, J. C. (2000). One frog, two frog, red frog, blue frog: factors affecting children’s syntactic choices in production and comprehension. Journal of Psycholinguistic Research, 29(6), 597–626.  https://doi.org/10.1023/A:1026468209238.Google Scholar
  18. Kamide, Y., Altmann, G. T. M., & Haywood, S. L. (2003). The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language, 49(1), 133–156.  https://doi.org/10.1016/S0749-596X(03)00023-8.Google Scholar
  19. Kimmelman, V. (2012). Word order in Russian sign language. Sign Language Studies, 12(3), 414–445.  https://doi.org/10.1353/sls.2012.0001.Google Scholar
  20. Knoeferle, P., Carminati, M. N., Abashidze, D., & Essig, K. (2011). Preferential inspection of recent real-world events over future events: evidence from eye tracking during spoken sentence comprehension. Frontiers in Psychology, 2, 376.  https://doi.org/10.3389/fpsyg.2011.00376.Google Scholar
  21. Knoeferle, P., & Crocker, M. W. (2009). Constituent order and semantic parallelism in online comprehension: Eye-tracking evidence from German. The Quarterly Journal of Experimental Psychology, 62(12), 2338–2371.  https://doi.org/10.1080/17470210902790070.Google Scholar
  22. Lieberman, A. M., Borovsky, A., Hatrak, M., & Mayberry, R. I. (2015). Real-time processing of ASL signs: delayed first language acquisition affects organization of the mental Lexicon. Journal of Experimental Psychology. Learning, Memory, and Cognition, 41(4), 1130–1139.  https://doi.org/10.1037/xlm0000088.Google Scholar
  23. Lieberman, A. M., Borovsky, A., & Mayberry, R. I. (2018). Prediction in a visual language: Real-time sentence processing in American Sign Language across development. Language, Cognition and Neuroscience, 33(4), 387–401.  https://doi.org/10.1080/23273798.2017.1411961.Google Scholar
  24. Lieberman, A. M., Hatrak, M., & Mayberry, R. I. (2014). Learning to look for language: Development of joint attention in young deaf children. Language Learning and Development, 10(1), 19–35.  https://doi.org/10.1080/15475441.2012.760381.Google Scholar
  25. MacDonald, K., LaMarr, T., Corina, D., Marchman, V. A., & Fernald, A. (2018). Real-time lexical comprehension in young children learning American Sign Language. Developmental science, 21(6), e12672.  https://doi.org/10.31234/osf.io/zht6g.Google Scholar
  26. Mani, N., & Huettig, F. (2012). Prediction during language processing is a piece of cake—But only for skilled producers. Journal of Experimental Psychology: Human Perception and Performance, 38(4), 843–847.  https://doi.org/10.1037/a0029284.Google Scholar
  27. Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG-and MEG-data. Journal of Neuroscience Methods, 164(1), 177–190.  https://doi.org/10.1016/j.jneumeth.2007.03.024.Google Scholar
  28. Mayberry, R. I., & Squires, B. (2006). Sign language: acquisition. In K. Brown (Ed.), Encyclopedia of language and linguistics (pp. 291–296). Boston: Elsevier.Google Scholar
  29. Melkman, R., Tversky, B., & Baratz, D. (1981). Developmental trends in the use of perceptual and conceptual attributes in grouping, clustering, and retrieval. Journal of Experimental Child Psychology, 31(3), 470–486.  https://doi.org/10.1016/0022-0965(81)90031-x.Google Scholar
  30. Nation, K., Marshall, C. M., & Altmann, G. T. (2003). Investigating individual differences in children’s real-time sentence comprehension using language-mediated eye movements. Journal of Experimental Child Psychology, 86(4), 314–329.  https://doi.org/10.1016/j.jecp.2003.09.001.Google Scholar
  31. Neidle, C., & Nash, J. (2012). The noun phrase. In R. Pfau, M. Steinbach, & B. Woll (Eds.), Sign language: An international handbook (pp. 265–292). Berlin, Boston: De Gruyter Mouton.Google Scholar
  32. Neville, H. J., & Lawson, D. (1987). Attention to central and peripheral visual space in a movement detection task. III. Separate effects of auditory deprivation and acquisition of a visual language. Brain Research, 405(2), 284–294.  https://doi.org/10.1016/0006-8993(87)90297-6.Google Scholar
  33. Ninio, A. (2004). Young children’s difficulty with adjectives modifying nouns. Journal of Child Language, 31(2), 255–285.  https://doi.org/10.1017/S0305000904006191.Google Scholar
  34. Nordmeyer, A. E., & Frank, M. C. (2014). The role of context in young children’s comprehension of negation. Journal of Memory and Language, 77, 25–39.  https://doi.org/10.1016/j.jml.2014.08.002.Google Scholar
  35. Odom, R. D., & Guzman, R. D. (1972). Development of hierarchies of dimensional salience. Developmental Psychology, 6(2), 271–287.  https://doi.org/10.1037/h0032096.Google Scholar
  36. Pavani, F., & Bottari, D. (2012). Visual abilities in individuals with profound deafness a critical review. In M. M. Murray & M. T. Wallace (Eds.), The neural bases of multisensory processes (pp. 423–448). Boca Raton: CRC Press/Taylor & Francis.Google Scholar
  37. Rubio-Fernandez, P., Mollica, F., & Jara-Ettinger, J. (2018). English and Spanish speakers exploit language structure to increase communicative efficiencyhttps://psyarxiv.com/gf8qx.
  38. Rubio-Fernández, P., Wienholz, A., Kirby, S., & Lieberman, A. M. (2019). ASL signers vary adjective position to maximize efficiency: A reference production study. Poster presented at the 32nd Annual CUNY Conference on Human Sentence Processing, Boulder, CO.Google Scholar
  39. Sandler, W., & Lillo-Martin, D. (2006). Sign language and linguistic universals. Cambrigde: Cambridge University Press.Google Scholar
  40. Sedivy, J. C. (2003). Pragmatic versus form-based accounts of referential contrast: Evidence for effects of informativity expectations. Journal of Psycholinguistic Research, 32(1), 3–23.  https://doi.org/10.1023/A:1021928914454.Google Scholar
  41. Sedivy, J. C., Tanenhaus, M. K., Chambers, C. G., & Carlson, G. N. (1999). Achieving incremental semantic interpretation through contextual representation. Cognition, 71(2), 109–147.  https://doi.org/10.1016/s0010-0277(99)00025-6.Google Scholar
  42. Snedeker, J., & Trueswell, J. C. (2004). The developing constraints on parsing decisions: The role of lexical-biases and referential scenes in child and adult sentence processing. Cognitive Psychology, 49(3), 238–299.  https://doi.org/10.1016/j.cogpsych.2004.03.001.Google Scholar
  43. Spivey, M. J., Tanenhaus, M. K., Eberhard, K. M., & Sedivy, J. C. (2002). Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution. Cognitive Psychology, 45(4), 447–481.  https://doi.org/10.1016/s0010-0285(02)00503-0.Google Scholar
  44. Spivey, M. J., Tyler, M. J., Eberhard, K. M., & Tanenhaus, M. K. (2001). Linguistically mediated visual search. Psychological Science, 12(4), 282–286.  https://doi.org/10.1111/1467-9280.00352.Google Scholar
  45. Sutton-Spence, R., & Woll, B. (1999). The linguistics of British Sign Language: An introduction. Cambrigde: Cambridge University Press.Google Scholar
  46. Swisher, M. V. (2000). Learning to converse: How deaf mothers support the development of attention and conversational skills in their young deaf children. In P. Spencer, C. J. Erting, & M. Marschark (Eds.), The deaf child in the family and at school (pp. 21–40). Mahwah: Erlbaum.Google Scholar
  47. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632–1634.  https://doi.org/10.1126/science.7777863.Google Scholar
  48. Tribushinina, E., & Mak, W. M. (2016). Three-year-olds can predict a noun based on an attributive adjective: evidence from eye-tracking. Journal of Child Language, 43(2), 425–441.  https://doi.org/10.1017/s0305000915000173.Google Scholar
  49. Trueswell, J. C., Sekerina, I., Hill, N. M., & Logrip, M. L. (1999). The kindergarten-path effect: studying on-line sentence processing in young children. Cognition, 73(2), 89–134.  https://doi.org/10.1016/S0010-0277(99)00032-3.Google Scholar
  50. Waxman, R. P., & Spencer, P. E. (1997). What mothers do to support infant visual attention: sensitivities to age and hearing status. Journal of Deaf Studies and Deaf Education, 2(2), 104–114.  https://doi.org/10.1093/oxfordjournals.deafed.a014311.Google Scholar
  51. Weisleder, A., & Fernald, A. (2009). Real-time processing of postnominal adjectives by Latino children learning Spanish as a first language. In Proceedings of the 24th Annual Boston University Conference on Language Development (pp. 611–621).Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Wheelock College of Education and Human DevelopmentBoston UniversityBostonUSA

Personalised recommendations