Perceptual biases for multimodal cues in chimpanzee (Pan troglodytes) affect recognition


The ability of organisms to discriminate social signals, such as affective displays, using different sensory modalities is important for social communication. However, a major problem for understanding the evolution and integration of multimodal signals is determining how humans and animals attend to different sensory modalities, and these different modalities contribute to the perception and categorization of social signals. Using a matching-to-sample procedure, chimpanzees discriminated videos of conspecifics’ facial expressions that contained only auditory or only visual cues by selecting one of two facial expression photographs that matched the expression category represented by the sample. Other videos were edited to contain incongruent sensory cues, i.e., visual features of one expression but auditory features of another. In these cases, subjects were free to select the expression that matched either the auditory or visual modality, whichever was more salient for that expression type. Results showed that chimpanzees were able to discriminate facial expressions using only auditory or visual cues, and when these modalities were mixed. However, in these latter trials, depending on the expression category, clear preferences for either the visual or auditory modality emerged. Pant-hoots and play faces were discriminated preferentially using the auditory modality, while screams were discriminated preferentially using the visual modality. Therefore, depending on the type of expressive display, the auditory and visual modalities were differentially salient in ways that appear consistent with the ethological importance of that display’s social function.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2 a


  1. 1.

    Note that the play faces used in this study were always accompanied by laughter vocalizations, although this is not always the case in naturally occurring behavior.


  1. Bauer HR, Philip M (1983) Facial and vocal individual recognition in the common chimpanzee. Psychol Rec 33:161–170

  2. Caron AJ, Caron RF, MacLean DJ (1988) Infant discrimination of naturalistic emotional expressions: the role of face and voice. Child Dev 59:604–616

  3. Ekman P, Friesen WV, Ellsworth P (1972) Emotion in the human face: guidelines for research and an integration of findings. Pergamon, New York

  4. Ghazanfar AA, Logothetis NK (2003) Facial expressions linked to monkey calls. Nature 423:937–938

  5. Gouzoules H, Gouzoules S (2000) Agonistic screams differ among four species of macaques: the significance of motivation-structural rules. Anim Behav 59:501–515

  6. Gouzoules S, Gouzoules H, Marler P (1984) Rhesus monkey (Macaca mulatta) screams: representational signalling in the recruitment of agonistic aid. Anim Behav 32:182–193

  7. Hashiya K (1999) Auditory-visual inter-modal recognition of conspecifics by a chimpanzee (Pan troglodytes). Primate Res 15:333–342

  8. Hashiya K, Kojima S (2001) Acquisition of auditory-visual inter-modal matching-to-sample by a chimpanzee (Pan troglodytes): Comparison with visual-visual intra-modal matching. Anim Cogn 4:231–239

  9. Hooff JARAM van (1967) The facial displays of the Catarrhine monkeys and apes. In: Morris D (ed) Primate ethology. Aldine, Chicago, pp 7–68

  10. Kuhl PK, Meltzoff AN (1982) The bi-modal perception of speech in infancy. Science 218:1138–1141

  11. McGurk H, McDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748

  12. Meltzoff AN (1999) Origins of theory of mind, cognition and communication. J Commun Disord 32:251–269

  13. Mitani JC, Gros-Louis J, Macedonia JM (1996) Selection for acoustic individuality within the vocal repertoire of wild chimpanzees. Int J Primatol 17:569–581

  14. Morton ES (1982) Grading, discreteness, redundancy, and motivation-structural rules. In: Kroodsma DE, Miller EH, Ouellet H (eds) Acoustic communication in birds. Academic Press, New York, pp 183–212

  15. Parr LA (2001) Cognitive and physiological markers of emotional awareness in chimpanzees. Anim Cogn 4:223–229

  16. Parr L, Maestripieri D (2003) Nonvocal communication in nonhuman primates. In: Maestripieri D (ed) Primate psychology: the mind and behavior of human and nonhuman primates. Chicago University Press, Chicago, pp 324–358

  17. Parr LA, Hopkins WD, de Waal FBM (1998) The perception of facial expressions in chimpanzees (Pan troglodytes). Evol Commun 2:1–23

  18. Parr LA, Winslow JT, Hopkins WD, de Waal FBM (2000) Recognizing facial cues: individual recognition in chimpanzees (Pan troglodytes) and rhesus monkeys (Macaca mulatta). J Comp Psychol 114:47–60

  19. Parr LA, Preuschoft S, de Waal FBM (2002) Afterword: research on facial emotion in chimpanzees, 75 years since Kohts. In: de Waal FBM (ed) Infant chimpanzee and human child. Oxford University Press, New York, pp 411–452

  20. Partan S, Marler P (1999) Communication goes multimodal. Science 283:1272–1273

  21. Rowe C (1999) Receiver psychology and the evolution of multicomponent signals. Anim Behav 58:921–931

  22. Rowe C (2001) Sound improves visual discrimination learning in avian predators. Proc R Soc Lond B 269:1353–1357

  23. Siebert E, Parr LA (2002) Structural and contextual analysis of chimpanzee screams. Poster presented at the New York Academy of Sciences, Emotions Inside Out: 130 years after Darwin’s the Expression of the Emotions in Man and Animals, New York, N.Y., 16 November

  24. Soken NH, Pick AD (1992) Intermodal perception of happy and angry expressive behaviors by seven-month-old infants. Child Dev 63:787–795

  25. Vick SJ, Waller B, Smith-Pasqualini M, Parr LA, Bard KA (2003) The chimpanzee facial affect coding system (FACS): preliminary findings. 10th European conference on facial expression, measurement and meaning, Rimini, Italy, 18–20 September

  26. Walker AS (1982) Intermodal perception of expressive behaviors by human infants. J Exp Child Psychol 33:514–535

  27. Walker-Andrews AS (1986) Intermodal perception of expressive behaviors: relation of eye and voice? Dev Psychol 22:373–377

  28. Walker-Andrews AS (1997) Infants’ perception of expressive behaviors: differentiation of multimodal information. Psychol Bull 121:437–456

Download references


This investigation was supported by RR-00165 from the NIH/NCRR to the Yerkes Regional Primate Research Center. The Yerkes Primate Center is fully accredited by the American Association for Accreditation of Laboratory Animal Care. Thanks to the Living Links Center, Emory University, for the use of photographic material, and the animal care staff at the Yerkes Primate Center. Todd Preuss, Stuart Zola, and three anonymous reviewers provided helpful comments on earlier versions of this manuscript.

Author information

Correspondence to Lisa A. Parr.

Electronic Supplementary Material

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Parr, L.A. Perceptual biases for multimodal cues in chimpanzee (Pan troglodytes) affect recognition. Anim Cogn 7, 171–178 (2004) doi:10.1007/s10071-004-0207-1

Download citation


  • Communication
  • Multimodal signals
  • Facial expressions
  • Chimpanzee