Perception of Conflicting Audio-Visual Speech: an Examination across Spanish and German

Abstract

A corpus of discrepant audio-visual (AV) German and Spanish stimuli, including CV-syllables, CCV-syllables and real words, was created in order to explore possible interlanguage differences in the magnitude and type of illusions reported in bimodal speech perception. For this purpose, audio-only (A), visual-only (V), and AV tests were run under four different conditions: both German and Spanish perceivers were tested with both a German and a Spanish talker. Subjects were asked to report what they perceived in an open choice test. Results showed that both German and Spanish perceivers were subject to visual influence in the bimodal perception of speech. However, differences were found in the response patterns across the four conditions. They can be partly explained by differences in the phonetic systems and in the phonotactic constraints of the two languages. Finally, the magnitude of visual influence with real words was higher in the nonnative language.