Effect of pitch–space correspondence on sound-induced visual motion perception
- 554 Downloads
The brain tends to associate specific features of stimuli across sensory modalities. The pitch of a sound is for example associated with spatial elevation such that higher-pitched sounds are felt as being “up” in space and lower-pitched sounds as being “down.” Here we investigated whether changes in the pitch of sounds could be effective for visual motion perception similar to those in the location of sounds. We demonstrated that only sounds that alternate in up/down location induced illusory vertical motion of a static visual stimulus, while sounds that alternate in higher/lower pitch did not induce this illusion. The pitch of a sound did not even modulate the visual motion perception induced by sounds alternating in up/down location. Interestingly, though, sounds alternating in higher/lower pitch could become a driver for visual motion if they were paired in a previous exposure phase with vertical visual apparent motion. Thus, only after prolonged exposure, the pitch of a sound became an inducer for upper/lower visual motion. This occurred even if during exposure the pitch and location of the sounds were paired in an incongruent fashion. These findings indicate that pitch–space correspondence is not so strong to drive or modulate visual motion perception. However, associative exposure could increase the saliency of pitch–space relationships and then the pitch could induce visual motion perception by itself.
KeywordsCrossmodal correspondence Multisensory perception Auditory space Pitch Visual motion perception
We thank Wouter D.H. Stumpel for his technical supports. We are grateful to anonymous reviewers for their valuable and insightful comments and suggestions for early versions of the manuscript. This research was supported by the Ministry of Education, Culture, Sports, Science and Technology, Grant-in-Aid for Specially Promoted Research (No. 19001004) and Rikkyo University Special Fund for Research.
- Calvert GA, Spence C, Stein BE (eds) (2004) The handbook of multisensory processing. MIT Press, CambridgeGoogle Scholar
- Dolscheid S, Shayan S, Majid A, Casasanto D. (2011) The thickness of musical pitch: psychophysical evidence for the Whorfian hypothesis. In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society, pp 537–542Google Scholar
- Ernst MO (2005) A Bayesian view on multimodal cue integration. In: Knoblich G, Thornton I, Grosjean M, Shiffrar M (eds) Perception of the human body perception from the inside out. Oxford University Press, New York, pp 105–131Google Scholar
- Evans KK, Treisman A (2010) Natural cross-modal mappings between visual and auditory features. J Vis 10(1):6: 1–12Google Scholar
- Macmillan NA, Creelman CD (2004) Detection theory: a user’s guide, 2nd edn. Lawrence Erlbaum Associates Inc, New JerseyGoogle Scholar
- Marks LE (2004) Cross-modal interactions in speeded classification. In: Calvert GA, Spence C, Stein BE (eds) Handbook of multisensory processes. MIT Press, Cambridge, pp 85–105Google Scholar