Experimental Brain Research

, Volume 237, Issue 12, pp 3089–3098 | Cite as

Temporal sequence discrimination within and across senses: do we really hear what we see?

  • Daniel BratzkeEmail author
  • Rolf Ulrich
Research Article


Previous evidence suggests that people “hear” visual stimuli when encoding temporal information. This suggestion is based on the observation that auditory distractor information can strongly affect discrimination performance for visual temporal sequences. The present study aimed to replicate and extend this finding by investigating sequence discrimination within and across the two modalities. In two experimental series, participants judged whether two subsequently presented temporal sequences, a standard sequence followed by a comparison sequence, were identical or not. In Experimental Series A, irrelevant distractor information was presented simultaneously with the standard sequence. In Series B, the distraction appeared in the retention interval between the standard sequence and the comparison sequence. The results showed that auditory distraction impaired performance irrespective of whether the target sequences were auditory or visual, whereas visual distraction only impaired the discrimination of visual target sequences. Furthermore, auditory distraction was always at least as effective as visual distraction, irrespective of standard modality. Generally, discrimination performance was much better for auditory than for visual sequences. Overall, the present results are consistent with the idea that people code visual temporal information in the auditory modality. Moreover, the present study also suggests that such cross-modal interference effects should be interpreted cautiously with respect to their underlying timing mechanism because of the basic differences in temporal sensitivity between the two modalities.


Cross-modal timing Rhythm discrimination Cross-modal encoding Time perception 



We thank Karolin Schuh for her assistance with data collection. This work was supported by a grant from the Deutsche Forschungsgemeinschaft (UL 116/14-1).


  1. Bausenhart KM, de la Rosa MD, Ulrich R (2014) Multimodal integration of time: visual and auditory contributions to perceived duration and sensitivity. Exp Psychol 61:310–322CrossRefGoogle Scholar
  2. Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433–436CrossRefGoogle Scholar
  3. Bratzke D, Seifried T, Ulrich R (2012) Perceptual learning in temporal discrimination: asymmetric cross-modal transfer from audition to vision. Exp Brain Res 221:205–210CrossRefGoogle Scholar
  4. Bratzke D, Schröter H, Ulrich R (2014) The role of consolidation for perceptual learning in temporal discrimination within and across modalities. Acta Psychol 147:75–79CrossRefGoogle Scholar
  5. Bratzke D, Quinn KR, Ulrich R, Bausenhart KM (2016) Representations of temporal information in short-term memory: are they modality-specific? Acta Psychol 170:163–167CrossRefGoogle Scholar
  6. Burr D, Banks MS, Morrone MC (2009) Auditory dominance over vision in the perception of interval duration. Exp Brain Res 198:49–57CrossRefGoogle Scholar
  7. Chen L, Zhou X (2014) Fast transfer of crossmodal time interval training. Exp Brain Res 232:1855–1864CrossRefGoogle Scholar
  8. Coltheart M (1980) Iconic memory and visible persistence. Percept Psychophys 27:183–228CrossRefGoogle Scholar
  9. Cowan N, Aaronson D, Glanzer M, Balota D, Clifton R, Pollatsek A, Crowder R (1984) On short and long auditory stores. Psychol Bull 96:341–370CrossRefGoogle Scholar
  10. Fendrich R, Corballis PM (2001) The temporal cross-capture of audition and vision. Percept Psychophys 63:719–725CrossRefGoogle Scholar
  11. Glenberg AM, Mann S, Altman L, Forman T, Procise S (1989) Modality effects in the coding and reproduction of rhythms. Mem Cogn 17:373–383CrossRefGoogle Scholar
  12. Grahn JA (2012) See what I hear? Beat perception in auditory and visual rhythms. Exp Brain Res 220:51–61CrossRefGoogle Scholar
  13. Grondin S (1993) Duration discrimination of empty and filled intervals marked by auditory and visual signals. Percept Psychophys 54:383–394CrossRefGoogle Scholar
  14. Grondin S, Ulrich R (2011) Duration discrimination performance: no cross-modal transfer from audition to vision even after massive perceptual learning. In: Vatakis A, Esposito E, Giagkou M, Cummins F, Papadelis G (eds) Multidisciplinary aspects of time and time perception, vol 6789. LNAI. Springer, Berlin, pp 92–100CrossRefGoogle Scholar
  15. Guttman SE, Gilroy LA, Blake R (2005) Hearing what the eyes see: auditory encoding of visual temporal sequences. Psychol Sci 16:228–235CrossRefGoogle Scholar
  16. Kanai R, Lloyd H, Bueti D, Walsh V (2011) Modality-independent role of the primary auditory cortex in time estimation. Exp Brain Res 209:465–471CrossRefGoogle Scholar
  17. Kleiner M, Brainard DH, Pelli D (2007) What’s new in psychtoolbox-3? Perception, 36 ECVP Abstract SupplementGoogle Scholar
  18. Lapid E, Ulrich R, Rammsayer T (2009) Perceptual learning in auditory temporal discrimination: no evidence for a cross-modal transfer to the visual modality. Psychon Bull Rev 16:382–389CrossRefGoogle Scholar
  19. McAuley JD, Henry MJ (2010) Modality effects in rhythm processing: auditory encoding of visual rhythms is neither obligatory nor automatic. Atten Percept Perform 72:1377–1389CrossRefGoogle Scholar
  20. Morey R (2008) Confidence intervals from normalized data: a correction to Cousineau (2005). Tutor Quant Methods Psychol 4:61–64CrossRefGoogle Scholar
  21. Ortega L, Guzman-Martinez E, Grabowecky M, Suzuki S (2014) Audition dominates vision in duration perception irrespective of salience, attention, and temporal discriminability. Atten Percept Psychophys 76:1485–1502CrossRefGoogle Scholar
  22. Pelli D (1997) The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat Vis 4:437–442CrossRefGoogle Scholar
  23. Rattat AC, Picard D (2012) Short-term memory for auditory and visual durations: evidence for selective interference effects. Psychol Res 76:32–40CrossRefGoogle Scholar
  24. Repp BH, Penel A (2002) Auditory dominance in temporal processing: new evidence from synchronization with simultaneous visual and auditory sequences. J Exp Psychol Hum Percept Perform 28:1085–1099CrossRefGoogle Scholar
  25. Ulrich R, Nitschke J, Rammsayer T (2006) Crossmodal temporal discrimination: assessing the predictions of a general pacemaker-counter model. Percept Psychophys 68:1140–1152CrossRefGoogle Scholar
  26. Welch RB, Warren DH (1980) Immediate perceptual response to intersensory discrepancy. Psychol Bull 88:638–667CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of TübingenTübingenGermany

Personalised recommendations