Research into the capacity of audiovisual integration has previously assessed whether capacity is strictly limited to a single item, or whether it can exceed one item under certain environmental conditions. More recently, investigations have turned to examining the effects of various stimulus factors on capacity. Across two experiments, we looked at a number of factors that were expected to play a modulatory role on capacity. Experiment 1 deployed a manipulation of illusory polygons, revealing an increase in audiovisual capacity, even in an absence of visual connections. This demonstrates that exceeding the capacity of 1 does not only represent a functional increase in the binding of a singular, complex visual object, but that it can also represent binding of multiple simpler objects. Findings also support the hypothesis that capacity modulates quantitatively, but not qualitatively, with respect to speed of presentation. Experiment 2 examined the effects of different sound types (sine tones or white noise) and of different spatial visual field sizes on the capacity of audiovisual integration. The results indicate that capacity is maximized when stimuli are presented in a smaller circle (7.5°) if alongside a sine tone, and when presented in a larger circle (18.5°) alongside white noise. These results suggest that audiovisual integration capacity is dependent on the combination of sound type and visual spatial field size. The combination of these results reveal additional phenomenological features of the capacity of audiovisual integration, and provides impetus for further research into applications of the findings.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
The pattern of results using the full data set (N = 42) was similar to that of the trimmed data set (N = 26), with a significant main effect of SOA (p < .001, ηp2 = .617), along with a significant polygon × SOA interaction (p = .001, ηp2 = .098). One difference between the full data and trimmed data set is that the main effect of polygon crossed the standard threshold of significance (p = .064, ηp2 = .087), although the pattern of results did not change.
The data as a function of SOA only, along with the linear trend analysis, were reported in Wilbiks and Dyson (2018) in order to satisfy a reviewer’s query. However, the remainder of the data analysis in Experiment 1, and the entirety of Experiment 2 have not been reported elsewhere.
We thank a reviewer for this recommendation for further study.
The pattern of results using the full data set (N = 35) was similar to that of the trimmed data set (N = 26), with a significant main effect of SOA (p < .001, ηp2 = .622), along with a significant sound type × circle diameter interaction (p = .049, ηp2 = .131).
Alvarez, G. A., Cavanagh, P. (2004). The capacity of visual short-term memory is set both by visual information load and by number of objects. Psychological Science, 15, 2. doi:https://doi.org/10.1111/j.0963-7214.2004.01502006.x
Awh, E., Barton, B., & Vogel, E. K. (2007). Visual working memory represents a fixed number of items regardless of complexity. Psychological Science, 18, 7. doi:https://doi.org/10.1111/j.1467-9280.2007.01949.x
Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., & Clubley, E. (2001). The autism-spectrum quotient (AQ) : Evidence from asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31(1), 5-17.
Baum, S. H., Stevenson, R. A., & Wallace, M. T. (2015). Behavioral, perceptual, and neural alterations in sensory and multisensory function in autism spectrum disorder. Progress in Neurobiology, 134, 140-160.
Bertelson, P., Pavani, F., Ladavas, E., Vroomen, J., & de Gelder, B. (2000). Ventriloquism in patients with unilateral visual neglect. Neuropsychologia, 38(12), 1634-1642. doi:https://doi.org/10.1016/s0028-3932(00)00067-1
Botta, F., Lupianez, J., & Sanabria, D. (2013). Visual unimodal grouping mediates auditory attentional bias in visuo-spatial working memory. Acta Psychologica, 144, 104-111. doi: https://doi.org/10.1016/j.actapsy.2013.05.010
Botta, F., Santangelo, V., Raffone, A., Lupianez, J., & Belardinelli, M. O. (2010). Exogenous and endogenous spatial attention effects on visuospatial working memory. The Quarterly Journal of Experimental Psychology, 63(8), 1590-1602. doi: https://doi.org/10.1080/17470210903443836
Botta, F., Santangelo, V., Raffone, A., Sanabria, D., Lupianez, J., & Belardinelli, M. O. (2011). Multisensory integration affects visuo-spatial working memory. Journal of Experimental Psychology: Human Perception and Performance, 37(4), 1099-1109. doi: https://doi.org/10.1037/a0023513
Castiello, U., & Umilta, C. (1989). Size of the attentional focus and efficiency of processing. Acta Psychologica, 73(3), 195-209. doi:https://doi.org/10.1016/00016918(90)90022-8
Cowan, N. (2001). The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behavioral Brain Science, 24, 87-114. doi:https://doi.org/10.1017/S0140525X01003922
Cowan, N. (2010). The magical mystery four: How is working memory capacity limited, and why? Current Directions in Psychological Science, 19(1), 51-57.
Deruelle, C., Rondan, C., Gepner, B., & Fagot, J. (2006). Processing of compound stimuli by children with autism and Asperger syndrome. International Journal of Psychology, 41(2), 97-106.
Eng, H. Y., Chen, D., & Jiang, Y. (2005). Visual working memory for simple and complex visual stimuli. Psychonomic Bulletin & Review, 12, 1127-1133. doi:https://doi.org/10.3758/BF03206454
Fernell, E., Eriksson, M. A., & Gillberg, C. (2013). Early diagnosis of autism and impact on prognosis: a narrative review. Clinical Epidemiology, 5, 33-43.
Fujisaki, W., Koene, A., Arnold, D., Johnston, A., & Nishida, S. (2006). Visual search for a target changing in synchrony with an auditory signal. Proceedings of the Royal Society B: Biological Sciences, 273(1588), 865-874. doi:https://doi.org/10.1098/rspb.2005.3327
Gleiss, S., & Kayser, C. (2014). Acoustic noise improves visual perception and modulates occipital oscillatory states. Journal of Cognitive Neuroscience, 26(4), 699-711. doi: https://doi.org/10.1162/jocn_a_00524
Gobet, F., Lane, P. C. R., Croker, S., Cheng, P. C.-H., Jones, G., Oliver, I., & Pine, J. M. (2001) Chunking mechanisms in human learning. Trends in Cognitive Sciences, 5236–243. doi:https://doi.org/10.1016/S1364-6613(00)01662-4
Golomb, J. D., McDavitt, J. R., Ruf, B. M., Chen, J. I., Saricicek, A., Maloney, K. H., ... & Bhagwagar, Z. (2009). Enhanced visual motion perception in major depressive disorder. Journal of Neuroscience, 29(28), 9072-9077.
Hommel, B. (1998). Event files: Evidence for automatic integration of stimulus-response episodes. Visual Cognition, 5, 183-216.
Hommel, B. (2004). Event files: Feature binding in and across perception and action. Trends in Cognitive Sciences, 8, 494-500.
Hommel, B. & Colzato, L. (2004). Visual attention and the temporal dynamics of feature integration. Visual Cognition, 11, 483-521.
Kanizsa, G. (1976). Subjective contours. Scientific American, 23(4), 48-52. doi:https://doi.org/10.1038/scientificamerican0476-48
Kawachi, Y., Grove, P. M., & Sakurai, K. (2014). A single auditory tone alters the perception of multiple visual events. Journal of Vision, 14(8), 16-16. doi:https://doi.org/10.1167/14.8.16
Kösem, A., & van Wassenhove, V. (2012). Temporal Structure in Audiovisual Sensory Selection. PLoS ONE, 7(7), 1-11. doi:https://doi.org/10.1371/journal.pone.0040936
Laberge, D., & Brown, V. (1986). Variations in size of the visual field in which targets are presented: An attentional range effect. Perception & Psychophysics, 40(3), 188-200. doi:https://doi.org/10.3758/bf03203016
Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, 279-81. doi:https://doi.org/10.1038/36846
Manjarrez, E., Mendez, I., Martinez, L., Flores, A., & Mirasso, C. R. (2007). Effects of auditory noise on the psychophysical detection of visual signals: Cross-modal stochastic resonance. Neuroscience Letters, 415, 231-236. doi: https://doi.org/10.1016/j/neulet.2007.01.030
Miller, G. A. (1956). The magical number seven, plus or minus two some limits on our capacity for processing information. Psychological Review, 101, 343-352. doi:https://doi.org/10.1037/h0043158
Mottron, L., Dawson, M., Soulieres, I., Hubert, B., & Burack, J. (2006). Enhanced perceptual functioning in autism: an update, and eight principles of autistic perception. Journal of Autism and Developmental Disorders, 36(1), 27-43.
Nicholls, M. E. R., Loveless, K. M., Thomas, N. A., Loetscher, T., & Churches, T. (2015). Some participants may be better than others: Sustained attention and motivation are higher early in the semester. The Quarterly Journal of Experimental Psychology, 68(1), 10-18.
Olivers, C. N. L., Awh, E., & Van der Burg, E. (2016). The capacity to detect synchronous audiovisual events is severely limited: Evidence from mixture modelling. Journal of Experimental Psychology: Human Perception and Performance, 42, 2115-2124.
Pylyshyn, Z. W. (2001). Visual indexes, preconceptual objects, and situated vision. Cognition, 80, 127-58. doi:https://doi.org/10.1016/S0010-0277(00)00156-6
Sargent, J., Dopkins, S., Philbeck, J., & Chickha, D. (2010). Chunking in spatial memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 576-589. doi: https://doi.org/10.1037/a0017528
Schutz, M., & Gillard, J. (2018). Generalizing audio-visual integration: what kinds of stimuli have we been using? Paper presented at the meeting of the International Multisensory Research Forum, Toronto, ON, Canada.
Schutz, M., & Vaisberg, J. M. (2014). Surveying the temporal structure of sounds used in Music Perception. Music Perception, 31(3), 288-296.
Seibold, V. C. (2018). Do alerting signals increase the size of the attentional focus? Attention, Perception, & Psychophysics, 80(2), 402-425. doi:https://doi.org/10.3758/s13414-017-1451-1
Seitz, A. R., Kim, R., & Shams, L. (2006). Sound facilitates visual learning. Current Biology, 16(14), 1422-1427.
Stevenson, R. A., Zemtsov, R. K., & Wallace, M. T. (2012). Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. Journal of Experimental Psychology: Human Perception and Performance, 38(6), 1517-1529.
Usher, M., & Feingold, M. (2000). Stochastic resonance in the speed of memory retrieval. Biological Cybernetics, 83, 11-16.
Van der Burg, E., Awh, E., & Olivers, C. N. (2013). The capacity of audiovisual integration is limited to one item. Psychological Science, 24(3), 345-351. doi:https://doi.org/10.1177/0956797612452865
Van der Burg, E., Cass, J., Olivers, C., Theeuwes, J., & Alais, D. (2010). Efficient visual search from synchronized auditory signals requires transient audiovisual events. Journal of Vision, 10(7), 855-855. doi:https://doi.org/10.1167/10.7.855
Van der Burg, E., Olivers, C. N., Bronkhorst, A. W., & Theeuwes, J. (2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34(5), 1053-1065. doi:https://doi.org/10.1037/0096-1518.104.22.1683
Van der Stoep, N., Van der Stigchel, S., Nijboer, T. C. W., & Van der Smagt, M. J. (2016). Audiovisual integration in near and far space: effects of changes in distance and stimulus effectiveness. Experimental Brain Research, 234, 1175-1188. doi: https://doi.org/10.1007/s00221-015-4248-2
Van Wanrooij, M. M., Bremen, P., & Opstal, J. A. (2010) Acquired prior knowledge modulates audiovisual integration. European Journal of Neuroscience, 31, 1763-71. doi:https://doi.org/10.1111/j.1460-9568.2010.07198.x.
Vogel, E. K., Woodman, G. F., & Luck, S. J. (2001). Storage of features, conjunctions, and objects in visual working memory. Psychonomic Bulletin & Review, 27, 92-114. doi:https://doi.org/10.1037//0096-1522.214.171.124
Vroomen, J., & de Gelder, B. (2000). Sound enhances visual perception: Cross-modal effects of auditory organization on vision. Journal of Experimental Psychology: Human Perception and Performance, 26(5), 1583-1590. doi:https://doi.org/10.1037//0096-15126.96.36.1993
Weinbach, N., & Henik, A. (2012). The relationship between alertness and executive control. Journal of Experimental Psychology: Human Perception and Performance, 38(6), 1530-1540. doi:https://doi.org/10.1037/a0027875
Weinbach, N., & Henik, A. (2014). Alerting enhances attentional bias for salient stimuli; evidence from a global/local processing task. Cognition, 133, 414-419.
Wilbiks, J. M. P., & Dyson, B. J. (2013). Effects of temporal asynchrony and stimulus magnitude on competitive audio-visual binding. Attention, Perception, & Psychophysics, 75(8), 1883-1891.
Wilbiks, J. M. P., & Dyson, B. J. (2016). The dynamics and neural correlates of audio-visual integration capacity as determined by temporal unpredictability, proactive interference, and SOA. PLOS One, 11(12). doi:https://doi.org/10.1371/journal.pone.0168304
Wilbiks, J. M. P., & Dyson, B. J. (2018). The contribution of perceptual factors and training on varying audiovisual integration capacity. Journal of Experimental Psychology: Human Perception and Performance, 44(6), 871-884.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Wilbiks, J.M.P., Pavilanis, A.D.S. & Rioux, D.M. Audiovisual integration capacity modulates as a function of illusory visual contours, visual display circumference, and sound type. Atten Percept Psychophys 82, 1971–1986 (2020). https://doi.org/10.3758/s13414-019-01882-6
- Multisensory processing
- Visual perception