Abstract
The brain integrates incoming sensory signals to a degree that depends on the signals’ redundancy. Redundancy—which is commonly high when signals originate from a common physical object or event—is estimated by the brain from the signals’ spatial and/or temporal correspondence. Here we tested whether verbally instructed knowledge of non-redundancy can also be used to reduce the strength of the sensory integration. We used a cursor-control task in which cursor motions in the frontoparallel plane were controlled by hand movements in the horizontal plane, yet with a small and randomly varying visuomotor rotation that created spatial discrepancies between hand and cursor positions. Consistent with previous studies, we found mutual biases in the hand and cursor position judgments, indicating partial sensory integration. The integration was reduced in strength, but not eliminated, after participants were verbally informed about the non-redundancy (i.e., the spatial discrepancies) in the hand and cursor positions. Comparisons with model predictions excluded confounding bottom-up effects of the non-redundancy instruction. Our findings thus show that participants have top-down control over the degree to which they integrate sensory information. Additionally, we found that the magnitude of this top-down modulatory capability is a reliable individual trait. A comparison between participants with and without video-gaming experience tentatively suggested a relation between top-down modulation of integration strength and attentional control.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adams, W.J. (2016). The Development of audio-visual integration for temporal judgements. PLoS Computational Biology, 12, e1004865. https://doi.org/10.1371/journal.pcbi.1004865.
Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257–262. https://doi.org/10.1016/j.cub.2004.01.029.
Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15, 839–843. https://doi.org/10.1016/j.cub.2005.03.046.
Astle, D.E., & Scerif, G. (2009). Using developmental cognitive neuroscience to study behavioral and attentional control. Developmental Psychobiology, 51, 107–118. https://doi.org/10.1002/dev.20350.
Bavelier, D., Achtman, R.L., Mani, M., & Föcker, J. (2012). Neural bases of selective attention in action video game players. Vision Research, 61, 132–143. https://doi.org/10.1016/j.visres.2011.08.007.
Bertelson, P., Vroomen, J., Gelder, B.D., & Driver, J. (2000). The ventriloquist effect does not depend on the direction of deliberate visual attention. Perception Psychophysics, 62, 321–332. https://doi.org/10.3758/BF03205552.
Bresciani, J.-P., Dammeier, F., & Ernst, M.O. (2006). Vision and touch are automatically integrated for the perception of sequences of events. Journal of Vision, 6, 554–564. https://doi.org/10.1167/6.5.2.
Chen, Y.-C., & Spence, C. (2017) Assessing the role of the ‘Unity Assumption’ on multisensory integration: A review. Frontiers Psychology. https://doi.org/10.3389/fpsyg.2017.00445.
Cheng, K., Shettleworth, S.J., Huttenlocher, J., & Rieser, J.J. (2007). Bayesian integration of spatial information. Psychological Bulletin, 133, 625–637. https://doi.org/10.1037/0033-2909.133.4.625.
Chun, M.M., Golomb, J.D., & Turk-Browne, N.B. (2011). A taxonomy of external and internal attention. Annual Review of Psychology, 62, 73–101. https://doi.org/10.1146/annurev.psych.093008.100427.
Colonius, H., & Diederich, A. (2017) Formal models and quantitative measures of multisensory integration: A selective overview. European Journal of Neuroscience. https://doi.org/10.1111/ejn.13813.
Cuppini, C., Shams, L., Magosso, E., & Ursino, M. (2017). A biologically inspired neurocomputational model for audiovisual integration and causal inference. European Journal of Neuroscience, 46, 2481–2498. https://doi.org/10.1111/ejn.13725.
Debats, N.B., Ernst, M.O., & Heuer, H. (2017a). Perceptual attraction in tool use: evidence for a reliability-based weighting mechanism. Journal of Neurophysiology, 117, 1569–1580. https://doi.org/10.1152/jn.00724.2016.
Debats, N.B., Ernst, M.O., & Heuer, H. (2017b). Kinematic cross-correlation induces sensory integration across separate objects. European Journal of Neuroscience, 46, 2826–2834. https://doi.org/10.1111/ejn.13758.
Debats, N.B., & Heuer, H. (2018). Optimal integration of actions and their visual effects is based on both online and prior causality evidence. Scientific Reports, 8, 9796. https://doi.org/10.1038/s41598-018-28251-x.
Donohue, S.E., Woldorff, M.G., & Mitroff, S.R. (2010). Video game players show more precise multisensory temporal processing abilities. Attention, Perception, & Psychophysics, 72, 1120–1129. https://doi.org/10.3758/APP.72.4.1120.
Ernst, M.O. (2006). A Bayesian view on multimodal cue integration. In G. Knoblich, I.M. Thornton, M. Grosjean & M. Shiffrar (Eds.), Human body perception from the inside out (pp. 105–131). New York: Oxford University Press.
Ernst, M.O. (2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7, 1–14. https://doi.org/10.1167/7.5.7.
Ernst, M.O. (2012). Optimal multisensory integration: assumptions and limits. In B.E. Stein (Ed.), The new handbook of multisensory processes (pp. 1084–1124). Cambridge: The MIT Press.
Ernst, M.O., & Banks, M.S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429–433. https://doi.org/10.1038/415429a.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. https://doi.org/10.3758/BF03193146.
Fourneret, P., & Jeannerod, M. (1998). Limited conscious monitoring of motor performance in normal subjects. Neuropsychologia, 36, 1133–1140.
Gau, R., & Noppeney, U. (2016). How prior expectations shape multisensory perception. Neuroimage, 124, 876–886. https://doi.org/10.1016/j.neuroimage.2015.09.045.
Geerligs, L., Saliasi, E., Maurits, N.M., et al. (2014). Brain mechanisms underlying the effects of aging on different aspects of selective attention. Neuroimage, 91, 52–62. https://doi.org/10.1016/j.neuroimage.2014.01.029.
Gopher, D., Well, M., & Bareket, T. (1994). Transfer of skill from a computer game trainer to flight. Human Factors, 36, 387–405. https://doi.org/10.1177/001872089403600301.
Hartcher-O’Brien, J., Soto-Faraco, S., & Adam, R. (2017) Editorial: A matter of bottom-up or top-down processes: The role of attention in multisensory integration. Frontiers in Integrative Neuroscience. https://doi.org/10.3389/fnint.2017.00005.
Helbig, H.B., & Ernst, M.O. (2007). Knowledge about a common source can promote visual-haptic integration. Perception, 36, 1523–1533.
Helbig, H.B., & Ernst, M.O. (2008). Visual-haptic cue weighting is independent of modality-specific attention. Journal of Vision, 8, 21.1–21.16. https://doi.org/10.1167/8.1.21.
Heuer, A., & Schubö, A. (2017). Selective weighting of action-related feature dimensions in visual working memory. Psychonomic Bulletin & Review, 24, 1129–1134. https://doi.org/10.3758/s13423-016-1209-0.
Heuer, A., & Schubö, A. (2018). Separate and combined effects of action relevance and motivational value on visual working memory. Journal of Vision, 18, 14–14. https://doi.org/10.1167/18.5.14.
Heuer, H. (2006). Übung. In K. Pawlik (Ed.), Handbuch psychologie (pp. 179–193). Berlin: Springer.
Hugenschmidt, C.E., Mozolic, J.L., & Laurienti, P.J. (2009). Suppression of multisensory integration by modality-specific attention in aging. Neuroreport, 20, 349–353. https://doi.org/10.1097/WNR.0b013e328323ab07.
Kirsch, W., Pfister, R., & Kunde, W. (2016). Spatial action-effect binding. Attention, Perception, & Psychophysics, 78, 133–142. https://doi.org/10.3758/s13414-015-0997-z.
Klatzky, R.L., Lederman, S.J., & Langseth, S. (2003). Watching a cursor distorts haptically guided reproduction of mouse movement. Journal of Experimental Psychology Application, 9, 228–235. https://doi.org/10.1037/1076-898X.9.4.228.
Kleiner, M., Brainard, D., & Pelli, D. (2007). What’s new in psychtoolbox-3? Perception, 36(1), 14.
Körding, K.P., Beierholm, U., Ma, W.J., et al. (2007). Causal inference in multisensory perception. PLoS One, 2, e943. https://doi.org/10.1371/journal.pone.0000943.
Kuling, I.A., van der Graaff, M.C.W., Brenner, E., & Smeets, J.B.J. (2017). Matching locations is not just matching sensory representations. Experimental Brain Research, 235, 533–545. https://doi.org/10.1007/s00221-016-4815-1.
Ladwig, S., Sutter, C., & Müsseler, J. (2013). Intra- and intermodal integration of discrepant visual and proprioceptive action effects. Experimental Brain Research, 231, 457–468. https://doi.org/10.1007/s00221-013-3710-2.
Laurienti, P.J., Burdette, J.H., Maldjian, J.A., & Wallace, M. T. (2006). Enhanced multisensory integration in older adults. Neurobiology of Aging, 27, 1155–1163. https://doi.org/10.1016/j.neurobiolaging.2005.05.024.
Levene, H. (1960). Robust tests for equality of variances. In I. Olkin, G. Ghurye, W. Hoeffding, et al. (Eds.), Contributions to probability and statistics: Essays in honor of harold hotelling (pp. 278–292). Palo Alto: Stanford University Press.
Macaluso, E., Noppeney, U., Talsma, D., et al. (2016). The curious incident of attention in multisensory integration: Bottom-up vs. top-down. Multisensory Research, 29, 557–583. https://doi.org/10.1163/22134808-00002528.
Mozolic, J.L., Hugenschmidt, C.E., Peiffer, A.M., & Laurienti, P.J. (2008). Modality-specific selective attention attenuates multisensory integration. Experimental Brain Research, 184, 39–52. https://doi.org/10.1007/s00221-007-1080-3.
Müsseler, J., & Sutter, C. (2009). Perceiving one’s own movements when using a tool. Consciousness and Cognition, 18, 359–365. https://doi.org/10.1016/j.concog.2009.02.004.
Odegaard, B., Wozny, D.R., & Shams, L. (2016). The effects of selective and divided attention on sensory precision and integration. Neuroscience Letters, 614, 24–28. https://doi.org/10.1016/j.neulet.2015.12.039.
Ohl, S., & Rolfs, M. (2017). Saccadic eye movements impose a natural bottleneck on visual short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43, 736–748. https://doi.org/10.1037/xlm0000338.
Parise, C.V., & Ernst, M.O. (2016). Correlation detection as a general mechanism for multisensory integration. Nature Communications, 7, 11543. https://doi.org/10.1038/ncomms11543.
Raghunathan, T.E., Rosenthal, R., & Rubin, D.B. (1996). Comparing correlated but nonoverlapping correlations. Psychological Methods, 1, 178–183. https://doi.org/10.1037/1082-989X.1.2.178.
Rand, M.K., & Heuer, H. (2013). Implicit and explicit representations of hand position in tool use. PLoS One, 8, e68471. https://doi.org/10.1371/journal.pone.0068471.
Rand, M.K., & Heuer, H. (2016) Effects of reliability and global context on explicit and implicit measures of sensed hand position in cursor-control tasks. Frontiers Psychology. https://doi.org/10.3389/fpsyg.2015.02056.
Rand, M.K., & Heuer, H. (2018). Dissociating explicit and implicit measures of sensed hand position in tool use: Effect of relative frequency of judging different objects. Attention, Perception, & Psychophysics, 80, 211–221. https://doi.org/10.3758/s13414-017-1438-y.
Rand, M.K., Wang, L., Müsseler, J., & Heuer, H. (2013). Vision and proprioception in action monitoring by young and older adults. Neurobiology Aging, 34, 1864–1872. https://doi.org/10.1016/j.neurobiolaging.2013.01.021.
Shams, L., & Beierholm, U.R. (2010). Causal inference in perception. Trends in Cognitive Science, 14, 425–432. https://doi.org/10.1016/j.tics.2010.07.001.
Shapiro, S.S., & Wilk, M.B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52, 591–611. https://doi.org/10.2307/2333709.
Sülzenbrück, S., & Heuer, H. (2009). Functional independence of explicit and implicit motor adjustments. Consciousness and Cognition, 18, 145–159. https://doi.org/10.1016/j.concog.2008.12.001.
Takahashi, C., Diedrichsen, J., & Watt, S.J. (2009). Integration of vision and haptics during tool use. Journal of Vision, 9, 3–13. https://doi.org/10.1167/9.6.3.
Talsma, D., Doty, T.J., & Woldorff, M.G. (2007). Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? Cerebral Cortex 17, 679–690. https://doi.org/10.1093/cercor/bhk016.
Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M.G. (2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Science, 14, 400–410. https://doi.org/10.1016/j.tics.2010.06.008.
Unsworth, N. (2015). Consistency of attentional control as an important cognitive trait: A latent variable analysis. Intelligence, 49, 110–128. https://doi.org/10.1016/j.intell.2015.01.005.
Unsworth, N., Redick, T.S., McMillan, B.D., et al. (2015). Is playing video games related to cognitive abilities? Psychological Science, 26, 759–774. https://doi.org/10.1177/0956797615570367.
van Dam, L.C.J., Parise, C.V., & Ernst, M.O. (2014) Modeling multisensory integration. In: D.J. Bennett, C.S. Hill (Eds.), Sensory integration and the unity of consciousness (pp. 209–229). Cambridge: The MIT Press.
Vercillo, T., & Gori, M. (2015) Attention to sound improves auditory reliability in audio-tactile spatial optimal integration. Frontiers in Integrative Neuroscience. https://doi.org/10.3389/fnint.2015.00034.
Vroomen, J., Bertelson, P., & Gelder, B.D. (2001). The ventriloquist effect does not depend on the direction of automatic visual attention. Perception & Psychophysics, 63, 651–659. https://doi.org/10.3758/BF03194427.
Welch, R.B., & Warren, D.H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638–667.
Wozny, D.R., Beierholm, U.R., & Shams, L. (2008). Human trimodal perception follows optimal statistical inference. Journal of Vision, 8, 24. https://doi.org/10.1167/8.3.24.
Wozny, D.R., Beierholm, U.R., & Shams, L. (2010). Probability matching as a computational strategy used in perception. PLoS Computational Biology, 6, e1000871. https://doi.org/10.1371/journal.pcbi.1000871.
Acknowledgements
We thank A. Oppenborn, F. Steinbeck, and C. Osterbrink for assistance in data collection.
Funding
This study was funded by the Deutsche Forschungsgemeinschaft (DFG) Grant HE1187/19-1.
Author information
Authors and Affiliations
Contributions
NBD and HH designed the experiment; NBD and HH analyzed the data; NBD and HH wrote the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that no conflicts of interest exist.
Ethical approval
All procedures were approved by the Bielefeld University Ethics Committee and in accordance with the 1964 Helsinki declaration.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Debats, N.B., Heuer, H. Explicit knowledge of sensory non-redundancy can reduce the strength of multisensory integration. Psychological Research 84, 890–906 (2020). https://doi.org/10.1007/s00426-018-1116-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00426-018-1116-2