Multisensory integration and behavioral stability

  • Charlotte RoyEmail author
  • Simone Dalla Bella
  • Simon Pla
  • Julien Lagarde
Original Article


Information coming from multiple senses, as compared to a single one, typically enhances our performance. The multisensory improvement has been extensively examined in perception studies, as well as in tasks involving a motor response like a simple reaction time. However, how this effect extends to more complex behavior, typically involving the coordination of movements, such as bimanual coordination or walking, is still unclear. A critical element in achieving motor coordination in complex behavior is its stability. Reaching a stable state in the coordination pattern allows to sustain complex behavior over time (e.g., without interruption or negative consequences, like falling). This study focuses on the relation between stability in the coordination of movement patterns, like walking, and multisensory improvement. Participants walk with unimodal and audio-tactile metronomes presented either at their preferred rate or at a slower walking rate, the instruction being to synchronize their steps to the metronomes. Walking at a slower rate makes gait more variable than walking at the preferred rate. Interestingly however, the multimodal stimuli enhance the stability of motor coordination but only in the slower condition. Thus, the reduced stability of the coordination pattern (at a slower gait rate) prompts the sensorimotor system to capitalize on multimodal stimulation. These findings provide evidence of a new link between multisensory improvement and behavioral stability, in the context of ecological sensorimotor task.


Compliance with ethical standards

Conflict of interest

The authors Charlotte Roy, Simone Dalla Bella, Simon Pla, and Julien Lagarde declare that they have not conflict of interest.

Ethical approval

The study was in accordance with the ethical standards of the national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. Albouy, P., Lévêque, Y., Hyde, K. L., Bouchet, P., Tillmann, B., & Caclin, A. (2015). Boosting pitch encoding with audiovisual interactions in congenital amusia. Neuropsychologia,67, 111–120. Scholar
  2. Ammirante, P., Patel, A. D., & Russo, F. A. (2016). Synchronizing to auditory and tactile metronomes: a test of the auditory-motor enhancement hypothesis. Psychonomic Bulletin and Review. Scholar
  3. Armstrong, A., & Issartel, J. (2014). Sensorimotor synchronization with audio-visual stimuli: limited multisensory integration. Experimental Brain Research,232(11), 3453–3463. Scholar
  4. Aschersleben, G., & Bertelson, P. (2003). Temporal ventriloquism: crossmodal interaction on the time dimension. 2. Evidence from sensorimotor synchronization. International Journal of Psychophysiology,50, 157–163. Scholar
  5. Blais, M., Albaret, J.-M., & Tallet, J. (2015). Is there a link between sensorimotor coordination and inter-manual coordination? Differential effects of auditory and/or visual rhythmic stimulations. Experimental Brain Research,233(11), 3261–3269. Scholar
  6. Blais, M., Martin, E., Albaret, J.-M., & Tallet, J. (2014). Preservation of perceptual integration improves temporal stability of bimanual coordination in the elderly: An evidence of age-related brain plasticity. Behavioural Brain Research,275, 34–42. Scholar
  7. Bolognini, N., Convento, S., Rossetti, A., & Merabet, L. B. (2013). Multisensory processing after a brain damage: Clues on post-injury crossmodal plasticity from neuropsychology. Neuroscience and Biobehavioral Reviews,37(3), 269–278. Scholar
  8. Bressler, S., & Kelso, J. (2001). Cortical coordination dynamics. Trends in Cognitive Sciences,5(1), 26–36. Scholar
  9. Bruijn, S. M., Meijer, O. G., Beek, P. J., & Van Dieën, J. H. (2013). Assessing the stability of human locomotion : a review of current measures. Journal of the Royal Society, Interface/the Royal Society. Scholar
  10. Bruijn, S. M., van Dieën, J. H., Meijer, O. G., & Beek, P. J. (2009). Is slow walking more stable? Journal of Biomechanics,42(10), 1506–1512. Scholar
  11. Caclin, A., Bouchet, P., Djoulah, F., Pirat, E., Pernier, J., & Giard, M. H. (2011). Auditory enhancement of visual perception at threshold depends on visual abilities. Brain Research,1396, 35–44. Scholar
  12. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Hillsdale: Erlbaum Associates.Google Scholar
  13. Colonius, H., & Diederich, A. (2004). Multisensory interaction in saccadic reaction time: a time-window-of-integration model. Journal of Cognitive Neuroscience,16(6), 1000–1009. Scholar
  14. Dakos, V., Van Nes, E. H., D’Odorico, P., & Scheffer, M. (2012). Robustness of variance and autocorrelation as indicators of critical slowing down. Ecology,93(2), 264–271.CrossRefGoogle Scholar
  15. Dalla Bella, S. (2018). Music and movement: Towards a translational approach. Neurophysiologie Clinique, 48(6), 377–386.CrossRefGoogle Scholar
  16. Dingwell, J., & Marin, L. (2006). Kinematic variability and local dynamic stability of upper body motions when walking at different speeds. Journal of Biomechanics,39(3), 444–452. Scholar
  17. Elliott, M. T., Wing, A. M., & Welchman, A. E. (2010). Multisensory cues improve sensorimotor synchronisation. European Journal of Neuroscience,31(10), 1828–1835. Scholar
  18. Elliott, Mark T., Wing, A. M., & Welchman, A. E. (2011). The effect of ageing on multisensory integration for the control of movement timing. Experimental Brain Research,213(2–3), 291–298. Scholar
  19. Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature,415(6870), 429–433. Scholar
  20. Ernst, M. O., & Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences,8(4), 162–169. Scholar
  21. Frassinetti, F., Bolognini, N., Bottari, D., Bonora, A., & Làdavas, E. (2005). Audiovisual integration in patients with visual deficit. Journal of Cognitive Neuroscience,17(9), 1442–1452. Scholar
  22. Gardiner, C. W. (2003). Handbook of stochastic methods for physics, chemistry and the natural sciences. Berlin: Springer.Google Scholar
  23. Giard, M. H., & Peronnet, F. (1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience,11(5), 473–490. Scholar
  24. Haken, H., Kelso, J. S., & Bunz, H. (1985). A theoretical model of phase transitions in human hand movements. Biological Cybernetics,51(5), 347–356.CrossRefGoogle Scholar
  25. Hershenson, M. (1962). Reaction time as a measure of intersensory facilitation. Journal of Experimental Psychology,63(3), 289.CrossRefGoogle Scholar
  26. Huys, R., Perdikis, D., & Jirsa, V. K. (2014). Functional architectures and structured flows on manifolds: A dynamical framework for motor behavior. Psychological Review,121(3), 302.CrossRefGoogle Scholar
  27. Jordan, K., Challis, J. H., & Newell, K. M. (2007). Walking speed influences on gait cycle variability. Gait and Posture,26(1), 128–134. Scholar
  28. Kato, M., & Konishi, Y. (2006). Auditory dominance in the error correction process: A synchronized tapping study. Brain Research,1084(1), 115–122. Scholar
  29. Kelso, J. A. S. (1995). Dynamic patterns: the self-organization of brain and behavior. Cambridge: MIT Press.Google Scholar
  30. Kelso, J. A. S., Fink, P. W., DeLaplain, C. R., & Carson, R. G. (2001). Haptic information stabilizes and destabilizes coordination dynamics. Proceedings of the Royal Society B: Biological Sciences, 268(1472), 1207–1213. Scholar
  31. Lagarde, J., & Kelso, J. A. S. (2006). Binding of movement, sound and touch: multimodal coordination dynamics. Experimental Brain Research,173(4), 673–688. Scholar
  32. Lagarde, J., Zelic, G., & Mottet, D. (2012). Segregated audio-tactile events destabilize the bimanual coordination of distinct rhythms. Experimental Brain Research,219(3), 409–419. Scholar
  33. Large, E. W., & Jones, M. R. (1999). The dynamics of attending: How people track time-varying events. Psychological Review,106(1), 119.CrossRefGoogle Scholar
  34. Lovelace, C., Stein, B., & Wallace, M. (2003). An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Cognitive Brain Research,17(2), 447–453.CrossRefGoogle Scholar
  35. Meredith, M., & Stein, B. (1986). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56(3), 640–662. Scholar
  36. Meredith, M., Nemitz, J., & Stein, B. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. The Journal of Neuroscience, 7(10), 3215. Scholar
  37. Murray, M. M., Molholm, S., Michel, C. M., Heslenfeld, D. J., Ritter, W., Javitt, D. C., & Foxe, J. J. (2005). Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cerebral Cortex,15(7), 963–974. Scholar
  38. Nelson, W., Hettinger, L., Cunningham, J., Brickman, B., Haas, M., & McKinley, R. (1998). Effects of localized auditory information on visual target detection performance using a helmet-mounted display. Human Factors,40(3), 452–460.CrossRefGoogle Scholar
  39. Repp, B. H. (2005). Sensorimotor synchronization: A review of the tapping literature. Psychonomic Bulletin and Review,12(6), 969–992. Scholar
  40. Repp, B. H., & Penel, A. (2002). Auditory dominance in temporal processing: new evidence from synchronization with simultaneous visual and auditory sequences. Journal of Experimental Psychology Human Perception and Performance,28(5), 1085–1099. Scholar
  41. Repp, B., & Penel, A. (2004). Rhythmic movement is attracted more strongly to auditory than to visual rhythms. Psychological Research Psychologische Forschung. Scholar
  42. Repp, B. H., & Su, Y.-H. (2013). Sensorimotor synchronization: A review of recent research (2006–2012). Psychonomic Bulletin and Review,20(3), 403–452. Scholar
  43. Rowland, B. A., Quessy, S., Stanford, T. R., & Stein, B. E. (2007). Multisensory integration shortens physiological response latencies. Journal of Neuroscience,27(22), 5879–5884. Scholar
  44. Roy, C., Dalla Bella, S., & Lagarde, J. (2017a). To bridge or not to bridge the multisensory time gap: bimanual coordination to sound and touch with temporal lags. Experimental Brain Research,235(1), 135–151. Scholar
  45. Roy, C., Lagarde, J., Dotov, D., & Dalla Bella, S. (2017b). Walking to a multisensory beat. Brain and Cognition, 113, 172–183. Scholar
  46. Schöner, G., Haken, H., & Kelso, J. (1986). A stochastic theory of phase transitions in human hand movement. Biological Cybernetics,53(4), 247–257. Scholar
  47. Schöner, G., & Kelso, J. A. (1988). Dynamic pattern generation in behavioral and neural systems. Science, 239(4847), 1513–1520.CrossRefGoogle Scholar
  48. Sejdić, E., Fu, Y., Pak, A., Fairley, J. A., & Chau, T. (2012). The effects of rhythmic sensory cues on the temporal dynamics of human gait. PLoS One. Scholar
  49. Sowiński, J., & Dalla Bella, S. (2013). Poor synchronization to the beat may result from deficient auditory-motor mapping. Neuropsychologia,51(10), 1952–1963. Scholar
  50. Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge: MIT Press.Google Scholar
  51. Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nature Reviews Neuroscience,9(4), 255–266. Scholar
  52. Strogatz, S. H. (2018). Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering. Boca Raton: CRC Press.CrossRefGoogle Scholar
  53. Terrier, P., & Dériaz, O. (2012). Persistent and anti-persistent pattern in stride-to-stride variability of treadmill walking: influence of rhythmic auditory cueing. Human Movement Science,31(6), 1585–1597. Scholar
  54. Varlet, M., Marin, L., Issartel, J., Schmidt, R. C., & Bardy, B. G. (2012). Continuity of visual and auditory rhythms influences sensorimotor coordination. PLoS One. Scholar
  55. Vroomen, J., & Keetels, M. (2010). Perception of intersensory synchrony: A tutorial review. Attention, Perception and Psychophysics,72(4), 871–884. Scholar
  56. Warren, W. H. (2006). The dynamics of perception and action. Psychological Review,113(2), 358.CrossRefGoogle Scholar
  57. Wright, R. L., & Elliott, M. T. (2014). Stepping to phase-perturbed metronome cues: multisensory advantage in movement synchrony but not correction. Frontiers in Human Neuroscience,8(September), 1–7. Scholar
  58. Yamasaki, M., Sasaki, T., Tsuzuki, S., & Torii, M. (1984). Stereotyped pattern of lower limb movement during level and grade walking on treadmill. The Annals of Physiological Anthropology,3(4), 291–296. Scholar
  59. Zelic, Gregory, Mottet, D., & Lagarde, J. (2012). Behavioral impact of unisensory and multisensory audio-tactile events: Pros and cons for interlimb coordination in juggling. PLoS One,7(2), e32308. Scholar
  60. Zelic, G., Mottet, D., & Lagarde, J. (2016). Perceptuo-motor compatibility governs multisensory integration in bimanual coordination dynamics. Experimental Brain Research,234(2), 463–474. Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.EuroMov LaboratoryMontpellier UniversityMontpellierFrance
  2. 2.Applied Cognitive Psychology LaboratoryUlm UniversityUlmGermany
  3. 3.International Laboratory for Brain, Music, and Sound Research (BRAMS)MontrealCanada
  4. 4.Department of PsychologyUniversity of MontrealMontrealCanada

Personalised recommendations