Preliminary Experiment Combining Virtual Reality Haptic Shoes and Audio Synthesis

  • Rolf Nordahl
  • Amir Berrezag
  • Smilen Dimitrov
  • Luca Turchet
  • Vincent Hayward
  • Stefania Serafin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6192)


We describe a system that provides combined auditory and haptic sensations to simulate walking on different grounds. It uses a physical model that drives haptic transducers embedded in sandals and headphones. The model represents walking interactions with solid surfaces that can creak, or be covered with crumpling material. In a preliminary discrimination experiment, 15 participants were asked to recognize four different surfaces in a list of sixteen possibilities and under three different conditions, haptics only, audition only and combined haptic-audition. The results indicate that subjects are able to recognize most of the stimuli in the audition only condition, and some of the material properties such as hardness in the haptics only condition. The combination of auditory and haptic cues did not improve recognition significantly.


physical models walking sounds audio-haptic interaction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lederman, S.: Auditory texture perception. Perception (1979)Google Scholar
  2. 2.
    Jousmaki, V., Hari, R.: Parchment-skin illusion: sound-biased touch. Current Biology 8(6), R190–R191 (1998)CrossRefGoogle Scholar
  3. 3.
    Shimojo, S., Shams, L.: Sensory modalities are not separate modalities: plasticity and interactions. Current Opinion in Neurobiology 11(4), 505–509 (2001)CrossRefGoogle Scholar
  4. 4.
    Bresciani, J.P., Ernst, M.O., Drewing, K., Bouyer, G., Maury, V., Kheddar, A.: Feeling what you hear: Auditory signals can modulate tactile tap perception. Experimental Brain Research 162, 172–180 (2005)CrossRefGoogle Scholar
  5. 5.
    Ramstein, C., Hayward, V.: The pantograph: A large workspace haptic device for a multi-modal human-computer interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2004, ACM/SIGCHI Companion-4/94, pp. 57–58 (1994)Google Scholar
  6. 6.
    DiFranco, D., Beauregard, G.L., Srinivasan, M.: The effect of auditory cues on the haptic perception of stiffness in virtual environments. In: Proceedings of the ASME Dynamic Systems and Control Division (1997)Google Scholar
  7. 7.
    DiFilippo, D., Pai, D.K.: Contact interaction with integrated audio and haptics. In: Proceedings of the International Conference on Auditory Display, ICAD (2000)Google Scholar
  8. 8.
    Pai, D., Doel, K., James, D., Lang, J., Lloyd, J., Richmond, J., Yau, S.: Scanning physical interaction behavior of 3d objects. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 87–96 (2001)Google Scholar
  9. 9.
    Sreng, J., Bergez, F., Legarrec, J., Lécuyer, A., Andriot, C.: Using an event-based approach to improve the multimodal rendering of 6dof virtual contact. In: Proceedings of ACM Symposium on Virtual Reality Software and Technology (ACM VRST), pp. 173–179 (2007)Google Scholar
  10. 10.
    Avanzini, F., Crosato, P.: Integrating physically based sound models in a multimodal rendering architecture. The Journal of Visualization and Computer Animation 17(3-4), 411–419 (2006)Google Scholar
  11. 11.
    Giordano, B.L., Mcadams, S., Visell, Y., Cooperstock, J., Yao, H.Y., Hayward, V.: Non-visual identification of walking grounds. Journal of the Acoustical Society of America 123(5), 3412 (2008)CrossRefGoogle Scholar
  12. 12.
    Hayward, V., MacLean, K.E.: Do it yourself haptics, part-i. IEEE Robotics and Automation Magazine 14(4), 88–104 (2007)CrossRefGoogle Scholar
  13. 13.
    Iwata, H., Yano, H., Tomioka, H.: Powered shoes. In: ACM SIGGRAPH 2006 Emerging technologies, p. 28 (2006)Google Scholar
  14. 14.
    Schmidt, H., Hesse, S., Bernhardt, R., Krüger, J.: Hapticwalker—a novel haptic foot device. ACM Transactions on Applied Perception 2(2), 166–180 (2005)CrossRefGoogle Scholar
  15. 15.
    Visell, Y., Law, A., Cooperstock, J.R.: Touch is everywhere: Floor surfaces as ambient haptic interfaces. IEEE Transactions on Haptics 2, 148–159 (2009)CrossRefGoogle Scholar
  16. 16.
    Fu, X., Li, D.: Haptic shoes: representing information by vibration. In: Proceedings of the 2005 Asia-Pacific Symposium on Information Visualisation, pp. 47–50 (2005)Google Scholar
  17. 17.
    Magana, M., Velazquez, R.: On-shoe tactile display. In: IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE 2008), pp. 114–119 (2008)Google Scholar
  18. 18.
    Nordahl, R., Serafin, S., Turchet, L.: Sound synthesis and evaluation of interactive footsteps for virtual reality applications. In: Proc. IEEE VR 2010 (2010)Google Scholar
  19. 19.
    Avanzini, F., Serafin, S., Rocchesso, D.: Interactive simulation of rigid body interaction with friction-induced sound generation. IEEE Transactions on Speech and Audio Processing 13(5 Part 2), 1073–1081 (2005)CrossRefGoogle Scholar
  20. 20.
    Hunt, K.H., Crossley, F.R.E.: Coefficient of restitution interpreted as damping in vibroimpact. ASME Journal of Applied Mechanics 42(2), 440–445 (1975)CrossRefGoogle Scholar
  21. 21.
    Avanzini, F., Rocchesso, D.: Modeling collision sounds: Non-linear contact force. In: Proc. COST-G6 Conf. Digital Audio Effects (DAFx 2001), pp. 61–66 (2001)Google Scholar
  22. 22.
    Dupont, P., Hayward, V., Armstrong, B., Altpeter, F.: Single state elastoplastic friction models. IEEE Transactions on Automatic Control 47(5), 787–792 (2002)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Cook, P.: Physically Informed Sonic Modeling (PhISM): Synthesis of Percussive Sounds. Computer Music Journal 21(3), 38–49 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Rolf Nordahl
    • 1
  • Amir Berrezag
    • 2
  • Smilen Dimitrov
    • 1
  • Luca Turchet
    • 1
  • Vincent Hayward
    • 2
  • Stefania Serafin
    • 1
  1. 1.Aalborg University Copenhagen, MedialogyBallerupDenmark
  2. 2.Institut des Systèmes Intelligents et de Robotiqueupmc Univ Paris 06ParisFrance

Personalised recommendations