Temporal characteristics of perceived reality of multimodal contents

  • Shuichi SakamotoEmail author
  • Hiroyuki Yagyu
  • Zhenglie Cui
  • Tomoko Ohtani
  • Yôiti Suzuki
  • Jiro Gyoba
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 63)


Understanding the mechanism related to how people perceive reality is crucially important for the development of high-definition communication systems in multimedia contents. Various sensory information is well known to affect perceived reality from multimodal contents. For this purpose, it is important to present multimodal information with proper timing. This study investigated temporal characteristics of perceived reality induced by multimodal contents including vibration information. Results indicate that temporal characteristics of perceived reality depended on the timing of the events in the content, e.g. the timing with which the train just passes through an observation point. Moreover, the sense of presence was more sensitive to the timing of the presentation of sensory information with long contents.


sense of presence sense of verisimilitude multimodal integration stimulus onset asynchrony reality perception 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abe K., Ozawa K., Suzuki Y., Sone T.: Comparison of the effects of verbal versus visual information about sound sources on the perception of environmental sounds. Acta Acustica united with Acustica 92, 51–61 (2006)Google Scholar
  2. 2.
    Honda A., Kanda T., Shibata H., Asai N., Teramoto W., Sakamoto S., Iwaya Y., Suzuki Y., Gyoba J.: Determinants of senses of presence and verisimilitude in audio-visual contents. Journal of the Virtual Reality Society of Japan 18(1), 93–101 (2013) (in Japanese)Google Scholar
  3. 3.
    Kim S., Sakanashi H., Martens W.L.: Multimodal representation of electronic piano sound for pianoplayers. In: INTER-NOISE 2011, 6-page manuscript (2011)Google Scholar
  4. 4.
    Meyer G.F., Wang L.T., Timson E., Perfect P.: White MD Objective fidelity evaluation in multisensory virtual environments: Auditory cue fidelity in flight simulation. PLoS One 7(9), e44381 (2012)Google Scholar
  5. 5.
    Sakamoto S., Hasegawa G., Honda A., Iwaya Y., Suzuki Y., Gyoba J.: Body vibration effects on perceived reality with multi-modal contents. ITE Trans. on MTA 2(1), 46–50 (2014)Google Scholar
  6. 6.
    Slater M., Steed A., McCarthy J., Maringelli F.: The influence of body movement on subjective presence in virtual environments. Human Factors 40(3), 469–477 (1998)Google Scholar
  7. 7.
    Teramoto W., Yoshida K., Hidaka S., Asai N., Gyoba J., Sakamoto S., Iwaya Y., Suzuki Y.: Spatiotemporal characteristics responsible for high vraisemblance. Journal of the Virtual Reality Society of Japan 15(3), 483–486 (2010) (in Japanese)Google Scholar
  8. 8.
    Teramoto W., Yoshida K., Asai N., Hidaka S., Gyoba J., Suzuki Y.: What is “sense of presence”? A nonresearcher’s understanding of sense of presence. Journal of the Virtual Reality Society of Japan 15(1), 7–16 (2010) (in Japanese)Google Scholar
  9. 9.
    Tato T., Namba S., Kuwano S.: Continuous judgment of level-fluctuating noise, In: INTERNOISE 94, pp. 1081–1084 (1994)Google Scholar
  10. 10.
    Witmer B.G., Singer M.J.: Measuring presence in virtual environments: A presence questionnaire. Presence 7(3), 225–240 (1998)Google Scholar
  11. 11.
    Woszczyk W., Cooperstock J., Roston J., Martens W.L.: Shake, rattle, and roll: Getting immersed in multisensory, interactive music via broadband networks. J. Audio Eng. Soc. 53, 336–344 (2005)Google Scholar
  12. 12.
    Yagyu H., Cui Z., Sakamoto S., Ohtani T., Suzuki Y., Gyoba J.: Effects of multimodal contents on the temporal shifts in reality perception. Journal of the Virtual Reality Society of Japan 20(3), pp. 199–208 (2015) (in Japanese)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Shuichi Sakamoto
    • 1
    Email author
  • Hiroyuki Yagyu
    • 1
  • Zhenglie Cui
    • 1
  • Tomoko Ohtani
    • 1
  • Yôiti Suzuki
    • 1
  • Jiro Gyoba
    • 2
  1. 1.Research Institute of Electrical Communication and Graduate School of Information ScienceTohoku UniversityAoba-ku, SendaiJapan
  2. 2.Graduate School of Arts and LettersTohoku UniversityAoba-ku, SendaiJapan

Personalised recommendations