Advertisement

Automatic synthesis of explosion sound synchronized with animation

  • Shiguang LiuEmail author
  • Si Gao
Original Article
  • 24 Downloads

Abstract

Sound is an essential element for enhancing the realness of virtual world. Currently, there are some remarkable works on the synthesis of fluid sound, such as fire and water. However, little attention has been paid to synthesizing explosion sound. This paper proposes an automatic method for synthesizing explosion sounds that are synchronized with the visual phenomena of explosion animations, including fireball generation and flame combustion. Such two types of visual animation correspond to two types of sound, which we name as explosive sound and combustion noise, respectively. For the synthesis of explosive sound, firstly, the occurrence time and duration of explosion sound are determined according to the dynamic process of fuel consumption, and then, the corresponding explosive sound is extracted from the recording examples according to the high-frequency content. For the combustion noise, we propose a synthesis method of combustion noise on the basis of the timbre similarity between sound examples and low-frequency combustion noise generated by a physical method. Finally, the two types of sound are blended respecting the occurrence and duration of the explosions and combustions parts detected in the visual stream. Our experiments and the user study show the results of our method and demonstrate the effectiveness of our method.

Keywords

Audiovisual synchronization Explosive sound Combustion noise Sound synthesis Immersive virtual reality 

Notes

Acknowledgements

Funding was provided by Natural Science Foundation of China (Grant Nos. 61672375 and 61170118).

Supplementary material

Supplementary material 1 (wmv 9315 KB)

References

  1. Agostino DS (1999) Synthesis of environmental sound textures by iterated nonlinear functions. In: Digital audio effects. pp 109–117Google Scholar
  2. An SS, James DL, Marschner S (2012) Motion-driven concatenative synthesis of cloth sounds. ACM Trans Graph 31(4):1–10Google Scholar
  3. Chadwick JN, James DL (2011) Animating fire with sound. ACM Trans Graph 30(4):84–92Google Scholar
  4. Crighton D, Dowling A, Ffowcs-Williams J, Heckl M, Leppington F, Bartram JF (1992) Modern methods in analytical acoustics. Springer, BerlinGoogle Scholar
  5. Dobashi Y, Kaneda K, Yamashita H, Okita T, Nishita T (2000) A simple, efficient method for realistic animation of clouds. In: ACM SIGGRAPH. pp 19–28Google Scholar
  6. Dobashi Y, Yamamoto T, Nishita T (2003) Real-time rendering of aerodynamic sound using sound textures based on computational fluid dynamics. In: ACM SIGGRAPH. pp 732–740Google Scholar
  7. Dobashi Y, Yamamoto T, Nishita T (2004) Synthesizing sound from turbulent field using sound textures for interactive fluid simulation. Comput Graph Forum 23(3):539–545Google Scholar
  8. Doel KVD (2005) Physically-based models for liquid sounds. ACM Trans Appl Percept 2(4):534–546Google Scholar
  9. Dubnov S, Barjoseph Z, Ran EY, Lischinski D, Werman M (2002) Synthesizing sound textures through wavelet tree learning. IEEE Comput Graph Appl 22(4):38–48Google Scholar
  10. Feldman BE, O’brien JF, Arikan O (2003) Animating suspended particle explosions. ACM Trans Graph 22(3):708–715zbMATHGoogle Scholar
  11. Feng G, Liu S (2018) Detail-preserving sph fluid control with deformation constraints. Comput Animat Virtual Worlds 29(1):e1781MathSciNetGoogle Scholar
  12. Ihme M, Pitsch H, Bodony D (2009) Radiation of noise in turbulent non-premixed flames. Proc Combust Inst 32(1):1545–1553Google Scholar
  13. Kersten S, Purwins H (2013) Fire texture sound re-synthesis using sparse decomposition and noise modelling. In: Digital audio effects. pp 1–5Google Scholar
  14. Liu S, Xiong Y (2013) Fast and stable simulation of virtual water scenes with interactions. Virtual Real 17(1):77–88Google Scholar
  15. Liu S, Yu Z (2015) Sounding fire for immersive virtual reality. Virtual Real 19(3–4):291–302Google Scholar
  16. Longuethiggins MS (1990) An analytic model of sound production by raindrops. Fluid Mech 214:395–410MathSciNetGoogle Scholar
  17. Marelli D, Aramaki M, Kronland-Martinet R, Verron C (2012) An efficient time–frequency method for synthesizing noisy sounds with short transients and narrow spectral components. IEEE Trans Audio Speech Lang Process 20(4):1400–1408Google Scholar
  18. Moss W, Yeh H, Hong JM, Lin MC, Manocha D (2010) Sounding liquids: automatic sound synthesis from fluid simulation. ACM Trans Graph 29(3):1–13Google Scholar
  19. Nguyen DQ, Fedkiw R, Jensen HW (2002) Physically based modeling and animation of fire. ACM Trans Graph 21(3):721–728Google Scholar
  20. O’Brien JF, Bargteil AW, Hodgins JK (2002) Graphical modeling and animation of ductile fracture. ACM Trans Graph 21(3):291–294Google Scholar
  21. Powell A (2003) Theory of vortex sound. Cambridge University Press, CambridgeGoogle Scholar
  22. Prosperetti A (1988) Bubble dynamics in oceanic ambient noise. Springer, DordrechtGoogle Scholar
  23. Roads C (2004) Microsound. The MIT Press, CambridgeGoogle Scholar
  24. Sato S, Mizutani K, Dobashi Y, Nishita T, Yamamoto T (2017) Feedback control of fire simulation based on computational fluid dynamics. Comput Animat Virtual Worlds 28(3–4):e1766Google Scholar
  25. Schreck C, Rohmer D, James DL, Hahmann S, Cani MP (2016) Real-time sound synthesis for paper material based on geometric analysis. In: ACM SIGGRAPH. pp 211–220Google Scholar
  26. Schwarz D (2011) State of the art in sound texture synthesis. In: Digital audio effects (DAFx-12). pp 151–171Google Scholar
  27. Schwarz D, Caramiaux B (2014) Interactive sound texture synthesis through semi-automatic user annotations. In: Lecture notes in computer science. pp 372–392Google Scholar
  28. Schwarz D, O’Leary S (2015) Smooth granular sound texture synthesis by control of timbral similarity. In: Sound and music computing. pp 471 – 476Google Scholar
  29. Schwarz D, Schnell N (2008) Descriptor-based sound texture sampling. In: Sound and music computing. pp 510–515Google Scholar
  30. Smith S, Ericson E (2009) Using immersive game-based virtual reality to teach fire-safety skills to children. Virtual Real 13(2):87–99Google Scholar
  31. Stam J (1999) Stable fluids. In: Proceedings of the 26th annual conference on computer graphics and interactive techniques. pp 121–128Google Scholar
  32. Stam J (2000) Interacting with smoke and fire in real time. Commun ACM 43(7):76–83Google Scholar
  33. Stam J, Stam J, Jensen HW (2001) Visual simulation of smoke. In: ACM SIGGRAPH. pp 15–22Google Scholar
  34. Wang K, Liu S (2018) Example-based synthesis for sound of ocean waves caused by bubble dynamics. Comput Animat Virtual Worlds 29(4):e1835Google Scholar
  35. Wang K, Cheng H, Liu S (2017) Efficient sound synthesis for natural scenes. In: IEEE virtual reality. pp 303–304Google Scholar
  36. Yin Q, Liu S (2018) Sounding solid combustibles: non-premixed flame sound synthesis for different solid combustibles. IEEE Trans Vis Comput Graph 24(2):1179–1189Google Scholar
  37. Yngve GD, O’Brien JF, Hodgins JK (2000) Animating explosions. In: ACM SIGGRAPH. pp 29–36Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Division of Intelligence and ComputingSchool of Computer Science and TechnologyTianjinPeople’s Republic of China

Personalised recommendations