Automatic synthesis of explosion sound synchronized with animation

Abstract

Sound is an essential element for enhancing the realness of virtual world. Currently, there are some remarkable works on the synthesis of fluid sound, such as fire and water. However, little attention has been paid to synthesizing explosion sound. This paper proposes an automatic method for synthesizing explosion sounds that are synchronized with the visual phenomena of explosion animations, including fireball generation and flame combustion. Such two types of visual animation correspond to two types of sound, which we name as explosive sound and combustion noise, respectively. For the synthesis of explosive sound, firstly, the occurrence time and duration of explosion sound are determined according to the dynamic process of fuel consumption, and then, the corresponding explosive sound is extracted from the recording examples according to the high-frequency content. For the combustion noise, we propose a synthesis method of combustion noise on the basis of the timbre similarity between sound examples and low-frequency combustion noise generated by a physical method. Finally, the two types of sound are blended respecting the occurrence and duration of the explosions and combustions parts detected in the visual stream. Our experiments and the user study show the results of our method and demonstrate the effectiveness of our method.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

References

  1. Agostino DS (1999) Synthesis of environmental sound textures by iterated nonlinear functions. In: Digital audio effects. pp 109–117

  2. An SS, James DL, Marschner S (2012) Motion-driven concatenative synthesis of cloth sounds. ACM Trans Graph 31(4):1–10

    Article  Google Scholar 

  3. Chadwick JN, James DL (2011) Animating fire with sound. ACM Trans Graph 30(4):84–92

    Article  Google Scholar 

  4. Crighton D, Dowling A, Ffowcs-Williams J, Heckl M, Leppington F, Bartram JF (1992) Modern methods in analytical acoustics. Springer, Berlin

    Book  Google Scholar 

  5. Dobashi Y, Kaneda K, Yamashita H, Okita T, Nishita T (2000) A simple, efficient method for realistic animation of clouds. In: ACM SIGGRAPH. pp 19–28

  6. Dobashi Y, Yamamoto T, Nishita T (2003) Real-time rendering of aerodynamic sound using sound textures based on computational fluid dynamics. In: ACM SIGGRAPH. pp 732–740

  7. Dobashi Y, Yamamoto T, Nishita T (2004) Synthesizing sound from turbulent field using sound textures for interactive fluid simulation. Comput Graph Forum 23(3):539–545

    Article  Google Scholar 

  8. Doel KVD (2005) Physically-based models for liquid sounds. ACM Trans Appl Percept 2(4):534–546

    Article  Google Scholar 

  9. Dubnov S, Barjoseph Z, Ran EY, Lischinski D, Werman M (2002) Synthesizing sound textures through wavelet tree learning. IEEE Comput Graph Appl 22(4):38–48

    Article  Google Scholar 

  10. Feldman BE, O’brien JF, Arikan O (2003) Animating suspended particle explosions. ACM Trans Graph 22(3):708–715

    MATH  Article  Google Scholar 

  11. Feng G, Liu S (2018) Detail-preserving sph fluid control with deformation constraints. Comput Animat Virtual Worlds 29(1):e1781

    Article  Google Scholar 

  12. Ihme M, Pitsch H, Bodony D (2009) Radiation of noise in turbulent non-premixed flames. Proc Combust Inst 32(1):1545–1553

    Article  Google Scholar 

  13. Kersten S, Purwins H (2013) Fire texture sound re-synthesis using sparse decomposition and noise modelling. In: Digital audio effects. pp 1–5

  14. Liu S, Xiong Y (2013) Fast and stable simulation of virtual water scenes with interactions. Virtual Real 17(1):77–88

    Article  Google Scholar 

  15. Liu S, Yu Z (2015) Sounding fire for immersive virtual reality. Virtual Real 19(3–4):291–302

    Article  Google Scholar 

  16. Longuethiggins MS (1990) An analytic model of sound production by raindrops. Fluid Mech 214:395–410

    MathSciNet  Article  Google Scholar 

  17. Marelli D, Aramaki M, Kronland-Martinet R, Verron C (2012) An efficient time–frequency method for synthesizing noisy sounds with short transients and narrow spectral components. IEEE Trans Audio Speech Lang Process 20(4):1400–1408

    Article  Google Scholar 

  18. Moss W, Yeh H, Hong JM, Lin MC, Manocha D (2010) Sounding liquids: automatic sound synthesis from fluid simulation. ACM Trans Graph 29(3):1–13

    Article  Google Scholar 

  19. Nguyen DQ, Fedkiw R, Jensen HW (2002) Physically based modeling and animation of fire. ACM Trans Graph 21(3):721–728

    Article  Google Scholar 

  20. O’Brien JF, Bargteil AW, Hodgins JK (2002) Graphical modeling and animation of ductile fracture. ACM Trans Graph 21(3):291–294

    Article  Google Scholar 

  21. Powell A (2003) Theory of vortex sound. Cambridge University Press, Cambridge

    Google Scholar 

  22. Prosperetti A (1988) Bubble dynamics in oceanic ambient noise. Springer, Dordrecht

    Book  Google Scholar 

  23. Roads C (2004) Microsound. The MIT Press, Cambridge

    Google Scholar 

  24. Sato S, Mizutani K, Dobashi Y, Nishita T, Yamamoto T (2017) Feedback control of fire simulation based on computational fluid dynamics. Comput Animat Virtual Worlds 28(3–4):e1766

    Article  Google Scholar 

  25. Schreck C, Rohmer D, James DL, Hahmann S, Cani MP (2016) Real-time sound synthesis for paper material based on geometric analysis. In: ACM SIGGRAPH. pp 211–220

  26. Schwarz D (2011) State of the art in sound texture synthesis. In: Digital audio effects (DAFx-12). pp 151–171

  27. Schwarz D, Caramiaux B (2014) Interactive sound texture synthesis through semi-automatic user annotations. In: Lecture notes in computer science. pp 372–392

  28. Schwarz D, O’Leary S (2015) Smooth granular sound texture synthesis by control of timbral similarity. In: Sound and music computing. pp 471 – 476

  29. Schwarz D, Schnell N (2008) Descriptor-based sound texture sampling. In: Sound and music computing. pp 510–515

  30. Smith S, Ericson E (2009) Using immersive game-based virtual reality to teach fire-safety skills to children. Virtual Real 13(2):87–99

    Article  Google Scholar 

  31. Stam J (1999) Stable fluids. In: Proceedings of the 26th annual conference on computer graphics and interactive techniques. pp 121–128

  32. Stam J (2000) Interacting with smoke and fire in real time. Commun ACM 43(7):76–83

    Article  Google Scholar 

  33. Stam J, Stam J, Jensen HW (2001) Visual simulation of smoke. In: ACM SIGGRAPH. pp 15–22

  34. Wang K, Liu S (2018) Example-based synthesis for sound of ocean waves caused by bubble dynamics. Comput Animat Virtual Worlds 29(4):e1835

    Article  Google Scholar 

  35. Wang K, Cheng H, Liu S (2017) Efficient sound synthesis for natural scenes. In: IEEE virtual reality. pp 303–304

  36. Yin Q, Liu S (2018) Sounding solid combustibles: non-premixed flame sound synthesis for different solid combustibles. IEEE Trans Vis Comput Graph 24(2):1179–1189

    Article  Google Scholar 

  37. Yngve GD, O’Brien JF, Hodgins JK (2000) Animating explosions. In: ACM SIGGRAPH. pp 29–36

Download references

Acknowledgements

Funding was provided by Natural Science Foundation of China (Grant Nos. 61672375 and 61170118).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Shiguang Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (wmv 9315 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Gao, S. Automatic synthesis of explosion sound synchronized with animation. Virtual Reality 24, 469–481 (2020). https://doi.org/10.1007/s10055-019-00408-7

Download citation

Keywords

  • Audiovisual synchronization
  • Explosive sound
  • Combustion noise
  • Sound synthesis
  • Immersive virtual reality