Enabling Explorative Visualization with Full Temporal Resolution via In Situ Calculation of Temporal Intervals

  • Nicole MarsagliaEmail author
  • Shaomeng Li
  • Hank ChildsEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11203)


We explore a technique for saving full spatiotemporal simulation data for visualization and analysis. While such data is typically prohibitively large to store, we consider an in situ reduction approach that takes advantage of temporal coherence to make storage sizes tractable in some cases. Rather than limiting our data reduction to individual time slices or time windows, our algorithms act on individual locations and save data to disk as temporal intervals. Our results show that the efficacy of piecewise approximations varies based on the desired error bound guarantee and tumultuousness of the time-varying data. We ran our in situ algorithms for one simulation and experienced promising results compared to the traditional paradigm. We also compared the results to two data reduction operators: wavelets and SZ.


  1. 1.
    Hydrodynamics Challenge Problem, Lawrence Livermore National Laboratory. Technical report LLNL-TR-490254Google Scholar
  2. 2.
    Baker, A.H., Xu, H., Hammerling, D.M., Li, S., Clyne, J.P.: Toward a multi-method approach: lossy data compression for climate simulation data. In: Kunkel, J.M., Yokota, R., Taufer, M., Shalf, J. (eds.) ISC High Performance 2017. LNCS, vol. 10524, pp. 30–42. Springer, Cham (2017). Scholar
  3. 3.
    Baker, A.H., et al.: Evaluating lossy data compression on climate simulation data within a large ensemble. Geosci. Model Dev. 9(12), 4381–4403 (2016)CrossRefGoogle Scholar
  4. 4.
    Baker, A.H., et al.: A methodology for evaluating the impact of data compression on climate simulation data. In: Proceedings of the 23rd International Symposium on High-performance Parallel and Distributed Computing, HPDC 2014, pp. 203–214. ACM, New York (2014)Google Scholar
  5. 5.
    Bertram, M., Duchaineau, M.A., Hamann, B., Joy, K.I.: Bicubic subdivision-surface wavelets for large-scale isosurface representation and visualization. In: Proceedings of the Conference on Visualization 2000, pp. 389–396. IEEE Computer Society Press (2000)Google Scholar
  6. 6.
    Burtscher, M., Mukka, H., Yang, A., Hesaaraki, F.: Real-time synthesis of compression algorithms for scientific data. In: International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2016, pp. 264–275, November 2016Google Scholar
  7. 7.
    Burtscher, M., Ratanaworabhan, P.: FPC: a high-speed compressor for double-precision floating-point data. IEEE Trans. Comput. 58(1), 18–31 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Chang, C., et al.: Compressed ion temperature gradient turbulence in diverted tokamak edge. Phys. Plasmas (1994-present) 16(5), 056108 (2009)CrossRefGoogle Scholar
  9. 9.
    Chen, D., Chiang, Y.J., Memon, N., Wu, X.: Lossless geometry compression for steady-state and time-varying irregular grids. In: Santos, B.S., Ertl, T., Joy, K. (eds.) EUROVIS - Eurographics/IEEE VGTC Symposium on Visualization. The Eurographics Association (2006)Google Scholar
  10. 10.
    Clyne, J., Mininni, P., Norton, A., Rast, M.: Interactive desktop analysis of high resolution simulations: application to turbulent plume dynamics and current sheet formation. New J. Phys. 9(8), 301 (2007)CrossRefGoogle Scholar
  11. 11.
    Clyne, J., Rast, M.: A prototype discovery environment for analyzing and visualizing terascale turbulent fluid flow simulations. In: Electronic Imaging 2005, pp. 284–294. International Society for Optics and Photonics (2005)Google Scholar
  12. 12.
    Di, S., Cappello, F.: Fast error-bounded lossy HPC data compression with SZ. In: Proceedings of the IPDPS. IEEE (2016)Google Scholar
  13. 13.
    Fernandes, O., Frey, S., Sadlo, F., Ertl, T.: Space-time volumetric depth images for in-situ visualization. In: 2014 IEEE 4th Symposium on Large Data Analysis and Visualization (LDAV), pp. 59–65, November 2014Google Scholar
  14. 14.
    Fout, N., Ma, K.L.: An adaptive prediction-based approach to lossless compression of floating-point volume data. IEEE Trans. Vis. Comput. Graph. 18(12), 2295–2304 (2012)CrossRefGoogle Scholar
  15. 15.
    Gomez, L.A.B., Cappello, F.: Improving floating point compression through binary masks. In: 2013 IEEE International Conference on Big Data, pp. 326–331. IEEE (2013)Google Scholar
  16. 16.
    Guthe, S., Strasser, W.: Real-time decompression and visualization of animated volume data. In: Proceedings of IEEE Visualization (VIS 2001), pp. 349–572, October 2001Google Scholar
  17. 17.
    Hübbe, N., Kunkel, J.: Reducing the HPC-datastorage footprint with mafisc—multidimensional adaptive filtering improved scientific data compression. Comput. Sci. - Res. Dev. 28(2), 231–239 (2013)CrossRefGoogle Scholar
  18. 18.
    Ihm, I., Park, S.: Wavelet-based 3D compression scheme for interactive visualization of very large volume data. In: Computer Graphics Forum, vol. 18, pp. 3–15. Wiley Online Library (1999)Google Scholar
  19. 19.
    Iverson, J., Kamath, C., Karypis, G.: Fast and effective lossy compression algorithms for scientific datasets. In: Kaklamanis, C., Papatheodorou, T., Spirakis, P.G. (eds.) Euro-Par 2012. LNCS, vol. 7484, pp. 843–856. Springer, Heidelberg (2012). Scholar
  20. 20.
    Keogh, E., Chu, S., Hart, D., Pazzani, M.: An online algorithm for segmenting time series. In: Proceedings of the 2001 IEEE International Conference on Data Mining, pp. 289–296 (2001)Google Scholar
  21. 21.
    Kim, T.Y., Shin, Y.G.: An efficient wavelet-based compression method for volume rendering. In: Proceedings of the Seventh Pacific Conference on Computer Graphics and Applications, pp. 147–156. IEEE (1999)Google Scholar
  22. 22.
    Lakshminarasimhan, S., et al.: Compressing the incompressible with ISABELA: in-situ reduction of spatio-temporal data. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011. LNCS, vol. 6852, pp. 366–379. Springer, Heidelberg (2011). Scholar
  23. 23.
    Lakshminarasimhan, S., et al.: ISABELA for effective in situ compression of scientific data. Concurr. Comput.: Pract. Exp. 25(4), 524–540 (2013)CrossRefGoogle Scholar
  24. 24.
    Lee, D., Sim, A., Choi, J., Wu, K.: Novel data reduction based on statistical similarity. In: Proceedings of the 28th International Conference on Scientific and Statistical Database Management, p. 21. ACM (2016)Google Scholar
  25. 25.
    Lehmann, H., Jung, B.: In-situ multi-resolution and temporal data compression for visual exploration of large-scale scientific simulations. In: 2014 IEEE 4th Symposium on Large Data Analysis and Visualization (LDAV), pp. 51–58, November 2014Google Scholar
  26. 26.
    Li, S., Gruchalla, K., Potter, K., Clyne, J., Childs, H.: Evaluating the efficacy of wavelet configurations on turbulent-flow data. In: 2015 IEEE 5th Symposium on Large Data Analysis and Visualization (LDAV), pp. 81–89 (2015)Google Scholar
  27. 27.
    Li, S., Larsen, M., Clyne, J., Childs, H.: Performance impacts of in situ wavelet compression on scientific simulations. In: Proceedings of the In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization Workshop, ISAV 2017. ACM, New York (2017)Google Scholar
  28. 28.
    Li, S., Marsaglia, N., Chen, V., Sewell, C., Clyne, J., Childs, H.: Achieving portable performance for wavelet compression using data parallel primitives. In: Telea, A., Bennett, J. (eds.) Eurographics Symposium on Parallel Graphics and Visualization. The Eurographics Association (2017).
  29. 29.
    Li, S., Marsaglia, N., Garth, C., Woodring, J., Clyne, J., Childs, H.: Data reduction techniques for simulation, visualization and data analysis. Comput. Graph. Forum, March 2018. Scholar
  30. 30.
    Li, S., Sane, S., Orf, L., Mininni, P., Clyne, J., Childs, H.: Spatiotemporal wavelet compression for visualization of scientific simulation data. In: 2017 IEEE International Conference on Cluster Computing (CLUSTER), pp. 216–227, September 2017Google Scholar
  31. 31.
    Lindstrom, P.: Fixed-rate compressed floating-point arrays. IEEE Trans. Vis. Comput. Graph. 20(12), 2674–2683 (2014)CrossRefGoogle Scholar
  32. 32.
    Lindstrom, P.: Error distributions of lossy floating-point compressors. In: Joint Statistical Meetings, October 2017Google Scholar
  33. 33.
    Lindstrom, P., Isenburg, M.: Fast and efficient compression of floating-point data. IEEE Trans. Vis. Comput. Graph. 12(5), 1245–1250 (2006)CrossRefGoogle Scholar
  34. 34.
    Ma, J., Murphy, D., O’Mathuna, C., Hayes, M., Provan, G.: Visualizing uncertainty in multi-resolution volumetric data using marching cubes. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, pp. 489–496. ACM (2012)Google Scholar
  35. 35.
    Mininni, P., Alexakis, A., Pouquet, A.: Large-scale flow effects, energy transfer, and self-similarity on turbulence. Phys. Rev. E 74(1), 016303 (2006)CrossRefGoogle Scholar
  36. 36.
    Olanda, R., Pérez, M., Orduña, J.M., Rueda, S.: Terrain data compression using wavelet-tiled pyramids for online 3D terrain visualization. Int. J. Geogr. Inf. Sci. 28(2), 407–425 (2014)CrossRefGoogle Scholar
  37. 37.
    Orf, L., Wilhelmson, R., Wicker, L.: Visualization of a simulated long-track EF5 tornado embedded within a supercell thunderstorm. Parallel Comput. 55, 28–34 (2016)CrossRefGoogle Scholar
  38. 38.
    Rodler, F.F.: Wavelet based 3D compression with fast random access for very large volume data. In: Seventh Pacific Conference on Computer Graphics and Applications, pp. 108–117. IEEE (1999)Google Scholar
  39. 39.
    Sakai, R., Sasaki, D., Obayashi, S., Nakahashi, K.: Wavelet-based data compression for flow simulation on block-structured Cartesian mesh. Int. J. Numer. Methods Fluids 73(5), 462–476 (2013)CrossRefGoogle Scholar
  40. 40.
    Schendel, E.R., et al.: ISOBAR preconditioner for effective and high-throughput lossless data compression. In: Proceedings - International Conference on Data Engineering, pp. 138–149 (2012)Google Scholar
  41. 41.
    Tao, D., Di, S., Chen, Z., Capello, F.: Significantly improving lossy compression for scientific data sets based on multidimensional prediction and error-controlled quantization. In: IEEE International Parallel and Distributed Processing Symposium (2017, to appear)Google Scholar
  42. 42.
    Treib, M., Burger, K., Reichl, F., Meneveau, C., Szalay, A., Westermann, R.: Turbulence visualization at the terascale on desktop PCs. IEEE Trans. Vis. Comput. Graph. 18(12), 2169–2177 (2012)CrossRefGoogle Scholar
  43. 43.
    Woodring, J., Mniszewski, S., Brislawn, C., DeMarle, D., Ahrens, J.: Revisiting wavelet compression for large-scale climate data using jpeg 2000 and ensuring data precision. In: IEEE Symposium on Large Data Analysis and Visualization (LDAV), pp. 31–38. IEEE (2011)Google Scholar
  44. 44.
    Yang, A., Mukka, H., Hesaaraki, F., Burtscher, M.: MPC: a massively parallel compression algorithm for scientific data. In: 2015 IEEE International Conference on Cluster Computing, pp. 381–389, September 2015Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.University of OregonEugeneUSA
  2. 2.National Center for Atmospheric ResearchBoulderUSA

Personalised recommendations