Binning Based Data Reduction for Vector Field Data of a Particle-In-Cell Fusion Simulation

  • James KressEmail author
  • Jong Choi
  • Scott Klasky
  • Michael Churchill
  • Hank Childs
  • David Pugmire
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11203)


With this work, we explore the feasibility of using in situ data binning techniques to achieve significant data reductions for particle data, and study the associated errors for several post-hoc analysis techniques. We perform an application study in collaboration with fusion simulation scientists on data sets up to 489 GB per time step. We consider multiple ways to carry out the binning, and determine which techniques work the best for this simulation. With the best techniques we demonstrate reduction factors as large as 109x with low error percentage.


In situ Data reduction Visualization 



This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725.


  1. 1.
    Ahern, S., et al.: Scientific discovery at the exascale. In: Report from the DOE ASCR 2011 Workshop on Exascale Data Management (2011)Google Scholar
  2. 2.
    Bauer, A.C., et al.: In Situ Methods, Infrastructures, and Applications on High Performance Computing Platforms, a State-of-the-art (STAR) Report. In: Computer Graphics Forum, Proceedings of Eurovis 2016, vol. 35, no. 3, June 2016. lBNL-1005709Google Scholar
  3. 3.
    Chang, C., et al.: Compressed ion temperature gradient turbulence in diverted tokamak edge. Phys. Plasmas (1994-Present) 16(5), 056108 (2009)CrossRefGoogle Scholar
  4. 4.
    Childs, H., et al.: Extreme scaling of production visualization software on diverse architectures. IEEE Comput. Graph. Appl. 30(3), 22–31 (2010)CrossRefGoogle Scholar
  5. 5.
    Childs, H., et al.: Visualization at extreme scale concurrency. In: Bethel, E.W., Childs, H., Hansen, C. (eds.) High Performance Visualization: Enabling Extreme-Scale Scientific Insight. CRC Press, Boca Raton (2012)Google Scholar
  6. 6.
    Fabian, N., et al.: The paraview coprocessing library: a scalable, general purpose in situ visualization library. In: 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV), pp. 89–96. IEEE (2011)Google Scholar
  7. 7.
    Kress, J., Churchill, R.M., Klasky, S., Kim, M., Childs, H., Pugmire, D.: Preparing for in situ processing on upcoming leading-edge supercomputers. Supercomput. Front. Innov. 3(4), 49–65 (2016)Google Scholar
  8. 8.
    Kress, J., Pugmire, D., Klasky, S., Childs, H.: Visualization and analysis requirements for in situ processing for a large-scale fusion simulation code. In: Proceedings of the 2nd Workshop on In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization, pp. 45–50. IEEE Press (2016)Google Scholar
  9. 9.
    Liu, Q., et al.: Hello adios: the challenges and lessons of developing leadership class i/o frameworks. Concurr. Comput.: Pract. Exp. 26(7), 1453–1473 (2014). Scholar
  10. 10.
    Lo, L., Sewell, C., Ahrens, J.P.: Piston: a portable cross-platform framework for data-parallel visualization operators. In: EGPGV, pp. 11–20 (2012)Google Scholar
  11. 11.
    Lofstead, J.F., Klasky, S., Schwan, K., Podhorszki, N., Jin, C.: Flexible io and integration for scientific codes through the adaptable io system (adios). In: Proceedings of the 6th International Workshop on Challenges of Large Applications in Distributed Environments, CLADE 2008, pp. 15–24. ACM, New York (2008).
  12. 12.
    Meredith, J.S., Ahern, S., Pugmire, D., Sisneros, R.: EAVL: the extreme-scale analysis and visualization library. In: Eurographics Symposium on Parallel Graphics and Visualization, pp. 21–30. The Eurographics Association (2012)Google Scholar
  13. 13.
    Moreland, K., Ayachit, U., Geveci, B., Ma, K.L.: Dax toolkit: a proposed framework for data analysis and visualization at extreme scale. In: 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV), pp. 97–104, October 2011Google Scholar
  14. 14.
    Moreland, K., et al.: VTK-m: accelerating the visualization toolkit for massively threaded architectures. IEEE Comput. Graph. Appl. (CG&A) 36(3), 48–58 (2016)CrossRefGoogle Scholar
  15. 15.
    Neuroth, T., Sauer, F., Wang, W., Ethier, S., Ma, K.L.: Scalable visualization of discrete velocity decompositions using spatially organized histograms. In: 2015 IEEE 5th Symposium on Large Data Analysis and Visualization (LDAV), pp. 65–72. IEEE (2015)Google Scholar
  16. 16.
    Oldfield, R.A., Widener, P., Maccabe, A.B., Ward, L., Kordenbrock, T.: Effcient data-movement for lightweight i/o. In: 2006 IEEE International Conference on Cluster Computing, pp. 1–9, September 2006.
  17. 17.
    Pugmire, D., Kress, J., Meredith, J., Podhorszki, N., Choi, J., Klasky, S.: Towards scalable visualization plugins for data staging workows. In: Big Data Analytics: Challenges and Opportunities (BDAC 2014) Workshop at Supercomputing Conference, November 2014Google Scholar
  18. 18.
    Reach, C., North, C.: Bandlimited olap cubes for interactive big data visualization. In: 2015 IEEE 5th Symposium on Large Data Analysis and Visualization (LDAV), pp. 107–114. IEEE (2015)Google Scholar
  19. 19.
    Schatz, K., Müller, C., Krone, M., Schneider, J., Reina, G., Ertl, T.: Interactive visual exploration of a trillion particles. In: 2016 IEEE 6th Symposium on Large Data Analysis and Visualization (LDAV), pp. 56–64. IEEE (2016)Google Scholar
  20. 20.
    Tchoua, R., et al.: Adios visualization schema: a first step towards improving interdisciplinary collaboration in high performance computing. In: 2013 IEEE 9th International Conference on eScience (eScience), pp. 27–34. IEEE (2013)Google Scholar
  21. 21.
    Vishwanath, V., Hereld, M., Papka, M.: Toward simulation-time data analysis and i/o acceleration on leadership-class systems. In: 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV), pp. 9–14 (2011).
  22. 22.
    Whitlock, B., Favre, J.M., Meredith, J.S.: Parallel in situ coupling of simulation with a fully featured visualization system. In: Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization, pp. 101–109. Eurographics Association (2011)Google Scholar
  23. 23.
    Ye, Y.C., et al.: In situ generated probability distribution functions for interactive post hoc visualization and analysis. In: 2016 IEEE 6th Symposium on Large Data Analysis and Visualization (LDAV), pp. 65–74. IEEE (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • James Kress
    • 1
    • 2
    Email author
  • Jong Choi
    • 1
  • Scott Klasky
    • 1
  • Michael Churchill
    • 3
  • Hank Childs
    • 2
  • David Pugmire
    • 1
  1. 1.Oak Ridge National LaboratoryOak RidgeUSA
  2. 2.University of OregonEugeneUSA
  3. 3.Princeton Plasma Physics LaboratoryPrincetonUSA

Personalised recommendations