Abstract
This paper presents an in situ framework focused on time-varying simulations, and uses a novel temporal buffer for storing simulation results sampled at user-defined intervals. This framework has been designed to provide flexible data processing and visualization capabilities in modern HPC operational environments composed of powerful front-end systems, for pre-and post-processing purposes, along with traditional back-end HPC systems. The temporal buffer is implemented using the functionalities provided by Open Address Space (OpAS) library, which enables asynchronous one-sided communication from outside processes to any exposed memory region on the simulator side. This buffer can store time-varying simulation results, and can be processed via in situ approaches with different proximities. We present a prototype of our framework, and code integration process with a target simulation code. The proposed in situ framework utilizes separate files to describe the initialization and execution codes, which are in the form of Python scripts. This framework also enables the runtime modification of these Python-based files, thus providing greater flexibility to the users, not only for data processing, such as visualization and analysis, but also for the simulation steering.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ayachit, U., et al.: The SENSEI generic in situ interface. In: Proceedings of the 2nd Workshop on In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization, ISAV 2016, pp. 40–44. IEEE Press, Piscataway (2016). https://doi.org/10.1109/ISAV.2016.13
Bauer, A.C., et al.: In situ methods, infrastructures, and applications on high performance computing platforms. Comput. Graph Forum 35(3), 577–597 (2016). https://doi.org/10.1111/cgf.12930. https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.12930
Fareed, H., Singler, J.R.: Error Analysis of an Incremental POD Algorithm for PDE Simulation Data. ArXiv e-prints, March 2018
Hunter, J.D.: Matplotlib: a 2D graphics environment. Comput Sci Eng. 9(3), 90–95 (2007). https://doi.org/10.1109/MCSE.2007.55
Research Institute for Information Technology, K.U.: Supercomputer System ITO. https://www.cc.kyushu-u.ac.jp/scp/eng/system/01_into.html. Accessed 15 May 2018
Kress, J., Klasky, S., Podhorszki, N., Choi, J., Childs, H., Pugmire, D.: Loosely coupled in situ visualization: a perspective on why it’s here to stay. In: Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, pp. 1-6. ISAV 2015. ACM, New York (2015). https://doi.org/10.1145/2828612.2828623, http://doi.acm.org/10.1145/2828612.2828623
Nanri, T.: Proposal of interface for runtime memory manipulation of applications via PGAS-based communication library. In: Workshop on PGAS programming models: Experiences and Implementations, HPC Asia 2018, 31 January 2018
Ono, K., Kawashima, Y., Kawanabe, T.: Data centric framework for large-scale high-performance parallel computation. Procedia Comput. Sci. 29, 2336–2350 (2014). https://doi.org/10.1016/j.procs.2014.05.218, http://www.sciencedirect.com/science/article/pii/S1877050914003950. 2014 International Conference on Computational Science
Uchida, T.: LES investigation of terrain-induced turbulence in complex terrain and economic effects of wind turbine control. Energies 11(6), 1530 (2018). https://doi.org/10.3390/en11061530
Acknowledgement
This research has used the computational resources of the K computer at RIKEN Center for Computational Science (R-CCS) in Kobe, Japan. This work is partially supported by the “Joint Usage/Research Center for Interdisciplinary Large-scale Information Infrastructures” in Japan (Project ID: jh180060-NAH), and also by the Japanese Ministry of Education, Culture, Sports, Science and Technology (MEXT) as a social and scientific priority issue (Development of Innovative Design and Production Processes that Lead the Way for the Manufacturing Industry in the Near Future) to be tackled by using the post-K supercomputer.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Ono, K. et al. (2018). Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets. In: Yokota, R., Weiland, M., Shalf, J., Alam, S. (eds) High Performance Computing. ISC High Performance 2018. Lecture Notes in Computer Science(), vol 11203. Springer, Cham. https://doi.org/10.1007/978-3-030-02465-9_17
Download citation
DOI: https://doi.org/10.1007/978-3-030-02465-9_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-02464-2
Online ISBN: 978-3-030-02465-9
eBook Packages: Computer ScienceComputer Science (R0)