ppOpen-HPC/pK-Open-HPC: Application Development Framework with Automatic Tuning (AT)

  • Kengo NakajimaEmail author
  • Masaharu Matsumoto
  • Masatoshi Kawai
  • Takahiro Katagiri
  • Takashi Arakawa
  • Hisashi Yashiro
  • Akihiro Ida


ppOpen-HPC and pK-Open-HPC are open source infrastructures for development and execution of large-scale scientific applications on post-petascale (pp) supercomputers with automatic tuning (AT). Both of ppOpen-HPC and pK-Open-HPC focus on parallel computers based on many-core architectures and consist of various types of libraries covering general procedures for scientific computations. The source code, developed on a PC with a single processor, is linked with these libraries, and the parallel code generated is optimized for post-petascale systems. In this article, recent achievements and progress of the ppOpen-HPC and pK-Open-HPC project are summarized.


ppOpen-HPC pK-Open-HPC Post-petascale systems Automatic tuning Parallel computing 



This work is supported by Core Research for Evolutional Science and Technology (CREST), the Japan Science and Technology Agency (JST), Japan, and the German Priority Programme 1648 Software for Exascale Computing (SPPEXA-II). Authors would like to thank Professor Gerhard Wellein (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany) and members of the ESSEX-II project.


  1. 1.
    ppOpen-HPC.: Accessed 23 Mar 2018
  2. 2.
    Nakajima, K., Satoh, M., Furumura, T., Okuda, H., Iwashita, T., Sakaguchi, H., Katagiri, T., Matsumoto, M., Ohshima, M., Jitsumoto, H., Arakawa, T., Mori, F., Kitayama, T., Ida, A., Matsuo, M.Y.: ppOpen-HPC: open source infrastructure for development and execution of large-scale scientific applications on post-peta-scale supercomputers with automatic tuning (AT). In: Optimization in the Real World -Towards Solving Real-Worlds Optimization Problems, Mathematics for Industry, vol. 13, pp. 15–35. Springer, Beijing (2015)Google Scholar
  3. 3.
    Joint Center for Advanced High Performance Computing (JCAHPC): Accessed 23 Mar 2018
  4. 4.
    Post-Peta CREST.: Accessed 23 Mar 2018
  5. 5.
    ESSEX.: Accessed 23 Mar 23 2018
  6. 6.
    SPPEXA.: Accessed 23 Mar 23 2018
  7. 7.
    Dezeeuw, D., Powell, K.G.: An adaptively refined Cartesian mesh solver for the Euler equations. J. Comput. Phys. 104, 56–68 (1993)CrossRefGoogle Scholar
  8. 8.
    Matsumoto, M., Mori, F., Ohshima, S., Jitsumoto, H., Katagiri, T., Nakajima, K.: Implementation and evaluation of an AMR framework for FDM applications. Proc. Comput. Sci. 29, 936–946 (2014)CrossRefGoogle Scholar
  9. 9.
    Usui, H., Nagara, A., Nunami, M., Matsumoto, M.: Development of a computational framework for block-based AMR simulations. Proc. Comput. Sci. 29, 2351–2359 (2014)CrossRefGoogle Scholar
  10. 10.
    Miyashita, M., Matsumoto, M., Kubota, K.: The development of adaptive mesh refinement technique for hybrid kinetic/fluid plasma simulation. In: Proceedings of 2015 International Conference on Numerical Simulation of Plasmas (ICNSP2015) (2015)Google Scholar
  11. 11.
    Usui, H., Kito, S., Nunami, M., Matsumoto, M.: Application of block-structured adaptive mesh refinement to particle simulation. Proc. Comput. Sci. 108, 2527–2536 (2017)CrossRefGoogle Scholar
  12. 12.
    Sakurai, T., Sugiura, H.: A projection method for generalized eigenvalue problems using numerical integration. J. Comput. Appl. Math. 159(1), 119–128 (2003)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Galgon, M., Lukas, K., Lang, B.: The FEAST algorithm for large eigenvalue problems. Proc. Parallel Appl. Math. Mech. (PAMM). 11(1), 747–748 (2011)CrossRefGoogle Scholar
  14. 14.
    Kawai, M., Ida, A., Nakajima, K.: Modified IC Preconditioner of CG method for ill-conditioned problems. IPSJ SIG, Technical Report 2017-HPC-158-9 (2017.) (in Japanese)Google Scholar
  15. 15.
    Kawai, M., Ida, A., Nakajima, K.: Hierarchical parallelization of multi-coloring algorithms for block IC, preconditioners. In: IEEE 19th International Conference on High Performance Computing and Communications (HPCC), pp. 138–145 (2017)Google Scholar
  16. 16.
    Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)CrossRefGoogle Scholar
  17. 17.
    Iwashita, T., Shimasaki, M.: Algebraic multicolor ordering for parallelized ICCG solver in finite-element analyses. IEEE Trans. Magn. 38(2), 429–432 (2002)CrossRefGoogle Scholar
  18. 18.
    Davis, T.A., Hu, Y.: The University of Florida sparse matrix collection. ACM Trans. Math. Softw. (TOMS). 38(1), 1–25 (2011)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Katagiri, T., Ohshima, S., Matsumoto, M.: Directive-based auto-tuning for the finite difference method on the Xeon Phi. In: IEEE Proceedings of IPDPSW2015, pp. 1221–1230 (2015)Google Scholar
  20. 20.
    Katagiri, T., Ohshima, S., Matsumoto, M.: Auto-tuning on NUMA and many-core environments with an FDM code. In: IEEE Proceedings of IPDPSW2017, pp. 1399–1407 (2017)Google Scholar
  21. 21.
    Washington, W.M., Buja, L., Craig, A.: The computational future for climate and earth sys-tem models: on the path to petaflop and beyond. Phil. Trans. R. Soc. A. 367, 833–846 (2009)CrossRefGoogle Scholar
  22. 22.
    Satoh, M., Tomita, H., Yashiro, H., Miura, H., Kodama, C., Seiki, T., Noda, A., Yamada, Y., Goto, D., Sawada, M., Miyoshi, T., Niwa, Y., Hara, M., Ohno, T., Iga, S., Arakawa, T., Inoue, T., Kubokawa, H.: The non-hydrostatic icosahedral atmospheric model: description and development. Prog Earth Planet Sci. 1, (2014).
  23. 23.
    Hasumi, H.: CCSR Ocean Component Model (COCO) Version 4.0. Accessed 23 Mar 2018
  24. 24.
    Jones, P.H.: First and second order conservative remapping schemes for grids in spherical coordinates. Mon. Weather Rev. 127, 2204–2210 (1999)CrossRefGoogle Scholar
  25. 25.
    Miyakawa, T., Yashiro, H., Suzuki, T., Tatebe, H., Satoh, M.: A Madden-Julian oscillation-event remotely accelerates ocean upwelling to abruptly terminate the 1997/1998 super El Nino. Geophys. Res. Lett. 44, 9489 (2017). CrossRefGoogle Scholar
  26. 26.
    Iwashita, T., Ida, A., Mifune, T., Takahashi, Y.: Software framework for parallel BEM analyses with H-matrices using MPI and OpenMP. Proc. Comput. Sci. 108C, 2200–2209 (2017)CrossRefGoogle Scholar
  27. 27.
    Ida, A., Iwashita, T., Mifune, T., Takahashi, Y.: Parallel hierarchical matrices with adaptive cross approximation on symmetric multiprocessing clusters. J. Inf. Process. 22(4), 642–650 (2014)Google Scholar
  28. 28.
    Ida, A., Nakashima, H., Kawai, M.: Parallel hierarchical matrices with block low-rank representation on distributed memory computer systems. In: ACM Proceedings of the International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia 2018), pp. 232–240 (2018)CrossRefGoogle Scholar
  29. 29.
    Ida, A.: Lattice H-matrices on distributed-memory systems. In: Proceedings of IEEE International Parallel and Distributed Processing Symposium (IPDPS) (2018). in pressGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Kengo Nakajima
    • 1
    Email author
  • Masaharu Matsumoto
    • 1
  • Masatoshi Kawai
    • 1
  • Takahiro Katagiri
    • 2
  • Takashi Arakawa
    • 3
  • Hisashi Yashiro
    • 4
  • Akihiro Ida
    • 1
  1. 1.The University of TokyoTokyoJapan
  2. 2.Nagoya UniversityNagoyaJapan
  3. 3.Research Organization for Information Science and Technology (RIST)TokyoJapan
  4. 4.RIKEN Advanced Institute for Computational Science (AICS)KobeJapan

Personalised recommendations