Exploring Scientific Application Performance Using Large Scale Object Storage

  • Steven Wei-der ChienEmail author
  • Stefano Markidis
  • Rami Karim
  • Erwin Laure
  • Sai Narasimhamurthy
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11203)


One of the major performance and scalability bottlenecks in large scientific applications is parallel reading and writing to supercomputer I/O systems. The usage of parallel file systems and consistency requirements of POSIX, that all the traditional HPC parallel I/O interfaces adhere to, pose limitations to the scalability of scientific applications. Object storage is a widely used storage technology in cloud computing and is more frequently proposed for HPC workload to address and improve the current scalability and performance of I/O in scientific applications. While object storage is a promising technology, it is still unclear how scientific applications will use object storage and what the main performance benefits will be. This work addresses these questions, by emulating an object storage used by a traditional scientific application and evaluating potential performance benefits. We show that scientific applications can benefit from the usage of object storage on large scales.


Scientific applications Object storage Parallel I/O HPC HDF5 



Funding for the work is received from the European Commission H2020 program, Grant Agreement No. 671500 (SAGE).


  1. 1.
  2. 2.
    Lustre: A scalable, high-performance file system. Cluster File Systems Inc., White Paper (2002)Google Scholar
  3. 3.
    IEEE standard for information technology-portable operating system interface (POSIX(R)) base specifications, issue 7. IEEE Std 1003.1-2017 (Revision of IEEE Std 1003.1-2008) (2018)Google Scholar
  4. 4.
    Bergman, K., et al.: Exascale computing study: technology challenges in achieving exascale systems. Defense Advanced Research Projects Agency Information Processing Techniques Office (DARPA IPTO), Technical Report 15 (2008)Google Scholar
  5. 5.
    Borrill, J., Oliker, L., Shalf, J., Shan, H.: Investigation of leading HPC I/O performance using a scientific-application derived benchmark. In: Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, SC 2007, pp. 1–12 (2007)Google Scholar
  6. 6.
    Breitenfeld, M.S., et al.: DAOS for extreme-scale systems in scientific applications. arXiv preprint arXiv:1712.00423 (2017)
  7. 7.
    Byna, S., Chaarawi, M., Koziol, Q., Mainzer, J., Willmore, F.: Tuning HDF5 subfiling performance on parallel file systems (2017)Google Scholar
  8. 8.
    Carns, P., Latham, R., Ross, R., Iskra, K., Lang, S., Riley, K.: 24/7 characterization of petascale I/O workloads. In: IEEE International Conference on Cluster Computing and Workshops, CLUSTER 2009, pp. 1–10. IEEE (2009)Google Scholar
  9. 9.
    Carns, P.H., Ligon III, W.B., Ross, R.B., Thakur, R.: PVFS: a parallel file system for Linux clusters. In: Proceedings of the 4th Annual Linux Showcase and Conference, vol. 4. p. 28 (2000).
  10. 10.
  11. 11.
    Factor, M., Meth, K., Naor, D., Rodeh, O., Satran, J.: Object storage: the future building block for storage systems. In: Local to Global Data Interoperability-Challenges and Technologies, pp. 119–123. IEEE (2005)Google Scholar
  12. 12.
    Folk, M., Heber, G., Koziol, Q., Pourmal, E., Robinson, D.: An overview of the HDF5 technology suite and its applications. In: Proceedings of the EDBT/ICDT2011 Workshop on Array Databases, pp. 36–47. ACM (2011)Google Scholar
  13. 13.
    Li, J., et al.: Parallel netCDF: a high-performance scientific I/O interface. In: 2003 ACM/IEEE Conference on Supercomputing, p. 39 (2003)Google Scholar
  14. 14.
    Mohindra, A., Devarakonda, M.: Distributed token management in Calypso file system. In: Proceedings of 1994 6th IEEE Symposium on Parallel and Distributed Processing, pp. 290–297 (1994)Google Scholar
  15. 15.
    Narasimhamurthy, S., et al.: The SAGE project: a storage centric approach for exascale computing. In: Proceedings of Computing Frontiers. ACM (2018)Google Scholar
  16. 16.
    Narasimhamurthy, S., et al.: SAGE: percipient storage for exascale data centric computing. Parallel Comput. (2018).
  17. 17.
    Peng, I.B., Gioiosa, R., Kestor, G., Cicotti, P., Laure, E., Markidis, S.: Exploring the performance benefit of hybrid memory system on HPC environments. In: 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 683–692. IEEE (2017)Google Scholar
  18. 18.
    Peng, I.B., Markidis, S., Laure, E., Kestor, G., Gioiosa, R.: Exploring application performance on emerging hybrid-memory supercomputers. In: 2016 IEEE 18th International Conference on High Performance Computing and Communications, IEEE 14th International Conferenceon Smart City, IEEE 2nd International Conference on Data Science and Systems (HPCC/SmartCity/DSS), pp. 473–480. IEEE (2016)Google Scholar
  19. 19.
    Rivas-Gomez, S., et al.: MPI windows on storage for HPC applications. In: Proceedings of the 24th European MPI Users’ Group Meeting, p. 15. ACM (2017)Google Scholar
  20. 20.
    Schmuck, F.B., Haskin, R.L.: GPFS: a shared-disk file system for large computing clusters. In: Proceedings of the Conference on File and Storage Technologies, FAST 2002, pp. 231–244 (2002)Google Scholar
  21. 21.
    Schwan, P., et al.: Lustre: building a file system for 1000-node clusters. In: Proceedings of the 2003 Linux Symposium, vol. 2003, pp. 380–386 (2003)Google Scholar
  22. 22.
    Shan, H., Antypas, K., Shalf, J.: Characterizing and predicting the I/O performance of HPC applications using a parameterized synthetic benchmark. In: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, p. 42. IEEE Press (2008)Google Scholar
  23. 23.
    Thakur, R., Gropp, W., Lusk, E.: On implementing MPI-IO portably and withhigh performance. In: Proceedings of the Sixth Workshop on I/O in Paralleland Distributed Systems, pp. 23–32 (1999)Google Scholar
  24. 24.
    The HDF Group: Hierarchical Data Format, version 5 (1997).
  25. 25.
    The HDF Group: HDF5 Virtual Dataset (2014).
  26. 26.
    Wang, F., et al.: File system workload analysis for large scale scientific computing applications. Technical report (2004)Google Scholar
  27. 27.
    Weil, S.A., Brandt, S.A., Miller, E.L., Long, D.D., Maltzahn, C.: Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th symposium on Operating systems design and implementation, pp. 307–320. USENIX Association (2006)Google Scholar
  28. 28.
    Xu, C., et al.: LIOProf: exposing Lustre file system behavior for I/O middleware (2016).
  29. 29.
    Yu, W., Vetter, J., Canon, R.S., Jiang, S.: Exploiting Lustre file joining for effective collective I/O. In: Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2007), pp. 267–274 (2007)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Steven Wei-der Chien
    • 1
    Email author
  • Stefano Markidis
    • 1
  • Rami Karim
    • 1
  • Erwin Laure
    • 1
  • Sai Narasimhamurthy
    • 2
  1. 1.KTH Royal Institute of TechnologyStockholmSweden
  2. 2.Seagate Systems UKHavantUK

Personalised recommendations