Early Evaluation of the “Infinite Memory Engine” Burst Buffer Solution

  • Wolfram SchenckEmail author
  • Salem El Sayed
  • Maciej Foszczynski
  • Wilhelm Homberg
  • Dirk Pleiter
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9945)


Hierarchical storage architectures are required to meet both, capacity and bandwidth requirements for future high-end storage architectures. In this paper we present the results of an evaluation of an emerging technology, DataDirect Networks’ (DDN) Infinite Memory Engine (IME). IME allows to realize a fast buffer in front of a large capacity storage system. We collected benchmarking data with IOR and with the HPC application NEST. The IOR bandwidth results show how well network bandwidth towards such fast buffer can be exploited compared to the external storage system. The NEST benchmarks clearly demonstrate that IME can reduce I/O-induced load imbalance between MPI ranks to a minimum while speeding up I/O as a whole by a considerable factor.


Burst buffer Storage Infinite Memory Engine (IME) GPFS NEST IOR Performance analysis 



We would like to thank DDN for making an IME test system available at Jülich Supercomputing Centre. In particular, we gracefully acknowledge the continuous support by Tommaso Cecchi and Toine Beckers.


  1. 1.
    Gray, J., Shenoy, P.: Rules of thumb in data engineering, pp. 3–10 (2000)Google Scholar
  2. 2.
    Bent, J., Grider, G., Kettering, B., Manzanares, A., McClelland, M., Torres, A., Torrez, A.: Storage challenges at Los Alamos national lab. In: 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–5, April 2012Google Scholar
  3. 3.
    Moody, A., Bronevetsky, G., Mohror, K., de Supinski, B.: Design, modeling, and evaluation of a scalable multi-level checkpointing system. In: 2010 International Conference for High Performance Computing, Networking, Storage and Analysis (SC), pp. 1–11, November 2010Google Scholar
  4. 4.
    Bent, J., Gibson, G., Grider, G., McClelland, B., Nowoczynski, P., Nunez, J., Polte, M., Wingate, M.: PLFs: a checkpoint filesystem for parallel applications. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, pp. 1–12, November 2009Google Scholar
  5. 5.
    El Sayed, S., Graf, S., Hennecke, M., Pleiter, D., Schwarz, G., Schick, H., Stephan, M.: Using GPFS to manage NVRAM-based storage cache. In: Kunkel, J.M., Ludwig, T., Meuer, H.W. (eds.) ISC 2013. LNCS, vol. 7905, pp. 435–446. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  6. 6.
    Liu, N., Cope, J., Carns, P., Carothers, C., Ross, R., Grider, G., Crume, A., Maltzahn, C.: On the role of burst buffers in leadership-class storage systems. In: 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–11, April 2012Google Scholar
  7. 7.
    Carns, P., Harms, K., Allcock, W., Bacon, C., Lang, S., Latham, R., Ross, R.: Understanding and improving computational science storage access through continuous characterization. In: 2011 IEEE 27th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–14, May 2011Google Scholar
  8. 8.
    Kannan, S., Gavrilovska, A., Schwan, K., Milojicic, D., Talwar, V.: Using active NVRAM for I/O staging. In: PDAC 2011, pp. 15–22. ACM, New York (2011)Google Scholar
  9. 9.
    He, J., Jagatheesan, A., Gupta, S., Bennett, J., Snavely, A.: Dash: a recipe for a flash-based data intensive supercomputer. In: 2010 International Conference for High Performance Computing, Networking, Storage and Analysis (SC), pp. 1–11, November 2010Google Scholar
  10. 10.
    Abbasi, H., Wolf, M., Eisenhauer, G., Klasky, S., Schwan, K., Zheng, F.: DataStager: scalable data staging services for petascale applications. In: HPDC 2009, pp. 39–48. ACM, New York (2009)Google Scholar
  11. 11.
    Docan, C., Parashar, M., Klasky, S.: Enabling high-speed asynchronous data extraction and transfer using dart. Concur. Comput. Pract. Exper. 22(9), 1181–1204 (2010)Google Scholar
  12. 12.
    Gewaltig, M.O., Diesmann, M.: NEST (Neural Simulation Tool). Scholarpedia 2(4), 1430 (2007)CrossRefGoogle Scholar
  13. 13.
    Kunkel, S., Schmidt, M., Eppler, J.M., Plesser, H.E., Masumoto, G., Igarashi, J., Ishii, S., Fukai, T., Morrison, A., Diesmann, M., Helias, M.: Spiking network simulation code for petascale computers. Front. Neuroinform. 8, Article number 78 (2014)Google Scholar
  14. 14.
    Schumann, T., Frings, W., Peyser, A., Schenck, W., Thust, K., Eppler, J.M.: Modeling the I/O behavior of the NEST simulator using a proxy. In: Elgeti, S., Simon, J.W. (eds.) Conference Proceedings of the YIC GACM 2015, Aachen (Germany), pp. 213–216. Publication Server of RWTH Aachen University (2015)Google Scholar
  15. 15.
    Schenck, W., Adinetz, A.V., Zaytsev, Y.V., Pleiter, D., Morrison, A.: Performance model for large-scale neural simulations with NEST. In: SC14 Conference for Supercomputing (Extended Poster Abstracts), New Orleans (LA), November 2014Google Scholar
  16. 16.
    Morrison, A., Aertsen, A., Diesmann, M.: Spike-timing dependent plasticity in balanced random networks. Neural Comput. 19(6), 1437–1467 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Lujan, J., et al.: APEX workflows. Technical report, LANL, NERSC, SNL (2015)Google Scholar
  18. 18.
    Rozier, E.W.D., Belluomini, W., Deenadhayalan, V., Hafner, J., Rao, K., Zhou, P.: Evaluating the impact of undetected disk errors in raid systems. In: IEEE/IFIP International Conference on Dependable Systems Networks, DSN 2009, pp. 83–92, June 2009Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Wolfram Schenck
    • 1
    Email author
  • Salem El Sayed
    • 2
  • Maciej Foszczynski
    • 2
  • Wilhelm Homberg
    • 2
  • Dirk Pleiter
    • 2
  1. 1.Faculty of Engineering and MathematicsBielefeld University of Applied SciencesBielefeldGermany
  2. 2.Jülich Supercomputing CentreForschungszentrum JülichJülichGermany

Personalised recommendations