Early Experiences Porting the NAMD and VMD Molecular Simulation and Analysis Software to GPU-Accelerated OpenPOWER Platforms

  • John E. StoneEmail author
  • Antti-Pekka Hynninen
  • James C. Phillips
  • Klaus Schulten
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9945)


All-atom molecular dynamics simulations of biomolecules provide a powerful tool for exploring the structure and dynamics of large protein complexes within realistic cellular environments. Unfortunately, such simulations are extremely demanding in terms of their computational requirements, and they present many challenges in terms of preparation, simulation methodology, and analysis and visualization of results. We describe our early experiences porting the popular molecular dynamics simulation program NAMD and the simulation preparation, analysis, and visualization tool VMD to GPU-accelerated OpenPOWER hardware platforms. We report our experiences with compiler-provided autovectorization and compare with hand-coded vector intrinsics for the POWER8 CPU. We explore the performance benefits obtained from unique POWER8 architectural features such as 8-way SMT and its value for particular molecular modeling tasks. Finally, we evaluate the performance of several GPU-accelerated molecular modeling kernels and relate them to other hardware platforms.


Fast Fourier Transform Neighbor List Rabbit Hemorrhagic Disease Virus Particle Mesh Ewald Pairwise Force 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors acknowledge support from NIH grants 9P41GM104601 and 5R01GM098243-02, the CUDA Center of Excellence at the University of Illinois, the Blue Waters sustained-petascale computing project supported by NSF awards OCI-0725070 and ACI-1238993, the state of Illinois, “The Computational Microscope” NSF PRAC awards OCI-0832673 and ACI-1440026, and the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory supported by the Department of Energy under Contract DE-AC05-00OR22725.


  1. 1.
    Zhao, G., Perilla, J.R., Yufenyuy, E.L., Meng, X., Chen, B., Ning, J., Ahn, J., Gronenborn, A.M., Schulten, K., Aiken, C., Zhang, P.: Mature HIV-1 capsid structure by cryo-electron microscopy and all-atom molecular dynamics. Nature 497, 643–646 (2013)CrossRefGoogle Scholar
  2. 2.
    Liu, C., Perilla, J.R., Ning, J., Lu, M., Hou, G., Ramalho, R., Bedwell, G., Byeon, I.J., Ahn, J., Shi, J., Gronenborn, A., Prevelige, P., Rousso, I., Aiken, C., Polenova, T., Schulten, K., Zhang, P.: Cyclophilin A stabilizes HIV-1 capsid through a novel non-canonical binding site. Nat. Commun. 7, Article no. 10714, 10 pages (2016)Google Scholar
  3. 3.
    Sothiselvam, S., Liu, B., Han, W., Klepacki, D., Atkinson, G.C., Brauer, A., Remm, M., Tenson, T., Schulten, K., Vázquez-Laslop, N., Mankin, A.S.: Macrolide antibiotics allosterically predispose the ribosome for translation arrest. Proc. Natl. Acad. Sci. USA 111, 9804–9809 (2014)CrossRefGoogle Scholar
  4. 4.
    Mendes, C.L., Bode, B., Bauer, G.H., Enos, J., Beldica, C., Kramer, W.T.: Deploying a large petascale system: the Blue Waters experience. Procedia Comput. Sci. 29, 198–209 (2014)CrossRefGoogle Scholar
  5. 5.
    Joubert, W., Archibald, R., Berrill, M., Brown, W.M., Eisenbach, M., Grout, R., Larkin, J., Levesque, J., Messer, B., Norman, M., Philip, B., Sankaran, R., Tharrington, A., Turner, J.: Accelerated application development: the ORNL Titan experience. Comput. Electr. Eng. 46, 123–138 (2015)CrossRefGoogle Scholar
  6. 6.
    Phillips, J.C., Braun, R., Wang, W., Gumbart, J., Tajkhorshid, E., Villa, E., Chipot, C., Skeel, R.D., Kale, L., Schulten, K.: Scalable molecular dynamics with NAMD. J. Comp. Chem. 26, 1781–1802 (2005)CrossRefGoogle Scholar
  7. 7.
    Phillips, J.C., Stone, J.E., Schulten, K.: Adapting a message-driven parallel application to GPU-accelerated clusters. In: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, SC 2008, 9 pages. IEEE Press, Piscataway, NJ, USA (2008)Google Scholar
  8. 8.
    Humphrey, W., Dalke, A., Schulten, K.: VMD - visual molecular dynamics. J. Mol. Graph. 14, 33–38 (1996)CrossRefGoogle Scholar
  9. 9.
    Stone, J.E., Isralewitz, B., Schulten, K.: Early experiences scaling VMD molecular visualization and analysis jobs on Blue Waters. In: Extreme Scaling Workshop (XSW 2013), pp. 43–50 (2013)Google Scholar
  10. 10.
    Stone, J.E., Sener, M., Vandivort, K.L., Barragan, A., Singharoy, A., Teo, I., Ribeiro, J.V., Isralewitz, B., Liu, B., Goh, B.C., Phillips, J.C., MacGregor-Chatwin, C., Johnson, M.P., Kourkoutis, L.F., Hunter, C.N., Schulten, K.: Atomic detail visualization of photosynthetic membranes with GPU-accelerated ray tracing. Parallel Comput. 55, 17–27 (2016)CrossRefGoogle Scholar
  11. 11.
    Götz, A.W., Williamson, M.J., Xu, D., Poole, D., Grand, S.L., Walker, R.C.: Routine microsecond molecular dynamics simulations with AMBER on GPUs. 1. Generalized Born. J. Chem. Theory Comput. 8, 1542–1555 (2012)CrossRefGoogle Scholar
  12. 12.
    Essmann, U., Perera, L., Berkowitz, M.L., Darden, T., Lee, H., Pedersen, L.G.: A smooth particle mesh Ewald method. J. Chem. Phys. 103, 8577–8593 (1995)CrossRefGoogle Scholar
  13. 13.
    Darden, T., York, D., Pedersen, L.: Particle mesh Ewald: an N\(\cdot \)log(N) method for Ewald sums in large systems. J. Chem. Phys. 98, 10089–10092 (1993)CrossRefGoogle Scholar
  14. 14.
    Stone, J.E., Messmer, P., Sisneros, R., Schulten, K.: High performance molecular visualization: In-situ and parallel rendering with EGL. In: 2016 IEEE International Parallel and Distributed Processing Symposium Workshop (IPDPSW) (2016, in Press)Google Scholar
  15. 15.
    Stone, J.E., Vandivort, K.L., Schulten, K.: GPU-accelerated molecular visualization on petascale supercomputing platforms. In: Proceedings of the 8th International Workshop on Ultrascale Visualization. UltraVis 2013, pp. 6:1–6:8. ACM, New York (2013)Google Scholar
  16. 16.
    Stone, J.E., McGreevy, R., Isralewitz, B., Schulten, K.: GPU-accelerated analysis and visualization of large structures solved by molecular dynamics flexible fitting. Faraday Discuss. 169, 265–283 (2014)CrossRefGoogle Scholar
  17. 17.
    Phillips, J.C., Stone, J.E., Vandivort, K.L., Armstrong, T.G., Wozniak, J.M., Wilde, M., Schulten, K.: Petascale Tcl with NAMD, VMD, and Swift/T. In: Workshop on High Performance Technical Computing in Dynamic Languages, SC 2014, pp. 6–17. IEEE Press (2014)Google Scholar
  18. 18.
    Ribeiro, J.V., Bernardi, R.C., Rudack, T., Stone, J.E., Phillips, J.C., Freddolino, P.L., Schulten, K.: QwikMD-integrative molecular dynamics toolkit for novices and experts. Sci. Rep. 6, 26536 (2016)CrossRefGoogle Scholar
  19. 19.
    Pronk, S., Páll, S., Schulz, R., Larsson, P., Bjelkmar, P., Apostolov, R., Shirts, M.R., Smith, J.C., Kasson, P.M., van der Spoel, D., Hess, B., Lindahl, E.: Gromacs 4.5: a high-throughput and highly parallel open source molecular simulation toolkit. Bioinformatics 29, 845–854 (2013)CrossRefGoogle Scholar
  20. 20.
    Vermaas, J.V., Hardy, D.J., Stone, J.E., Tajkhorshid, E., Kohlmeyer, A.: TopoGromacs: automated topology conversion from CHARMM to GROMACS within VMD. J. Chem. Inf. Model. (2016, in Press)Google Scholar
  21. 21.
    Stone, J.E.: An efficient library for parallel ray tracing and animation. Master’s thesis, Computer Science Department, University of Missouri-Rolla (1998)Google Scholar
  22. 22.
    Parker, S.G., Bigler, J., Dietrich, A., Friedrich, H., Hoberock, J., Luebke, D., McAllister, D., McGuire, M., Morley, K., Robison, A., Stich, M.: OptiX: a general purpose ray tracing engine. In: ACM SIGGRAPH 2010 papers, SIGGRAPH 2010, pp. 66:1–66:13. ACM, New York (2010)Google Scholar
  23. 23.
    Wald, I., Woop, S., Benthin, C., Johnson, G.S., Ernst, M.: Embree: a kernel framework for efficient CPU ray tracing. ACM Trans. Graph. 33, 143:1–143:8 (2014)CrossRefGoogle Scholar
  24. 24.
    Nickolls, J., Buck, I., Garland, M., Skadron, K.: Scalable parallel programming with CUDA. ACM Queue 6, 40–53 (2008)CrossRefGoogle Scholar
  25. 25.
    Pharr, M., Mark, W.: ispc: A SPMD compiler for high-performance CPU programming. In: Innovative Parallel Computing (InPar 2012), pp. 1–13 (2012)Google Scholar
  26. 26.
    Stone, J.E., Sherman, W.R., Schulten, K.: Immersive molecular visualization with omnidirectional stereoscopic ray tracing and remote rendering. In: 2016 IEEE International Parallel and Distributed Processing Symposium Workshop (IPDPSW) (2016, in Press)Google Scholar
  27. 27.
    Wang, X., Xu, F., Liu, J., Gao, B., Liu, Y., Zhai, Y., Ma, J., Zhang, K., Baker, T.S., Schulten, K., Zheng, D., Pang, H., Sun, F.: Atomic model of rabbit hemorrhagic disease virus by cryo-electron microscopy and crystallography. PLoS Pathog. 9, e1003132 (2013). (14 pages)CrossRefGoogle Scholar
  28. 28.
    Stone, J.E., Saam, J., Hardy, D.J., Vandivort, K.L., Hwu, W.W., Schulten, K.: High performance computation and interactive display of molecular orbitals on GPUs and multi-core CPUs. In: Proceedings of the 2nd Workshop on General-Purpose Processing on Graphics Processing Units, ACM International Conference Proceeding Series, vol. 383, pp. 9–18. ACM, New York (2009)Google Scholar
  29. 29.
    Stone, J.E., Hardy, D.J., Saam, J., Vandivort, K.L., Schulten, K.: GPU-accelerated computation and interactive display of molecular orbitals. In: Hwu, W. (ed.) GPU Computing Gems, pp. 5–18. Morgan Kaufmann Publishers, San Francisco (2011)CrossRefGoogle Scholar
  30. 30.
    Stone, J.E., Hallock, M.J., Phillips, J.C., Peterson, J.R., Luthey-Schulten, Z., Schulten, K.: Evaluation of emerging energy-efficient heterogeneous computing platforms for biomolecular and cellular simulation workloads. In: 2016 IEEE International Parallel and Distributed Processing Symposium Workshop (IPDPSW) (2016, in Press)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • John E. Stone
    • 1
    Email author
  • Antti-Pekka Hynninen
    • 2
  • James C. Phillips
    • 1
  • Klaus Schulten
    • 1
    • 3
  1. 1.Beckman Institute for Advanced Science and TechnologyUniversity of Illinois at Urbana-ChampaignUrbanaUSA
  2. 2.Oak Ridge Leadership Computing FacilityOak Ridge National LaboratoryOak RidgeUSA
  3. 3.Department of PhysicsUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations