Building Nonparametric n-Body Force Fields Using Gaussian Process Regression

  • Aldo GlielmoEmail author
  • Claudio Zeni
  • Ádám Fekete
  • Alessandro De Vita
Part of the Lecture Notes in Physics book series (LNP, volume 968)


Constructing a classical potential suited to simulate a given atomic system is a remarkably difficult task. This chapter presents a framework under which this problem can be tackled, based on the Bayesian construction of nonparametric force fields of a given order using Gaussian process (GP) priors. The formalism of GP regression is first reviewed, particularly in relation to its application in learning local atomic energies and forces. For accurate regression, it is fundamental to incorporate prior knowledge into the GP kernel function. To this end, this chapter details how properties of smoothness, invariance and interaction order of a force field can be encoded into corresponding kernel properties. A range of kernels is then proposed, possessing all the required properties and an adjustable parameter n governing the interaction order modelled. The order n best suited to describe a given system can be found automatically within the Bayesian framework by maximisation of the marginal likelihood. The procedure is first tested on a toy model of known interaction and later applied to two real materials described at the DFT level of accuracy. The models automatically selected for the two materials were found to be in agreement with physical intuition. More in general, it was found that lower order (simpler) models should be chosen when the data are not sufficient to resolve more complex interactions. Low n GPs can be further sped up by orders of magnitude by constructing the corresponding tabulated force field, here named “MFF”.



The authors acknowledge funding by the Engineering and Physical Sciences Research Council (EPSRC) through the Centre for Doctoral Training “Cross Disciplinary Approaches to Non-Equilibrium Systems” (CANES, Grant No. EP/L015854/1) and by the Office of Naval Research Global (ONRG Award No. N62909-15-1-N079). The authors thank the UK Materials and Molecular Modelling Hub for computational resources, which is partially funded by EPSRC (EP/P020194/1). ADV acknowledges further support by the EPSRC HEmS Grant No. EP/L014742/1 and by the European Union’s Horizon 2020 research and innovation program (Grant No. 676580, The NOMAD Laboratory, a European Centre of Excellence). We, AG, CZ and AF, are immensely grateful to Alessandro De Vita for having devoted, with inexhaustible energy and passion, an extra-ordinary amount of his time and brilliance towards our personal and professional growth.


  1. 1.
    D.H. Wolpert, Neural Comput. 8(7), 1341 (1996)CrossRefGoogle Scholar
  2. 2.
    A.J. Skinner, J.Q. Broughton, Modell. Simul. Mater. Sci. Eng. 3(3), 371 (1995)ADSCrossRefGoogle Scholar
  3. 3.
    J. Behler, M. Parrinello, Phys. Rev. Lett. 98(14), 146401 (2007)ADSCrossRefGoogle Scholar
  4. 4.
    R. Kondor (2018). Preprint. arXiv:1803.01588Google Scholar
  5. 5.
    M. Gastegger, P. Marquetand, J. Chem. Theory Comput. 11(5), 2187 (2015)CrossRefGoogle Scholar
  6. 6.
    S. Manzhos, R. Dawes, T. Carrington, Int. J. Quantum Chem. 115(16), 1012 (2014)CrossRefGoogle Scholar
  7. 7.
    P. Geiger, C. Dellago, J. Chem. Phys. 139(16), 164105 (2013)ADSCrossRefGoogle Scholar
  8. 8.
    N. Kuritz, G. Gordon, A. Natan, Phys. Rev. B 98(9), 094109 (2018)ADSCrossRefGoogle Scholar
  9. 9.
    K.T. Schütt, F. Arbabzadah, S. Chmiela, K.R. Müller, A. Tkatchenko, Nat. Commun. 8, 13890 (2017)ADSCrossRefGoogle Scholar
  10. 10.
    N. Lubbers, J.S. Smith, K. Barros, J. Chem. Phys. 148(24), 241715 (2018)ADSCrossRefGoogle Scholar
  11. 11.
    K.T. Schütt, H.E. Sauceda, P.J. Kindermans, A. Tkatchenko, K.R. Müller, J. Chem. Phys. 148(24), 241722 (2018)ADSCrossRefGoogle Scholar
  12. 12.
    A.P. Bartók, M.C. Payne, R. Kondor, G. Csányi, Phys. Rev. Lett. 104(13), 136403 (2010)ADSCrossRefGoogle Scholar
  13. 13.
    Z. Li, J.R. Kermode, A. De Vita, Phys. Rev. Lett. 114(9), 096405 (2015)ADSCrossRefGoogle Scholar
  14. 14.
    A. Glielmo, P. Sollich, A. De Vita, Phys. Rev. B 95(21), 214302 (2017)ADSCrossRefGoogle Scholar
  15. 15.
    A. Glielmo, C. Zeni, A. De Vita, Phys. Rev. B 97(18), 1 (2018)CrossRefGoogle Scholar
  16. 16.
    C. Zeni, K. Rossi, A. Glielmo, Á. Fekete, N. Gaston, F. Baletto, A. De Vita, J. Chem. Phys. 148(24), 241739 (2018)ADSCrossRefGoogle Scholar
  17. 17.
    W.J. Szlachta, A.P. Bartók, G. Csányi, Phys. Rev. B 90(10), 104108 (2014)ADSCrossRefGoogle Scholar
  18. 18.
    A.P. Thompson, L.P. Swiler, C.R. Trott, S.M. Foiles, G.J. Tucker, J. Comput. Phys. 285(C), 316 (2015)ADSMathSciNetCrossRefGoogle Scholar
  19. 19.
    A.V. Shapeev, Multiscale Model. Simul. 14(3), 1153 (2016)MathSciNetCrossRefGoogle Scholar
  20. 20.
    A. Takahashi, A. Seko, I. Tanaka, J. Chem. Phys. 148(23), 234106 (2018)ADSCrossRefGoogle Scholar
  21. 21.
    A.P. Bartók, R. Kondor, G. Csányi, Phys. Rev. B 87(18), 184115 (2013)ADSCrossRefGoogle Scholar
  22. 22.
    W.H. Jefferys, J.O. Berger, Am. Sci. 80(1), 64 (1992)ADSGoogle Scholar
  23. 23.
    C.E. Rasmussen, Z. Ghahramani, in Proceedings of the 13th International Conference on Neural Information Processing Systems (NIPS’00) (MIT Press, Cambridge, 2000), pp. 276–282Google Scholar
  24. 24.
    Z. Ghahramani, Nature 521(7553), 452 (2015)ADSCrossRefGoogle Scholar
  25. 25.
    V.N. Vapnik, A.Y. Chervonenkis, in Measures of Complexity (Springer, Cham, 2015), pp. 11–30zbMATHGoogle Scholar
  26. 26.
    V.N. Vapnik, Statistical Learning Theory (Wiley, Hoboken, 1998)zbMATHGoogle Scholar
  27. 27.
    M.J. Kearns, U.V. Vazirani, An Introduction to Computational Learning Theory (MIT Press, Cambridge, 1994)CrossRefGoogle Scholar
  28. 28.
    T. Suzuki, in Proceedings of the 25th Annual Conference on Learning Theory, ed. by S. Mannor, N. Srebro, R.C. Williamson. Proceedings of Machine Learning Research, vol. 23 (PMLR, Edinburgh, 2012), pp. 8.1–8.20Google Scholar
  29. 29.
    C. Zeni, F. Ádám, A. Glielmo, MFF: a Python package for building nonparametric force fields from machine learning (2018).
  30. 30.
    R.P. Feynman, Phys. Rev. 56(4), 340 (1939)ADSCrossRefGoogle Scholar
  31. 31.
    V. Botu, R. Ramprasad, Phys. Rev. B 92(9), 094306 (2015)ADSCrossRefGoogle Scholar
  32. 32.
    I. Kruglov, O. Sergeev, A. Yanilkin, A.R. Oganov, Sci. Rep. 7(1), 1–7 (2017)CrossRefGoogle Scholar
  33. 33.
    G. Ferré, J.B. Maillet, G. Stoltz, J. Chem. Phys. 143(10), 104114 (2015)ADSCrossRefGoogle Scholar
  34. 34.
    A.P. Bartók, G. Csányi, Int. J. Quantum Chem. 115(16), 1051 (2015)CrossRefGoogle Scholar
  35. 35.
    C.K.I. Williams, C.E. Rasmussen, Gaussian Processes for Machine Learning (MIT Press, Cambridge, 2006)zbMATHGoogle Scholar
  36. 36.
    I. Macêdo, R. Castro, Learning Divergence-Free and Curl-Free Vector Fields with Matrix-Valued Kernels (Instituto Nacional de Matematica Pura e Aplicada, Rio de Janeiro, 2008)Google Scholar
  37. 37.
    C.M. Bishop, in Pattern Recognition and Machine Learning. Information Science and Statistics (Springer, New York, 2006)Google Scholar
  38. 38.
    M. Rupp, A. Tkatchenko, K.R. Müller, O.A. von Lilienfeld, Phys. Rev. Lett. 108(5), 058301 (2012)ADSCrossRefGoogle Scholar
  39. 39.
    M. Rupp, Int. J. Quantum Chem. 115(16), 1058 (2015)MathSciNetCrossRefGoogle Scholar
  40. 40.
    K. Hansen, G. Montavon, F. Biegler, S. Fazli, M. Rupp, M. Scheffler, O.A. von Lilienfeld, A. Tkatchenko, K.R. Müller, J. Chem. Theory Comput. 9(8), 3404 (2013)CrossRefGoogle Scholar
  41. 41.
    S. Chmiela, A. Tkatchenko, H.E. Sauceda, I. Poltavsky, K.T. Schütt, K.R. Müller, Sci. Adv. 3(5), e1603015 (2017)ADSCrossRefGoogle Scholar
  42. 42.
    V.L. Deringer, G. Csányi, Phys. Rev. B 95(9), 094203 (2017)ADSCrossRefGoogle Scholar
  43. 43.
    H. Huo, M. Rupp (2017). Preprint. arXiv:1704.06439Google Scholar
  44. 44.
    A.P. Bartók, M.J. Gillan, F.R. Manby, G. Csányi, Phys. Rev. B 88(5), 054104 (2013)ADSCrossRefGoogle Scholar
  45. 45.
    J. Behler, J. Chem. Phys. 134(7), 074106 (2011)ADSCrossRefGoogle Scholar
  46. 46.
    B. Haasdonk, H. Burkhardt, Mach. Learn. 68(1), 35 (2007)CrossRefGoogle Scholar
  47. 47.
    C.A. Micchelli, M. Pontil, in Advances in Neural Information Processing Systems (University at Albany State University of New York, Albany, 2005)Google Scholar
  48. 48.
    C.A. Micchelli, M. Pontil, Neural Comput. 17(1), 177 (2005)MathSciNetCrossRefGoogle Scholar
  49. 49.
    T. Bereau, R.A. DiStasio, A. Tkatchenko, O.A. von Lilienfeld, J. Chem. Phys. 148(24), 241706 (2018)ADSCrossRefGoogle Scholar
  50. 50.
    A. Grisafi, D.M. Wilkins, G. Csányi, M. Ceriotti, Phys. Rev. Lett. 120, 036002 (2018). ADSCrossRefGoogle Scholar
  51. 51.
    S.K. Reddy, S.C. Straight, P. Bajaj, C. Huy Pham, M. Riera, D.R. Moberg, M.A. Morales, C. Knight, A.W. Götz, F. Paesani, J. Chem. Phys. 145(19), 194504 (2016)ADSCrossRefGoogle Scholar
  52. 52.
    G.A. Cisneros, K.T. Wikfeldt, L. Ojamäe, J. Lu, Y. Xu, H. Torabifard, A.P. Bartók, G. Csányi, V. Molinero, F. Paesani, Chem. Rev. 116(13), 7501 (2016)CrossRefGoogle Scholar
  53. 53.
    F.H. Stillinger, T.A. Weber, Phys. Rev. B31(8), 5262 (1985)ADSCrossRefGoogle Scholar
  54. 54.
    J. Tersoff, Phys. Rev. B 37(12), 6991 (1988)ADSCrossRefGoogle Scholar
  55. 55.
    K. Yao, J.E. Herr, J. Parkhill, J. Chem. Phys. 146(1), 014106 (2017)ADSCrossRefGoogle Scholar
  56. 56.
    K. Hornik, Neural Netw. 6(8), 1069 (1993)CrossRefGoogle Scholar
  57. 57.
    R.A. Jacobs, M.I. Jordan, S.J. Nowlan, G.E. Hinton, Neural Comput. 3(1), 79 (1991)CrossRefGoogle Scholar
  58. 58.
    C.E. Rasmussen, Z. Ghahramani, in Advances in Neural Information Processing Systems (UCL, London, 2002)Google Scholar
  59. 59.
    S. De, A.P. Bartók, G. Csányi, M. Ceriotti, Phys. Chem. Chem. Phys. 18, 13754 (2016)CrossRefGoogle Scholar
  60. 60.
    L.M. Ghiringhelli, J. Vybiral, S.V. Levchenko, C. Draxl, M. Scheffler, Phys. Rev. Lett. 114(10), 105503 (2015)ADSCrossRefGoogle Scholar
  61. 61.
    J. Mavračić, F.C. Mocanu, V.L. Deringer, G. Csányi, S.R. Elliott, J. Phys. Chem. Lett. 9(11), 2985 (2018)CrossRefGoogle Scholar
  62. 62.
    S. De, F. Musil, T. Ingram, C. Baldauf, M. Ceriotti, J. Cheminf. 9(1), 1–14 (2017)CrossRefGoogle Scholar
  63. 63.
    L. Breiman, Mach. Learn. 24(2), 123 (1996)Google Scholar
  64. 64.
    O. Sagi, L. Rokach, Wiley Interdiscip. Rev. Data Min. Knowl. Disc. 8(4), e1249 (2018)Google Scholar
  65. 65.
    M. Sewell, Technical Report RN/11/02 (Department of Computer Science, UCL, London, 2008)Google Scholar
  66. 66.
    I. Kruglov, O. Sergeev, A. Yanilkin, A.R. Oganov, Sci. Rep. 7(1), 8512 (2017)ADSCrossRefGoogle Scholar

Copyright information

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Aldo Glielmo
    • 1
    Email author
  • Claudio Zeni
    • 1
  • Ádám Fekete
    • 1
  • Alessandro De Vita
    • 1
    • 2
  1. 1.Department of PhysicsKing’s College LondonLondonUK
  2. 2.Dipartimento di Ingegneria e ArchitetturaUniversity of TriesteTriesteItaly

Personalised recommendations