Abstract
We review and compare physics-informed learning models built upon Gaussian processes and deep neural networks for solving forward and inverse problems governed by linear and nonlinear partial differential equations. We define a unified data model on which Gaussian processes, physics-informed Gaussian processes, neural networks, and physics-informed neural networks are based. We develop continuous-time and discrete-time models to facilitate different application scenarios. We present a connection between a Gaussian process and an infinitely wide neural network, which enables us to obtain a “best” kernel, which is determined directly by the data. We demonstrate the implementation of physics-informed Gaussian processes and physics-informed neural networks using a pedagogical example. Additionally, we compare physics-informed Gaussian processes and physics-informed neural networks for two nonlinear partial differential equations, i.e. the 1D Burgers’ equation and the 2D Navier–Stokes, and provide guidance in choosing the proper machine learning model according to the problem type, i.e. forward or inverse problem, and the availability of data. These new methods for solving partial differential equations governing multi-physics problems do not require any grid, and they are simple to implement, and agnostic to specific application. Hence, we expect that variants and proper extensions of these methods will find broad applicability in the near future across different scientific disciplines but also in industrial applications.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
M. Raissi, P. Perdikaris, G.E. Karniadakis, J. Comput. Phys. 348, 683 (2017)
M. Raissi, P. Perdikaris, G.E. Karniadakis, J. Comput. Phys. 335, 736 (2017)
M. Raissi, P. Perdikaris, G.E. Karniadakis, SIAM, J. Sci. Comput. 40, A172 (2018)
M. Raissi, G.E. Karniadakis, J. Comput. Phys. 357, 125 (2018)
Z. Ghahramani, Nature 521(7553), 452 (2015)
M.I. Jordan, T.M. Mitchell, Science 349(6245), 255 (2015)
M. Dissanayake, N. Phan-Thien, Commun. Numer. Meth. Eng. 10, 195 (1994)
B.P. van Milligen, V. Tribaldos, J. Jiménez, Phys. Rev. Lett. 75, 3594 (1995)
I.E. Lagaris, A. Likas, D.I. Fotiadis, I.E.E.E. Trans, Neural Netw. 9, 987 (1998)
M. Raissi, P. Perdikaris, G.E. Karniadakis, J. Comput. Phys. 378, 686 (2019)
C.A. Micchelli, Y. Xu, H. Zhang, J. Mach. Learn. Res. 7, 2651 (2006)
K. Hornik, M. Stinchcombe, H. White, Neural Netw. 2, 359 (1989)
G. Pang, Python codes for the numerical examples in this chapter (2019), https://github.com/Pang1987/pedagogical-example-PIGP-PINN. Accessed 27 Jan 2020
C.E. Rasmussen, in Advanced Lectures on Machine Learning, ed. by O. Bousquet, U. von Luxburg, G. Rätsch (Springer, Berlin, 2003), p. 63
C. Finn, P. Abbeel, S. Levine, in 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, ed. by D. Precup, Y.W. Teh (Sidney, 2017), p. 1126
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, J. Mach, Learn. Res. 15, 1929 (2014)
S. Särkkä, in Artificial Neural Networks and Machine Learning – ICANN 2011, 21st International Conference on Artificial Neural Networks, Espoo, Finland, June 2011. Lecture Notes in Computer Science, vol. 6791, ed. by T. Honkela, W. Duch, M. Girolami, S. Kaski (Springer, Berlin, 2011), p. 151
M. Abadi et al., in Proceedings of OSDI ’16. 12th USENIX Symposium on Operating Systems Design and Implementation, Savannah, USA (2016), p. 265
G. Pang, L. Lu, G.E. Karniadakis, SIAM, J. Sci. Comput. 41, A2603 (2019)
G. Pang, L. Yang, G.E. Karniadakis, J. Comput. Phys. 384, 270 (2019)
I.M. Sobol’, Zh Vychisl, Mat. Mat. Fiz. 7, 784 (1967)
D. Duvenaud, Ph.D. thesis (University of Cambridge, 2014)
B. Zoph, Q.V. Le, Neural architecture search with reinforcement learning, arXiv:1611.01578
S.H. Rudy, S.L. Brunton, J.L. Proctor, J.N. Kutz, Sci. Adv. 3, e1602614 (2017)
M. Raissi, A. Yazdani, G.E. Karniadakis, Hidden fluid mechanics: a Navier-Stokes informed deep learning framework for assimilating flow visualization data, arXiv:1808.04327
A.G. Wilson, Z. Hu, R. Salakhutdinov, E.P. Xing, in Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, AISTATS 2016. JMLR Workshop and Conference Proceedings, vol. 51, ed. by A. Gretton, C.C. Robert. Cádiz (Spain, 2016), p. 370
A. Wilson, H. Nickisch, in Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 37, ed. by F. Bach, D. Blei (Lille, France, 2015), p. 1775
X. Meng, Z. Li, D. Zhang, G.E. Karniadakis, PPINN: parareal physics-informed neural network for time-dependent PDEs, arXiv:1909.10145
L. Lu, X. Meng, Z. Mao, G.E. Karniadakis, DeepXDE: a deep learning library for solving differential equations, arXiv:1907.04502
L. Le Gratiet, Ph.D. thesis (Université Paris Diderot, 2013)
G. Pang, P. Perdikaris, W. Cai, G.E. Karniadakis, J. Comput. Phys. 348, 694 (2017)
X. Meng, G.E. Karniadakis, J. Comput. Phys. 401, 109020 (2020)
J. Pathak, B. Hunt, M. Girvan, Z. Lu, E. Ott, Phys. Rev. Lett. 120, 024102 (2018)
J. Huang, SC19: NVIDIA CEO Jensen Huang on the expanding universe of HPC, https://www.youtube.com/watch?v=69nEEpdEJzU. Accessed 27 Jan 2020
Acknowledgements
This work was partially supported by AFOSR (FA9550-17-1-0013), Army Research Laboratory (W911NF-12-2-0023), and PHILMS (DE-SC0019453).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Pang, G., Karniadakis, G.E. (2020). Physics-Informed Learning Machines for Partial Differential Equations: Gaussian Processes Versus Neural Networks. In: Kevrekidis, P., Cuevas-Maraver, J., Saxena, A. (eds) Emerging Frontiers in Nonlinear Science. Nonlinear Systems and Complexity, vol 32. Springer, Cham. https://doi.org/10.1007/978-3-030-44992-6_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-44992-6_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-44991-9
Online ISBN: 978-3-030-44992-6
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)