A Deep-Learning-Based Geological Parameterization for History Matching Complex Models

  • Yimin LiuEmail author
  • Wenyue Sun
  • Louis J. Durlofsky


A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN–PCA method is inspired by recent developments in computer vision using deep learning. CNN–PCA can be viewed as a generalization of an existing optimization-based PCA (O-PCA) method. Both CNN–PCA and O-PCA entail post-processing a PCA model to better honor complex geological features. In CNN–PCA, rather than use a histogram-based regularization as in O-PCA, a new regularization involving a set of metrics for multipoint statistics is introduced. The metrics are based on summary statistics of the nonlinear filter responses of geological models to a pre-trained deep CNN. In addition, in the CNN–PCA formulation presented here, a convolutional neural network is trained as an explicit transform function that can post-process PCA models quickly. CNN–PCA is shown to provide both unconditional and conditional realizations that honor the geological features present in reference SGeMS geostatistical realizations for a binary channelized system. Flow statistics obtained through simulation of random CNN–PCA models closely match results for random SGeMS models for a demanding case in which O-PCA models lead to significant discrepancies. Results for history matching are also presented. In this assessment CNN–PCA is applied with derivative-free optimization, and a subspace randomized maximum likelihood method is used to provide multiple posterior models. Data assimilation and significant uncertainty reduction are achieved for existing wells, and physically reasonable predictions are also obtained for new wells. Finally, the CNN–PCA method is extended to a more complex nonstationary bimodal deltaic fan system, and is shown to provide high-quality realizations for this challenging example.


Geological parameterization History matching Deep learning Principal component analysis 



We thank the industrial affiliates of the Stanford Smart Fields Consortium for financial support. We are grateful to Hai Vo for providing O-PCA code and geological models and for useful suggestions, and to Hang Zhang and Abhishek Kadian for their open-source PyTorch implementation of the neural style transfer and fast neural style transfer algorithms. We also acknowledge the teaching staff for the Stanford CS231N course for offering guidance and providing computing resources on Google Cloud.


  1. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. Paper presented at the 34th international conference on machine learning, Sydney, Australia, 6–11 August, pp 214–223Google Scholar
  2. Astrakova A, Oliver DS (2015) Conditioning truncated pluri-Gaussian models to facies observations in ensemble-Kalman-based data assimilation. Math Geosci 47(47):345–367Google Scholar
  3. Canchumuni SA, Emerick AA, Pacheco MA (2017) Integration of ensemble data assimilation and deep learning for history matching facies models. Paper OTC-28015-MS, presented at the OTC Brasil, Rio de Janeiro, Brazil, 24–26 OctoberGoogle Scholar
  4. Canchumuni SA, Emerick AA, Pacheco M (2018) History matching channelized facies models using ensemble smoother with a deep learning parameterization. Paper presented at the 15th European conference on the mathematics of oil recovery, Barcelona, Spain, 3–6 SeptemberGoogle Scholar
  5. Chan S, Elsheikh AH (2017) Parametrization and generation of geological models with generative adversarial networks. arXiv preprint arXiv:1708.01810
  6. Chan S, Elsheikh AH (2018) Parametric generation of conditional geological realizations using generative neural networks. arXiv preprint arXiv:1807.05207
  7. Chang H, Zhang D, Lu Z (2010) History matching of facies distribution with the EnKF and level set parameterization. J Comput Phys 229(20):8011–8030Google Scholar
  8. Chen C, Gao G, Ramirez BA, Vink JC, Girardi AM (2016) Assisted history matching of channelized models by use of pluri-principal-component analysis. SPE J 21(05):1793–1812Google Scholar
  9. Deng J, Dong W, Socher R, Li LJ, Li K, Li FF (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp 248–255Google Scholar
  10. Dimitrakopoulos R, Mustapha H, Gloaguen E (2010) High-order statistics of spatial random fields: exploring spatial cumulants for modeling complex non-Gaussian and non-linear phenomena. Math Geosci 42(1):65–99Google Scholar
  11. Dupont E, Zhang T, Tilke P, Liang L, Bailey W (2018) Generating realistic geology conditioned on physical measurements with generative adversarial networks. arXiv preprint arXiv:1802.03065
  12. Echeverría Ciaurri D, Mukerji T, Santos ET (2009) Robust scheme for inversion of seismic and production data for reservoir facies modeling. Paper SEG-2009-2432, presented at the SEG annual meeting, Houston, Texas, 25–30 OctoberGoogle Scholar
  13. Emerick AA (2016) Investigation on principal component analysis parameterizations for history matching channelized facies models with ensemble-based data assimilation. Math Geosci 49(1):85–120Google Scholar
  14. Gatys LA, Ecker AS, Bethge M (2015) Texture synthesis using convolutional neural networks. In: Advances in neural information processing systems, pp 262–270Google Scholar
  15. Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2414–2423Google Scholar
  16. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, CambridgeGoogle Scholar
  17. Hakim-Elahi S, Jafarpour B (2017) A distance transform for continuous parameterization of discrete geologic facies for subsurface flow model calibration. Water Resour Res 53(10):8226–8249Google Scholar
  18. Insuasty E, Van den Hof PMJ, Weiland S, Jansen JD (2017) Low-dimensional tensor representations for the estimation of petrophysical reservoir parameters. Paper SPE-182707-MS presented at the SPE Reservoir Simulation Conference, Montgomery, Texas, 20–22 FebruaryGoogle Scholar
  19. Isebor OJ, Echeverría Ciaurri D, Durlofsky LJ (2014) Generalized field-development optimization with derivative-free procedures. SPE J 19(05):891–908Google Scholar
  20. Jafarpour B, Goyal VK, McLaughlin DB, Freeman WT (2010) Compressed history matching: exploiting transform-domain sparsity for regularization of nonlinear dynamic data integration problems. Math Geosci 42(1):1–27Google Scholar
  21. Johnson J, Alahi A, Li FF (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision, pp 694–711Google Scholar
  22. Kadian A (2018) Pytorch implementation of an algorithm for artistic style transfer. Accessed 1 May 2018
  23. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. Accessed 1 May 2018
  24. Kitanidis PK (1986) Parameter uncertainty in estimation of spatial functions: Bayesian analysis. Water Resour Res 22(4):499–507Google Scholar
  25. Laloy E, Hérault R, Lee J, Jacques D, Linde N (2017) Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network. Adv Water Resour 110:387–405Google Scholar
  26. Laloy E, Hérault R, Jacques D, Linde N (2018) Training-image based geostatistical inversion using a spatial generative adversarial neural network. Water Resour Res 54(1):381–406Google Scholar
  27. Liu Y (2017) Multilevel strategy for O-PCA-based history matching using mesh adaptive direct search. Master’s thesis, Stanford UniversityGoogle Scholar
  28. Lu P, Horne RN (2000) A multiresolution approach to reservoir parameter estimation using wavelet analysis. Paper SPE-62985-MS, presented at the SPE annual technical conference and exhibition, Dallas, TX, 1–4 OctoberGoogle Scholar
  29. Mosser L, Dubrule O, Blunt MJ (2017) Reconstruction of three-dimensional porous media using generative adversarial neural networks. Phys Rev E 96(4):043,309Google Scholar
  30. Mosser L, Dubrule O, Blunt MJ (2018) Conditioning of three-dimensional generative adversarial networks for pore and reservoir-scale models. arXiv preprint arXiv:1802.05622
  31. Oliver DS (1996) Multiple realizations of the permeability field from well test data. SPE J 1(2):145–154Google Scholar
  32. Pardo-Igúzquiza E, Dowd PA (2003) CONNEC3D: a computer program for connectivity analysis of 3D random set models. Comput Geosci 29(6):775–785Google Scholar
  33. Paszke A, Gross S, Chintala S, Chanan G, Yang E (2017) Automatic differentiation in PyTorch. NIPS 2017 workshopGoogle Scholar
  34. Ping J, Zhang D (2013) History matching of fracture distributions by ensemble Kalman filter combined with vector based level set parameterization. J Pet Sci Eng 108:288–303Google Scholar
  35. Remy N, Boucher A, Wu J (2009) Applied geostatistics with SGeMS: a user’s guide. Cambridge University Press, CambridgeGoogle Scholar
  36. Reynolds AC, He N, Chu L, Oliver DS (1996) Reparameterization techniques for generating reservoir descriptions conditioned to variograms and well-test pressure data. SPE J 1(4):413–426Google Scholar
  37. Rwechungura R, Dadashpour M, Kleppe J (2011) Application of particle swarm optimization for parameter estimation integrating production and time lapse seismic data. Paper SPE-146199-MS, presented at the SPE offshore Europe conference and exhibition, Aberdeen, UK, 6–8 SeptemberGoogle Scholar
  38. Sarma P, Durlofsky LJ, Aziz K, Chen WH (2006) Efficient real-time reservoir management using adjoint-based optimal control and model updating. Comput Geosci 10(1):3–36Google Scholar
  39. Sarma P, Durlofsky LJ, Aziz K (2008) Kernel principal component analysis for efficient, differentiable parameterization of multipoint geostatistics. Math Geosci 40(1):3–32Google Scholar
  40. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
  41. Strebelle S (2002) Conditional simulation of complex geological structures using multiple-point statistics. Math Geol 34(1):1–21Google Scholar
  42. Tavakoli R, Reynolds AC (2010) History matching with parametrization based on the SVD of a dimensionless sensitivity matrix. SPE J 15(02):495–508Google Scholar
  43. Torquato S, Beasley JD, Chiew YC (1988) Two-point cluster function for continuum percolation. J Chem Phys 88(10):6540–6547Google Scholar
  44. Tran D, Bourdev L, Fergus R, Torresani L, Paluri M (2015) Learning spatiotemporal features with 3D convolutional networks. Paper presented at the 2015 IEEE international conference on computer vision, Santiago, Chile, 13–16 December, pp 4489–4497Google Scholar
  45. Vo HX, Durlofsky LJ (2014) A new differentiable parameterization based on principal component analysis for the low-dimensional representation of complex geological models. Math Geosci 46(7):775–813Google Scholar
  46. Vo HX, Durlofsky LJ (2015) Data assimilation and uncertainty assessment for complex geological models using a new PCA-based parameterization. Comput Geosci 19(4):747–767Google Scholar
  47. Vo HX, Durlofsky LJ (2016) Regularized kernel PCA for the efficient parameterization of complex geological models. J Comput Phys 322:859–881Google Scholar
  48. Wang Y, Arns CH, Rahman SS, Arns JY (2018) Porous structure reconstruction using convolutional neural networks. Math Geosci 50(7):781–799Google Scholar
  49. Zhang T, Switzer P, Journel A (2006) Filter-based classification of training image patterns for spatial simulation. Math Geol 38(1):63–80Google Scholar

Copyright information

© International Association for Mathematical Geosciences 2019

Authors and Affiliations

  1. 1.Department of Energy Resources EngineeringStanford UniversityStanfordUSA

Personalised recommendations