Computational Geosciences

, Volume 22, Issue 1, pp 145–161 | Cite as

Calibration of imperfect models to biased observations

  • Dean S. Oliver
  • Miguel Alfonzo
Original Paper


The problem of assimilating biased and inaccurate observations into inadequate models of the physical systems from which the observations were taken is common in the petroleum and groundwater fields. When large amounts of data are assimilated without accounting for model error and observation bias, predictions tend to be both overconfident and incorrect. In this paper, we propose a workflow for calibration of imperfect models to biased observations that involves model construction, model calibration, model criticism and model improvement. Model criticism is based on computation of model diagnostics which provide an indication of the validity of assumptions. During the model improvement step, we advocate identification of additional physically motivated parameters based on examination of data mismatch after calibration and addition of bias correction terms. If model diagnostics indicates the presence of residual model error after parameters have been added, then we advocate estimation of a “total” observation error covariance matrix, whose purpose is to reduce weighting of observations that cannot be matched because of deficiency of the model. Although the target applications of this methodology are in the subsurface, we illustrate the approach with two simplified examples involving prediction of the future velocity of fall of a sphere from models calibrated to a short-time series of biased measurements with independent additive random noise. The models into which the data are assimilated contain model errors due to neglect of physical processes and neglect of uncertainty in parameters. In every case, the estimated total error covariance is larger than the true observation covariance implying that the observations need not be matched to the accuracy of the measuring instrument. Predictions are much improved when all model improvement steps were taken.


Model calibration History matching Model error Observation bias Predictability Randomized maximum likelihood Data assimilation Model improvement 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



Primary support for the authors has been provided by the CIPR/IRIS cooperative research project “4D Seismic History Matching” which is funded by industry partners Eni, Petrobras, and Total, as well as the Research Council of Norway through the PETROMAKS2 program.


  1. 1.
    Aanonsen, S.I., Aavatsmark, I., Barkve, T., Cominelli, A., Gonard, R., Gosselin, O., Kolasinski, M., Reme, H.: Effect of scale dependent data correlations in an integrated history matching loop combining production data and 4D seismic data (SPE 79665). In: Proc SPE RSS, The Woodlands (2003)Google Scholar
  2. 2.
    Aanonsen, S.I., Eydinov, D.: A multiscale method for distributed parameter estimation with application to reservoir history matching. Comput. Geosci. 10(1), 97–117 (2006)CrossRefGoogle Scholar
  3. 3.
    Achenbach, E.: Influence of surface roughness on cross-flow around a circular cylinder. J. Fluid Mech. 46(2), 321–335 (1971)CrossRefGoogle Scholar
  4. 4.
    Adam, L., Batzle, M., Brevik, I.: Gassmann’s fluid substitution and shear modulus variability in carbonates at laboratory seismic and ultrasonic frequencies. Geophysics 71(6), F173–F183 (2006)CrossRefGoogle Scholar
  5. 5.
    Auligné, T., McNally, A.P., Dee, D.P.: Adaptive bias correction for satellite data in a numerical weather prediction system. Q. J. Roy. Meteor. Soc. 133(624), 631–642 (2007)CrossRefGoogle Scholar
  6. 6.
    Aziz, K., Settari, A.: Petroleum reservoir simulation. Elsevier Applied Science Publishers, London (1979)Google Scholar
  7. 7.
    Bardsley, J., Solonen, A., Haario, H., Laine, M.: Randomize-then-optimize: a method for sampling from posterior distributions in nonlinear inverse problems. SIAM J. Sci. Comput. 36(4), A1895–A1910 (2014)CrossRefGoogle Scholar
  8. 8.
    Bayarri, M.J., Berger, J.O.: P values for composite null models. J. Am. Stat. Assoc. 95(452), 1127–1142 (2000)Google Scholar
  9. 9.
    Bayarri, M.J., Berger, J.O., Paulo, R., Sacks, J., Cafeo, J.A., Cavendish, J., Lin, C.-H., Tu, J.: A framework for validation of computer models. Technometrics 49(2), 138–154 (2007)CrossRefGoogle Scholar
  10. 10.
    Bennett, N.D., Croke, B.F., Guariso, G., Guillaume, J.H., Hamilton, S.H., Jakeman, A.J., Marsili-Libelli, S., Newham, L.T., Norton, J.P., Perrin, C., Pierce, S.A., Robson, B., Seppelt, R., Voinov, A.A., Fath, B.D., Andreassian, V.: Characterising performance of environmental models. Environ. Model Softw. 40, 1–20 (2013)CrossRefGoogle Scholar
  11. 11.
    Box, G.E.P., Tiao, G.C.: Bayesian inference in statistical analysis. Addison-Wesley Publishing Company (1973)Google Scholar
  12. 12.
    Brajard, J., Sirven, J., Talagrand, O.: Ensemble variational data assimilation with a shallow-water model: preliminary results. In: EGU General Assembly 2016, Held 17–22 April in Vienna, Austria (2016)Google Scholar
  13. 13.
    Capen, E.C.: The difficulty of assessing uncertainty. J. Pet. Technol. 28(8), 843–850 (1976)CrossRefGoogle Scholar
  14. 14.
    Chapnik, B., Desroziers, G., Rabier, F., Talagrand, O.: Properties and first application of an error-statistics tuning method in variational assimilation. Q. J. Roy. Meteor. Soc. 130(601), 2253–2275 (2004)CrossRefGoogle Scholar
  15. 15.
    Chapnik, B., Desroziers, G., Rabier, F., Talagrand, O.: Diagnosis and tuning of observational error in a quasi-operational data assimilation setting. Q. J. R. Meteorol. Soc. 132(615), 543–565 (2006)CrossRefGoogle Scholar
  16. 16.
    Chen, Y., Oliver, D.S.: Parameterization techniques to improve mass conservation and data assimilation for ensemble Kalman filter (SPE 133560). In: SPE Western Regional Meeting, 27–29 May 2010, Anaheim, California, USA (2010)Google Scholar
  17. 17.
    Chen, Y., Oliver, D.S.: Levenberg-Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification. Comput. Geosci. 17(4), 689–703 (2013)CrossRefGoogle Scholar
  18. 18.
    Chen, Y., Oliver, D.S.: History matching of the Norne full-field model with an iterative ensemble smoother. SPE Reserv. Eval. Eng. 17(2), 244–256 (2014)CrossRefGoogle Scholar
  19. 19.
    Chilès, J.-P., Delfiner, P.: Geostatistics: modeling spatial uncertainty, 2nd edn. Wiley, New York (2012)CrossRefGoogle Scholar
  20. 20.
    Christensen, R., Johnson, W., Branscum, A., Hanson, T.E.: Bayesian ideas and data analysis: an introduction for scientists and statisticians. CRC Press (2011)Google Scholar
  21. 21.
    Cordua, K.S., Nielsen, L., Looms, M.C., Hansen, T.M., Binley, A.: Quantifying the influence of static-like errors in least-squares-based inversion and sequential simulation of cross-borehole ground penetrating radar data. J. Appl. Geophys. 68(1), 71–84 (2009)CrossRefGoogle Scholar
  22. 22.
    Dake, L.P.: The practice of reservoir engineering, volume 36 of Developments in Petroleum Science. Elsevier (1994)Google Scholar
  23. 23.
    Dee, D.: On-line estimation of error covariance parameters for atmospheric data assimilation. Mon. Weather Rev. 123(4), 1128–1145 (1995)CrossRefGoogle Scholar
  24. 24.
    Dee, D.P.: Bias and data assimilation. Q. J. R. Meteorol. Soc. 131(613), 3323–3343 (2005)CrossRefGoogle Scholar
  25. 25.
    Dee, D.P., da Silva, A.M.: Maximum-likelihood estimation of forecast and observation error covariance parameters. Part I: methodology. Mon. Weather. Rev. 127(8), 1822–1834 (1999)CrossRefGoogle Scholar
  26. 26.
    Desroziers, G., Berre, L., Chabot, V., Chapnik, B.: A posteriori diagnostics in an ensemble of perturbed analyses. Mon. Weather Rev. 137(10), 3420–3436 (2009)CrossRefGoogle Scholar
  27. 27.
    Desroziers, G., Berre, L., Chapnik, B., Poli, P.: Diagnosis of observation, background and analysis-error statistics in observation space. Q. J. R. Meteorol. Soc. 131(613, C), 3385–3396 (2005)CrossRefGoogle Scholar
  28. 28.
    Desroziers, G., Ivanov, S.: Diagnosis and adaptive tuning of observation-error parameters in a variational assimilation. Q. J. R. Meteorol. Soc. 127(574, B), 1433–1452 (2001)CrossRefGoogle Scholar
  29. 29.
    Doherty, J.: Model-based environmental decision-making. (2010)
  30. 30.
    Doherty, J., Christensen, S.: Use of paired simple and complex models to reduce predictive bias and quantify uncertainty. Water Resour. Res. 47(12), W12534 (2011)CrossRefGoogle Scholar
  31. 31.
    Doherty, J., Welter, D.: A short exploration of structural noise. Water Resour. Res. 46(5), W05525 (2010)CrossRefGoogle Scholar
  32. 32.
    Draper, D.: Assessment and propagation of model uncertainty. J. R. Stat. Soc. Ser. B Methodol. 57(1), 45–97 (1995)Google Scholar
  33. 33.
    Evin, G., Thyer, M., Kavetski, D., McInerney, D., Kuczera, G.: Comparison of joint versus postprocessor approaches for hydrological uncertainty estimation accounting for error autocorrelation and heteroscedasticity. Water Resour. Res. 50(3), 2350–2375 (2014)CrossRefGoogle Scholar
  34. 34.
    Fertig, E., Baek, S.-J., Hunt, B., Ott, E., Szunyogh, I., Aravéquia, J., Kalnay, E., Li, H., Liu, J.: Observation bias correction with an ensemble Kalman filter. Tellus A 61(2) (2009)Google Scholar
  35. 35.
    Garthwaite, P.H., Kadane, J.B., O’Hagan, A.: Statistical methods for eliciting probability distributions. J. Am. Stat. Assoc. 100(470), 680–701 (2005)CrossRefGoogle Scholar
  36. 36.
    Gelman, A., Meng, X.-L.: Model checking and model improvement. In: Gilks, W.R., Richardson, S., Spiegelhalter, D.J. (eds.) Markov Chain Monte Carlo in Practice, pp. 189–201. Chapman & Hall, New York (1996)Google Scholar
  37. 37.
    Gelman, A., Meng, X.-L., Stern, H.: Posterior predictive assessment of model fitness via realized discrepancies. Stat. Sin. 6(4), 733–760 (1996)Google Scholar
  38. 38.
    Gelman, A., Shalizi, C.R.: Philosophy and the practice of Bayesian statistics. Br. J. Math. Stat. Psychol. 66(1), 8–38 (2013)CrossRefGoogle Scholar
  39. 39.
    Hansen, T.M., Cordua, K.S., Jacobsen, B.H., Mosegaard, K.: Accounting for imperfect forward modeling in geophysical inverse problems — exemplified for crosshole tomography. Geophysics 79(3), H1–H21 (2014)CrossRefGoogle Scholar
  40. 40.
    Higdon, D., Gattiker, J., Williams, B., Rightley, M.: Computer model calibration using high-dimensional output. J. Am. Stat. Assoc. 103(482), 570–583 (2008)CrossRefGoogle Scholar
  41. 41.
    Hunt, R.J., Doherty, J., Tonkin, M.J.: Are models too simple? Arguments for increased parameterization. Ground Water 45(3), 254–262 (2007)CrossRefGoogle Scholar
  42. 42.
    Jafarpour, B., Khodabakhshi, M.: A probability conditioning method (PCM) for nonlinear flow data integration into multipoint statistical facies simulation. Math. Geosci. 43, 133–164 (2011)CrossRefGoogle Scholar
  43. 43.
    Kennedy, M.C., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc. Ser. B (Stat Methodol.) 63(3), 425–464 (2001)CrossRefGoogle Scholar
  44. 44.
    Kersting, A.B., Efurd, D.W., Finnegan, D.L., Rokop, D.J., Smith, D.K., Thompson, J.L.: Migration of plutonium in ground water at the Nevada Test Site. Nature 397(6714), 56–59 (1999)CrossRefGoogle Scholar
  45. 45.
    Kitanidis, P.K.: Quasi-linear geostatistical theory for inversing. Water Resour. Res. 31(10), 2411–2419 (1995)CrossRefGoogle Scholar
  46. 46.
    Kynn, M.: The heuristics and biases bias in expert elicitation. J. R. Stat. Soc. A. Stat. Soc. 171(1), 239–264 (2008)Google Scholar
  47. 47.
    Ledoit, O., Wolf, M.: A well-conditioned estimator for large-dimensional covariance matrices. J. Multivar. Anal. 88(2), 365–411 (2004)CrossRefGoogle Scholar
  48. 48.
    Lødøen, O.P., Omre, H.: Scale-corrected ensemble Kalman filtering applied to production history conditioning in reservoir evaluation. SPE J. 13(2), 177–194 (2008)CrossRefGoogle Scholar
  49. 49.
    Lyotard, N., Shew, W.L., Bocquet, L., Pinton, J.-F.: Polymer and surface roughness effects on the drag crisis for falling spheres. Eur. Phys. J. B 60(4), 469–476 (2007)CrossRefGoogle Scholar
  50. 50.
    Mathai, A.M., Provost, S.B.: Quadratic forms in random variables: theory and applications. Marcel Dekker, Inc., New York (1992)Google Scholar
  51. 51.
    Mavko, G., Mukerji, T.: Bounds on low-frequency seismic velocities in partially saturated rocks. Geophysics 63(3), 918–924 (1998)CrossRefGoogle Scholar
  52. 52.
    Michel, Y.: Diagnostics on the cost-function in variational assimilations for meteorological models. Nonlinear Processes Geophys. 21(1), 187–199 (2014)CrossRefGoogle Scholar
  53. 53.
    Miyoshi, T., Kalnay, E., Li, H.: Estimating and including observation-error correlations in data assimilation. Inverse Prob. Sci. Eng. 21(3), 387–398 (2013)CrossRefGoogle Scholar
  54. 54.
    Moore, C., Doherty, J.: Role of the calibration process in reducing model predictive error. Water Resour. Res. 41(5), W05020 (2005)CrossRefGoogle Scholar
  55. 55.
    Mordant, N., Metz, P., Pinton, J.-F., Michel, O.: Acoustical technique for Lagrangian velocity measurement. Rev. Sci. Instrum. 76(2), 1–7 (2005)CrossRefGoogle Scholar
  56. 56.
    National Research Council: Assessing the reliability of complex models: mathematical and statistical foundations of verification, validation, and uncertainty quantification. National Academies Press. Committee on Mathematical Foundations of VV&UQ (2012)Google Scholar
  57. 57.
    O’Hagan, A.: Eliciting expert beliefs in substantial practical applications. Journal of the Royal Statistical Society. Series D (The Statistician) 47(1), 21–35 (1998)CrossRefGoogle Scholar
  58. 58.
    Oliver, D.S.: Minimization for conditional simulation: Relationship to optimal transport. J. Comput. Phys. 265(0), 1–15 (2014)CrossRefGoogle Scholar
  59. 59.
    Oliver, D.S.: Metropolized randomized maximum likelihood for improved sampling from multimodal distributions. SIAM/ASA Journal on Uncertainty Quantification 5(1), 259–277 (2017)CrossRefGoogle Scholar
  60. 60.
    Oliver, D.S., Chen, Y.: Recent progress on reservoir history matching: a review. Comput. Geosci. 15(1), 185–221 (2011)CrossRefGoogle Scholar
  61. 61.
    Oliver, D.S., He, N., Reynolds, A.C.: Conditioning permeability fields to pressure data. In: Proceedings of the European Conference on the Mathematics of Oil Recovery, V, pp. 1–11 (1996)Google Scholar
  62. 62.
    Oliver, D.S., Reynolds, A.C., Liu, N.: Inverse theory for petroleum reservoir characterization and history matching. Cambridge University Press, Cambridge (2008)CrossRefGoogle Scholar
  63. 63.
    Omre, H., Lødøen, O.P.: Improved production forecasts and history matching using approximate fluid-flow simulators. SPE J. 9(3), 339–351 (2004)CrossRefGoogle Scholar
  64. 64.
    Peters, L., Arts, R.J., Brouwer, G.K., Geel, C.R., Cullick, S., Lorentzen, R.J., Chen, Y., Dunlop, K.N.B., Vossepoel, F.C., Xu, R., Sarma, P., Alhutali, A.H., Reynolds, A.C.: Results of the Brugge benchmark study for flooding optimization and history matching. SPE Reserv. Evalu. Eng. 13(3), 391–405 (2010)CrossRefGoogle Scholar
  65. 65.
    Pringle, M.J., Lark, R.M.: Spatial analysis of model error, illustrated by soil Carbon dioxide emissions. Vadose Zone J. 5(1), 168–183 (2006)CrossRefGoogle Scholar
  66. 66.
    Rood, R.B.: The role of the model in the data assimilation system. In: Lahoz, W., Khattatov, B., Menard, R. (eds.) Data Assimilation: Making Sense of Observations, pp. 351–379. Springer (2010)Google Scholar
  67. 67.
    Rubin, D.B.: Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Stat. 12(4), 1151–1172 (1984)CrossRefGoogle Scholar
  68. 68.
    Silverton, A., Warner, M., Umpleby, A., Morgan, J., Irabor, K.: Non- physical water density as a proxy to improve data fit during acoustic FWI. In: 76Th EAGE Conference and Exhibition 2014 (2014)Google Scholar
  69. 69.
    Skjervheim, J.-A., Evensen, G., Aanonsen, S.I., Ruud, B.O., Johansen, T.A.: Incorporating 4D seismic data in reservoir simulation models using ensemble Kalman filter. SPE J. 12(3), 282–292 (2007)CrossRefGoogle Scholar
  70. 70.
    Smith, L.A.: Disentangling uncertainty and error: on the predictability of nonlinear systems. In: Mees, A.I. (ed.) Nonlinear Dynamics and Statistics, pp. 31–64. Birkhäuser (2000)Google Scholar
  71. 71.
    Sønstabø, J.K., Hellevik, L.R.: Numerical methods for engineers: a digital compendium. (2015)
  72. 72.
    Stewart, L.M., Dance, S.L., Nichols, N.K.: Correlated observation errors in data assimilation. Int. J. Numer. Methods Fluids 56(8), 1521–1527 (2008)CrossRefGoogle Scholar
  73. 73.
    Talagrand, O.: A posteriori verification of analysis and assimilation algorithms. In: Proceedings of Workshop on Diagnosis of Data Assimilations Systems, 2–4 Nov 1998, ECMFW, Reading, UK, pp. 17–28 (1999)Google Scholar
  74. 74.
    Talagrand, O.: Evaluation of assimilation algorithms. In: Lahoz, W., Khattatov, B., Menard, R. (eds.) Data Assimilation: Making Sense of Observations, pp. 217–240. Springer, Berlin (2010)CrossRefGoogle Scholar
  75. 75.
    Tarantola, A.: Inverse Problem Theory: Methods for Data Fitting and Model Parameter Estimation. Elsevier, Amsterdam (1987)Google Scholar
  76. 76.
    Ueno, G., Nakamura, N.: Bayesian estimation of the observation-error covariance matrix in ensemble-based filters. Q. J. Roy. Meteorol. Soc. 142(698), 2055–2080 (2016)CrossRefGoogle Scholar
  77. 77.
    Van Trees, H.L.: Detection, estimation, and modulation theory. Wiley (2004)Google Scholar
  78. 78.
    Waller, J.A., Dance, S.L., Nichols, N.K.: Theoretical insight into diagnosing observation error correlations using observation-minus-background and observation-minus-analysis statistics. Q. J. Roy. Meteorol. Soc. 142(694), 418–431 (2016)CrossRefGoogle Scholar
  79. 79.
    Watson, T.A., Doherty, J.E., Christensen, S.: Parameter and predictive outcomes of model simplification. Water Resour. Res. 49(7), 3952–3977 (2013)CrossRefGoogle Scholar
  80. 80.
    Williams, G.J.J., Mansfield, M., MacDonald, D.G., Bush, M.D., et al.: Top-down reservoir modelling. In: SPE Annual Technical Conference and Exhibition Held in Houston, Texas, 26–29 September. Society of Petroleum Engineers (2004)Google Scholar
  81. 81.
    Zhao, Y., Li, G., Reynolds, A.C.: Characterization of the measurement error in time-lapse seismic data and production data with an EM algorithm. Oil Gas Sci. Technol. 62(2, Sp. Iss. SI), 181–193 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Uni Research CIPRBergenNorway
  2. 2.University of BergenBergenNorway

Personalised recommendations