Skip to main content
SpringerLink
Log in

Theoretical connections between optimization algorithms based on an approximate gradient

  • ORIGINAL PAPER
  • Published:
Computational Geosciences Aims and scope Submit manuscript

Abstract

Performing a line search method in the direction given by the simplex gradient is a well-known method in the mathematical optimization community. For reservoir engineering optimization problems, both a modification of the simultaneous perturbation stochastic approximation (SPSA) and ensemble-based optimization (EnOpt) have recently been applied for estimating optimal well controls in the production optimization step of closed-loop reservoir management. The modified SPSA algorithm has also been applied to assisted history-matching problems. A recent comparison of the performance of EnOpt and a SPSA-type algorithm (G-SPSA) for a set of production optimization test problems showed that the two algorithms resulted in similar estimates of the optimal net-present-value and required roughly the same amount of computational time to achieve these estimates. Here, we show that, theoretically, this result is not surprising. In fact, we show that both the simplex, preconditioned simplex, and EnOpt algorithms can be derived directly from a modified SPSA-type algorithm where the preconditioned simplex algorithm is presented for the first time in this paper. We also show that the expectation of all these preconditioned stochastic gradients is a first-order approximation of the preconditioning covariance matrix times the true gradient or a covariance matrix squared times the true gradient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bangerth, W., Klie, H., Wheeler, M., Stoffa, P., Sen, M.: On optimization algorithm for the reservoir oil well placement problem. Comput. Geosci. 10, 303–319 (2006)

    Article  Google Scholar 

  2. Bortz, D., Kelley, C.: The simplex gradient and noisy optimization problems. In: Borggaard, J.T., Burns, J., Cliff, E., Schreck, S. (eds.) Computational Methods for Optimal Design and Control. Progress in Systems and Control, vol. 24, p. 7790. Birkhauser (1998)

  3. Brouwer, D.R., Nævdal, G., Jansen, J.D., Vefring, E.H., van Kruijsdijk, C.P.J.W.: Improved reservoir management through optimal control and continuous model updating. In: Proceedings of the SPE Annual Technical Conference and Exhibition, 26-29 September, SPE 90149. Houston (2004)

  4. Chen, C., Li, G., Reynolds, A.C.: Closed-loop reservoir management on the Brugge test case. Comput. Geosci. 14(4), 691–703 (2010)

    Article  Google Scholar 

  5. Chen, C., Li, G., Reynolds, A.C.: Robust constrained optimization of short and long-term NPV for closed-loop reservoir management. SPEJ 17(3), 849–864 (2011)

    Google Scholar 

  6. Chen, Y., Oliver, D.: Ensemble-based closed-loop optimization applied to Brugge field. SPE Reserv. Eval. Eng. 13(1), 56–71 (2010)

    Google Scholar 

  7. Chen, Y., Oliver, D.S., Zhang, D.: Efficient ensemble-based closed-loop production optimization. SPE J. 14(4), 634–645 (2009)

    Google Scholar 

  8. Conn, A.R., Scheinberg, S., Vicente, L.N.: Introduction to Derivative-Free Optimization. SIAM, Philadelphia (2008)

    Google Scholar 

  9. Custódio, A.L., Vicente, L.N.: Using sampling and simplex derivatives in pattern search methods. SIAM J. Optim. 18, 537–555 (2007)

    Article  Google Scholar 

  10. de Montleau, P., Cominelli, A., Neylong, K., Rowan, D., Pallister, I., Tesaker, O., Nygard, I.: Production optimization under constraints using adjoint gradients. In: Proceedings of 10th European Conference on the Mathematics of Oil Recovery (2006)

  11. Do, S.: Application SPSA-Type Algorithms to Production Optimization. Ph.D. thesis, The University of Tulsa, Tulsa (2012)

  12. Fonseca, R., Leeuwenburgh, O., den Hof, P.V., Jansen, J.D.: Improving the ensemble optimization procedure through covariance matrix adaptation (CMA-EnOpt). In: Proceedings of the SPE Reservoir Simulation Symposium, SPE 163657 (2013)

  13. Gao, G., Reynolds, A.C.: An improved implementation of the LBFGS algorithm for automatic history matching. SPE J. 11(1), 5–17 (2006)

    Google Scholar 

  14. Golub, G.H., van Loan, C.F.: Matrix Computations, 2nd edn. The Johns Hopkins University Press, Baltimore (1989)

    Google Scholar 

  15. Gu, Y., Oliver, D.S.: An iterative ensemble Kalman filter for multiphase fluid flow data assimilation. SPE J. 12(4), 438–446 (2007)

    Google Scholar 

  16. Jansen, J., Brouwer, D., Naevdal, G., van Kruijsdijk, C.: Closed-loop reservoir management. First Break 23, 43–48 (2005)

    Google Scholar 

  17. Jansen, J.D., Douma, S.D., Brouwer, D.R., den Hof, P.M.J.V., Heemink, A.W.: Closed-loop reservoir management. In: Proceedings of the SPE Reservoir Simulation Symposium. The Woodlands, 24 February, SPE 119098. Texas (2009)

  18. Kelley, C.: Detection and remediation of stagnation in Nelder–Mead algorithm using a sufficient decrease condition. SIAM J. Optim. 10, 43–55 (1999)

    Article  Google Scholar 

  19. Kraaijevanger, J.F.B.M., Egberts, P.J.P., Valstar, J.R., Buurman, H.W.: Optimal waterflood design using the adjoint method. In: Proceedings of the SPE Reservoir Simulation Symposium, p. 15. SPE 105764 (2007)

  20. Li, G., Reynolds, A.C.: Uncertainty quantification of reservoir performance predictions using a stochastic optimization algorithm. Comput. Geosci. 15(3), 451–462 (2011)

    Article  Google Scholar 

  21. Li, R., Reynolds, A.C., Oliver, D.S.: History matching of three-phase flow production data. SPE J. 8(4), 328–340 (2003)

    Google Scholar 

  22. Li, R., Reynolds, A.C., Oliver, D.S.: Sensitivity coefficients for three-phase flow history matching. J. Can. Pet. Tech. 42(4), 70–77 (2003)

    Google Scholar 

  23. Lorentzen, R.J., Berg, A.M., Nævdal, G., Vefring, E.H.: A new approach for dynamic optimization of waterflooding problems. In: Proceedings of the SPE Intelligent Energy Conference and Exhibition, SPE 99690 (2006)

  24. Nævdal, G., Brower, D.R., Jansen, J.-D.: Waterflooding using closed-loop control. Comput. Geosci. 10(1), 37–60 (2006)

    Article  Google Scholar 

  25. Oliver, D.S., Chen, Y.: Recent progress on reservoir history matching: a review. Comput. Geosci. 15(1), 185–221 (2010)

    Article  Google Scholar 

  26. Oliver, D.S., Reynolds, A.C., Liu, N.: Inverse Theory for Petroleum Reservoir Characterization and history Matching. Cambridge University Press, Cambridge (2008)

    Book  Google Scholar 

  27. Peters, L., Arts, R., Brouwer, G., Geel, C., Cullick, S., Lorentzen, R., Chen, Y., Dunlop, K., Vossepoel, F., Xu, R., Sarma, P., Alhuthali, A., Reynolds, A.: Results of the Brugge benchmark study for flooding optimisation and history matching. SPE Reserv. Eval. Eng. 13(3), 391–405 (2010)

    Google Scholar 

  28. Reynolds, A.C., Zafari, M., Li, G.: Iterative forms of the ensemble Kalman filter. In: Proceedings of 10th European Conference on the Mathematics of Oil Recovery, 4–7 September. Amsterdam (2006)

  29. Rodrigues, J.R.P.: Calculating derivatives for history matching in reservoir simulators. In: Proceedings of the SPE Reservoir Simulation Symposium, p. 9. SPE 93445 (2005)

  30. Rodrigues, J.R.P.: Calculating derivatives for automatic history matching. Comput. Geosci. 10, 119–136 (2006)

    Article  Google Scholar 

  31. Sarma, P., Chen, W., Durlofsky, L., Aziz, K.: Production optimization with adjoint models under nonlinear control-state path inequality constraints. SPE Reserv. Eval. Eng. 11(2), 326–339 (2008)

    Google Scholar 

  32. Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37(3), 332–341 (1992)

    Article  Google Scholar 

  33. Spall, J.C.: Implementation of the simultaneous perturbation algorithm for stochastic optimization. IEEE Trans. Aerosp. Electron. Syst. 34(3), 817–823 (1998)

    Article  Google Scholar 

  34. Spall, J.C.: Stochastic optimization and the simultaneous perturbation method. In: Proceedings of the Winter Simulation Conference (1999)

  35. Spall, J.C.: Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans. Autom. Control 45(10), 1839–1853 (2000)

    Article  Google Scholar 

  36. Spall, J.C.: Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control. Wiley, Hoboken (2003)

    Book  Google Scholar 

  37. Tseng, P.: Fortified-descent simplicial search method: a general approach. SIAM J. Optim. 10, 269–288 (1999)

    Article  Google Scholar 

  38. van Essen, G., den Hof, P.V., Jansen, J.: Hierarchical long-term and short-term production optimization. In: Proceedings of the SPE Annual Technical Conference and Exhibition, SPE 124332 (2009)

  39. van Essen, G., den Hof, P.V., Jansen, J.: Hierarchical long-term and short-term production optimization. SPE J. 16(1), 191–199 (2011)

    Google Scholar 

  40. van Essen, G., Zandvliet, M., den Hof, P.V., Bosgra, O., Jansen, J.: Robust waterflooding optimization of multiple geological scenarios. SPE J. 14(1), 202–210 (2009)

    Google Scholar 

  41. Wang, C., Li, G., Reynolds, A.C.: Production optimization in closed-loop reservoir management. SPE J. 14(3), 506–523 (2009)

    Google Scholar 

  42. Wu, Z., Reynolds, A.C., Oliver, D.S.: Conditioning geostatistical models to two-phase production data. SPE J. 3(2), 142–155 (1999)

    Google Scholar 

  43. Yan, X., Reynolds, A.C.: An optimization algorithm based on combining finite difference approximations and stochastic gradients. In: Proceedings of the SPE Reservoir Simulation Symposium. The Woodlands, 18-20 February, SPE 163645. Texas (2013)

  44. Zakirov, I.S., Aanonsen, S.I., Zakirov, E.S., Palatnik, B.M.: Optimizating reservoir performance by automatic allocation of well rates. In: Proceedings of the 5th European Conference on the Mathematical Oil Recovery - Leoben, 35 September. Austria (1996)

  45. Zandvliet, M., Bosgra, O., Jasen, J., den Hof, P.V., Kraaijevvanger, J.: Bang-bang control and singular arcs in reservoir flooding. J. Pet. Sci. Eng. 58, 186–200 (2007)

    Article  Google Scholar 

  46. Zhang, F., Reynolds, A.C.: Optimization algorithms for automatic history matching of production data. In: Proceedings of 8th European Conference on the Mathematics of Oil Recovery, 36 September. Freiberg (2002)

  47. Zhao, H., Chen, C., Do, S., Li, G., Reynolds, A.: Maximization of a dynamic quadratic interpolation model for production optimization. In: Proceedings of the SPE Reservoir Simulation Symposium. The Woodlands, 2123 February (accepted for publication in SPE Journal), SPE 141317. Texas (2011)

Download references

Author information

Authors and Affiliations

  1. University of Tulsa, 800 South Tucker Drive, Tulsa, OK, 74104, USA

    Sy T. Do & Albert C. Reynolds

Authors
  1. Sy T. Do
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Albert C. Reynolds
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Albert C. Reynolds.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Do, S.T., Reynolds, A.C. Theoretical connections between optimization algorithms based on an approximate gradient. Comput Geosci 17, 959–973 (2013). https://doi.org/10.1007/s10596-013-9368-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10596-013-9368-9

Keywords

Search

Navigation