Computational Geosciences

, Volume 17, Issue 6, pp 959–973 | Cite as

Theoretical connections between optimization algorithms based on an approximate gradient

  • Sy T. Do
  • Albert C. Reynolds


Performing a line search method in the direction given by the simplex gradient is a well-known method in the mathematical optimization community. For reservoir engineering optimization problems, both a modification of the simultaneous perturbation stochastic approximation (SPSA) and ensemble-based optimization (EnOpt) have recently been applied for estimating optimal well controls in the production optimization step of closed-loop reservoir management. The modified SPSA algorithm has also been applied to assisted history-matching problems. A recent comparison of the performance of EnOpt and a SPSA-type algorithm (G-SPSA) for a set of production optimization test problems showed that the two algorithms resulted in similar estimates of the optimal net-present-value and required roughly the same amount of computational time to achieve these estimates. Here, we show that, theoretically, this result is not surprising. In fact, we show that both the simplex, preconditioned simplex, and EnOpt algorithms can be derived directly from a modified SPSA-type algorithm where the preconditioned simplex algorithm is presented for the first time in this paper. We also show that the expectation of all these preconditioned stochastic gradients is a first-order approximation of the preconditioning covariance matrix times the true gradient or a covariance matrix squared times the true gradient.


G-SPSA Ensemble-based optimization (EnOpt) Simplex gradient Optimal well control 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bangerth, W., Klie, H., Wheeler, M., Stoffa, P., Sen, M.: On optimization algorithm for the reservoir oil well placement problem. Comput. Geosci. 10, 303–319 (2006)CrossRefGoogle Scholar
  2. 2.
    Bortz, D., Kelley, C.: The simplex gradient and noisy optimization problems. In: Borggaard, J.T., Burns, J., Cliff, E., Schreck, S. (eds.) Computational Methods for Optimal Design and Control. Progress in Systems and Control, vol. 24, p. 7790. Birkhauser (1998)Google Scholar
  3. 3.
    Brouwer, D.R., Nævdal, G., Jansen, J.D., Vefring, E.H., van Kruijsdijk, C.P.J.W.: Improved reservoir management through optimal control and continuous model updating. In: Proceedings of the SPE Annual Technical Conference and Exhibition, 26-29 September, SPE 90149. Houston (2004)Google Scholar
  4. 4.
    Chen, C., Li, G., Reynolds, A.C.: Closed-loop reservoir management on the Brugge test case. Comput. Geosci. 14(4), 691–703 (2010)CrossRefGoogle Scholar
  5. 5.
    Chen, C., Li, G., Reynolds, A.C.: Robust constrained optimization of short and long-term NPV for closed-loop reservoir management. SPEJ 17(3), 849–864 (2011)Google Scholar
  6. 6.
    Chen, Y., Oliver, D.: Ensemble-based closed-loop optimization applied to Brugge field. SPE Reserv. Eval. Eng. 13(1), 56–71 (2010)Google Scholar
  7. 7.
    Chen, Y., Oliver, D.S., Zhang, D.: Efficient ensemble-based closed-loop production optimization. SPE J. 14(4), 634–645 (2009)Google Scholar
  8. 8.
    Conn, A.R., Scheinberg, S., Vicente, L.N.: Introduction to Derivative-Free Optimization. SIAM, Philadelphia (2008)Google Scholar
  9. 9.
    Custódio, A.L., Vicente, L.N.: Using sampling and simplex derivatives in pattern search methods. SIAM J. Optim. 18, 537–555 (2007)CrossRefGoogle Scholar
  10. 10.
    de Montleau, P., Cominelli, A., Neylong, K., Rowan, D., Pallister, I., Tesaker, O., Nygard, I.: Production optimization under constraints using adjoint gradients. In: Proceedings of 10th European Conference on the Mathematics of Oil Recovery (2006)Google Scholar
  11. 11.
    Do, S.: Application SPSA-Type Algorithms to Production Optimization. Ph.D. thesis, The University of Tulsa, Tulsa (2012)Google Scholar
  12. 12.
    Fonseca, R., Leeuwenburgh, O., den Hof, P.V., Jansen, J.D.: Improving the ensemble optimization procedure through covariance matrix adaptation (CMA-EnOpt). In: Proceedings of the SPE Reservoir Simulation Symposium, SPE 163657 (2013)Google Scholar
  13. 13.
    Gao, G., Reynolds, A.C.: An improved implementation of the LBFGS algorithm for automatic history matching. SPE J. 11(1), 5–17 (2006)Google Scholar
  14. 14.
    Golub, G.H., van Loan, C.F.: Matrix Computations, 2nd edn. The Johns Hopkins University Press, Baltimore (1989)Google Scholar
  15. 15.
    Gu, Y., Oliver, D.S.: An iterative ensemble Kalman filter for multiphase fluid flow data assimilation. SPE J. 12(4), 438–446 (2007)Google Scholar
  16. 16.
    Jansen, J., Brouwer, D., Naevdal, G., van Kruijsdijk, C.: Closed-loop reservoir management. First Break 23, 43–48 (2005)Google Scholar
  17. 17.
    Jansen, J.D., Douma, S.D., Brouwer, D.R., den Hof, P.M.J.V., Heemink, A.W.: Closed-loop reservoir management. In: Proceedings of the SPE Reservoir Simulation Symposium. The Woodlands, 24 February, SPE 119098. Texas (2009)Google Scholar
  18. 18.
    Kelley, C.: Detection and remediation of stagnation in Nelder–Mead algorithm using a sufficient decrease condition. SIAM J. Optim. 10, 43–55 (1999)CrossRefGoogle Scholar
  19. 19.
    Kraaijevanger, J.F.B.M., Egberts, P.J.P., Valstar, J.R., Buurman, H.W.: Optimal waterflood design using the adjoint method. In: Proceedings of the SPE Reservoir Simulation Symposium, p. 15. SPE 105764 (2007)Google Scholar
  20. 20.
    Li, G., Reynolds, A.C.: Uncertainty quantification of reservoir performance predictions using a stochastic optimization algorithm. Comput. Geosci. 15(3), 451–462 (2011)CrossRefGoogle Scholar
  21. 21.
    Li, R., Reynolds, A.C., Oliver, D.S.: History matching of three-phase flow production data. SPE J. 8(4), 328–340 (2003)Google Scholar
  22. 22.
    Li, R., Reynolds, A.C., Oliver, D.S.: Sensitivity coefficients for three-phase flow history matching. J. Can. Pet. Tech. 42(4), 70–77 (2003)Google Scholar
  23. 23.
    Lorentzen, R.J., Berg, A.M., Nævdal, G., Vefring, E.H.: A new approach for dynamic optimization of waterflooding problems. In: Proceedings of the SPE Intelligent Energy Conference and Exhibition, SPE 99690 (2006)Google Scholar
  24. 24.
    Nævdal, G., Brower, D.R., Jansen, J.-D.: Waterflooding using closed-loop control. Comput. Geosci. 10(1), 37–60 (2006)CrossRefGoogle Scholar
  25. 25.
    Oliver, D.S., Chen, Y.: Recent progress on reservoir history matching: a review. Comput. Geosci. 15(1), 185–221 (2010)CrossRefGoogle Scholar
  26. 26.
    Oliver, D.S., Reynolds, A.C., Liu, N.: Inverse Theory for Petroleum Reservoir Characterization and history Matching. Cambridge University Press, Cambridge (2008)CrossRefGoogle Scholar
  27. 27.
    Peters, L., Arts, R., Brouwer, G., Geel, C., Cullick, S., Lorentzen, R., Chen, Y., Dunlop, K., Vossepoel, F., Xu, R., Sarma, P., Alhuthali, A., Reynolds, A.: Results of the Brugge benchmark study for flooding optimisation and history matching. SPE Reserv. Eval. Eng. 13(3), 391–405 (2010)Google Scholar
  28. 28.
    Reynolds, A.C., Zafari, M., Li, G.: Iterative forms of the ensemble Kalman filter. In: Proceedings of 10th European Conference on the Mathematics of Oil Recovery, 4–7 September. Amsterdam (2006)Google Scholar
  29. 29.
    Rodrigues, J.R.P.: Calculating derivatives for history matching in reservoir simulators. In: Proceedings of the SPE Reservoir Simulation Symposium, p. 9. SPE 93445 (2005)Google Scholar
  30. 30.
    Rodrigues, J.R.P.: Calculating derivatives for automatic history matching. Comput. Geosci. 10, 119–136 (2006)CrossRefGoogle Scholar
  31. 31.
    Sarma, P., Chen, W., Durlofsky, L., Aziz, K.: Production optimization with adjoint models under nonlinear control-state path inequality constraints. SPE Reserv. Eval. Eng. 11(2), 326–339 (2008)Google Scholar
  32. 32.
    Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37(3), 332–341 (1992)CrossRefGoogle Scholar
  33. 33.
    Spall, J.C.: Implementation of the simultaneous perturbation algorithm for stochastic optimization. IEEE Trans. Aerosp. Electron. Syst. 34(3), 817–823 (1998)CrossRefGoogle Scholar
  34. 34.
    Spall, J.C.: Stochastic optimization and the simultaneous perturbation method. In: Proceedings of the Winter Simulation Conference (1999)Google Scholar
  35. 35.
    Spall, J.C.: Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans. Autom. Control 45(10), 1839–1853 (2000)CrossRefGoogle Scholar
  36. 36.
    Spall, J.C.: Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control. Wiley, Hoboken (2003)CrossRefGoogle Scholar
  37. 37.
    Tseng, P.: Fortified-descent simplicial search method: a general approach. SIAM J. Optim. 10, 269–288 (1999)CrossRefGoogle Scholar
  38. 38.
    van Essen, G., den Hof, P.V., Jansen, J.: Hierarchical long-term and short-term production optimization. In: Proceedings of the SPE Annual Technical Conference and Exhibition, SPE 124332 (2009)Google Scholar
  39. 39.
    van Essen, G., den Hof, P.V., Jansen, J.: Hierarchical long-term and short-term production optimization. SPE J. 16(1), 191–199 (2011)Google Scholar
  40. 40.
    van Essen, G., Zandvliet, M., den Hof, P.V., Bosgra, O., Jansen, J.: Robust waterflooding optimization of multiple geological scenarios. SPE J. 14(1), 202–210 (2009)Google Scholar
  41. 41.
    Wang, C., Li, G., Reynolds, A.C.: Production optimization in closed-loop reservoir management. SPE J. 14(3), 506–523 (2009)Google Scholar
  42. 42.
    Wu, Z., Reynolds, A.C., Oliver, D.S.: Conditioning geostatistical models to two-phase production data. SPE J. 3(2), 142–155 (1999)Google Scholar
  43. 43.
    Yan, X., Reynolds, A.C.: An optimization algorithm based on combining finite difference approximations and stochastic gradients. In: Proceedings of the SPE Reservoir Simulation Symposium. The Woodlands, 18-20 February, SPE 163645. Texas (2013)Google Scholar
  44. 44.
    Zakirov, I.S., Aanonsen, S.I., Zakirov, E.S., Palatnik, B.M.: Optimizating reservoir performance by automatic allocation of well rates. In: Proceedings of the 5th European Conference on the Mathematical Oil Recovery - Leoben, 35 September. Austria (1996)Google Scholar
  45. 45.
    Zandvliet, M., Bosgra, O., Jasen, J., den Hof, P.V., Kraaijevvanger, J.: Bang-bang control and singular arcs in reservoir flooding. J. Pet. Sci. Eng. 58, 186–200 (2007)CrossRefGoogle Scholar
  46. 46.
    Zhang, F., Reynolds, A.C.: Optimization algorithms for automatic history matching of production data. In: Proceedings of 8th European Conference on the Mathematics of Oil Recovery, 36 September. Freiberg (2002)Google Scholar
  47. 47.
    Zhao, H., Chen, C., Do, S., Li, G., Reynolds, A.: Maximization of a dynamic quadratic interpolation model for production optimization. In: Proceedings of the SPE Reservoir Simulation Symposium. The Woodlands, 2123 February (accepted for publication in SPE Journal), SPE 141317. Texas (2011)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.University of TulsaTulsaUSA

Personalised recommendations