Simulation methods for stochastic storage problems: a statistical learning perspective

  • Michael Ludkovski
  • Aditya MaheshwariEmail author
Original Paper


We consider solution of stochastic storage problems through regression Monte Carlo methods. Taking a statistical learning perspective, we develop the dynamic emulation algorithm (DEA) that unifies the different existing approaches in a single modular template. We then investigate the two central aspects of regression architecture and experimental design that constitute DEA. For the regression piece, we discuss various non-parametric approaches, in particular introducing the use of Gaussian process regression in the context of stochastic storage. For simulation design, we compare the performance of traditional design (grid discretization), against space-filling, and several adaptive alternatives. The overall DEA template is illustrated with multiple examples drawing from natural gas storage valuation and optimal control of back-up generator in a microgrid.


Regression Monte Carlo Simulation design Gaussian process regression Natural gas storage Microgrid control 



  1. 1.
    Alais, J.C., Carpentier, P., De Lara, M.: Multi-usage hydropower single dam management: chance-constrained optimization and stochastic viability. Energy Syst. 8(1), 7–30 (2017)CrossRefGoogle Scholar
  2. 2.
    Alasseur, C., Balata, A., Ben Aziza, S., Maheshwari, A., Tankov, P., Warin, X.: Regression Monte Carlo for Microgrid Management. Tech. Rep., (2018). arXiv:1802.10352
  3. 3.
    Balata, A., Palczewski, J.: Regress-Later Monte Carlo for optimal control of Markov processes. Tech. Rep., (2017). arXiv:1712.09705
  4. 4.
    Balata, A., Palczewski, J.: Regress-Later Monte Carlo for Optimal Inventory Control with applications in energy. Tech. Rep., (2017). arXiv:1703.06461
  5. 5.
    Bäuerle, N., Riess, V.: Gas storage valuation with regime switching. Energy Syst. 7(3), 499–528 (2016)CrossRefGoogle Scholar
  6. 6.
    Binois, M., Gramacy, R.B., Ludkovski, M.: Practical heteroscedastic gaussian process modeling for large simulation experiments. J. Comput. Gr. Stat. 27(4), 808–821 (2018)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Boogert, A., de Jong, C.: Gas storage valuation using a Monte Carlo method. J. Deriv. 15(3), 81–98 (2008)CrossRefGoogle Scholar
  8. 8.
    Boogert, A., de Jong, C.: Gas storage valuation using a multi-factor price process. J. Energy Markets 4, 29–52 (2011)CrossRefGoogle Scholar
  9. 9.
    Bouchard, B., Warin, X.: Monte Carlo valuation of American options: Facts and new algorithms to improve existing methods. In: Carmona, R.A., Del Moral, P., Hu, P., Oudjane, N. (eds.) Numerical Methods in Finance: Bordeaux, June 2010, pp. 215–255. Springer, Berlin Heidelberg, Berlin, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Carmona, R., Ludkovski, M.: Valuation of energy storage: an optimal switching approach. Quant. Financ. 10(4), 359–374 (2010)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Chen, Z., Forsyth, P.A.: A semi-Lagrangian approach for natural gas storage valuation and optimal operation. SIAM J. Sci. Comput. 30(1), 339–368 (2008)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Cong, F., Oosterlee, C.: Multi-period meanvariance portfolio optimization based on Monte Carlo simulation. J. Econo. Dyn. Control 64, 23–38 (2016)CrossRefGoogle Scholar
  13. 13.
    Davison, M., Zhao, G.: Optimal control of two-dam hydro facility. Syst. Eng. Procedia 3, 1–12 (2012)CrossRefGoogle Scholar
  14. 14.
    Denault, M., Simonato, J.G., Stentoft, L.: A simulation-and-regression approach for stochastic dynamic programs with endogenous state variables. Comput. Oper. Res. 40(11), 2760–2769 (2013)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Egloff, D.: Monte Carlo algorithms for optimal stopping and statistical learning. Ann. Appl. Probab. 15(2), 1396–1432 (2005)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Egloff, D., Kohler, M., Todorovic, N.: A dynamic look-ahead monte carlo algorithm for pricing bermudan options. Ann. Appl. Probab. 17(4), 1138–1171 (2007)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Felix, B.J., Weber, C.: Gas storage valuation applying numerically constructed recombining trees. Eur. J. Oper. Res. 216(1), 178–187 (2012)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Gramacy, R.B., Ludkovski, M.: Sequential design for optimal stopping problems. SIAM J. Financ. Math. 6(1), 748–775 (2015)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Han, J., E, W.: Deep Learning Approximation for Stochastic Control Problems. Tech. Rep., (2016). arXiv:1611.07422
  20. 20.
    Heymann, B., Bonnans, J.F., Martinon, P., Silva, F.J., Lanas, F., Jiménez-Estévez, G.: Continuous optimal control approaches to microgrid energy management. Energy Syst. 9(1), 59–77 (2018)CrossRefGoogle Scholar
  21. 21.
    Heymann, B., Bonnans, J.F., Silva, F., Jimenez, G.: A stochastic continuous time model for microgrid energy management. In: 2016 European Control Conference (ECC), pp. 2084–2089 (2016)Google Scholar
  22. 22.
    Heymann, B., Martinon, P., Bonnans, F.: Long term aging : an adaptative weights dynamic programming algorithm. Tech. Rep., hal 01349932 (2016)Google Scholar
  23. 23.
    Hinz, J., Tarnopolskaya, T., Yee, J.: Efficient algorithms of pathwise dynamic programming for decision optimization in mining operations. Ann. Oper. Res. (2018).
  24. 24.
    Jain, S., Oosterlee, C.W.: Pricing high-dimensional Bermudan options using the stochastic grid method. Int. J. Comput. Math. 89(9), 1186–1211 (2012)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Jain, S., Oosterlee, C.W.: The stochastic grid bundling method: efficient pricing of Bermudan options and their Greeks. Appl. Math. Comput. 269, 412–431 (2015)MathSciNetGoogle Scholar
  26. 26.
    Kharroubi, I., Langrené, N., Pham, H.: A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization. Monte Carlo Meth. Appl. 20, 145–165 (2014)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Langrené, N., Tarnopolskaya, T., Chen, W., Zhu, Z., Cooksey, M.: New regression Monte Carlo methods for high-dimensional real options problems in minerals industry. In: 21st International Congress on Modelling and Simulation, pp. 1077–1083 (2015)Google Scholar
  28. 28.
    Longstaff, F.A., Schwartz, E.S.: Valuing American options by simulation: a simple least-squares approach. Rev. Financ. Stud. 14(1), 113–147 (2001)CrossRefGoogle Scholar
  29. 29.
    Ludkovski, M.: Kriging metamodels and experimental design for Bermudan option pricing. J. Comput. Financ. 22(1), 37–77 (2018)CrossRefGoogle Scholar
  30. 30.
    Malyscheff, A.M., Trafalis, T.B.: Natural gas storage valuation via least squares Monte Carlo and support vector regression. Energy Syst. 8(4), 815–855 (2017)CrossRefGoogle Scholar
  31. 31.
    Mazieres, D., Boogert, A.: A radial basis function approach to gas storage valuation. J. Energy Mark. 6(2), 19–50 (2013)Google Scholar
  32. 32.
    Nadarajah, S., Margot, F., Secomandi, N.: Comparison of least squares Monte Carlo methods with applications to energy real options. Eur. J. Oper. Res. 256(1), 196–204 (2017)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Pham, H.: Optimal switching and free boundary problems. In: Continuous-time stochastic control and optimization with financial applications, pp. 95–137. Springer, Berlin, Heidelberg (2009)Google Scholar
  34. 34.
    Quintero, S.A.P., Ludkovski, M., Hespanha, J.P.: Stochastic optimal coordination of small uavs for target tracking using regression-based dynamic programming. J Intell. Robot. Syst. 82(1), 135–162 (2016)CrossRefGoogle Scholar
  35. 35.
    Thompson, M.: Natural gas storage valuation, optimization, market and credit risk management. J. Commod. Mark. 2(1), 26–44 (2016)CrossRefGoogle Scholar
  36. 36.
    Thompson, M., Davison, M., Rasmussen, H.: Natural gas storage valuation and optimization: a real options application. Naval Res. Logist. (NRL) 56(3), 226–238 (2009)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Tsitsiklis, J.N., Van Roy, B.: Regression methods for pricing complex American-style options. Trans. Neural Netw. 12(4), 694–703 (2001)CrossRefGoogle Scholar
  38. 38.
    Van-Ackooij, W., Warin, X.: On conditional cuts for Stochastic Dual Dynamic Programming. Tech. Rep., (2017). arXiv:1704.06205
  39. 39.
    Warin, X.: Gas storage hedging. In: Carmona, R.A., Del Moral, P., Hu, P., Oudjane, N. (eds.) Numerical methods in finance: Bordeaux, June 2010, pp. 421–445. Springer, Berlin, Heidelberg (2012)CrossRefGoogle Scholar
  40. 40.
    Zhang, R., Langrené, N., Tian, Y., Zhu, Z., Klebaner, F., Hamza, K.: Dynamic Portfolio Optimization with Liquidity Cost and Market Impact: A Simulation-and-Regression Approach. Tech. Rep., (2017). arXiv:1610.07694
  41. 41.
    Zhao, G., Davison, M.: Optimal control of hydroelectric facility incorporating pump storage. Renew. Energy 34(4), 1064–1077 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Statistics and Applied ProbabilitySouth Hall, University of CaliforniaSanta BarbaraUSA

Personalised recommendations