Statistical Inference of Stochastic Optimization Problems

  • Alexander Shapiro
Part of the Nonconvex Optimization and Its Applications book series (NOIA, volume 49)


We discuss in this paper statistical inference of Monte Carlo simulation based approximations of stochastic optimization problems, where the “true” objective function, and probably some of the constraints, are estimated, typically by averaging a random sample. The classical maximum likelihood estimation can be considered in that framework. Recently statistical analysis of such methods has been motivated by a development of simulation based optimization techniques. We investigate asymptotic properties of the optimal value and an optimal solution of the corresponding Monte Carlo simulation approximations by employing the so-called Delta method, and discuss some examples.


Stochastic Programming Random Element Order Expansion Unique Optimal Solution True Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Araujo and E. Giné, The Central Limit Theorem for Real and Banach Valued Random Variables. Wiley, New York, 1980.zbMATHGoogle Scholar
  2. [2]
    E. Beale, “On minimizing a convex function subject to linear inequalities”, J. Roy. Statist Soc, Ser. B, 17 (1955), 173–184.MathSciNetzbMATHGoogle Scholar
  3. [3]
    P. Billingsley, Convergence of Probability Measures. Wiley, New York, 1968.zbMATHGoogle Scholar
  4. [4]
    J.R. Birge and F. Louveaux, Introduction to Stochastic Programming. Springer, New York, 1997.zbMATHGoogle Scholar
  5. [5]
    J.F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems. Springer-Verlag, New York, 2000.zbMATHGoogle Scholar
  6. [6]
    J.A. Bucklew, Large Deviation Techniques in Decision, Simulation, and Estimation. Wiley, New York, 1990.Google Scholar
  7. [7]
    H. Chernoff, “On the distribution of the likelihood ratio ”, Ann. Math. Statist., 25 (1954), 573–578.MathSciNetzbMATHCrossRefGoogle Scholar
  8. [8]
    G. Dantzig, “Linear programming under uncertainty ”, Management Sci.,1 (1955), 197–206.MathSciNetzbMATHCrossRefGoogle Scholar
  9. [9]
    D. Dentcheva and W. Römisch, “Differential stability of two-stage stochastic programs ”, SI AM J. Optimization, to appear.Google Scholar
  10. [10]
    J. Dupacovä and R.J.B. Wets, “Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems, ” The Annals of Statistics 16 (1988) 1517–1549.MathSciNetCrossRefGoogle Scholar
  11. [11]
    R.D. Gill, “Non-and-semiparametric maximum likelihood estimators and the von Mises method (Part I)”, Scandinavian Journal of Statistics, 16 (1989), 97–124.zbMATHGoogle Scholar
  12. [12]
    R. Griibel, “The length of the short ”, Ann. Statist, 16 (1988), 619–628.MathSciNetCrossRefGoogle Scholar
  13. [13]
    RJ. Huber, “Robust estimation of a location parameter ”, Ann. Math. Statist., 35 (1964), 73–101.MathSciNetzbMATHCrossRefGoogle Scholar
  14. [14]
    P. J. Huber, “The behavior of maximum likelihood estimates under nonstandard conditions”, Proc. Fifth Berkeley Symp.Math. Statist. Probab., 1 (1967), 221–233, Univ. California Press.Google Scholar
  15. [15]
    P.J. Huber, Robust Statistics. Wiley, New York, 1981.zbMATHCrossRefGoogle Scholar
  16. [16]
    P. Kall and S.W. Wallace, Stochastic Programming. Wiley, Chichester, 1994.zbMATHGoogle Scholar
  17. [17]
    A.J. King, “Asymptotic behavior of solutions in stochastic optimization: Non-smooth analysis and the derivation of non-normal limit distributions”, Ph.D. dissertation, Dept. Applied Mathematics, Univ. Washington, 1986.Google Scholar
  18. [18]
    A.J. King, “Generalized delta theorems for multivalued mappings and measurable selections ”, Mathematics of Operations Research, 14 (1989), 720–736.MathSciNetzbMATHCrossRefGoogle Scholar
  19. [19]
    A.J. King and R.J.-B. Wets, “Epi-consistency of convex stochastic programs, ” Stochastics, 34 (1991), 83–92.MathSciNetzbMATHGoogle Scholar
  20. [20]
    A.J. King and R.T. Rockafellar, “Asymptotic theory for solutions in statistical estimation and stochastic programming”, Mathematics of Operations Research, 18 (1993), 148–162.MathSciNetzbMATHCrossRefGoogle Scholar
  21. [21]
    O.L. Mangasarian and S. Fromovitz, “The Fritz John necessary optimality conditions in the presence of equality and inequality constraints”, Journal of Mathematical Analysis and Applications, 7 (1967), pp. 37–47.MathSciNetCrossRefGoogle Scholar
  22. [22]
    E.L. Plambeck, B.R. Fu, S.M. Robinson and R. Suri, “Sample-Path Optimization of Convex Stochastic Performance Functions ”, Mathematical Programming, vol. 75 (1996), no. 2, 137–176.MathSciNetzbMATHCrossRefGoogle Scholar
  23. [23]
    D. Pollard, Convergence of Stochastic Processes. Springer-Verlag, New York, 1984.zbMATHCrossRefGoogle Scholar
  24. [24]
    C.R. Rao, Linear Statistical Inference and Its Applications. Wiley, New York, 1973.zbMATHCrossRefGoogle Scholar
  25. [25]
    S.M. Robinson, “Analysis of sample-path optimization, ” Math. Oper. Res., 21 (1996), 513–528.MathSciNetzbMATHCrossRefGoogle Scholar
  26. [26]
    R.T. Rockafellar, Convex Analysis. Princeton University Press, 1970.zbMATHGoogle Scholar
  27. [27]
    R.Y. Rubinstein and A. Shapiro, Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method. Wiley, New York, NY, 1993.zbMATHGoogle Scholar
  28. [28]
    A. Shapiro, “Asymptotic Properties of Statistical Estimators in Stochastic Programming, ” Annals of Statistics, 17 (1989), 841–858.MathSciNetzbMATHCrossRefGoogle Scholar
  29. [29]
    A. Shapiro, “On concepts of directional differentiability, ” Journal Optim. Theory and Appl., 66 (1990), 477–487.zbMATHCrossRefGoogle Scholar
  30. [30]
    A. Shapiro, “Asymptotic analysis of stochastic programs, ” Annals of Operations Research, 30 (1991), 169–186.MathSciNetzbMATHCrossRefGoogle Scholar
  31. [31]
    A. Shapiro, “Asymptotic Behavior of Optimal Solutions in Stochastic Programming,” Mathematics of Operations Research, 18 (1993), 829–845.MathSciNetzbMATHCrossRefGoogle Scholar
  32. [32]
    A. Shapiro and Y. Wardi, “Nondifferentiability of the steady-state function in Discrete Event Dynamic Systems ”, IEEE transactions on Automatic Control, 39 (1994), 1707–1711.MathSciNetzbMATHCrossRefGoogle Scholar
  33. [33]
    A. Shapiro and T. Homem-de-Mello, “A simulation-based approach to two-stage stochastic programming with recourse ”, Mathematical Programming, 81 (1998), 301–325.MathSciNetzbMATHGoogle Scholar
  34. [34]
    A. Shapiro and T. Homem-de-Mello, “On rate of convergence of Monte Carlo approximations of stochastic programs ”, SIAM J. Optimization, to appear.Google Scholar
  35. [35]
    J. Wang, “Distribution sensitivity analysis for stochastic programs with complete recourse ”, Mathematical Programming, 31 (1985), 286–297.MathSciNetzbMATHCrossRefGoogle Scholar
  36. [36]
    R.W. Wolff, Stochastic Modeling and the Theory of Queues. Prentice Hall, En-glewood Cliffs, NJ, 1989.zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2000

Authors and Affiliations

  • Alexander Shapiro
    • 1
  1. 1.Georgia Institute of TechnologyAtlantaUSA

Personalised recommendations