Optimization Letters

, Volume 12, Issue 3, pp 485–497 | Cite as

Variable sample size method for equality constrained optimization problems

  • Nataša Krejić
  • Nataša Krklec Jerinkić
  • Andrea Rožnjik
Original Paper


An equality constrained optimization problem with a deterministic objective function and constraints in the form of mathematical expectation is considered. The constraints are transformed into the Sample Average Approximation form resulting in deterministic problem. A method which combines a variable sample size procedure with line search is applied to a penalty reformulation. The method generates a sequence that converges towards first-order critical points. The final stage of the optimization procedure employs the full sample and the SAA problem is eventually solved with significantly smaller cost. Preliminary numerical results show that the proposed method can produce significant savings compared to SAA method and some heuristic sample update counterparts while generating a solution of the same quality.


Stochastic optimization Equality constraints Variable sample size Penalty method Line search 



We are grateful to the Associate Editor and reviewers whose comments helped us to improve the paper. N. Krejić and N. Krklec Jerinkić are supported by Serbian Ministry of Education, Science and Technological Development, Grant No. 174030.


  1. 1.
    Bastin, F.: Trust-Region Algorithms for Nonlinear Stochastic Programming and Mixed Logit Models. Ph.D. thesis, University of Namur, Belgium (2004)Google Scholar
  2. 2.
    Bastin, F., Cirillo, C., Toint, P.L.: An adaptive Monte Carlo algorithm for computing mixed logit estimators. Comput. Manag. Sci. 3(1), 55–79 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Deng, G., Ferris, M.C.: Variable-number sample path optimization. Math. Program. 117(12), 81–109 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Dolgopolik, M.V.: Smooth exact penalty functions: a general approach. Optim. Lett. 10, 635–648 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Friedlander, M.P., Schmidt, M.: Hybrid deterministic-stochastic methods for data fitting. SIAM J. Sci. Comput. 34(3), 13801405 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization. Academic Press, London (1997)zbMATHGoogle Scholar
  8. 8.
    Hock, W., Schittkowski, K.: Test Examples for Nonilnear Programming Codes. Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Berlin (1981)Google Scholar
  9. 9.
    Homem-de-Mello, T.: Variable-sample methods for stochastic optimization. ACM Trans. Model. Comput. Simul. 13(2), 108–133 (2003)CrossRefGoogle Scholar
  10. 10.
    Huyer, W., Neumaier, A.: A new exact penalty function. SIAM J. Optim. 13(4), 1141–1158 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Krejić, N., Krklec, N.: Line search methods with variable sample size for unconstrained optimization. J. Comput. Appl. Math. 245, 213–231 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Krejić, N., Krklec Jerinkić, N.: Nonmonotone line search methods with variable sample size. Numer. Algorithms 68(4), 711–739 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer Series in Operations Research. Springer, Berlin (1999)Google Scholar
  14. 14.
    Pasupathy, R.: On choosing parameters in retrospective-approximation algorithms for stochastic root finding and simulation optimization. Oper. Res. 58(4), 889–901 (2010)CrossRefzbMATHGoogle Scholar
  15. 15.
    Polak, E., Royset, J.O.: Eficient sample sizes in stochastic nonlinear programing. J. Comput. Appl. Math. 217(2), 301–310 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Shapiro, A.: Monte Carlo sampling methods. In: Stochastic Programming, Handbook in Operations Research and Management Science, vol. 10, pp. 353–425. Elsevier, Amsterdam (2003)Google Scholar
  17. 17.
    Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming: Modeling and Theory. MPS-SIAM Series on Optimization (2009)Google Scholar
  18. 18.
    Spall, J.C.: Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control. Wiley-Interscience Series in Discrete Mathematics, New Jersey (2003)Google Scholar
  19. 19.
    Wang, X., Ma, S., Yuan, Y.: Penalty Methods with Stochastic Approximation for Stochastic Nonlinear Programming, Technical report. arXiv:1312.2690 [math.OC] (2015)

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  1. 1.Department of Mathematics and InformaticsUniversity of Novi SadNovi SadSerbia
  2. 2.Faculty of Civil EngineeringUniversity of Novi SadSuboticaSerbia

Personalised recommendations