An inexact primal-dual algorithm for semi-infinite programming

Abstract

This paper considers an inexact primal-dual algorithm for semi-infinite programming (SIP) for which it provides general error bounds. We create a new prox function for nonnegative measures for the dual update, and it turns out to be a generalization of the Kullback-Leibler divergence. We show that, with a tolerance for small errors (approximation and regularization error), this algorithm achieves an \({\mathcal {O}}(1/\sqrt{K})\) rate of convergence in terms of the optimality gap and constraint violation, where K is the total number of iterations. We then use our general error bounds to analyze the convergence and sample complexity of a specific primal-dual SIP algorithm based on Monte Carlo sampling. Finally, we provide numerical experiments to demonstrate the performance of this algorithm.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. Aliprantis CD, Border KC (2006) Infinite dimensional analysis. A hitchhiker’s guide. Springer, Berlin

    Google Scholar 

  2. Anstreicher KM, Wolsey LA (2009) Two “well-known” properties of subgradient optimization. Math Program 120(1):213–220

    MathSciNet  MATH  Google Scholar 

  3. Betrò B (2004) An accelerated central cutting plane algorithm for linear semi-infinite programming. Math Program 101(3):479–495

    MathSciNet  MATH  Google Scholar 

  4. Bhat N, Farias V, Moallemi CC (2012) Non-parametric approximate dynamic programming via the kernel method. In: Advances in neural information processing systems, pp 386–394

  5. Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin

    Google Scholar 

  6. Bonnans JF, Shapiro A (2013) Perturbation analysis of optimization problems. Springer, Berlin

    Google Scholar 

  7. Bruck RE Jr (1977) On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in hilbert space. J Math Anal Appl 61(1):159–164

    MathSciNet  MATH  Google Scholar 

  8. Calafiore G, Campi MC (2005) Uncertain convex programs: randomized solutions and confidence levels. Math Program Ser A 102:25–46

    MathSciNet  MATH  Google Scholar 

  9. Campi MC, Garatti S (2008) The exact feasibility of randomized solutions of uncertain convex programs. SIAM J Optim 19(3):1211–1230

    MathSciNet  MATH  Google Scholar 

  10. de Farias DP, Van Roy B (2004) On constraint sampling in the linear programming approach to approximate dynamic programming. Math Oper Res 29(3):462–478

    MathSciNet  MATH  Google Scholar 

  11. Dentcheva D, Ruszczynski A (2003) Optimization with stochastic dominance constraints. SIAM J Optim 14(2):548–566

    MathSciNet  MATH  Google Scholar 

  12. Dentcheva D, Ruszczyński A (2009) Optimization with multivariate stochastic dominance constraints. Math Program 117(1):111–127

    MathSciNet  MATH  Google Scholar 

  13. Dentcheva D, Wolfhagen E (2015) Optimization with multivariate stochastic dominance constraints. SIAM J Optim 25(1):564–588

    MathSciNet  MATH  Google Scholar 

  14. Duchi J (2016) Introductory lectures on stochastic optimization. Park City Mathematics Institute, Graduate Summer School Lectures

  15. Esfahani PM, Sutter T, Lygeros J (2015) Performance bounds for the scenario approach and an extension to a class of non-convex programs. IEEE Tran Autom Control 60(1):46–58

    MathSciNet  MATH  Google Scholar 

  16. Fan K (1953) Minimax theorems. Proc Natl Acad Sci 39(1):42–47

    MathSciNet  MATH  Google Scholar 

  17. Fonseca I, Leoni G (2007) Modern methods in the calculus of variations: \(Lp\) Spaces. Springer, Berlin

    Google Scholar 

  18. Goberna MA, Lopez MA (2002) Linear semi-infinite programming theory: an updated survey. Eur J Oper Res 143(2):390–405

    MathSciNet  MATH  Google Scholar 

  19. Goberna MÁ, López MA (2013) Semi-infinite programming: recent advances, vol 57. Springer, Berlin

    Google Scholar 

  20. Goberna MA, López MA (2017) Recent contributions to linear semi-infinite optimization. 4OR 15(3):221–264

    MathSciNet  MATH  Google Scholar 

  21. Gramlich G, Hettich R, Sachs EW (1995) Local convergence of sqp methods in semi-infinite programming. SIAM J Optim 5(3):641–658

    MathSciNet  MATH  Google Scholar 

  22. Gribik PR (1979) A central-cutting-plane algorithm for semi-infinite programming problems. In: Semi-infinite Programming. Springer, Berlin, pp 66–82

  23. Gustavsson E, Patriksson M, Strömberg A-B (2015) Primal convergence from dual subgradient methods for convex optimization. Math Program 150(2):365–390

    MathSciNet  MATH  Google Scholar 

  24. Haskell WB, Shanthikumar JG, Shen ZM (2013) Optimization with a class of multivariate integral stochastic order constraints. Ann Oper Res 206(1):147–162

    MathSciNet  MATH  Google Scholar 

  25. Haskell WB, Shanthikumar JG, Shen ZM (2017) Primal-dual algorithms for optimization with stochastic dominance. SIAM J Optim 27(1):34–66

    MathSciNet  MATH  Google Scholar 

  26. Hettich R, Kortanek KO (1993) Semi-infinite programming: theory, methods, and applications. SIAM Rev 35(3):380–429

    MathSciNet  MATH  Google Scholar 

  27. Hu J, Mello TH, Mehrotra S (2012) Sample average approximation of stochastic dominance constrained programs. Math Program 133(1):171–201

    MathSciNet  MATH  Google Scholar 

  28. Ito S, Liu Y, Teo KL (2000) A dual parametrization method for convex semi-infinite programming. Ann Oper Res 98(1):189–213

    MathSciNet  MATH  Google Scholar 

  29. Kiwiel KC, Larsson T, Lindberg PO (2007) Lagrangian relaxation via ballstep subgradient methods. Math Oper Res 32(3):669–686

    MathSciNet  MATH  Google Scholar 

  30. Kortanek KO, No H (1993) A central cutting plane algorithm for convex semi-infinite programming problems. SIAM J Optim 3(4):901–918

    MathSciNet  MATH  Google Scholar 

  31. Lan G, Nemirovski A, Shapiro A (2012) Validation analysis of mirror descent stochastic approximation method. Math Program 134(2):425–458

    MathSciNet  MATH  Google Scholar 

  32. Larsson T, Patriksson M, Strömberg A-B (1999) Ergodic, primal convergence in dual subgradient schemes for convex programming. Math Program 86(2):283–312

    MathSciNet  MATH  Google Scholar 

  33. Li D-H, Qi L, Tam J, Soon-Yi W (2004) A smoothing newton method for semi-infinite programming. J Glob Optim 30(2–3):169–194

    MathSciNet  MATH  Google Scholar 

  34. Ling C, Ni Q, Qi L, Soon-Yi W (2010) A new smoothing newton-type algorithm for semi-infinite programming. J Global Optim 47(1):133–159

    MathSciNet  MATH  Google Scholar 

  35. Lin Q, Nadarajah S, Soheili N (2017) Revisiting approximate linear programming using a saddle point based reformulation and root finding solution approach

  36. Liu Y, Teo KL (2002) An adaptive dual parametrization algorithm for quadratic semi-infinite programming problems. J Global Optim 24(2):205–217

    MathSciNet  MATH  Google Scholar 

  37. Liu Y, Teo KL, Wu S-Y (2004) A new quadratic semi-infinite programming algorithm based on dual parametrization. J Global Optim 29(4):401–413

    MathSciNet  MATH  Google Scholar 

  38. López M, Still G (2007) Semi-infinite programming. Eur J Oper Res 180(2):491–518

    MathSciNet  MATH  Google Scholar 

  39. Luenberger DG (1997) Optimization by vector space methods. Wiley, New York

    Google Scholar 

  40. MacKay DJC (2003) Information theory, inference and learning algorithms. Cambridge University Press, Cambridge

    Google Scholar 

  41. Mahdavi M, Yang T, Jin R (2012) Online stochastic optimization with multiple objectives. arXiv preprint arXiv:1211.6013

  42. Mehrotra S, Papp D (2014) A cutting surface algorithm for semi-infinite convex programming with an application to moment robust optimization. SIAM J Optim 24(4):1670–1697

    MathSciNet  MATH  Google Scholar 

  43. Mello TH, Mehrotra S (2009) A cutting-surface method for uncertain linear programs with polyhedral stochastic dominance constraints. SIAM J Optim 20(3):1250–1273

    MathSciNet  MATH  Google Scholar 

  44. Mohajerin Esfahani P, Sutter T, Kuhn D, Lygeros J (2017) From infinite to finite programs: explicit error bounds with applications to approximate dynamic programming. arXiv preprint arXiv:1701.06379

  45. Nedić A, Ozdaglar A (2009) Approximate primal solutions and rate analysis for dual subgradient methods. SIAM J Optim 19(4):1757–1780

    MathSciNet  MATH  Google Scholar 

  46. Nedić A, Ozdaglar A (2009) Subgradient methods for saddle-point problems. J Optim Theory Appl 142(1):205–228

    MathSciNet  MATH  Google Scholar 

  47. Ni Q, Ling C, Qi L, Teo KL (2006) A truncated projected newton-type algorithm for large-scale semi-infinite programming. SIAM J Optim 16(4):1137–1154

    MathSciNet  MATH  Google Scholar 

  48. Noyan N, Rudolf G (2013) Optimization with multivariate conditional value-at-risk constraints. Oper Res 61(4):990–1013

    MathSciNet  MATH  Google Scholar 

  49. Noyan N, Rudolf G (2018) Optimization with stochastic preferences based on a general class of scalarization functions. Oper Res 66:463–486

    MathSciNet  Google Scholar 

  50. Pinsker MS (1964) Information and information stability of random variables and processes. Holden-Day, San Francisco

    Google Scholar 

  51. Qi L, Wu S-Y, Zhou G (2003) Semismooth newton methods for solving semi-infinite programming problems. J Global Optim 27(2–3):215–232

    MathSciNet  MATH  Google Scholar 

  52. Qi L, Ling C, Tong X, Zhou G (2009) A smoothing projected newton-type algorithm for semi-infinite programming. Comput Optim Appl 42(1):1–30

    MathSciNet  MATH  Google Scholar 

  53. Reemtsen R (1991) Discretization methods for the solution of semi-infinite programming problems. J Optim Theory Appl 71(1):85–103

    MathSciNet  MATH  Google Scholar 

  54. Reemtsen R, Görner S (1998) Numerical methods for semi-infinite programming: a survey. In Semi-infinite programming, Springer, pp 195–275

  55. Robert C, Casella G (2004) Monte Carlo statistical methods. Springer, New York

    Google Scholar 

  56. Shapiro A (2009) Semi-infinite programming, duality, discretization and optimality conditions. Optimization 58(2):133–161

    MathSciNet  MATH  Google Scholar 

  57. Still G (2001) Discretization in semi-infinite programming: the rate of convergence. Math Program 91(1):53–69

    MathSciNet  MATH  Google Scholar 

  58. Zhang L, Wu S-Y, López MA (2010) A new exchange method for convex semi-infinite programming. SIAM J Optim 20(6):2959–2977

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to William B. Haskell.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wei, B., Haskell, W.B. & Zhao, S. An inexact primal-dual algorithm for semi-infinite programming. Math Meth Oper Res 91, 501–544 (2020). https://doi.org/10.1007/s00186-019-00698-2

Download citation

Keywords

  • Semi-infinite programming
  • Primal-dual algorithms
  • Monte Carlo integration