Projection Methods in Conic Optimization

  • Didier Henrion
  • Jérôme Malick
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 166)


There exist efficient algorithms to project a point onto the intersection of a convex conic and an affine subspace. Those conic projections are in turn the work-horse of a range of algorithms in conic optimization, having a variety of applications in science, finance and engineering. This chapter reviews some of these algorithms, emphasizing the so-called regularization algorithms for linear conic optimization, and applications in polynomial optimization. This is a presentation of the material of several recent research articles; we aim here at clarifying the ideas, presenting them in a general framework, and pointing out important techniques.


Regularization Method Alternate Direction Method Polynomial Optimization Conic Projection Regularization Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The work of the first author was partly supported by research project 103/10/0628 of the Grant Agency of the Czech Republic and research programme MSM6840770038 of the Ministry of Education of the Czech Republic.


  1. 1.
    Al-Homidan, S.: Solving Hankel matrix approximation problem using semidefinite programming. Journal of Computational and Applied Mathematics 202(2), 304–314 (2007)CrossRefGoogle Scholar
  2. 2.
    Alizadeh, F., Haeberly, J.-P., Overton, M: Complementarity and nondegeneracy in semidefinite programming. Math. Program. 77(2), 111–128 (1997)Google Scholar
  3. 3.
    Anjos, M.F., Higham, N.J., Takouda, P.L., Wolkowicz, H.: A semidefinite programming approach for the nearest correlation matrix problem. Technical report, Combinatorics and Optimization, University of Waterloo, September 2003.Google Scholar
  4. 4.
    Arrow, K., Hurwicz, L., Uzawa, H.: Studies in Linear and Nonlinear Programming. Stanford University Press (1959)Google Scholar
  5. 5.
    Bolte, J., Daniilidis, A., Lewis, A.: Tame functions are semismooth. Math. Program. 117(1), 5–19 (2008)Google Scholar
  6. 6.
    Bertsekas, D.P.: Nonlinear Programming. Athena Scientific (1995)Google Scholar
  7. 7.
    Boyd, S., El Ghaoui, L., Feron, E., Balakrishnan, V.: Linear Matrix Inequalities in System and Control Theory. Studies in Applied Mathematics. SIAM (1994)Google Scholar
  8. 8.
    Bonnans, J.F., Gilbert, J.Ch., Lemaréchal, C., Sagastizábal, C.: Numerical Optimization. Springer Verlag (2003)Google Scholar
  9. 9.
    Borsdorf, R., Higham, N.: A preconditionned Newton algorithm for the nearest correlation matrix. IMA J. Numer. Anal. 30(1), 94–107 (2010)CrossRefGoogle Scholar
  10. 10.
    Bellman, R., Kalaba, R., Lockett, J.: Numerical Inversion of the Laplace Transform. Elsevier (1966)Google Scholar
  11. 11.
    Brigo, D., Mercurio, F.: Interest rate models: theory and practice. Springer-Verlag (2006)Google Scholar
  12. 12.
    Burer, S., Monteiro, R., Zhang, Y.: A computational study of a gradient-based log-barrier algorithm for a class of large-scale sdps. Mathematical Programming Series B, 359–379, 2001.Google Scholar
  13. 13.
    Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer Verlag (2000)Google Scholar
  14. 14.
    Boyd, S., Vandenberghe, L.: Convex optimization. Cambridge University Press (2004)Google Scholar
  15. 15.
    Burer, S., Vandenbussche, D.: Solving lift-and-project relaxations of binary integer programs. SIAM Journal on Optimization 16, 726–750 (2006)CrossRefGoogle Scholar
  16. 16.
    Boyd, S., Xiao, L.: Least-squares covariance matrix adjustment. SIAM Journal on Matrix Analysis and Applications 27(2) (2005)Google Scholar
  17. 17.
    Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Mathematical Programming 62(2), 261–275 (1993)CrossRefGoogle Scholar
  18. 18.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley (1983); reprinted by SIAM, (1983)Google Scholar
  19. 19.
    Dembo, R., Eisenstat, S., Steihaug, T.: Inexact Newton methods. SIAM Journal on Numerical Analysis 19(2) (1982)Google Scholar
  20. 20.
    Deutsch, F.: Best Approximation in Inner Product Spaces. Springer, New York (2001)Google Scholar
  21. 21.
    Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Amer. Math. Soc. 82, 421–439 (1956)CrossRefGoogle Scholar
  22. 22.
    Dukanovic, I., Rendl, F.: Semidefinite programming relaxations for graph coloring and maximal clique problems. Mathematical Programming 109, 345–365 (2007)CrossRefGoogle Scholar
  23. 23.
    Dumitrescu, B.: Positive Trigonometric Polynomials and Signal Processing Applications. Springer Verlag (2007)Google Scholar
  24. 24.
    Dykstra, R.L.: An algorithm for restricted least-square regression. Journal of the American Statistical Association 78, 837–842, (1983)CrossRefGoogle Scholar
  25. 25.
    Gilbert, J.Ch., Lemaréchal, C.: Some numerical experiments with variable-storage quasi-Newton algorithms. Mathematical Programming 45, 407–435 (1989)CrossRefGoogle Scholar
  26. 26.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Computers and Mathematics with Applications 2 (1976)Google Scholar
  27. 27.
    Gao, Y., Sun, D.: Calibrating least squares semidefinite programming with equality and inequality constraints. SIAM J. Matrix Anal. Appl. 31(3), 1432–1457 (2009)CrossRefGoogle Scholar
  28. 28.
    Gao, Y., Sun, D.: A majorized penalty approach for calibrating rank constrained correlation matrix problems. Technical report, Univ. of Singapore (2003)Google Scholar
  29. 29.
    Goemans, M., Williamson, D.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM 6, 1115–1145 (1995)CrossRefGoogle Scholar
  30. 30.
    Henrion, D., Garulli, A. (eds.): Positive polynomials in control. LNCIS, Springer Verlag (2005)Google Scholar
  31. 31.
    Higham, N.: Computing a nearest symmetric positive semidefinite matrix. Linear Algebra and its Applications 103, 103–118 (1988)CrossRefGoogle Scholar
  32. 32.
    Higham, N.: Computing a nearest symmetric correlation matrix — a problem from finance. IMA Journal of Numerical Analysis 22(3), 329–343 (2002)CrossRefGoogle Scholar
  33. 33.
    Henrion, D., Lasserre, J.-B.: Detecting global optimality and extracting solutions in gloptipoly. In: Henrion, D., Garulli, A., (eds.) Positive polynomials in control, LNCIS. Springer (2005)Google Scholar
  34. 34.
    Henrion, D., Malick, J.: Projection methods for conic feasibility problems; application to sum-of-squares decompositions. Optimization Methods and Software 26(1), 23–46 (2011)CrossRefGoogle Scholar
  35. 35.
    Hiriart-Urruty, J.-B. Lemaréchal, C.: Convex Analysis and Minimization Algorithms. Springer Verlag, Heidelberg (1993) Two volumes.Google Scholar
  36. 36.
    Hiriart-Urruty, J.-B., Strodiot, J.J., Nguyen, H.V.: Generalized hessian matrix and second-order optimality conditions for problems with C1, 1 data. Applied Mathematics and Optimization 11, 43–56 (1984)CrossRefGoogle Scholar
  37. 37.
    He, B., Xu, M., Yuan, X.: Solving large-scale least-squares covariance matrix problems by alternating direction methods. Technical report (2009)Google Scholar
  38. 38.
    Jarre, F., Rendl, F.: An augmented primal-dual method for linear conic problems. SIAM Journal on Optimization (2008)Google Scholar
  39. 39.
    Johnson, D.J., Trick, M.A. (eds.): Cliques, Coloring, and Satisfiability: Second DIMACS Implementation Challenge, Workshop, 11-13 October 1993. American Mathematical Society, Boston, MA, USA (1996)Google Scholar
  40. 40.
    Kočvara, M., Stingl, M.: PENNON: a code for convex nonlinear and semidefinite programming. Optimisation Methods and Sofware 18(3), 317–333 (2003)CrossRefGoogle Scholar
  41. 41.
    Kočvara, M., Stingl, M.: On the solution of large-scale sdp problems by the modified barrier method using iterative solvers. Math. Program. 109(2), 413–444 (2007)CrossRefGoogle Scholar
  42. 42.
    Lasserre, J.-B.: A sum of squares approximation of nonnegative polynomials. SIAM J. Optim. 16, 751–765 (2006)CrossRefGoogle Scholar
  43. 43.
    Lasserre, J.-B.: Moments, Positive Polynomials and Their Applications. Imperial College Press (2009)Google Scholar
  44. 44.
    Laurent, M.: Sums of squares, moment matrices and optimization over polynomials. In: M. Putinar and S. Sullivant, editors, Emerging Applications of Algebraic Geometry, IMA Volumes in Mathematics and its Applications. Springer (2009)Google Scholar
  45. 45.
    Lin, F., Jovanović, M.R.: Least-squares approximation of structured covariances. IEEE Trans. Automat. Control 54(7), 1643–1648 (2009)CrossRefGoogle Scholar
  46. 46.
    Lewis, A., Luke, D., Malick, J.: Local linear convergence for alternating and averaged nonconvex projections. Foundations of Computational Mathematics 9, 485–513 (2009)CrossRefGoogle Scholar
  47. 47.
    Lewis, A., Malick, J.: Alternating projections on manifolds. Mathematics of Operations Research 33(1), 216–234 (2008)CrossRefGoogle Scholar
  48. 48.
    Lovász, L.: On the Shannon capacity of a graph. IEEE Transactions on Information Theory 25, 1–7 (1979)CrossRefGoogle Scholar
  49. 49.
    Li, Q., Qi, H.: A sequential semismooth Newton method method for the nearest low-rank correlation matrix problem. Technical report (2003)Google Scholar
  50. 50.
    Malick, J.: A dual approach to semidefinite least-squares problems. SIAM Journal on Matrix Analysis and Applications 26(1), 272–284 (2004)CrossRefGoogle Scholar
  51. 51.
    B. Martinet. Régularisation d’inéquations variationnelles par approximations successives. Revue Française d’Informatique et Recherche Opérationnelle R-3, 154–179 (1970)Google Scholar
  52. 52.
    Malick, J., Povh, J., Rendl, F., Wiegele, A.: Regularization methods for semidefinite programming. SIAM Journal on Optimization 20(1), 336–356 (2009)CrossRefGoogle Scholar
  53. 53.
    Malick, J., Sendov, H.: Clarke generalized Jacobian of the projection onto the cone of positive semidefinite matrices. Set-Valued Analysis 14(3), 273–293 (2006)CrossRefGoogle Scholar
  54. 54.
    Micchelli, C.A., Utretas, F.I.: Smoothing and interpolation in a convex subset of an Hilbert space. SIAM J. Sci. Stat. Comput. 9, 728–746 (1988)CrossRefGoogle Scholar
  55. 55.
    Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005)CrossRefGoogle Scholar
  56. 56.
    Nie, J.: Regularization methods for sum of squares relaxations in large scale polynomial optimization. Technical report, ArXiv (2009)Google Scholar
  57. 57.
    Caprara, A., Oswald, M., Reinelt, G., Schwarz, R., Traversi, E.: Optimal linear arrangements using betweenness variables. Mathematical Programming (Series C) 3(3), 261–280, (2011)Google Scholar
  58. 58.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer Verlag, New York (2006)Google Scholar
  59. 59.
    Nouralishahi, M., Wu, C., Vandenberghe, L.: Model calibration for optical lithography via semidefinite programming. Optimization and Engineering 9, 19–35 (2008)CrossRefGoogle Scholar
  60. 60.
    Povh, J., Rendl, F., Wiegele, A.: A boundary point method to solve semidefinite programs. Computing 78, 277–286 (2006)CrossRefGoogle Scholar
  61. 61.
    Polyak, B.T., Tretjakov, N.V.: On an iterative method for linear programming and its economical interpretations. Èkonom. i Mat. Methody 8, 740–751 (1972)Google Scholar
  62. 62.
    Qi, L.Q., Sun, J.: A nonsmooth version of Newton’s method. Mathematical Programming 58(3), 353–367 (1993)CrossRefGoogle Scholar
  63. 63.
    Qi, H., Sun, D.: Quadratic convergence and numerical experiments of Newton’s method for computing the nearest correlation matrix. SIAM Journal on Matrix Analysis and Applications 28, 360–385 (2006)CrossRefGoogle Scholar
  64. 64.
    Qi, H., Sun, D.: An augmented Lagrangian dual approach for the h-weighted nearest correlation matrix problem. IMA Journal of Numerical Analysis (2003)Google Scholar
  65. 65.
    Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Mathematics of Operations Research 1, 97–116 (1976)CrossRefGoogle Scholar
  66. 66.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM Journal on Control and Optimization 14, 877–898 (1976)CrossRefGoogle Scholar
  67. 67.
    Schwertman, N., Allen, D.: Smoothing an indefinite variance-covariance matrix. Journal of Statistical Computation and Simulation 9, 183–194 (1979)CrossRefGoogle Scholar
  68. 68.
    Sun, D., Sun, J.: Semismooth matrix valued functions. Mathematics of Operations Research 57 (2002)Google Scholar
  69. 69.
    Sturm, J.F.: Using Sedumi 1.02, a Matlab toolbox for optimization over symmetric cones. Optmization Methods and Softwares 11-12, 625–653 (1999)Google Scholar
  70. 70.
    Saigal, R., Vandenberghe, L., Wolkowicz, H.: Handbook of Semidefinite-Programming. Kluwer (2000)Google Scholar
  71. 71.
    Sun, J., Zhang, S.: A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. European Journal of Operational Research 207(3), 1210–1220 (2003)CrossRefGoogle Scholar
  72. 72.
    Toh, K.: An inexact primal-dual path following algorithm for convex quadratic SDP. Math. Program. 112(1), 221–254 (2008)CrossRefGoogle Scholar
  73. 73.
    Tutuncu, R.H., Toh, K.C., Todd, M.J.: Solving semidefinite-quadratic-linear programs using SDPT3. Mathematical Programming B 95, 189–217 (2003)CrossRefGoogle Scholar
  74. 74.
    Tutuncu, R.H., Toh, K.C., Todd, M.J.: Inexact primal-dual path-following algorithms for a special class of convex quadratic SDP and related problems. Pacific Journal on Optimization (2006)Google Scholar
  75. 75.
    Zhao, X., Sun, D., Toh, K.: A Newton-CG augmented lagrangian method for semidefinite programming. SIAM J. Optim 20(4) (2003)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.CNRS, LAAS, ToulouseToulouseFrance
  2. 2.Faculty of Electrical EngineeringCzech Technical University in PraguePragueCzech Republic
  3. 3.CNRS, LJK, Grenoble, INRIASaint IsmierFrance

Personalised recommendations