An Overview on Projection-Type Methods for Convex Large-Scale Quadratic Programs

  • Valeria Ruggiero
  • Luca Zanni
Part of the Nonconvex Optimization and Its Applications book series (NOIA, volume 58)


A well-known approach for solving large and sparse linearly constrained quadratic programming (QP) problems is given by the splitting and projection methods. After a survey on these classical methods, we show that they can be unified in a general iterative scheme consisting in to solve a sequence of QP subproblems with the constraints of the original problem and an easily solvable Hessian matrix. A convergence theorem is given for this general scheme. In order to improve the numerical performance of these methods, we introduce two variants of a projection- type scheme that use a variable projection parameter at each step. The two variable projection methods differ in the strategy used to assure a sufficient decrease of the objective function at each iteration. We prove, under very general hypotheses, the convergence of these schemes and we propose two practical, nonexpensive and efficient updating rules for the projection parameter. An extensive numerical experimentation shows the effectiveness of the variable projection-type methods.


Convex quadratic programs large-scale problems projection type methods splitting methods 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    H.C. Andrews, B.R. Hunt, Digital Image Restoration, Prentice Hall, Englewood Press, New Jersey, 1977.Google Scholar
  2. [2]
    D.P. Bertsekas, Nonlinear Programming, Athena Scientific, Belmont, MA, 1995.zbMATHGoogle Scholar
  3. [3]
    I. Bongartz, A.R. Conn, N. Gould, Ph.L. Toint, CUTE: constrained and unconstrained testing environment, ACM Transaction on Mathematical Software 1995, 21, 123–160.zbMATHCrossRefGoogle Scholar
  4. [4]
    Y. Censor, S.A. Zenios, Parallel Optimization: Theory, Algorithms and Applications, Oxford University Press, New York, 1997.zbMATHGoogle Scholar
  5. [5]
    H.G. Chen, R.T. Rockafellar, Convergence rates in forward-backward splitting, SIAM Journal on Optimization 7 (1997), 421–444.MathSciNetzbMATHGoogle Scholar
  6. [6]
    C. Cortes, V.N. Vapnik, Support vector network, Machine Learning 20 (1995), 1–25.Google Scholar
  7. [7]
    C.W. Cryer, The solution of a quadratic programming problem using systematic overrelaxation, SIAM Journal on Control 9 (1971), 385–392.CrossRefMathSciNetzbMATHGoogle Scholar
  8. [8]
    S. Dafermos, Traffic equilibrium and variational inequalities, Transportation Science 14 (1980), 14:42–54.MathSciNetGoogle Scholar
  9. [9]
    P.L. De Angelis, P.M. Pardalos, G. Toraldo, Quadratic Programming with Box Constraints, In Developments in Global Optimization, I.M. Bomze et al. eds., A0A0A0 Kluwer Academic Publishers, Dordrecht, 1997.Google Scholar
  10. [10]
    N. Dyn, J. Ferguson, The numerical solution of equality-constrained quadratic programming problems, Mathematics of Computation 41 (1983), 165–170.MathSciNetzbMATHGoogle Scholar
  11. [11]
    C.S. Fisk, S. Nguyen, Solution algorithms for network equilibrium models with asymmetric user costs, Transportation Science 16 (1982), 361–381.MathSciNetCrossRefGoogle Scholar
  12. [12]
    E. Galligani, C 1 surface interpolation with constraints, Numerical Algorithms 5 (1993), 549–555.CrossRefMathSciNetzbMATHGoogle Scholar
  13. [13]
    E. Galligani, V. Ruggiero, L. Zanni, Splitting methods for quadratic optimization in data analysis, International Journal of Computer Mathematics 63 (1997), 289–307.MathSciNetGoogle Scholar
  14. [14]
    E. Galligani, V. Ruggiero, L. Zanni, Splitting methods for constrained quadratic programs in data analysis, Computers & Mathematics with Applications 32 (1996), 1–9.CrossRefMathSciNetzbMATHGoogle Scholar
  15. [15]
    E. Galligani, V. Ruggiero, L. Zanni, Splitting methods and parallel solution of constrained quadratic programs, Proceedings of Equilibrium Problems with Side Constraints. Lagrangean Theory and Duality II, 1996 May 17–18; Scilla, Italy; Rendiconti del Circolo Matematico di Palermo, Ser. II, Suppl. 48 (1997), 121–136.MathSciNetGoogle Scholar
  16. [16]
    E. Galligani, V. Ruggiero, L. Zanni, Parallel solution of large scale quadratic programs. Proceedings of High Performance Algoritms and Software in Nonlinear Optimization, June 4–6 1997, Italy; Kluwer Academic Publishers, Dordrecht, 1998.Google Scholar
  17. [17]
    P.E. Gill, W. Murray, M.A. Saunders, SNOPT: An SQP Algorithm for Large-scale Constrained Optimization, Numerical Analysis Report 96-2. Departement of Mathematics, University of California, San Diego, 1996.Google Scholar
  18. [18]
    B. He, Solving a class of linear projection equations, Numerische Mathematik 68 (1994), 71–80.CrossRefMathSciNetzbMATHGoogle Scholar
  19. [19]
    F. Leibftitz, E.W. Sachs, Numerical Solution of Parabolic State Constrained Control Problems Using SQP and Interior-Point-Methods, In Optimization: State of the Art, W.W. Hager et al. eds., Kluwer Academic Publishers, Dordrecht, 1994.Google Scholar
  20. [20]
    Y.Y. Lin, J.S. Pang, Iterative methods for large convex quadratic programs: a survey, SIAM Journal on Control and Optimization 25 (1987), 383–411.CrossRefMathSciNetzbMATHGoogle Scholar
  21. [21]
    Z.Q. Luo, P. Tseng, On the linear convergence of descent methods for convex essentially smooth minimization, SIAM Journal on Control and Optimization 30 (1992), 408–425.CrossRefMathSciNetzbMATHGoogle Scholar
  22. [22]
    O.L. Mangasarian, Solution of symmetric linear complementarity problems by iterative methods, Journal of Optimization Theory and Applications 22 (1977), 465–485.CrossRefMathSciNetzbMATHGoogle Scholar
  23. [23]
    O.L. Mangasarian, R. De Leone, Parallel successive overrelaxation methods for symmetric linear complementarity problems and Linear Programs, Journal of Optimization Theory and Applications 54 (1987), 437–446.CrossRefMathSciNetzbMATHGoogle Scholar
  24. [24]
    O.L. Mangasarian, R. De Leone, Parallel gradient projection successive overrelaxation for symmetric linear complementarity problems and linear programs, Annals of Operations Research 14 (1988), 41–59.CrossRefMathSciNetzbMATHGoogle Scholar
  25. [25]
    P. Marcotte, J.H. Wu, On the convergence of projection methods: application to the decomposition of affine variational inequalities, Journal of Optimization Theory and Applications 85 (1995), 347–362.CrossRefMathSciNetzbMATHGoogle Scholar
  26. [26]
    J.J. Morè, S.J. Wright, Optimization Software Guide, SIAM, Philadelphia, PA, 1993.zbMATHGoogle Scholar
  27. [27]
    W. Murray, Sequential quadratic programming methods for largescale problems, Computational Optimization and Applications 7 (1997). 127–142.CrossRefMathSciNetzbMATHGoogle Scholar
  28. [28]
    B.A. Murtagh, M.A. Saunders, MINOS 5.4 User’s Guide, Report SOL 83-20R. Department of Operations Research, Stanford University, 1995.Google Scholar
  29. [29]
    NAG Fortran Library Manual, Mark 18, 1998.Google Scholar
  30. [30]
    Y. Nesterov, A. Nemirovskii, Interior-Point Polynomial Algorithms in Convex Programming, SIAM, Philadelphia, PA, 1994.zbMATHGoogle Scholar
  31. [31]
    S.N. Nielsen, S.A. Zenios, Massively parallel algorithms for single constrained convex programs, ORSA Journal on Computing 4 (1992), 166–181.MathSciNetzbMATHGoogle Scholar
  32. [32]
    J.M. Ortega, W.C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.zbMATHGoogle Scholar
  33. [33]
    J.S. Pang, D. Chan, Iterative methods for variational and complementarity problems, Mathematical Programming 24 (1982), 284–313.CrossRefMathSciNetzbMATHGoogle Scholar
  34. [34]
    P.M. Pardalos, N. Kovoor, An algorithm for a single constrained class of quadratic programs subject to upper and lower bounds, Mathematical Programming 46 (1990), 321–328.CrossRefMathSciNetzbMATHGoogle Scholar
  35. [35]
    P.M. Pardalos, Y. Ye, C.G. Han, Computational aspects of an interior point algorithm for quadratic problems with box constraints, In Large-Scale Numerical Optimization, T. Coleman and Y. Li eds., SIAM Philadelphia, 1990.Google Scholar
  36. [36]
    M. Patriksson, The Traffic Assignment Problem: Models and Methods, Topics in Transportation, VSP, Utrecht, 1994.Google Scholar
  37. [37]
    M. Patriksson, Nonlinear Programming and Variational Inequality Problems: A Unified Approach. Kluwer Academic Publisher, Dordrecht, 1999.zbMATHGoogle Scholar
  38. [38]
    V. Ruggiero, L. Zanni, Modified projection algorithm for large strictly convex quadratic programs Journal of Optimization Theory and Applications 104 (2000), 281–299.CrossRefMathSciNetzbMATHGoogle Scholar
  39. [39]
    V. Ruggiero, L. Zanni, On the efficiency of splitting and projection methods for large strictly convex quadratic programs, In Nonlinear Optimization and Related Topics, G. Di Pillo ang F. Giannessi eds., Kluwer Academic Publishers, Dordrecht, 1999.Google Scholar
  40. [40]
    M.V. Solodov, P. Tseng, Modified projection-type methods for monotone variational inequalities SIAM Journal on Control and Optimization 34 (1996), 1814–1830.CrossRefMathSciNetzbMATHGoogle Scholar
  41. [41]
    G.W. Stewart, The efficient generation of random orthogonal matrices with an application to condition estimators, SIAM Journal on Numerical Analysis 17 (1980), 403–409.CrossRefMathSciNetzbMATHGoogle Scholar
  42. [42]
    R.J. Vanderbei, T.J. Carpenter, Symmetric indefinite systems for interior point methods, Mathematical Programming 58 (1993), 1–32.CrossRefMathSciNetzbMATHGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Valeria Ruggiero
    • 1
  • Luca Zanni
    • 2
  1. 1.Dipartimento di MatematicaUniversità di FerraraFerraraItaly
  2. 2.Dipartimento di MatematicaUniversità di Modena e Reggio EmiliaModenaItaly

Personalised recommendations