Journal of Optimization Theory and Applications

, Volume 164, Issue 1, pp 173–201 | Cite as

Convergence of the Surrogate Lagrangian Relaxation Method

  • Mikhail A. Bragin
  • Peter B. Luh
  • Joseph H. Yan
  • Nanpeng Yu
  • Gary A. Stern


Studies have shown that the surrogate subgradient method, to optimize non-smooth dual functions within the Lagrangian relaxation framework, can lead to significant computational improvements as compared to the subgradient method. The key idea is to obtain surrogate subgradient directions that form acute angles toward the optimal multipliers without fully minimizing the relaxed problem. The major difficulty of the method is its convergence, since the convergence proof and the practical implementation require the knowledge of the optimal dual value. Adaptive estimations of the optimal dual value may lead to divergence and the loss of the lower bound property for surrogate dual values. The main contribution of this paper is on the development of the surrogate Lagrangian relaxation method and its convergence proof to the optimal multipliers, without the knowledge of the optimal dual value and without fully optimizing the relaxed problem. Moreover, for practical implementations, a stepsizing formula that guarantees convergence without requiring the optimal dual value has been constructively developed. The key idea is to select stepsizes in a way that distances between Lagrange multipliers at consecutive iterations decrease, and as a result, Lagrange multipliers converge to a unique limit. At the same time, stepsizes are kept sufficiently large so that the algorithm does not terminate prematurely. At convergence, the lower-bound property of the surrogate dual is guaranteed. Testing results demonstrate that non-smooth dual functions can be efficiently optimized, and the new method leads to faster convergence as compared to other methods available for optimizing non-smooth dual functions, namely, the simple subgradient method, the subgradient-level method, and the incremental subgradient method.


Non-smooth optimization Subgradient methods Surrogate subgradient method Lagrangian relaxation Mixed-integer programming 

Mathematics Subject Classification (2000)




This work was supported in part by grants from Southern California Edison and by the National Science Foundation under Grant ECCS–1028870. The authors would like to acknowledge Congcong Wang and Yaowen Yu for their careful perusal of the paper, insightful comments and valuable suggestions during numerous discussions.


  1. 1.
    Ermoliev, Y.M.: Methods for solving nonlinear extremal problems. Cybernetics 2(4), 1–17 (1966)CrossRefGoogle Scholar
  2. 2.
    Polyak, B.T.: A general method of solving extremum problems. Sov. Math. Doklady 8, 593–597 (1967)MATHGoogle Scholar
  3. 3.
    Polyak, B.T.: Minimization of unsmooth functionals. USSR Comput. Math. Math. Phys. 9(3), 14–29 (1969). (in Russian)CrossRefGoogle Scholar
  4. 4.
    Shor, N.Z.: On the rate of convergence of the generalized gradient method. Cybernetics 4(3), 79–80 (1968)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Shor, N.Z.: Generalized gradient methods for non-smooth functions and their applications to mathematical programming problems. Econ. Math. Methods 12(2), 337–356 (1976). (in Russian)MATHGoogle Scholar
  6. 6.
    Goffin, J.-L.: On the finite convergence of the relaxation method for solving systems of inequalities. Operations Research Center Report ORC 71–36, University of California at Berkeley, Berkeley (1971).Google Scholar
  7. 7.
    Goffin, J.-L., Kiwiel, K.: Convergence of a simple subgradient level method. Math. Program. 85(1), 207–211 (1999)CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Nedic, A., Bertsekas, D.P.: Convergence rate of incremental subgradient algorithms. In: Uryasev, S., Pardalos, P.M. (eds.) Stochastic Optimization: Algorithms and Applications, pp. 263–304. Kluwer Academic, New York (2000)Google Scholar
  9. 9.
    Nedic, A., Bertsekas, D.: Incremental subgradient methods for nondifferentiable optimization. SIAM J. Optim. 56(1), 109–138 (2001)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005)CrossRefMATHMathSciNetGoogle Scholar
  11. 11.
    Bertsekas, D.P.: Incremental gradient, subgradient, and proximal methods for convex optimization: a survey. LIDS Technical Report no. 2848, MIT, (2010).Google Scholar
  12. 12.
    Bertsekas, D.P.: Incremental proximal methods for large scale convex optimization. Math. Program. 129, 163–195 (2011)CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Lemarechal, C., Nemirovskii, A.S., Nesterov, Y.E.: New variants of bundle methods. Math. Program. 69, 111–147 (1995)CrossRefMATHMathSciNetGoogle Scholar
  14. 14.
    Kaskavelis, C.A., Caramanis, M.C.: Efficient Lagrangian relaxation algorithms for industry size job-shop scheduling problems. IIE Trans. 30(11), 1085–1097 (1998)Google Scholar
  15. 15.
    Zhao, X., Luh, P.B., Wang, J.: Surrogate gradient algorithm for Lagrangian relaxation. J. Optim. Theory Appl. 100(3), 699–712 (1999)CrossRefMATHMathSciNetGoogle Scholar
  16. 16.
    Luh, P.B., Blankson, W.E., Chen, Y., Yan, J.H., Stern, G.A., Chang, S.C., Zhao, F.: Payment cost minimization auction for the deregulated electricity markets using surrogate optimization. IEEE Trans. Power Syst. 21(2), 568–578 (2006)CrossRefGoogle Scholar
  17. 17.
    Sun, T., Zhao, Q.C., Luh, P.B.: On the surrogate gradient algorithm for Lagrangian relaxation. J. Optim. Theory Appl. 133(3), 413–416 (2007)CrossRefMATHMathSciNetGoogle Scholar
  18. 18.
    Chang, T.S.: Comments on “Surrogate gradient algorithm for Lagrangian relaxation”. J. Optim. Theory Appl. 137(3), 691–697 (2008)CrossRefMATHMathSciNetGoogle Scholar
  19. 19.
    Bragin, M.A., Han, X., Luh, P.B., Yan, J.H.: Payment cost minimization using Lagrangian relaxation and modified surrogate optimization approach. In: Proceedings of the IEEE Power Engineering Society, General Meeting, Detroit, Michigan (2011)Google Scholar
  20. 20.
    Bragin, M.A., Luh, P.B., Yan, J.H., Yu, N., Han, X., Stern, G.A.: An efficient surrogate subgradient method within Lagrangian relaxation for the payment cost minimization problem. In: Proceedings of the IEEE Power Engineering Society, General Meeting, San Diego (2012)Google Scholar
  21. 21.
    Allen, E., Nelgason, R., Kennongton, J., Shettym, B.: A generalization of Polyak’s convergence result for subgradient optimization. Math. Program. 37(3), 309–317 (1987)CrossRefMATHGoogle Scholar
  22. 22.
    Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Massachusetts (2008)Google Scholar
  23. 23.
    Wah, B.W., Chen, Y.X.: Subgoal partitioning and global search for solving temporal planning problems in mixed space. Int. J. Artif. Intell. Tools 13(4), 767–790 (2004)CrossRefGoogle Scholar
  24. 24.
    Chu, P.C., Beasley, J.E.: A genetic algorithm for the generalized assignment problem. Comput. Oper. Res. 24(1), 17–23 (1997)CrossRefMATHMathSciNetGoogle Scholar
  25. 25.
    Yagiura, M., Yamaguchi, T., Ibaraki, T.: A variable depth search algorithm with branching search for the generalized assignment problem. Optim. Methods Softw. 10, 419–441 (1998)CrossRefMATHMathSciNetGoogle Scholar
  26. 26.
    Yagiura, M., Ibaraki, T., Glover, F.: A path relinking approach with ejection chains for the generalized assignment problem. Eur. J. Oper. Res. 169(2), 548–569 (2006)CrossRefMATHMathSciNetGoogle Scholar
  27. 27.
    Avella, P., Boccia, M., Vasilyev, I.: A computational study of exact knapsack separation for the generalized assignment problem. Comput. Optim. Appl. 45(3), 543–555 (2010)CrossRefMATHMathSciNetGoogle Scholar
  28. 28.
    Posta, M., Ferland, J.A., Michelon, P.: An exact method with variable fixing for solving the generalized assignment problem. Comput. Optim. Appl. 52(3), 629–644 (2012)CrossRefMATHMathSciNetGoogle Scholar
  29. 29.
    Özbakir, L., Baykasoglu, A., Tapkan, P.: Bees algorithm for generalized assignment problem. Appl. Math. Comput. 215(11), 3782–3795 (2010)CrossRefMATHMathSciNetGoogle Scholar
  30. 30.
    Asahiro, Y., Ishibashi, M., Yamashita, M.: Independent and cooperative parallel search methods for the generalized assignment problem. Optim. Methods Softw. 18(2), 129–141 (2003)CrossRefMATHMathSciNetGoogle Scholar
  31. 31.
    Laguna, M., Kelly, J.P., Gonzalez-Velarde, J.L., Glover, F.: Tabu search for the multilevel generalized assignment problem. Eur. J. Oper. Res. 82, 176–189 (1995)CrossRefMATHGoogle Scholar
  32. 32.
    Koopmans, T.C., Beckmann, M.J.: Assignment problems and the location of economic activities. Econometrica 25(1), 53–76 (1957)CrossRefMATHMathSciNetGoogle Scholar
  33. 33.
    Dickey, J.W., Hopkins, J.W.: Campus building arrangement using TOPAZ. Transp. Res. 6, 59–68 (1972)CrossRefGoogle Scholar
  34. 34.
    Elshafei, A.N.: Hospital layout as a quadratic assignment problem. Oper. Res. Q. 28, 167–179 (1977)CrossRefMATHGoogle Scholar
  35. 35.
    Geoffrion, A.M., Graves, G.W.: Scheduling parallel production lines with changeover costs: practical applications of a quadratic assignment/LP approach. Oper. Res. 24, 595–610 (1976)CrossRefMATHGoogle Scholar
  36. 36.
    Krarup, J., Pruzan, P.M.: Computer-aided layout design. Math. Program. Study 9, 75–94 (1978)CrossRefMathSciNetGoogle Scholar
  37. 37.
    Burkard, R.E., Offermann, J.: Entwurf von Schreibmaschinentastaturen mittels quadratischer Zuordnungsprobleme. Zeitschrift fur Oper. Res. 21(4), 121–132 (1977). (in German)Google Scholar
  38. 38.
    Christofides, N., Benavent, E.: An exact algorithm for the quadratic assignment problem. Oper. Res. 37(5), 760–768 (1989)CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Mikhail A. Bragin
    • 1
  • Peter B. Luh
    • 1
  • Joseph H. Yan
    • 2
  • Nanpeng Yu
    • 2
  • Gary A. Stern
    • 2
  1. 1.Department of Electrical and Computer EngineeringUniversity of ConnecticutStorrsUSA
  2. 2.Southern California EdisonRosemeadUSA

Personalised recommendations