Convergence of the Surrogate Lagrangian Relaxation Method
- 744 Downloads
Studies have shown that the surrogate subgradient method, to optimize non-smooth dual functions within the Lagrangian relaxation framework, can lead to significant computational improvements as compared to the subgradient method. The key idea is to obtain surrogate subgradient directions that form acute angles toward the optimal multipliers without fully minimizing the relaxed problem. The major difficulty of the method is its convergence, since the convergence proof and the practical implementation require the knowledge of the optimal dual value. Adaptive estimations of the optimal dual value may lead to divergence and the loss of the lower bound property for surrogate dual values. The main contribution of this paper is on the development of the surrogate Lagrangian relaxation method and its convergence proof to the optimal multipliers, without the knowledge of the optimal dual value and without fully optimizing the relaxed problem. Moreover, for practical implementations, a stepsizing formula that guarantees convergence without requiring the optimal dual value has been constructively developed. The key idea is to select stepsizes in a way that distances between Lagrange multipliers at consecutive iterations decrease, and as a result, Lagrange multipliers converge to a unique limit. At the same time, stepsizes are kept sufficiently large so that the algorithm does not terminate prematurely. At convergence, the lower-bound property of the surrogate dual is guaranteed. Testing results demonstrate that non-smooth dual functions can be efficiently optimized, and the new method leads to faster convergence as compared to other methods available for optimizing non-smooth dual functions, namely, the simple subgradient method, the subgradient-level method, and the incremental subgradient method.
KeywordsNon-smooth optimization Subgradient methods Surrogate subgradient method Lagrangian relaxation Mixed-integer programming
Mathematics Subject Classification (2000)90C25
This work was supported in part by grants from Southern California Edison and by the National Science Foundation under Grant ECCS–1028870. The authors would like to acknowledge Congcong Wang and Yaowen Yu for their careful perusal of the paper, insightful comments and valuable suggestions during numerous discussions.
- 6.Goffin, J.-L.: On the finite convergence of the relaxation method for solving systems of inequalities. Operations Research Center Report ORC 71–36, University of California at Berkeley, Berkeley (1971).Google Scholar
- 8.Nedic, A., Bertsekas, D.P.: Convergence rate of incremental subgradient algorithms. In: Uryasev, S., Pardalos, P.M. (eds.) Stochastic Optimization: Algorithms and Applications, pp. 263–304. Kluwer Academic, New York (2000)Google Scholar
- 11.Bertsekas, D.P.: Incremental gradient, subgradient, and proximal methods for convex optimization: a survey. LIDS Technical Report no. 2848, MIT, (2010).Google Scholar
- 14.Kaskavelis, C.A., Caramanis, M.C.: Efficient Lagrangian relaxation algorithms for industry size job-shop scheduling problems. IIE Trans. 30(11), 1085–1097 (1998)Google Scholar
- 19.Bragin, M.A., Han, X., Luh, P.B., Yan, J.H.: Payment cost minimization using Lagrangian relaxation and modified surrogate optimization approach. In: Proceedings of the IEEE Power Engineering Society, General Meeting, Detroit, Michigan (2011)Google Scholar
- 20.Bragin, M.A., Luh, P.B., Yan, J.H., Yu, N., Han, X., Stern, G.A.: An efficient surrogate subgradient method within Lagrangian relaxation for the payment cost minimization problem. In: Proceedings of the IEEE Power Engineering Society, General Meeting, San Diego (2012)Google Scholar
- 22.Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Massachusetts (2008)Google Scholar
- 37.Burkard, R.E., Offermann, J.: Entwurf von Schreibmaschinentastaturen mittels quadratischer Zuordnungsprobleme. Zeitschrift fur Oper. Res. 21(4), 121–132 (1977). (in German)Google Scholar