Mathematical Programming Computation

, Volume 10, Issue 2, pp 225–266 | Cite as

Algorithmic innovations and software for the dual decomposition method applied to stochastic mixed-integer programs

  • Kibaek Kim
  • Victor M. Zavala
Full Length Paper


We present algorithmic innovations for the dual decomposition method to address two-stage stochastic programs with mixed-integer recourse and provide an open-source software implementation that we call DSP. Our innovations include the incorporation of Benders-like cuts in a dual decomposition framework to tighten Lagrangian subproblems and aid the exclusion of infeasible first-stage solutions for problems without (relative) complete recourse. We also use an interior-point cutting-plane method with new termination criteria for solving the Lagrangian master problem. We prove that the algorithm converges to an optimal solution of the Lagrangian dual problem in a finite number of iterations, and we also prove that convergence can be achieved even if the master problem is solved suboptimally. DSP can solve instances specified in C code, SMPS files, and Julia script. DSP also implements a standard Benders decomposition method and a dual decomposition method based on subgradient dual updates that we use to perform benchmarks. We present extensive numerical results using SIPLIB instances and a large unit commitment problem to demonstrate that the proposed innovations provide significant improvements in the number of iterations and solution times. The software reviewed as part of this submission has been given the Digital Object Identifier (DOI)


Stochastic mixed-integer programming Decomposition Parallel Large-scale Open-source software 

Mathematics Subject Classification

90C15 90C11 49M27 65Y05 

Supplementary material


  1. 1.
    Achterberg, T.: SCIP: solving constraint integer programs. Math. Program. Comput. 1(1), 1–41 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Ahmed, S.: A scenario decomposition algorithm for 0–1 stochastic programs. Oper. Res. Lett. 41(6), 565–569 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Ahmed, S., Tawarmalani, M., Sahinidis, N.V.: A finite branch-and-bound algorithm for two-stage stochastic integer programs. Math. Program. 100(2), 355–377 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bezanson, J., Karpinski, S., Shah, V.B., Edelman, A.: Julia: a fast dynamic language for technical computing. arXiv preprint arXiv:1209.5145 (2012)
  5. 5.
    Birge, J.R., Dempster, M.A., Gassmann, H.I., Gunn, E.A., King, A.J., Wallace, S.W.: A standard input format for multiperiod stochastic linear programs. IIASA Laxenburg Austria (1987)Google Scholar
  6. 6.
    Birge, J.R., Louveaux, F.: Introduction to Stochastic Programming. Springer, Berlin (2011)CrossRefzbMATHGoogle Scholar
  7. 7.
    Bixby, R.E.: Solving real-world linear programs: a decade and more of progress. Oper. Res. 50(1), 3–15 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Carøe, C.C., Schultz, R.: Dual decomposition in stochastic integer programming. Oper. Res. Lett. 24(1–2), 37–45 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Crainic, T.G., Fu, X., Gendreau, M., Rei, W., Wallace, S.W.: Progressive hedging-based metaheuristics for stochastic network design. Networks 58(2), 114–124 (2011)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Dawande, M., Hooker, J.N.: Inference-based sensitivity analysis for mixed integer/linear programming. Oper. Res. 48(4), 623–634 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Fisher, M.L.: An applications oriented guide to lagrangian relaxation. Interfaces 15(2), 10–21 (1985)CrossRefGoogle Scholar
  12. 12.
    Fisher, M.L.: The Lagrangian relaxation method for solving integer programming problems. Manag. Sci. 50(12–supplement), 1861–1871 (2004)Google Scholar
  13. 13.
  14. 14.
  15. 15.
    Frangioni, A.: About lagrangian methods in integer optimization. Ann. Oper. Res. 139(1), 163–193 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Gade, D., Küçükyavuz, S., Sen, S.: Decomposition algorithms with parametric gomory cuts for two-stage stochastic integer programs. Math. Program. 144(1–2), 39–64 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Gamrath, G., Lübbecke, M.E.: Experiments with a generic Dantzig–Wolfe decomposition for integer programs. In: International Symposium on Experimental Algorithms, pp. 239–252. Springer (2010)Google Scholar
  18. 18.
    Gassmann, H.I., Schweitzer, E.: A comprehensive input format for stochastic linear programs. Ann. Oper. Res. 104(1–4), 89–125 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Geoffrion, A.M.: Lagrangean Relaxation for Integer Programming. Springer, Berlin (1974)CrossRefzbMATHGoogle Scholar
  20. 20.
    Gertz, E.M., Wright, S.J.: Object-oriented software for quadratic programming. ACM Trans. Math. Softw. (TOMS) 29(1), 58–81 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Goffin, J.L., Vial, J.P.: Cutting planes and column generation techniques with the projective algorithm. J. Optim. Theory Appl. 65(3), 409–429 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Gondzio, J.: Warm start of the primal-dual method applied in the cutting-plane scheme. Math. Program. 83(1–3), 125–143 (1998)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Gondzio, J., Gonzalez-Brevis, P., Munari, P.: New developments in the primal-dual column generation technique. Eur. J. Oper. Res. 224(1), 41–51 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Gondzio, J., Grothey, A.: A new unblocking technique to warmstart interior point methods based on sensitivity analysis. SIAM J. Optim. 19(3), 1184–1210 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Gondzio, J., Sarkissian, R.: Column generation with a primal-dual method. Relatorio tecnico, University of Geneva 102 (1996)Google Scholar
  26. 26.
    Guo, G., Hackebeil, G., Ryan, S.M., Watson, J.P., Woodruff, D.L.: Integration of progressive hedging and dual decomposition in stochastic integer programs. Oper. Res. Lett. 43(3), 311–316 (2015)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Gurobi Optimization, Inc.: Gurobi optimizer reference manual (2015).
  28. 28.
    Helmberg, C.: ConicBundle. (2004)
  29. 29.
    IBM Corp.: IBM ILOG CPLEX Optimization Studio 12.6.1 (2014).
  30. 30.
    Kim, K., Mehrotra, S.: A two-stage stochastic integer programming approach to integrated staffing and scheduling with application to nurse management. Oper. Res. 63(6), 1431–1451 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    King, A.: Stochastic Modeling Interface (2007).
  32. 32.
    Kleywegt, A.J., Shapiro, A., Homem-de Mello, T.: The sample average approximation method for stochastic discrete optimization. SIAM J. Optim. 12(2), 479–502 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Laporte, G., Louveaux, F.V.: The integer L-shaped method for stochastic integer programs with complete recourse. Oper. Res. Lett. 13(3), 133–142 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Lee, C., Liu, C., Mehrotra, S., Shahidehpour, M.: Modeling transmission line constraints in two-stage robust unit commitment problem. IEEE Trans. Power Syst. 29(3), 1221–1231 (2014)CrossRefGoogle Scholar
  35. 35.
    Linderoth, J., Wright, S.: Decomposition algorithms for stochastic programming on a computational grid. Comput. Optim. Appl. 24(2–3), 207–250 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Løkketangen, A., Woodruff, D.L.: Progressive hedging and tabu search applied to mixed integer (0, 1) multistage stochastic programming. J. Heuristics 2(2), 111–128 (1996)CrossRefzbMATHGoogle Scholar
  37. 37.
    Lubin, M., Dunning, I.: Computing in operations research using Julia. arXiv preprint arXiv:1312.1431 (2013)
  38. 38.
    Lubin, M., Martin, K., Petra, C.G., Sandıkçı, B.: On parallelizing dual decomposition in stochastic integer programming. Oper. Res. Lett. 41(3), 252–258 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Lubin, M., Petra, C.G., Anitescu, M., Zavala, V.: Scalable stochastic optimization of complex energy systems. In: 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC), pp. 1–10. IEEE (2011)Google Scholar
  40. 40.
    Lulli, G., Sen, S.: A branch-and-price algorithm for multistage stochastic integer programming with application to stochastic batch-sizing problems. Manag. Sci. 50(6), 786–796 (2004)CrossRefzbMATHGoogle Scholar
  41. 41.
    Märkert, A., Gollmer, R.: Users Guide to ddsip–a C package for the dual decomposition of two-stage stochastic programs with mixed-integer recourse (2014)Google Scholar
  42. 42.
    Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2(4), 575–601 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Mitchell, J.E.: Computational experience with an interior point cutting plane algorithm. SIAM J. Optim. 10(4), 1212–1227 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    OptiRisk Systems: FortSP: a stochastic programming solver, version 1.2 (2014).
  45. 45.
    Papavasiliou, A., Oren, S.S.: Multiarea stochastic unit commitment for high wind penetration in a transmission constrained network. Oper. Res. 61(3), 578–592 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Papavasiliou, A., Oren, S.S., O’Neill, R.P.: Reserve requirements for wind power integration: a scenario-based stochastic programming framework. IEEE Trans. Power Syst. 26(4), 2197–2206 (2011)CrossRefGoogle Scholar
  47. 47.
    Ralphs, T.K., Galati, M.V.: Decomposition in integer linear programming. Integer Program. Theory Pract. 3, 57–110 (2005)Google Scholar
  48. 48.
    Ralphs, T.K., Hassanzadeh, A.: A generalization of Benders algorithm for two-stage stochastic optimization problems with mixed integer recourse. Technical Report 14T-005, Department of Industrial and Systems Engineering, Lehigh University (2014)Google Scholar
  49. 49.
    Saltzman, M., Ladányi, L., Ralphs, T.: The COIN-OR open solver interface: technology overview. In: CORS/INFORMS Conference. Banff (2004)Google Scholar
  50. 50.
    Santoso, T., Ahmed, S., Goetschalckx, M., Shapiro, A.: A stochastic programming approach for supply chain network design under uncertainty. Eur. J. Oper. Res. 167(1), 96–115 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Sen, S., Higle, J.L.: The C\(^3\) theorem and a D\(^2\) algorithm for large scale stochastic mixed-integer programming: set convexification. Math. Program. 104(1), 1–20 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  52. 52.
    Sherali, H.D., Fraticelli, B.M.: A modification of Benders’ decomposition algorithm for discrete subproblems: an approach for stochastic programs with integer recourse. J. Global Optim. 22(1–4), 319–342 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  53. 53.
    Shinano, Y., Achterberg, T., Berthold, T., Heinz, S., Koch, T.: ParaSCIP: a parallel extension of SCIP. In: Competence in High Performance Computing 2010, pp. 135–148. Springer (2011)Google Scholar
  54. 54.
    Shinano, Y., Heinz, S., Vigerske, S., Winkler, M.: Fiberscip-a shared memory parallelization of scip, pp. 13–55. Zuse Institute Berlin, Technical Report ZR (2013)Google Scholar
  55. 55.
    Tarhan, B., Grossmann, I.E.: Improving dual bound for stochastic MILP models using sensitivity analysis. Working paper (2015)Google Scholar
  56. 56.
    Watson, J.P., Woodruff, D.L.: Progressive hedging innovations for a class of stochastic mixed-integer resource allocation problems. CMS 8(4), 355–370 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  57. 57.
    Watson, J.P., Woodruff, D.L., Hart, W.E.: PySP: modeling and solving stochastic programs in Python. Math. Program. Comput. 4(2), 109–149 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  58. 58.
    Wunderling, R.: Paralleler und objektorientierter Simplex-Algorithmus. Ph.D. Thesis, Technische Universität Berlin (1996).
  59. 59.
    Zhang, M., Kucukyavuz, S.: Finitely convergent decomposition algorithms for two-stage stochastic pure integer programs. SIAM J. Optim. 24(4), 1933–1951 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  60. 60.
    Zverovich, V., Fábián, C.I., Ellison, E.F., Mitra, G.: A computational study of a solver system for processing two-stage stochastic LPs with enhanced Benders’ decomposition. Math. Program. Comput. 4(3), 211–238 (2012)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany and The Mathematical Programming Society 2017

Authors and Affiliations

  1. 1.Mathematics and Computer Science DivisionArgonne National LaboratoryLemontUSA
  2. 2.Department of Chemical and Biological EngineeringUniversity of Wisconsin-MadisonMadisonUSA

Personalised recommendations