Journal of Global Optimization

, Volume 73, Issue 1, pp 113–151 | Cite as

Convergence-order analysis for differential-inequalities-based bounds and relaxations of the solutions of ODEs

  • Spencer D. Schaber
  • Joseph K. Scott
  • Paul I. BartonEmail author


For the performance of global optimization algorithms, the rate of convergence of convex relaxations to the objective and constraint functions is critical. We extend results from Bompadre and Mitsos (J Glob Optim 52(1):1–28, 2012) to characterize the convergence rate of parametric bounds and relaxations of the solutions of ordinary differential equations (ODEs). Such bounds and relaxations are used for global dynamic optimization and are computed using auxiliary ODE systems that use interval arithmetic and McCormick relaxations. Two ODE relaxation methods (Scott et al. in Optim Control Appl Methods 34(2):145–163, 2013; Scott and Barton in J Glob Optim 57:143–176, 2013) are shown to give second-order convergence, yet they can behave very differently from each other in practice. As time progresses, the prefactor in the convergence-order bound tends to grow much more slowly for one of these methods, and can even decrease over time, yielding global optimization procedures that require significantly less computation time.


Deterministic global optimization Convergence-order analysis Nonconvex optimization Dynamic optimization 

Mathematics Subject Classification

34A40 49M20 49M37 65L05 65L20 90C26 



We gratefully acknowledge funding from Novartis Pharmaceuticals and helpful comments from the anonymous reviewers.


  1. 1.
    Alefeld, G., Mayer, G.: Interval analysis: theory and applications. J. Comput. Appl. Math. 121(1–2), 421–464 (2000)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Banga, J.R., Alonso, A.A., Singh, R.P.: Stochastic dynamic optimization of batch and semicontinuous bioprocesses. Biotechnol. Prog. 13, 326–335 (1997)Google Scholar
  3. 3.
    Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)zbMATHGoogle Scholar
  4. 4.
    Bompadre, A., Mitsos, A.: Convergence rate of McCormick relaxations. J. Glob. Optim. 52(1), 1–28 (2012)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Bompadre, A., Mitsos, A., Chachuat, B.: Convergence analysis of Taylor models and McCormick–Taylor models. J. Glob. Optim. 57, 75–114 (2013)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Cervantes, A.M., Wächter, A., Tütüncü, R.H., Biegler, L.T.: A reduced space interior point strategy for optimization of differential algebraic systems. Comput. Chem. Eng. 24, 39–51 (2000)Google Scholar
  7. 7.
    Chachuat, B., Barton, P.I., Singer, A.B.: Global methods for dynamic optimization and mixed-integer dynamic optimization. Ind. Eng. Chem. Res. 45(25), 8373–8392 (2006)Google Scholar
  8. 8.
    Chachuat, B., Villanueva, M.: Bounding the solutions of parametric ODEs: when Taylor models meet differential inequalities. In: Bogle, I.D.L., Fairweather, M. (eds.) 22 European Symposium on Computer Aided Process Engineering, volume 30 of Comput. Aided Chem. Eng., pp. 1307–1311. Elsevier Science BV (2012)Google Scholar
  9. 9.
    Dahlquist, G.: Stability and error bounds in the numerical integration of ordinary differential equations. PhD thesis, University of Stockholm (1958)Google Scholar
  10. 10.
    Du, K., Kearfott, R.B.: The cluster problem in multivariate global optimization. J. Glob. Optim. 5(3), 253–265 (1994)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Esposito, W.R., Floudas, C.A.: Global optimization for the parameter estimation of differential-algebraic systems. Ind. Eng. Chem. Res. 39, 1291–1310 (2000)Google Scholar
  12. 12.
    Filippov, A.F.: Differential Equations with Discontinuous Righthand Sides. Kluwer Academic Publishers, Dordrecht (1988)Google Scholar
  13. 13.
    Hairer, E., Norsett, S.P., Wanner, G.: Solving Ordinary Differential Equations I. Springer, Berlin (1993)zbMATHGoogle Scholar
  14. 14.
    Harrison, G.: Dynamic models with uncertain parameters. In: Avula, X. (ed.) Proc. First Int. Conf. Math. Model., vol. 1, pp. 295–304. University of Missouri, Rolla (1977)Google Scholar
  15. 15.
    Harwood, S.M., Barton, P.I.: Efficient polyhedral enclosures for the reachable set of nonlinear control systems. Math. Control Signal 28(1), 8 (2016)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Harwood, S.M., Scott, J.K., Barton, P.I.: Bounds on reachable sets using ordinary differential equations with linear programs embedded. IMA J. Math. Control I 33(2), 519–541 (2016)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Houska, B., Logist, F., Van Impe, J., Diehl, M.: Robust optimization of nonlinear dynamic systems with application to a jacketed tubular reactor. J. Process Control 22(6), 1152–1160 (2012)Google Scholar
  18. 18.
    Houska, B., Villanueva, M.E., Chachuat, B.: A validated integration algorithm for nonlinear ODEs using Taylor models and ellipsoidal calculus. In: 52nd IEEE Conf. Decis. Control, pp. 484–489 (2013)Google Scholar
  19. 19.
    Houska, B., Villanueva, M.E., Chachuat, B.: Stable set-valued integration of nonlinear dynamic systems using affine set-parameterizations. SIAM J. Numer. Anal. 53(5), 2307–2328 (2015)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Khalil, H.K.: Nonlinear Systems, 3rd edn. Prentice-Hall, Upper Saddle River (2002)zbMATHGoogle Scholar
  21. 21.
    Khan, K.A., Watson, H.A.J., Barton, P.I.: Differentiable mccormick relaxations. J. Glob. Optim. 67(4), 687–729 (2017)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Krogh, B.H., Thorpe, C.E.: Integrated path planning and dynamic steering control for autonomous vehicles. In: Proceedings of 1986 IEEE Int. Conf. on Robot. Autom., vol. 3, pp. 1664–1669. IEEE (1986)Google Scholar
  23. 23.
    Leineweber, D.B., Bauer, I., Bock, H.G., Schlöder, J.P.: An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization. Part 1: theoretical aspects. Comput. Chem. Eng. 27, 157–166 (2003)Google Scholar
  24. 24.
    Lin, Y., Stadtherr, M.A.: Deterministic global optimization for parameter estimation of dynamic systems. Ind. Eng. Chem. Res. 45(25), 8438–8448 (2006)Google Scholar
  25. 25.
    Lin, Y., Stadtherr, M.A.: Deterministic global optimization of nonlinear dynamic systems. AIChE J. 53(4), 866–875 (2007)Google Scholar
  26. 26.
    Lin, Y., Stadtherr, M.A.: Validated solutions of initial value problems for parametric ODEs. Appl. Numer. Math. 57(10), 1145–1162 (2007)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Løvik, I., Hillestad, M., Hertzberg, T.: Long term dynamic optimization of a catalytic reactor system. Comput. Chem. Eng. 22, S707–S710 (1998)Google Scholar
  28. 28.
    Luus, R., Dittrich, J., Keil, F.J.: Multiplicity of solutions in the optimization of a bifunctional catalyst blend in a tubular reactor. Can. J. Chem. Eng. 70(4), 780–785 (1992)Google Scholar
  29. 29.
    Maravall, D., de Lope, J.: Multi-objective dynamic optimization with genetic algorithms for automatic parking. Soft Comput. 11(3), 249–257 (2007)Google Scholar
  30. 30.
    McCormick, G.P.: Computability of global solutions to factorable nonconvex programs: part I—convex underestimating problems. Math. Program. 10(1), 147–175 (1976)zbMATHGoogle Scholar
  31. 31.
    Moles, C.G., Mendes, P., Banga, J.R.: Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Res. 13(11), 2467–2474 (2003)Google Scholar
  32. 32.
    Moore, R.E.: Interval arithmetic and automatic error analysis in digital computing. PhD thesis, Stanford University (1962)Google Scholar
  33. 33.
    Moore, R.E.: Methods and Applications of Interval Analysis. SIAM, Philadelphia (1979)zbMATHGoogle Scholar
  34. 34.
    Moore, R.E., Kearfott, R.B., Cloud, M.J.: Introduction to Interval Analysis. Society for Industrial and Applied Mathematics, Philadelphia (2009)zbMATHGoogle Scholar
  35. 35.
    Müller, M.: Über die Eindeutigkeit der Integrale eines Systems gewöhnlicher Differentialgleichungen und die Konvergenz einer Gattung von Verfahren zur Approximation dieser Integrale. Sitz.-Ber. Heidelberger Akad. Wiss. Math.-Naturwiss. Kl. 9, 3–38 (1927)zbMATHGoogle Scholar
  36. 36.
    Najman, J., Mitsos, A.: Convergence analysis of multivariate mccormick relaxations. J. Glob. Optim. 66(4), 597–628 (2016)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Neumaier, A.: Taylor forms-use and limits. Reliab. Comput. 9(1), 43–79 (2003). MathSciNetzbMATHGoogle Scholar
  38. 38.
    Pongpunwattana, A., Rysdyk, R.: Real-time planning for multiple autonomous vehicles in dynamic uncertain environments. J. Aerosp. Comput. Inf. Commun. 1(12), 580–604 (2004)Google Scholar
  39. 39.
    Prata, A., Oldenburg, J., Kroll, A., Marquardt, W.: Integrated scheduling and dynamic optimization of grade transitions for a continuous polymerization reactor. Comput. Chem. Eng. 32(3), 463–476 (2008)Google Scholar
  40. 40.
    Rodriguez-Fernandez, M., Egea, J.A., Banga, J.R.: Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems. BMC Bioinformatics 7, 483 (2006)Google Scholar
  41. 41.
    Sahlodin, A.M.: Global optimization of dynamic process systems using complete search methods. PhD thesis, McMaster University (2013)Google Scholar
  42. 42.
    Sahlodin, A.M., Chachuat, B.: Convex/concave relaxations of parametric ODEs using Taylor models. Comput. Chem. Eng. 35(5), 844–857 (2011)zbMATHGoogle Scholar
  43. 43.
    Sahlodin, A.M., Chachuat, B.: Discretize-then-relax approach for convex/concave relaxations of the solutions of parametric ODEs. Appl. Numer. Math. 61(7), 803–820 (2011)MathSciNetzbMATHGoogle Scholar
  44. 44.
    Schaber, S.D.: Tools for dynamic model development. PhD thesis, Massachusetts Institute of Technology (2014)Google Scholar
  45. 45.
    Schöbel, A., Scholz, D.: The theoretical and empirical rate of convergence for geometric branch-and-bound methods. J. Glob. Optim. 48, 473–495 (2010)MathSciNetzbMATHGoogle Scholar
  46. 46.
    Scholz, D.: Theoretical rate of convergence for interval inclusion functions. J. Glob. Optim. 53(4), 749–767 (2012)MathSciNetzbMATHGoogle Scholar
  47. 47.
    Scott, J.K.: Reachability analysis and deterministic global optimization of differential-algebraic systems. PhD thesis, Massachusetts Institute of Technology (2012)Google Scholar
  48. 48.
    Scott, J.K., Barton, P.I.: Tight, efficient bounds on the solutions of chemical kinetics models. Comput. Chem. Eng. 34(5), 717–731 (2010)Google Scholar
  49. 49.
    Scott, J.K., Barton, P.I.: Bounds on the reachable sets of nonlinear control systems. Automatica 49(1), 93–100 (2013)MathSciNetzbMATHGoogle Scholar
  50. 50.
    Scott, J.K., Barton, P.I.: Convex and concave relaxations for the parametric solutions of semi-explicit index-one differential-algebraic equations. J. Optim. Theory Appl. 156, 617–649 (2013)MathSciNetzbMATHGoogle Scholar
  51. 51.
    Scott, J.K., Barton, P.I.: Improved relaxations for the parametric solutions of ODEs using differential inequalities. J. Glob. Optim. 57, 143–176 (2013)MathSciNetzbMATHGoogle Scholar
  52. 52.
    Scott, J.K., Barton, P.I.: Interval bounds on the solutions of semi-explicit index-one DAEs. Part 1: analysis. Numer. Math. 125(1), 1–25 (2013)MathSciNetzbMATHGoogle Scholar
  53. 53.
    Scott, J.K., Barton, P.I.: Interval bounds on the solutions of semi-explicit index-one DAEs. Part 2: computation. Numer. Math. 125(1), 27–60 (2013)MathSciNetzbMATHGoogle Scholar
  54. 54.
    Scott, J.K., Chachuat, B., Barton, P.I.: Nonlinear convex and concave relaxations for the solutions of parametric ODEs. Optim. Control Appl. Methods 34(2), 145–163 (2013)MathSciNetzbMATHGoogle Scholar
  55. 55.
    Scott, J.K., Stuber, M.D., Barton, P.I.: Generalized McCormick relaxations. J. Glob. Optim. 51(4), 569–606 (2011)MathSciNetzbMATHGoogle Scholar
  56. 56.
    Shen, K., Scott, J.K.: Rapid and accurate reachability analysis for nonlinear dynamic systems by exploiting model redundancy. Comput. Chem. Eng. 106, 596–608 (2017)Google Scholar
  57. 57.
    Singer, A.B., Barton, P.I.: Bounding the solutions of parameter dependent nonlinear ordinary differential equations. SIAM J. Sci. Comput. 27(6), 2167 (2006)MathSciNetzbMATHGoogle Scholar
  58. 58.
    Singer, A.B., Barton, P.I.: Global optimization with nonlinear ordinary differential equations. J. Glob. Optim. 34(2), 159–190 (2006). MathSciNetzbMATHGoogle Scholar
  59. 59.
    Singer, A.B., Taylor, J.W., Barton, P.I., Green, W.H.: Global dynamic optimization for parameter estimation in chemical kinetics. J. Phys. Chem. A 110(3), 971–976 (2006)Google Scholar
  60. 60.
    Söderlind, G.: The logarithmic norm. History and modern theory. BIT Numer. Math. 46(3), 631–652 (2006)MathSciNetzbMATHGoogle Scholar
  61. 61.
    Tjoa, I.-B., Biegler, L.T.: Simultaneous solution and optimization strategies for parameter estimation of differential-algebraic equation systems. Ind. Eng. Chem. Res. 30, 376–385 (1991)Google Scholar
  62. 62.
    Tulsyan, A., Barton, P.I.: Interval enclosures for reachable sets of chemical kinetic flow systems. Part 1: sparse transformation. Chem. Eng. Sci. 166, 334–344 (2017)Google Scholar
  63. 63.
    Tulsyan, A., Barton, P.I.: Interval enclosures for reachable sets of chemical kinetic flow systems. Part 2: direct-bounding method. Chem. Eng. Sci. 166, 345–357 (2017)Google Scholar
  64. 64.
    Tulsyan, A., Barton, P.I.: Interval enclosures for reachable sets of chemical kinetic flow systems. Part 3: indirect-bounding method. Chem. Eng. Sci. 166, 358–372 (2017)Google Scholar
  65. 65.
    Vassiliadis, V., Sargent, R., Pantelides, C.: Solution of a class of multistage dynamic optimization problems. 1. Problems without path constraints. Ind. Eng. Chem. Res. 33, 2111–2122 (1994)Google Scholar
  66. 66.
    Villanueva, M.E., Houska, B., Chachuat, B.: Unified framework for the propagation of continuous-time enclosures for parametric nonlinear ODEs. J. Glob. Optim. 62(3), 575–613 (2015)MathSciNetzbMATHGoogle Scholar
  67. 67.
    Wechsung, A., Schaber, S.D., Barton, P.I.: The cluster problem revisited. J. Glob. Optim. 58(3), 429–438 (2014)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Spencer D. Schaber
    • 1
  • Joseph K. Scott
    • 2
  • Paul I. Barton
    • 3
    Email author
  1. 1.Cargill, Inc.PlymouthUSA
  2. 2.Clemson UniversityClemsonUSA
  3. 3.Massachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations