Discrete Time Optimal Control Problems on Large Intervals

Part of the Advances in Mathematical Economics book series (MATHECON, volume 19)


In this paper we study the structure of approximate solutions of an autonomous nonconcave discrete-time optimal control system with a compact metric space of states. In the first part of the paper we discuss our recent results for systems described by a bounded upper semicontinuous objective function which determines an optimality criterion. These optimal control systems are discrete-time analogs of Lagrange problems in the calculus of variations. It is known that approximate solutions are determined mainly by the objective function, and are essentially independent of the choice of time interval and data, except in regions close to the endpoints of the time interval. Our goal is to study the structure of approximate solutions in regions close to the endpoints of the time intervals. The second part of the paper contains new results on the structure of solutions of optimal control systems which are discrete-time analogs of Bolza problems in the calculus of variations. These systems are described by a pair of objective functions which determines an optimality criterion.


Good program Optimal control problem Overtaking optimal program Turnpike property 


  1. 1.
    Anderson BDO, Moore JB (1971) Linear optimal control. Prentice-Hall, Englewood CliffsGoogle Scholar
  2. 2.
    Aseev SM, Veliov VM (2012) Maximum principle for infinite-horizon optimal control problems with dominating discount. Dyn Contin Discret Impuls Syst Ser B 19:43–63MathSciNetGoogle Scholar
  3. 3.
    Aubry S, Le Daeron PY (1983) The discrete Frenkel-Kontorova model and its extensions I. Physica D 8:381–422MathSciNetCrossRefGoogle Scholar
  4. 4.
    Baumeister J, Leitao A, Silva GN (2007) On the value function for nonautonomous optimal control problems with infinite horizon. Syst Control Lett 56:188–196MathSciNetCrossRefGoogle Scholar
  5. 5.
    Blot J (2009) Infinite-horizon Pontryagin principles without invertibility. J Nonlinear Convex Anal 10:177–189MathSciNetGoogle Scholar
  6. 6.
    Blot J, Cartigny P (2000) Optimality in infinite-horizon variational problems under sign conditions. J Optim Theory Appl 106:411–419MathSciNetCrossRefGoogle Scholar
  7. 7.
    Blot J, Hayek N (2014) Infinite-horizon optimal control in the discrete-time framework. Springer briefs in optimization. Springer, New YorkCrossRefGoogle Scholar
  8. 8.
    Cartigny P, Michel P (2003) On a sufficient transversality condition for infinite horizon optimal control problems. Autom J IFAC 39:1007–1010MathSciNetCrossRefGoogle Scholar
  9. 9.
    Gaitsgory V, Rossomakhine S, Thatcher N (2012) Approximate solution of the HJB inequality related to the infinite horizon optimal control problem with discounting. Dyn Contin Discret Impuls Syst Ser B 19:65–92MathSciNetGoogle Scholar
  10. 10.
    Hayek N (2011) Infinite horizon multiobjective optimal control problems in the discrete time case. Optimization 60:509–529MathSciNetCrossRefGoogle Scholar
  11. 11.
    Jasso-Fuentes H, Hernandez-Lerma O (2008) Characterizations of overtaking optimality for controlled diffusion processes. Appl Math Optim 57:349–369MathSciNetCrossRefGoogle Scholar
  12. 12.
    Kolokoltsov V, Yang W (2012) The turnpike theorems for Markov games. Dyn Games Appl 2:294–312MathSciNetCrossRefGoogle Scholar
  13. 13.
    Leizarowitz A (1985) Infinite horizon autonomous systems with unbounded cost. Appl Math Opt 13:19–43MathSciNetCrossRefGoogle Scholar
  14. 14.
    Leizarowitz A (1986) Tracking nonperiodic trajectories with the overtaking criterion. Appl Math Opt 14:155–171MathSciNetCrossRefGoogle Scholar
  15. 15.
    Leizarowitz A, Mizel VJ (1989) One dimensional infinite horizon variational problems arising in continuum mechanics. Arch Ration Mech Anal 106:161–194MathSciNetCrossRefGoogle Scholar
  16. 16.
    Lykina V, Pickenhain S, Wagner M (2008) Different interpretations of the improper integral objective in an infinite horizon control problem. J Math Anal Appl 340:498–510MathSciNetCrossRefGoogle Scholar
  17. 17.
    Makarov VL, Rubinov AM (1977) Mathematical theory of economic dynamics and equilibria. Springer, New YorkCrossRefGoogle Scholar
  18. 18.
    Malinowska AB, Martins N, Torres DFM (2011) Transversality conditions for infinite horizon variational problems on time scales. Optim Lett 5:41–53MathSciNetCrossRefGoogle Scholar
  19. 19.
    Marcus M, Zaslavski AJ (1999) The structure of extremals of a class of second order variational problems. Ann Inst Henri Poincaré Anal Non Lineare 16:593–629MathSciNetCrossRefGoogle Scholar
  20. 20.
    McKenzie LW (1976) Turnpike theory. Econometrica 44:841–866MathSciNetCrossRefGoogle Scholar
  21. 21.
    Mordukhovich BS (2011) Optimal control and feedback design of state-constrained parabolic systems in uncertainly conditions. Appl Anal 90:1075–1109MathSciNetCrossRefGoogle Scholar
  22. 22.
    Mordukhovich BS, Shvartsman I (2004) Optimization and feedback control of constrained parabolic systems under uncertain perturbations. In: de Queiroz MS, Malisoff M, Wolensk P (eds) Optimal control, stabilization and nonsmooth analysis. Lecture notes in control and information sciences. Springer, Berlin/New York, pp 121–132CrossRefGoogle Scholar
  23. 23.
    Ocana Anaya E, Cartigny P, Loisel P (2009) Singular infinite horizon calculus of variations. Applications to fisheries management. J Nonlinear Convex Anal 10:157–176MathSciNetGoogle Scholar
  24. 24.
    Pickenhain S, Lykina V, Wagner M (2008) On the lower semicontinuity of functionals involving Lebesgue or improper Riemann integrals in infinite horizon optimal control problems. Control Cybern 37:451–468MathSciNetGoogle Scholar
  25. 25.
    Porretta A, Zuazua E (2013) Long time versus steady state optimal control. SIAM J Control Optim 51:4242–4273MathSciNetCrossRefGoogle Scholar
  26. 26.
    Samuelson PA (1965) A catenary turnpike theorem involving consumption and the golden rule. Am Econ Rev 55:486–496Google Scholar
  27. 27.
    Zaslavski AJ (2006) Turnpike properties in the calculus of variations and optimal control. Springer, New YorkGoogle Scholar
  28. 28.
    Zaslavski AJ (2007) Turnpike results for a discrete-time optimal control systems arising in economic dynamics. Nonlinear Anal 67:2024–2049MathSciNetCrossRefGoogle Scholar
  29. 29.
    Zaslavski AJ (2009) Two turnpike results for a discrete-time optimal control systems. Nonlinear Anal 71:902–909MathSciNetCrossRefGoogle Scholar
  30. 30.
    Zaslavski AJ (2010) Stability of a turnpike phenomenon for a discrete-time optimal control systems. J Optim Theory Appl 145:597–612MathSciNetCrossRefGoogle Scholar
  31. 31.
    Zaslavski AJ (2011) Turnpike properties of approximate solutions for discrete-time control systems. Commun Math Anal 11:36–45MathSciNetGoogle Scholar
  32. 32.
    Zaslavski AJ (2011) Structure of approximate solutions for a class of optimal control systems. J Math Appl 34:1–14MathSciNetGoogle Scholar
  33. 33.
    Zaslavski AJ (2011) Stability of a turnpike phenomenon for a class of optimal control systems in metric spaces. Numer Algebra Control Optim 1:245–260MathSciNetCrossRefGoogle Scholar
  34. 34.
    Zaslavski AJ (2011) The existence and structure of approximate solutions of dynamic discrete time zero-sum games. J Nonlinear Convex Anal 12:49–68MathSciNetGoogle Scholar
  35. 35.
    Zaslavski AJ (2013) Nonconvex optimal control and variational problems. Springer optimization and its applications. Springer, New YorkCrossRefGoogle Scholar
  36. 36.
    Zaslavski AJ (2014) Turnpike properties of approximate solutions of nonconcave discrete-time optimal control problems. J Convex Anal 21:681–701MathSciNetGoogle Scholar
  37. 37.
    Zaslavski AJ (2014) Turnpike phenomenon and infinite horizon optimal control. Springer optimization and its applications. Springer, New YorkGoogle Scholar
  38. 38.
    Zaslavski AJ (2014) Turnpike properties for nonconcave problems. Adv Math Econ 18:101–134MathSciNetCrossRefGoogle Scholar
  39. 39.
    Zaslavski AJ (2014) Structure of solutions of discrete time optimal control roblems in the regions close to the endpoints. Set-Valued Var Anal 22:809–842MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Japan 2015

Authors and Affiliations

  1. 1.Department of MathematicsThe Technion – Israel Institute of TechnologyTechnion City, HaifaIsrael

Personalised recommendations