Skip to main content
SpringerLink
Log in

Time-Inconsistent Optimal Control Problems and Related Issues

  • Chapter
  • First Online:
Modeling, Stochastic Control, Optimization, and Applications

Part of the book series: The IMA Volumes in Mathematics and its Applications ((IMA,volume 164))

Abstract

Classical stochastic optimal control problems are time-consistent, by which it means that an optimal control selected at a given initial pair remains optimal thereafter, along the optimal pair. When the discount is non-exponential and/or the probability is subjective, the corresponding optimal control problem is time-inconsistent, in general. In this paper, we survey recent results in the area and briefly present some of our on-going works.

Jiongmin Yong is partially supported by NSF Grant DMS-1812921.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A. Adams, L. Cherchye, B. De Rock, and E. Verriest, Consume now or later? Time-inconsistency, collective choice and revealed preference, Amer. Econ. Review, 104 (2012), 4147–4183.

    Article  Google Scholar 

  2. J. Ai, L. Zhao, and W. Zhu, Equilibrium and welfare in insurance markets with time-inconsistent consumers, preprint.

    Google Scholar 

  3. Z. Akin, The role of time-inconsistent preferences in intertemporal investment decisions and bargaining, preprint.

    Google Scholar 

  4. Z. Akin, Time inconsistency and learning in bargaining game, Int. J. Game Theory, 36 (2007), 275–299.

    Article  MathSciNet  MATH  Google Scholar 

  5. M. Allais, Le comportement de l’homme rationnel devant de risque, critique des postulats et axiomes de l’école Américaine, Econometrica, 21 (1953), 503–546.

    Google Scholar 

  6. J. Andreoni and C. Sprenger, Risk preferences are not time preferences, Amer. Econ. Review, 102 (2012), 3357–3376.

    Article  Google Scholar 

  7. E. Bayraktar, A. Cosso and H. Pham, Randomized dynamic programming principle and Feynman-Kac representation for optimal control of McKean-Vlasov dynamics, Trans AMS, 370 (2017), 2115-2160.

    Article  MathSciNet  MATH  Google Scholar 

  8. T. Björk and A. Murgoci, A theory of Markovian time-inconsistent stochastic control in discrete time, Finance Stoch., 18 (2014), 545–592.

    Article  MathSciNet  MATH  Google Scholar 

  9. T. Björk, M. Khapko, and A. Murgoci, On time-inconsistent stochastic control in continuous time, Finance Stoch., 21 (2017), 331–360.

    Article  MathSciNet  MATH  Google Scholar 

  10. T. Björk, A. Murgoci, and X. Y. Zhou, Mean-variance portfolio optimization with state dependent risk aversion, Math. Finance, 24 (2014), 1–24.

    Article  MathSciNet  MATH  Google Scholar 

  11. M. K. Brunnermeier, F. Papakonstantinou, and J. A. Parker, Optimal time-inconsistent beliefs: misplanning, procrastination, and commitment, Management Sci., 63 (2017), 1318–1340.

    Article  Google Scholar 

  12. F. N. Caliendo, Time-inconsistent preferences and social security: Revisited in continuous time, J. Econ. Dynamic & Control, 35 (2011), 668–675.

    Article  MathSciNet  MATH  Google Scholar 

  13. R. Carmona and F. Delarue, Proabilistic Theory of Mean Field Games with Applications I, Mean Field FBSDEs, Control, and Games, Springer, 2018.

    Google Scholar 

  14. A. Caplin and J. Leahy, The recursive approach to time inconsistency, J. Economic Theory, 131 (2006), 134–156.

    Article  MathSciNet  MATH  Google Scholar 

  15. F. Cherbonnier, Optimal insurance for time-inconsistent agents, preprint.

    Google Scholar 

  16. G. Choquet, Theory of capacities, Ann. Inst. Fourier, 5 (1954), 131–296.

    Article  MathSciNet  MATH  Google Scholar 

  17. B. de Finetti, Theory of Probability: A Critical Introductory Treatment, Vol. 1, John Wiley & Sons, New York.

    Google Scholar 

  18. M. Dodd, Obesity and time-inconsistent preferences, Obesity Research & Clinical Practice, 2 (2008), 83–89.

    Article  Google Scholar 

  19. A. Dominiak and J.-P. Lefort, Unambiguous events and dynamic Choquet preferences, Econ. Theory, 46 (2011), 401–425.

    Google Scholar 

  20. Y. Dong and R. Sircar, Time-inconsistent portfolio investment problems, Springer Proc. Math. Stat., 100, Stoch. Anal. Appl., 239–281.

    Google Scholar 

  21. D. Duffie, L. Epstein, Stochastic differential utility, Econometrica, 60 (1992), 353–394.

    Article  MathSciNet  MATH  Google Scholar 

  22. I. Ekeland and A. Lazrak, The golden rule when preferences are time-inconsistent, Math. Finance Econ., 4 (2010), 29–55.

    Article  MathSciNet  MATH  Google Scholar 

  23. I. Ekeland, Y. Long, and Q. Zhou, A new class of problems in the calculus of variations, Regular and Chaotic Dynamics, 18 (2013), 553–584.

    Article  MathSciNet  MATH  Google Scholar 

  24. I. Ekeland, O. Mbodji, and T. A. Pirvu, Time-consistent portfolio management, SIAM J. Fin. Math., 3 (2012), 1–32.

    Article  MathSciNet  MATH  Google Scholar 

  25. I. Ekeland and T. A. Pirvu, Investment and consumption without commitment, Math. Finance Econ., 2 (2007), 57–68.

    Article  MathSciNet  MATH  Google Scholar 

  26. J. D. El Baghdady, Equilibrim strategies for time-inconsistent stochastic optimal control of asset allocation, PhD Dissertation, KTH Royal Institute of Technology, Stockholm, Sweden, 2017.

    Google Scholar 

  27. N. El Karoui, S. Peng, and M. C. Quenez, Backward stochastic differential equations in finance, Math. Finance, 7 (1997), 1–71.

    Google Scholar 

  28. N. El Karoui, S. Peng, and M. C. Quenez, A dynamic maximum principle for the optimization of recursive utilities under constraints, Ann. Appl. Probab., 11 (2001), 664–693.

    Google Scholar 

  29. D. Ellsberg, Risk, ambiguity and the Savage axioms, Quarterly Journal of Economics, 75 (1961), 643–649.

    Article  MATH  MathSciNet  Google Scholar 

  30. T. S. Findley and F. N. Caliendo, Short horizons, time inconsistency and optimal social security, Int. Tax & Public Finance, 16 (2009), 487–513.

    Article  Google Scholar 

  31. T. S. Findley and J. A. Feigenbaum, J. A. Quasi-hyperbolic discounting and the existence of time-inconsistent retirement, Theoretical Econ. Lett., 3 (2013), 119–123.

    Article  Google Scholar 

  32. S. Frederick, G. Loewnstein, and T. O’Donoghue, Time discounting and time preference: A critical review J. Economic Literature, 40 (2002), 351–401.

    Google Scholar 

  33. S. M. Goldman, Consistent plans, Review of Economic Studies, 47 (1980), 533–537.

    Google Scholar 

  34. S. L. Green, Time inconsistency, self-control, and remembrance, Faith & Economics, 42 (2003), 51–60.

    Google Scholar 

  35. S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences, J. Financial Ecnomics, 84 (2007), 2–39.

    Article  Google Scholar 

  36. L. Guo and F. N. Caliendo, Time-inconsistent preferences and time-inconsistent policies, J. Math. Econ., 51 (2011), 102–108.

    Article  MathSciNet  MATH  Google Scholar 

  37. Y. Halevy, Time consistency: stationarity and time invariance, Econometrica, 83 (2015), 335–352.

    Article  MathSciNet  MATH  Google Scholar 

  38. X. He and X. Y. Zhou, Portfolio choice via quantiles, Mathematical Finance, 21 (2011), 203–231.

    Google Scholar 

  39. P. J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model, Economic Theory, 29 (2006), 591–619.

    Google Scholar 

  40. G. Heutel, Optimal policy instruments for externality-producing durable goods under time inconsistency, preprint.

    Google Scholar 

  41. M. Hinnosaar, Time inconsistency and alcohol sales restrictions, European Economic Review, 87 (2012), 108–131.

    Article  Google Scholar 

  42. S. J. Hoch and G. F. Loewenstein, Time-inconsistent preferences and consumer self-control, J. Consumer Research, 17 (1991), 492–507.

    Article  Google Scholar 

  43. Y. Hu, H. Jin, and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control, SIAM J. Control Optim., 50 (2012), 1548–1572.

    Article  MathSciNet  MATH  Google Scholar 

  44. Y. Hu, H. Jin, and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control: characterization and uniqueness of equilibrium, SIAM J. Control Optim., 55 (2017), 1261–1279.

    Article  MathSciNet  MATH  Google Scholar 

  45. D. Hume, A Treatise of Human Nature, First Edition, New York, Oxford Univ. Press 1978.

    Google Scholar 

  46. P. Ireland, Does the time-inconsistency problem explain the behavior of inflation in the United States? J. Monetary Economics, 44 (1999), 279–291.

    Article  MathSciNet  Google Scholar 

  47. M. O. Jackson and L. Yariv, Collective dynamic choice: The necessity of time inconsistency, Amer. Econ. J. Microeconomics, 7 (2015), 159–178.

    Google Scholar 

  48. H. Jin and X. Y. Zhou, Behavioral portfolio selection in continuous time, Mathematical Finance, (2008), 385–426.

    Article  MathSciNet  MATH  Google Scholar 

  49. D. Jones and A. Mahajan, Time-inconsistency and savings: Experimental evidence from low-income tax filers, Center for Financial Security, University of Wisconsin-Madison, 2011.

    Google Scholar 

  50. D. Kahneman and A. Tversky, Prospective theory: an analysis of decision under risk, Econometrica, 47 (1979), 263–291.

    Article  MathSciNet  MATH  Google Scholar 

  51. C. Karnam, J. Ma, and J. Zhang, Dynamic approaches for some time-inconsistent optimization problems, Ann. Appl. Probab., 27 (2017), 3435–3477.

    Article  MathSciNet  MATH  Google Scholar 

  52. J. Klenberg and S. Oren, Time-inconsistent planning: a computational problem in behavioral economics, In Proc. 15th ACM Conference on Economics and Computation. ACM, 547–564.

    Google Scholar 

  53. L. Karp, Global warming and hyperbolic discounting, J. Public Econ., 8 (2005), 261–282.

    Article  Google Scholar 

  54. L. Karp, Non-constant discounting in continuous time, J. Economic Theory, 132 (2007), 557–568.

    Article  MathSciNet  MATH  Google Scholar 

  55. L. Karp and I. H. Lee, Time-consistent policies, J. Economic Theory, 112 (2003), 353–364.

    Google Scholar 

  56. T. Komatsubara, Illiquid securities and time-inconsistent preferences, preprint.

    Google Scholar 

  57. F. Kydlang and E. Prescott, Rules rather than discretion: the inconsistency of optimal plans, J. Political Economy, 85 (1977), 473–491.

    Google Scholar 

  58. D. Laibson, Golden eggs and hyperbolic discounting, Quartely J. Economics, 112 (1987), 443–477.

    Article  MATH  Google Scholar 

  59. K. Lindensjö, Time-inconsistent stochastic control: solving the extended HJB system is a necessary condition for regular equilibria, Research Report, No.20, Math. Stat., Stockholm Univ.

    Google Scholar 

  60. L. L. Lopes, Between hope and fear: The psychology of risk, Advances in Experimental Social Psychology, 20 (1987), 286–313.

    Google Scholar 

  61. H. Mei and J. Yong, Equilibrium strategies for time-inconsistent stochastic switching systems, COCV, to appear.

    Google Scholar 

  62. E. G. J. Luttmer and T. Mariotti, Efficiency and equilibrium when preferences are time-inconsistent, J. Economic Theory, 132 (2007), 493–506.

    Article  MathSciNet  MATH  Google Scholar 

  63. J. Ma and J. Yong, Forward-Backward Stochastic Differential Equations and Their Applications, Lecture Notes in Math., Vol.1702, Springer-Verlag.

    Google Scholar 

  64. A. Mahajan and A. Tarozzi, Time inconsistency, expectations and technology adoption: The case of insecticide treated nets, preprint.

    Google Scholar 

  65. J. Marin-Solano, Time-consistent equilibria in a differential game model with time inconsistent preferences and partial cooperation, Dynamic games in economics, Dynamic Modeling and Econometrics in Economics and Finance, 16 (2014), 219–238.

    Google Scholar 

  66. J. Marin-Solano and J. Navas, Non-constant discounting in finite horizon: the free terminal time case, J. Economic Dynamics and Control, 33 (2009), 666–675.

    Article  MathSciNet  MATH  Google Scholar 

  67. J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors, European J. Operational Research, 201 (2010), 860–872.

    Article  MathSciNet  MATH  Google Scholar 

  68. J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon, Automatica J. IFAC, 47 (2011), 2626–2638.

    Article  MathSciNet  MATH  Google Scholar 

  69. R. Mehra and E. C. Prescott, The equality premium: A puzzle, J. Monetary Econ., 15 (1985), 145–161.

    Google Scholar 

  70. T. Michielson, Environmental catastrophes under time-inconsistent preferences, preprint.

    Google Scholar 

  71. C. W. Miller, Methods for optimal stochastic control and optimal stopping problems featuring time-inconsistency, PhD dissertation, UC Berkeley.

    Google Scholar 

  72. Y. Narukawa and T. Murofushi, Decision modeling using the Choquet integral, Lecture Notes in Computer Science, 3131 (2004), 183–193.

    Google Scholar 

  73. N. Netzer, Evolution of time preferences and attitudes towards risk, Amer. Econ. Review, 99 (2009), 937–955.

    Article  Google Scholar 

  74. L. Mou and J. Yong, A variational formula for stochastic controls and some applications, Pure & Appl. Math. Quarterly, 3 (2007), 539–567.

    Article  MathSciNet  MATH  Google Scholar 

  75. T. O’Donoghue and M. Rabin, Doing it now or later, Amer. Economic Review, 103–124.

    Article  Google Scholar 

  76. T. O’Donoqhue and M. Rabin. Procrastination on long-term projects, J. Economic Behavior & Organization, 66 (2008), 161–175.

    Google Scholar 

  77. I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume, History of Political Economy, 35 (2003), 241–268.

    Article  MathSciNet  Google Scholar 

  78. E. Pardoux and S. Peng, Adapted solution of a backward stochastic differential equation, Syst. Control Lett., 14 (1990), 55–61.

    Article  MathSciNet  MATH  Google Scholar 

  79. B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing, Review of Economic Studies 40 (1973), 391–401.

    Article  MATH  Google Scholar 

  80. S. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966–979.

    Article  MathSciNet  MATH  Google Scholar 

  81. S. Peng, Backward SDE and related g-expectation. Backward stochastic differential equations (Paris, 1995–1996), 141–159, Pitman Res. Notes Math. Ser., 364, Longman, Harlow, 1997.

    Google Scholar 

  82. S. Peng, Nonlinear expectations, nonlinear evaluations and risk measures, Stochastic methods in finance, 165–253, Lecture Notes in Math., 1856, Springer, Berlin.

    Google Scholar 

  83. S. Peng, Backward stochastic differential equations, nonlinear expectation and their applications, Proc. ICM, Hyderabad, India.

    Google Scholar 

  84. H. Pham, Linear quadratic optimal control of conditional McKean-Vlasov equation with random coefficients and applications, Prob. Uncertainty & Quantitative Risk, 1 (2016): 7

    Google Scholar 

  85. H. Pham and X. Wei, Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics, SIAM J. Control Optim., 55 (2017), 1069–1101.

    Article  MathSciNet  MATH  Google Scholar 

  86. R. A. Pollak, Consistent planning, Review of Economic Studies, 35 (1968), 201–208.

    Google Scholar 

  87. D. Prelec and G. Loewenstein, Beyond time discounting, Marketing letters, 8 (1997), 97–108.

    Article  Google Scholar 

  88. F. P. Ramsey, The foundation of probability, Frank Plumpton Ramsey Papers, 1920–1930, ASP, 1983.01, Archives of Scientific Philosophy, Special Collections Department, Univ. of Pittsburgh.

    Google Scholar 

  89. K. I. Rohde, The hyperbolic factor: A measure of time inconsistency, J. Risk Uncertain, 41 (2010), 125–140.

    Article  MATH  Google Scholar 

  90. L. J. Savage, The Foundations of Statistics, Wiley, New York, 1954.

    Google Scholar 

  91. P. Schreiber and M. Weber, Time inconsistent preferences and the annuitization decision, J. Econ. Behavior & Organization, 129 (2016), 37–55.

    Article  Google Scholar 

  92. D. Schmeidler, Subjective probability and expected utility without additivity, Econometrica, 60 (1989), 1255–1272.

    Google Scholar 

  93. H. Shefrin and M. Statman, Behavioral portfolio theory, J. Financial & Quantitative Anal., 35 (2000), 127–151.

    Google Scholar 

  94. Y. Shi. T. Wang, and J. Yong, Mean-field backward stochastic Volterra integral equations, Discrete and Continuous Dynamic Systems, 18 (2013), 1929–1967.

    Article  MathSciNet  MATH  Google Scholar 

  95. H. Shui, Time inconsistency in the credit card market, preprint.

    Google Scholar 

  96. A. Smith, The Theory of Moral Sentiments, First Edition, Oxford Univ. Press, 1976.

    Google Scholar 

  97. R. H. Strotz, Myopia and inconsistency in dynamic utility maximization, Review of Economic Studies, 23 (1955–1956), 165–180.

    Article  Google Scholar 

  98. A. Tversky and D. Kahneman, The framing of decisions and the psychology of choice, Science, (New Series), 211 (1981), No. 4481, 453–458.

    Google Scholar 

  99. A. Tversky and D. Kahneman, Rational choice and the framing of decisions, J. Business, 59 (1986), S251–S278.

    Google Scholar 

  100. A. Tversky and D. Kahneman, Advances in prospect theory: cumulative representation of uncertainty, J. Risk Uncertainty, 5 (1992), 297–323.

    Article  MATH  Google Scholar 

  101. H. Wang and Z. Wu, Time-inconsistent optimal control problems with random coefficients and stochastic equilibrium HJB equation, Math. Control Rel. Fields, 5 (2015), 651–678.

    Article  MathSciNet  MATH  Google Scholar 

  102. S. Wang, V. R. Young, and H. H. Panjer, Axiomatic characterization of insurance prices, Insurance: Mathematics and Economics, 21 (1997), 173–183.

    MathSciNet  MATH  Google Scholar 

  103. J. Wei, Time-inconsistent optimal control problems with regime switching, Math. Control Rel. Fields, 7 (2017), 585–622.

    Article  MathSciNet  MATH  Google Scholar 

  104. Q. Wei, J. Yong, and Z. Yu, Time-inconsistent recursive stochastic optimal control problems, SIAM J. Control Optim., 55 (2017), 4156–4201.

    Article  MathSciNet  MATH  Google Scholar 

  105. M. E. Yaari, The dual theory of choice under risk, Econometrica, 55 (1987), 95–115.

    Article  MathSciNet  MATH  Google Scholar 

  106. M. Yilmaz, Contracting with a naive time-inconsistent agent: To exploit or not to exploit? Math. Soc. Sci., 77 (2013), 46–71.

    Google Scholar 

  107. J. Yong, Backward stochastic Volterra integral equations and some related problems, Stoch. Proc. Appl., 116 (2006), 779–795.

    Article  MathSciNet  MATH  Google Scholar 

  108. J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations, Prob. Theory Rel. Fields, 142 (2008), 21–77.

    Article  MathSciNet  MATH  Google Scholar 

  109. J. Yong, Optimality variational principle for optimal controls of forward-backward stochastic differential equations, SIAM J. Control & Optim., 48 (2010), 4119–4156.

    Google Scholar 

  110. J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem, Math. Control & Related Fields, 1 (2011), 83–118.

    Article  MathSciNet  MATH  Google Scholar 

  111. J. Yong, Deterministic time-inconsistent optimal control problems — An essentially cooperative approach, Acta Math. Appl. Sinica, 28 (2012), 1–30.

    Google Scholar 

  112. J. Yong, Time-inconsistent optimal control problems and the equilibrium HJB equation, Math. Control & Related Fields, 2 (2012), 271–329.

    Article  MathSciNet  MATH  Google Scholar 

  113. J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations — time-consistent solutions, Trans. AMS, 369 (2017), 5467–5523.

    Article  MathSciNet  MATH  Google Scholar 

  114. J. Yong, A linear-quadratic optimal control problem for mean-field stochastic differential equations, SIAM J. Control & Optim., 51 (2013), 2809–2838.

    Article  MathSciNet  MATH  Google Scholar 

  115. J. Yong, Time-inconsistent optimal control problems, Proc. ICM 2014, Section 16. Control Theory and Optimization, 947–969.

    Google Scholar 

  116. J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, 1999.

    Google Scholar 

  117. P. C. Yu, Optimal retirement policies with time-inconsistent agents, preprint.

    Google Scholar 

  118. Q. Zhao, On time-inconsistent investment and dividend problems, PhD Dissertation, East China Normal University, 2015.

    Google Scholar 

  119. X. Y. Zhou, Mathematicalising behavioural finance, Proc. ICM, Hyderabad, India, 2010.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Central Florida, Orlando, FL, 32816, USA

    Wei Yan & Jiongmin Yong

Authors
  1. Wei Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiongmin Yong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Yan .

Editor information

Editors and Affiliations

  1. Department of Mathematics, Wayne State University, Detroit, MI, USA

    George Yin

  2. Department of Mathematics, University of Georgia, Athens, GA, USA

    Qing Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Yan, W., Yong, J. (2019). Time-Inconsistent Optimal Control Problems and Related Issues. In: Yin, G., Zhang, Q. (eds) Modeling, Stochastic Control, Optimization, and Applications. The IMA Volumes in Mathematics and its Applications, vol 164. Springer, Cham. https://doi.org/10.1007/978-3-030-25498-8_22

Download citation

Publish with us

Policies and ethics

Search

Navigation