Abstract
In this chapter, we show that the degenerate points may separate the state space into different regions for multiple classes, and we discuss the optimization of multi-class degenerate diffusion processes. We also show that under some conditions, the performance function of finite-horizon optimization problems, or the potential function of the long-run average optimization problems, is semi-smooth at degenerate points and smooth at non-degenerate points. Thus, degenerate points coincide with semi-smooth points. Furthermore, there are some special features at the degenerate points: the local time at these points are zero, and the process can only move toward one direction. Therefore, the effect of semi-smoothness of a function can be ignored at these degenerate points in the Ito-Tanaka formula. With these special features in consideration, various optimization problems such as long-run average, finite-horizon, optimal stopping, and singular control, become simpler.
The formulation of a problem is often more essential than its solution, which may be merely a matter of mathematical or experimental skills [1].
Albert Einstein
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
These follow naturally from the Lipschitz condition.
- 2.
The lemma may follow directly from Theorem 2.2 by the confluencity of the process in \(( - \infty , 0]\).
- 3.
More w-ergodic classes may be reached for multidimensional processes.
- 4.
Since our derivation holds if [0, 1] is changed to [0, c] for any \(c>0\), this condition is equivalent to f(x) bounded in a neighborhood of 0, [0, c], for a \(c>0\).
- 5.
Under these conditions, the Lipschitz condition (3.9) holds in [a, b].
- 6.
Also called an antiderivative, \(h(x) := \int f(x)dx \) is defined as any function with \( \dot{h}(x) = f(x)\).
- 7.
Note the difference between \(\dot{\varPsi } (0_- )\) and \(\dot{\varPsi }_- (0)\).
- 8.
One needs to verify that both the denominator and numerator in H(x) at \(x=0\) are zero or infinite. This is true depending on \(\mu (0_\pm ) >0\) or \(<0\), as shown in the term \(\frac{2 \mu (x)}{\sigma ^2(x)}\) in the exponential. We will not go into the details.
- 9.
This proof applies to both cases with \(f(x) \not \equiv 0\) and \(f(x)\equiv 0\).
- 10.
These conditions are weaker than those in Lemma 4.5.
- 11.
In finite-horizon problems with (4.13), the potential function is the same as the performance measure \(\eta (x)\); however, in the infinite-horizon long-run average problem, they are different, i.e., \(\eta (x) \ne g(x)\). A potential function is the solution to a Poisson equation.
- 12.
Note for long-run average, the potential function g(x) in Theorem 4.3 is different from the performance function \(\eta (x)\).
References
Einstein A, Infeld L (1938) The evolution of physics. Cambridge University Press, Cambridge
Cao XR (2016) State classification of time nonhomogeneous Markov chains and average reward optimization of multi-chains. IEEE Trans Autom Control 61:3001–3015
Cao XR (2017) Stochastic feedback control with one-dimensional degenerate diffusions and non-smooth value functions. IEEE Trans Autom Control 62:6136–6151
Cao XR (2019) State classification and multi-class optimization of continuous-time and continuous-state Markov processes. IEEE Trans Autom Control 64:3632–3646
Cao XR (2018) Semi-smooth potentials of stochastic systems with degenerate diffusions. IEEE Trans Autom Control 63:3566–3572
Cao XR (2017) Relative time and stochastic control with non-smooth features. IEEE Trans Autom Control 62:837–852
Cao XR (2007) Stochastic learning and optimization - a sensitivity-based approach. Springer, Berlin
Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York
Bian B, Miao S, Zheng H (2011) Smooth value functions for a class of nonsmooth utility maximization problems. SIAM J Financ Math 2:727–747
Strulovicia B, Szydlowskib M (2015) On the smoothness of value functions and the existence of optimal strategies in diffusion models. J Econ Theory 159:1016–1055
Yong J, Zhou XY (1999) Stochastic controls - Hamilton systems and HJB equations. Springer, Berlin
Fleming WH, Soner HM (2006) Controlled Markov processes and viscosity solutions, 2nd edn. Springer, Berlin
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Cao, XR. (2020). Degenerate Diffusion Processes. In: Relative Optimization of Continuous-Time and Continuous-State Stochastic Systems. Communications and Control Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-41846-5_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-41846-5_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41845-8
Online ISBN: 978-3-030-41846-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)