Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Optimal stationary linear control of the Wiener process

  • 62 Accesses

  • 7 Citations

Abstract

In the present paper, we consider the following stochastic control problem: to minimize the average expected total cost

$$J(x,u) = \mathop {\lim \inf }\limits_{T \to \infty } (1/T)E_x^u \int_0^T {\left[ {\phi (\xi _t ) + |u_t (\xi )|} \right]} dt,$$

〈subject to

$$d\xi _t = u_1 (\xi )dt + dw_t , \xi _0 = x, |u| \leqslant 1,$$

(w t) a Wiener process, with all measurable functions on the past of the state process {ξ s ;st} and bounded by unity, admissible as controls. It is proved that, under very mild conditions on the running cost function φ(·), the optimal law is of the form

$$\begin{gathered} u_t^* (\xi ) = - sign\xi _t , |\xi _t | > b, \hfill \\ u_t^* (\xi ) = 0, |\xi _t | > b. \hfill \\ \end{gathered} $$

The cutoff pointb and the performance rate of the optimal lawu* are simultaneously determined in terms of the function φ(·) through a simple system of integrotranscendental equations.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Wonham, W. M.,Optimal Stationary Control of a Linear System with State-Dependent Noise, SIAM Journal on Control, Vol. 3, pp. 486–500, 1967.

  2. 2.

    Kushner, H. J.,Optimality Conditions for the Average Cost per Unit Time Problem with a Diffusion Model, SIAM Journal on Control and Optimization, Vol. 16, pp. 330–346, 1978.

  3. 3.

    Girsanov, I. V.,On Transforming a Certain Class of Stochastic Processes by Absolutely Continuous Substitution of Measures, Theory of Probability and Applications, Vol. 5, pp. 285–301, 1960.

  4. 4.

    Liptser, R. S., andShiryayev, A. N.,Statistics of Random Processes, Vol. 1, General Theory, Springer-Verlag, Berlin, Germany, 1977.

  5. 5.

    Zvonkin, A. K.,A Transformation of the Phase Space of a Diffusion Process that Removes the Drift, Mathematics of the USSR (Sbornik), Vol. 22, pp. 129–149, 1974.

  6. 6.

    Gihman, I. I., andSkorohod, A. V.,Stochastic Differential Equations, Springer-Verlag, Berlin, Germany, 1972.

  7. 7.

    Wonham, W. M.,Liapunov Criteria for Weak Stochastic Stability, Journal of Differential Equations, Vol. 2, pp. 195–207, 1966.

  8. 8.

    Khas'minskii, I. Z.,Ergodic Properties of Recurrent Diffusion Processes and Stabilization of the Solution to the Cauchy Problem for Parabolic Equations, Theory of Probability and Applications, Vol. 5, pp. 179–195, 1960.

  9. 9.

    Friedman, A.,Partial Differential Equations of Parabolic Type, Prentice Hall, Englewood Cliffs, New Jersey, 1964.

  10. 10.

    Fleming, W. H., andRishel, R. W.,Deterministic and Stochastic Optimal Control, Springer-Verlag, Berlin, Germany, 1975.

  11. 11.

    Ikeda, N., andWatanabe, S.,A Comparison Theorem for Solutions of Stochastic Differential Equations and Its Applications, Osaka Journal of Mathematics, Vol. 14, pp. 619–633, 1977.

  12. 12.

    Chow, Y. S., andTeicher, H.,Probability Theory: Independence, Interchangeability, Martingales, Springer-Verlag, Berlin, Germany, 1978.

Download references

Author information

Additional information

Research was supported in part by the Air Force Office of Scientific Research under Grant No. AF-AFOSR-77-3063, and in part by the National Science Foundation under Grant No. NSF-MCS-79-05811-A01.

Communicated by R. W. Rishel

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Beneš, V.E., Karatzas, I. Optimal stationary linear control of the Wiener process. J Optim Theory Appl 35, 611–633 (1981). https://doi.org/10.1007/BF00934934

Download citation

Key Words

  • Stationary stochastic control
  • Bellman equation
  • invariant measures
  • nonanticipative laws
  • dead-zone controllers