Skip to main content
SpringerLink
Log in

Stochastic Realization for Stochastic Control with Partial Observations

  • Chapter
Modeling, Estimation and Control

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 364))

  • 701 Accesses

Abstract

The purpose of this paper is to present a novel way to formulate control problems with partial observations of stochastic systems. Themethod is based on stochastic realization theory.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A. Arapostathis, V.S. Borkar, E. Fernández-Gaucherand, M.K. Ghosh, and S.I. Marcus. Discrete-time controlled Markov processes with average cost criterion: A survey. SIAM J. Control & Opt., 31:282–344, 1993.

    Article  MATH  Google Scholar 

  2. V.E. Benes and I. Karatzas. Filtering of diffusions controlled through their conditional measures. Stochastics, 13:1–23, 1984.

    MATH  MathSciNet  Google Scholar 

  3. A. Bensoussan and J.H. van Schuppen. Optimal control of partially observable stochastic systems with an exponential-of-integral performance index. SIAM J. Control Optim., 23:599–613, 1985.

    Article  MATH  MathSciNet  Google Scholar 

  4. D.P. Bertsekas. Dynamic programming and stochastic control. Academic Press, New York, 1976.

    MATH  Google Scholar 

  5. D.P. Bertsekas. Dynamic programming and optimal control, Volume I. Athena Scientific, Belmont, MA, 1995.

    Google Scholar 

  6. D.P. Bertsekas. Dynamic programming and optimal control, Volume II. Athena Scientific, Belmont, MA, 1995.

    MATH  Google Scholar 

  7. R.K. Boel and P. Varaiya. Optimal control of jump processes. SIAM J. Control Optim., 15:92–119, 1977.

    Article  MATH  MathSciNet  Google Scholar 

  8. M.H.A. Davis and P. Varaiya. Dynamic programming conditions for partially observable stochastic systems. SIAM J. Control, 11:226–261, 1973.

    Article  MATH  MathSciNet  Google Scholar 

  9. E. Fernandez-Gaucherand and A. Arapostathis. On partially observable Markov decision processes with an average cost criterion. In Proceedings of the 28th IEEE Conference on Decision and Control (CDC.1989), pages 1267–1272, New York, 1989. IEEE, IEEE Press.

    Google Scholar 

  10. E. Fernández-Gaucherand, A. Arapostathis, and S.I. Marcus. On the average cost optimality equation and the structure of optimal policies for partially observed Markov decision processes. Ann. Oper. Res., 29:439–470, 1991.

    Article  MATH  MathSciNet  Google Scholar 

  11. E. Fernandez-Gaucherand and S.I. Marcus. Risk-sensitive optimal control of hidden markov models: Structural results. IEEE Trans. Automatic Control, 42:1418–1422, 1997.

    Article  MATH  MathSciNet  Google Scholar 

  12. W.H. Fleming and R.W. Rishel. Deterministic and stochastic optimal control. Springer-Verlag, Berlin, 1975.

    MATH  Google Scholar 

  13. T.L. Gunckel. Optimum design of sampled-data systems with random parameters. Technical Report SEL TR 2102-2, Stanford Electron. Lab., Stanford, 1961.

    Google Scholar 

  14. D.H. Jacobson. Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games. IEEE Trans. Automatic Control, 18:124–131, 1973.

    Article  MATH  MathSciNet  Google Scholar 

  15. P.D. Joseph and J.T. Tou. On linear control theory. AIEE Trans. (Appl. Ind.), 80:193–196, 1961 ( Sep.).

    Google Scholar 

  16. P.R. Kumar and J.H. van Schuppen. On the optimal control of stochastic systems with an exponential-of-integral performance index. J. Math. Anal. Appl., 80:312–332, 1981.

    Article  MATH  MathSciNet  Google Scholar 

  17. P.R. Kumar and P. Varaiya. Stochastic systems: Estimation, identification, and adaptive control. Prentice Hall Inc., Englewood Cliffs, NJ, 1986.

    MATH  Google Scholar 

  18. A. Lindquist and G. Picci. A geometric approach to modelling and estimation of linear stochastic systems. J. Math. Systems, Estimation, and Control, 1:241–333, 1991.

    MathSciNet  Google Scholar 

  19. R. Rishel. Necessary and sufficient dynamic programming conditions for continuous time stochastic optimal control. SIAM J. Control & Opt., 8:559–571, 1970.

    Article  MATH  MathSciNet  Google Scholar 

  20. C. Striebel. Sufficient statistics in the optimum control of stochastic systems. J. Math. Anal. Appl., 12:576–592, 1965.

    Article  MATH  MathSciNet  Google Scholar 

  21. C. Striebel. Optimal control of discrete time stochastic systems, volume 110 of Lecture Notes in Economic and Mathematical Systems. Springer-Verlag, Berlin, 1975.

    MATH  Google Scholar 

  22. C. Striebel. Martingale conditions for the optimal control of continuous time stochastic systems. Stoc. Proc. Appl., 18:329–347, 1984.

    Article  MATH  MathSciNet  Google Scholar 

  23. J.H. van Schuppen. Stochastic realization problems. In J.M. Schumacher H. Nijmeijer, editor, Three decades of mathematical system theory, volume 135 of Lecture Notes in Control and Information Sciences, pages 480–523. Springer-Verlag, Berlin, 1989.

    Chapter  Google Scholar 

  24. P. Whittle. Risk-sensitive Linear/Quadratic/Gaussian control. Adv. Appl. Prob., 13:764–777, 1981.

    Article  MATH  MathSciNet  Google Scholar 

  25. H. Witsenhausen. Separation of estimation and control for discrete time systems. Proc. IEEE, 59:1557–1566, 1971.

    Article  MathSciNet  Google Scholar 

  26. H.S. Witsenhausen. On information structures, feedback and causality. SIAM J. Control, 9:149–160, 1971.

    Article  MATH  MathSciNet  Google Scholar 

  27. H.S. Witsenhausen. A standard form for sequential stochastic control. Math. Systems Theory, 7:5–11, 1973.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CWI, P.O.Box 94079, 1090 GB, Amsterdam, The Netherlands

    Jan H. van Schuppen

Authors
  1. Jan H. van Schuppen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartimento di Tecnica e Gestione dei Sistemi Industriali, Università di Padova sede di Vicenza, stradella San Nicola, 3, I-36100, Vicenza, Italy

    Alessandro Chiuso

  2. Dipartimento di Ingegneria dell’Informazione, Università di Padova, via Gradenigo, 6/B, I-35131, Padova, Italy

    Stefano Pinzoni  & Augusto Ferrante  & 

Additional information

The paper is dedicated to Giorgio Picci on the occasion of his 65th birthday for his inspiring contributions to stochastic realization and to system identification.

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

van Schuppen, J.H. (2007). Stochastic Realization for Stochastic Control with Partial Observations. In: Chiuso, A., Pinzoni, S., Ferrante, A. (eds) Modeling, Estimation and Control. Lecture Notes in Control and Information Sciences, vol 364. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73570-0_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-73570-0_23

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-73569-4

  • Online ISBN: 978-3-540-73570-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation