Abstract
Sufficient conditions are established under which the law of large numbers and related ergodic theorems hold for nonlinear stochastic systems operating under feedback. It is shown that these conditions hold whenever a moment condition is satisfied, which may be interpreted as a generalization of the martingale property.
If in addition a stochastic controllability condition holds, then it is shown that the underlying distributions governing the system converge to an invariant probability at a geometric rate.
These results are illustrated with general examples from linear, nonlinear, and adaptive control theory.
The key assumption used is that a Markov chain with stationary transition probabilities exists which serves as a state process for the closed loop system.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
Athreya, K.B. and P. Ney (1980) “Some aspects of ergodic theory and laws of large numbers for Harris recurrent Markov chains” Colloquia Mathematica Societatis János Bolyai 32 Nonparametric statistical inference, Budapest, Hungary, pp. 41–56.
Beneš, V.E. (1968) “Finite Regular Invariant Measures for Feller Processes”, J. Appl. Prob., 5, p.203.
Bhatia, N.P. and G.P. Szegö (1970) Stability theory of dynamical systems, Springer-Verlag Berlin, Heidelberg, New York.
Billingsley, P. (1968). Convergence of Probability Measures. John Wiley & Sons, N.Y.
Brown, J.B. (1976) Ergodic theory and topological dynamics, Academic Press, N.Y.
Caines, P.E. (1988) Linear stochastic systems, John Wiley & Sons, N.Y.
Chan, K.S. (1986) Topics in Nonlinear Time Series Analysis Thesis, Dept. of Mathematics, Princeton University.
Cogburn, R. (1975) “A uniform theory for sums of Markov chain transition probabilities”, Ann. Probab. 3 pp. 191–214.
Doob, J.L. (1953). Stochastic Processes. John Wiley & Sons, N.Y.
Foguel, S.R. (1969) “Positive operators on C(X)” Proceedings AMS 22 pp. 295–297.
Foster, F.G. (1953) “On the stochastic matrices associated with certain queueing processess” Annals of Mathematical Statistics, 24 pp. 355–360.
Goodwin, G.C., P.J. Ramadge and P.E. Caines, (1981). Discrete Time Stochastic Adaptive Control, SIAM J. Contr. Optimiz., 19, 829. Corrigendum, 20 No.6, 1982, p.893.
Guo, L. and S.P. Meyn (1988) “Short memory adaptive control” to appear in the International J. of Adaptive Control and Signal Processing.
Has'minskii, R.Z. (1980) Stochastic stability of differential equations sijthoff & Noordhoff Alphen an den Rijn, The Netherlands, Rockville, Maryland USA.
Jakubczyk, B. and E.D. Sontag (1988) “Controllability of nonlinear discrete time systems: A liealgebraic approach” to appear Siam J. Contr. Optimiz.
Kalman, R.E. and J.E. Bertram (1960) “Control system analysis and design by the second method of Lyapunov” Trans. ASME Ser. D: J. Basic Eng. 82 No. 2 pp. 371–400.
Kushner, H.J. (1967). Stochastic stability and control, Academic Press, N.Y.
Meyn S.P. (1988a) “Ergodic theorems for discrete time stochastic systems using a generalized stochastic Lyapunov function” submitted to Siam J. Control and Optimization.
Meyn S.P. (1988b) “Ergodic theory and topological dynamics for discrete time controlled systems” in preperation.
Meyn, S.P. and P.E. Caines, (1987) “A new approach to stochastic adaptive control”, IEEE Transactions on Automatic Control, AC-32, Number 3.
Meyn, S.P. and P.E. Caines (1988). Asymptotic Behavior of Stochastic Systems Possessing Markovian Realizations. submitted to Siam J. Control and Optimization.
Nummelin, E. (1984) General Irreducible Markov Chains and Non-Negative Operators Cambridge University Press, Cambridge.
Nummelin, E. and P. Tuominen. (1982) “Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory”. Stochastic Processes and their Applications, 12 187–202.
Orey, S. (1971) Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold Mathematical studies 34, Van Nostrand, London.
Popov, V.M. (1973) Hyperstability of Control Systems, Springer-Verlag, Berlin, and Editura Academici Bucuresti.
Revuz, D. (1975) Markov Chains, North Holland Publishing Co., Amsterdam, 1975.
Rosenblatt, M. (1971) Markov Processes: Structure and Asymptotic Behaviour. Springer-Verlag, Berlin, Heidelberg, New York.
Safonov, M.G. (1980) Stability and robustness of multivariable feedback systems MIT Press, Cambridge, Mass.
Solo, V. (1978) Time series recursions and stochastic approximation, Ph.D. dissertation, The Australian National University, Sept. 1978.
Sontag, E.D. (1983) “A Lyapunov-like characterization of asymptotic controllability” Siam J. Contr. Optimiz. 21 No. 3 pp. 462–471.
Tuominen, P. and R.L. Tweedie (1979) “Markov chains with continuous components” Proc. London Math. Society, Series 3, Vol. 38.
Tweedie, R.L. (1983) “Criteria for rates of convergence of Markov chains, with application to queueing and storage theory”, in Probability, Statistics and Analysis, London Math. Society Lecture Note Series 79 Ed. J.F.C. Kingman and G.E.H. Reuter, Cambridge University Press, Cambridge.
Zames, G. (1966) “On the Input-Output Stability of Time Varying Non-Linear Feedback Systems”, Part I: IEEE Transactions on Automatic Control, AC-11, Number 2, pp. 228–238, April 1966. Part II: IEEE Transactions on Automatic Control, AC-11, Number 2, pp. 465–476, April 1966.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1989 Springer-Verlag
About this paper
Cite this paper
Meyn, S.P., Caines, P.E. (1989). Stochastic controllability and stochastic Lyapunov functions with applications to adaptive and nonlinear systems. In: Christopeit, N., Helmes, K., Kohlmann, M. (eds) Stochastic Differential Systems. Lecture Notes in Control and Information Sciences, vol 126. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0043789
Download citation
DOI: https://doi.org/10.1007/BFb0043789
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-51299-8
Online ISBN: 978-3-540-46188-3
eBook Packages: Springer Book Archive