Abstract
This chapter develops asymptotic optimal controls of a class of hybrid linear quadratic Gaussian (LQG) systems that consist of a collection of diffusions coupled by a finite-state Markov chain. It is well known that LQG systems are most popular in the control systems community, especially owing to their simple structure. In addition, many nonlinear systems can be linearized locally to simplify the analysis. Many LQG systems stem from various applications in speech recognition, pattern recognition, signal processing, telecommunications, and manufacturing. Owing to their importance, there has been a growing interest in studying the control and optimization of such systems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
M. Abbad, J.A. Filar, and T.R. Bielecki, Algorithms for singularly perturbed limiting average Markov control problems, IEEE Trans. Automat. Control AC-37 (1992), 1421–1425.
R. Akella and P.R. Kumar, Optimal control of production rate in a failure-prone manufacturing system, IEEE Trans. Automat. Control AC-31 (1986), 116–126.
W.J. Anderson, Continuous-Time Markov Chains: An Application-Oriented Approach, Springer-Verlag, New York, 1991.
E. Altman, K.E. Avrachenkov, and R. Nunez-Queija, Pertrubation analysis for denumerable Markov chains with applications to queueing models, Adv. in Appl. Probab., 36 (2004), 839–853.
G. Badowski and G. Yin, Stability of hybrid dynamic systems containing singularly perturbed random processes, IEEE Trans. Automat. Control, 47 (2002), 2021–2031.
G. Barone-Adesi and R. Whaley, Efficient analytic approximation of American option values, J. Finance, 42 (1987), 301–320.
A. Bensoussan, J.L. Lion, and G.C. Papanicolaou, Asymptotic Analysis of Periodic Structures, North-Holland, Amsterdam, 1978.
A. Bensoussan, Perturbation Methods in Optimal Control, J. Wiley, Chichester, 1988.
L.D. Berkovitz, Optimal Control Theory, Springer-Verlag, New York, 1974.
A.T. Bharucha-Reid, Elements of the Theory of Markov Processes and Their Applications, McGraw-Hill, New York, 1960.
T.R. Bielecki and J.A. Filar, Singularly perturbed Markov control problem: Limiting average cost, Ann. Oper. Res. 28 (1991), 153–168.
T.R. Bielecki and P.R. Kumar, Optimality of zero-inventory policies for unreliable manufacturing systems, Oper. Res. 36 (1988), 532–541.
P. Billingsley, Convergence of Probability Measures, J. Wiley, New York, 1968.
T. Björk, Finite dimensional optimal filters for a class of Ito processes with jumping parameters, Stochastics, 4 (1980), 167–183.
W.P. Blair and D.D. Sworder, Feedback control of a class of linear discrete systems with jump parameters and quadratic cost criteria, Int. J. Control, 21 (1986), 833–841.
G.B. Blankenship and G.C. Papanicolaou, Stability and control of stochastic systems with wide band noise, SIAM J. Appl. Math. 34 (1978), 437–476.
H.A.P. Blom and Y. Bar-Shalom, The interacting multiple model algorithm for systems with Markovian switching coefficients, IEEE Trans. Automat. Control, AC-33 (1988), 780–783.
N.N. Bogoliubov and Y.A. Mitropolskii, Asymptotic Methods in the Theory of Nonlinear Oscillator, Gordon and Breach, New York, 1961.
E.K. Boukas and A. Haurie, Manufacturing flow control and preventive maintenance: A stochastic control approach, IEEE Trans. Automat. Control AC-35 (1990), 1024–1031.
P. Brémaud, Point Processes and Queues, Springer-Verlag, New York, 1981.
P.E. Caines and H.-F. Chen, Optimal adaptive LQG control for systems with finite state process parameters, IEEE Trans. Automat. Control, AC-30 (1985), 185–189.
S.L. Campbell, Singular perturbation of autonomous linear systems, II, J. Differential Equations 29 (1978), 362–373.
S.L. Campbell and N.J. Rose, Singular perturbation of autonomous linear systems, SIAM J. Math. Anal. 10 (1979), 542–551.
M. Caramanis and G. Liberopoulos, Perturbation analysis for the design of flexible manufacturing system flow controllers, Oper. Res. 40 (1992), 1107–1125.
M.-F. Chen, From Markov Chains to Non-equilibrium Particle Systems, 2nd ed., World Scientific, Singapore, 2004.
S. Chen, X. Li, and X.Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs, SIAM J. Control Optim. 36 (1998), 1685–1702.
C.L. Chiang, An Introduction to Stochastic Processes and Their Applications, Kreiger, Hungtington, 1980.
T.-S. Chiang and Y. Chow, A limit theorem for a class of inhomogeneous Markov processes, Ann. Probab. 17 (1989), 1483–1502.
P.L. Chow, J.L. Menaldi, and M. Robin, Additive control of stochastic linear systems with finite horizon, SIAM J. Control Optim. 23 (1985), 859–899.
Y.S. Chow and H. Teicher, Probability Theory, Springer-Verlag, New York, 1978.
K.L. Chung, Markov Chains with Stationary Transition Probabilities, 2nd Ed., Springer-Verlag, New York, 1967.
F. Clarke, Optimization and Non-smooth Analysis, Wiley Interscience, New York, 1983.
O.L.V. Costa and F. Dufour, Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes, Appl. Math. Optim., 63 (2011), 357–384.
O.L.V. Costa and F. Dufour, Singularly perturbed discounted Markov control processes in a general state space, SIAM J. Control Optim., 50 (2012), 720–747.
P.J. Courtois, Decomposability: Queueing and Computer System Applications, Academic Press, New York, NY, 1977.
D.R. Cox and H.D. Miller, The Theory of Stochastic Processes, J. Wiley, New York, 1965.
M.G. Crandall, C. Evans, and P.L. Lions, Some properties of viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc. 282 (1984), 487–501.
M.G. Crandall, H. Ishii, and P.L. Lions, User’s guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc. 27 (1992), 1–67.
M.G. Crandall and P.L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc. 277 (1983), 1–42.
I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional Conf. Ser. Appl. Math., SIAM, Philadelphia, PA, 1992.
M.H.A. Davis, Markov Models and Optimization, Chapman & Hall, London, 1993.
M.V. Day, Boundary local time and small parameter exit problems with characteristic boundaries, SIAM J. Math. Anal. 20 (1989), 222–248.
F. Delebecque, A reduction process for perturbed Markov chains, SIAM J. Appl. Math., 48 (1983), 325–350.
F. Delebecque and J. Quadrat, Optimal control for Markov chains admitting strong and weak interactions, Automatica 17 (1981), 281–296.
F. Delebecque, J. Quadrat, and P. Kokotovic, A unified view of aggregation and coherency in networks and Markov chains, Internat. J. Control 40 (1984), 939–952.
C. Derman, Finite State Markovian Decision Processes, Academic Press, New York, 1970.
G.B. Di Masi and Yu.M. Kabanov, The strong convergence of two-scale stochastic systems and singular perturbations of filtering equations, J. Math. Systems, Estimation Control 3 (1993), 207–224.
G.B. Di Masi and Yu.M. Kabanov, A first order approximation for the convergence of distributions of the Cox processes with fast Markov switchings, Stochastics Stochastics Rep. 54 (1995), 211–219.
J.L. Doob, Stochastic Processes, Wiley Classic Library Edition, Wiley, New York, 1990.
R.L. Dobrushin, Central limit theorem for nonstationary Markov chains, Theory Probab. Appl. 1 (1956), 65–80, 329–383.
E.B. Dynkin, Markov Processes, Springer-Verlag, Berlin, 1965.
N. Dunford and J.T. Schwartz, Linear Operators, Interscience, New York, 1958.
E.B. Dynkin and A.A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979.
W. Eckhaus, Asymptotic Analysis of Singular Perturbations, North-Holland, Amsterdam, 1979.
R.J. Elliott, Stochastic Calculus and Applications, Springer-Verlag, New York, 1982.
R.J. Elliott, Smoothing for a finite state Markov process, in Lecture Notes in Control and Inform. Sci., 69, 199–206, Springer-Verlag, New York, 1985.
R.J. Elliott, L. Aggoun, and J. Moore, Hidden Markov Models: Estimation and Control, Springer-Verlag, New York, 1995.
A. Erdélyi, Asymptotic Expansions, Dover, New York, 1956.
S.N. Ethier and T.G. Kurtz, Markov Processes: Characterization and Convergence, J. Wiley, New York, 1986.
W. Feller, An Introduction to Probability Theory and Its Applications, J. Wiley, New York, Vol. I, 1957; Vol. II, 1966.
W.H. Fleming, Functions of Several Variables, Addison-Wesley, Reading, 1965.
W.H. Fleming, Generalized solution in optimal stochastic control, in Proc. URI Conf. on Control, 147–165, Kingston, RI, 1982.
W.H. Fleming and R.W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, New York, 1975.
W.H. Fleming and H.M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1992.
W.H. Fleming, S.P. Sethi, and H.M. Soner, An optimal stochastic production planning problem with randomly fluctuating demand, SIAM J. Control Optim. 25 (1987), 1494–1502.
W.H. Fleming and Q. Zhang, Risk-sensitive production planning of a stochastic manufacturing system, SIAM J. Control Optim., 36 (1998), 1147–1170.
M.I. Friedlin and A.D. Wentzel, Random Perturbations of Dynamical Systems, Springer-Verlag, New York, 1984.
C.W. Gardiner, Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences, 2nd Ed., Springer-Verlag, Berlin, 1985.
V.G. Gaitsgori and A.A. Pervozvanskii, Aggregation of states in a Markov chain with weak interactions, Kybernetika 11 (1975), 91–98.
D. Geman and S. Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Machine Intelligence 6 (1984), 721–741.
S.B. Gershwin, Manufacturing Systems Engineering, Prentice-Hall, Englewood Cliffs, 1994.
M.K. Ghosh, A. Arapostathis, and S.I. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1952–1988.
I.I. Gihman and A.V. Skorohod, Introduction to the Theory of Random Processes, W.B. Saunders, Philadelphia, 1969.
P. Glasserman, Gradient Estimation via Perturbation Analysis, Kluwer, Boston, MA, 1991.
R. Goodman, Introduction to Stochastic Models, Benjamin/Cummings, Menlo Park, CA, 1988.
R.J. Griego and R. Hersh, Random evolutions, Markov chains, and systems of partial differential equations, Proc. Nat. Acad. Sci. U.S.A. 62 (1969), 305–308.
R.J. Griego and R. Hersh, Theory of random evolutions with applications to partial differential equations, Trans. Amer. Math. Soc. 156 (1971), 405–418.
X. Guo and O. Hernàndez-Lerma, Continuous-time Markov Decision Processes: Theory and Applications, Springer, Heidelberg, 2001.
J.K. Hale, Ordinary Differential Equations, R.E. Krieger Publishing Co., 2nd Ed., Malabar, 1980.
P. Hänggi, P. Talkner, and M. Borkovec, Reaction-rate theory: Fifty years after Kramers, Rev. Modern Phys. 62 (1990), 251–341.
J.M. Harrison and M.I. Reiman, Reflected Brownian motion on an orthant, Ann. Probab. 9 (1981), 302–308.
U.G. Haussmann and Q. Zhang, Stochastic adaptive control with small observation noise, Stochastics Stochastics Rep. 32 (1990), 109–144.
U.G. Haussmann and Q. Zhang, Discrete time stochastic adaptive control with small observation noise, Appl. Math. Optim. 25 (1992), 303–330.
Q. He, G. Yin, and Q. Zhang, Large Deviations for Two-Time-Scale Systems Driven by Nonhomogeneous Markov Chains and LQ Control Problems, SIAM J. Control Optim., 49, (2011), 1737–1765.
R. Hersh, Random evolutions: A survey of results and problems, Rocky Mountain J. Math. 4 (1974), 443–477.
F.S. Hillier and G.J. Lieberman, Introduction to Operations Research, McGraw-Hill, New York, 1989.
Y.C. Ho and X.R. Cao, Perturbation Analysis of Discrete Event Dynamic Systems, Kluwer, Boston, MA, 1991.
A. Hoyland and M. Rausand, System Reliability Theory: Models and Statistical Methods, J. Wiley, New York, 1994.
Z. Hou and Q. Guo, Homogeneous Denumerable Markov Processes, Springer-Verlag, Berlin, 1980.
V. Hutson and J.S. Pym, Applications of Functional Analysis and Operator Theory, Academic Press, London, 1980.
N. Ikeda and S. Watanabe, Stochastic Differential Equations and Diffusion Processes, North-Holland, Amsterdam, 1981.
A.M. Il’in, Matching of Asymptotic Expansions of Solutions of Boundary Value Problems, Trans. Math. Monographs, Vol. 102, Amer. Math. Soc., Providence, 1992.
A.M. Il’in and R.Z. Khasminskii, Asymptotic behavior of solutions of parabolic equations and ergodic properties of nonhomogeneous diffusion processes, Math. Sbornik. 60 (1963), 366–392.
A.M. Il’in, R.Z. Khasminskii, and G. Yin, Singularly perturbed switching diffusions: Rapid switchings and fast diffusions, J. Optim. Theory Appl. 102 (1999), 555–591.
M. Iosifescu, Finite Markov Processes and Their Applications, Wiley, Chichester, 1980.
H. Ishii, Uniqueness of unbounded viscosity solutions of Hamilton-Jacobi equations, Indiana Univ. Math. J. 33 (1984), 721–748.
Y. Ji and H.J. Chizeck, Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control, IEEE Trans. Automatic Control, 35 (1990), 777–788.
Y. Ji and H.J. Chizeck, Jump linear quadratic Gaussian control in continuous time, IEEE Trans. Automat. Control AC-37 (1992), 1884–1892.
J. Jiang and S.P. Sethi, A state aggregation approach to manufacturing systems having machines states with weak and strong interactions, Oper. Res. 39 (1991), 970–978.
Yu. Kabanov and S. Pergamenshchikov, Two-scale Stochastic Systems: Asymptotic Analysis and Control, Springer, New York, NY, 2003.
I.Ia. Kac and N.N. Krasovskii, On the stability of systems with random parameters, J. Appl. Math. Mech., 24 (1960), 1225–1246.
G. Kallianpur, Stochastic Filtering Theory, Springer-Verlag, New York, 1980.
D. Kannan, An Introduction to Stochastic Processes, North-Holland, New York, 1980.
S. Karlin and J. McGregor, The classification of birth and death processes, Trans. Amer. Math. Soc. 85 (1957), 489–546.
S. Karlin and H.M. Taylor, A First Course in Stochastic Processes, 2nd Ed., Academic Press, New York, 1975.
S. Karlin and H.M. Taylor, A Second Course in Stochastic Processes, Academic Press, New York, 1981.
J. Keilson, Green’s Function Methods in Probability Theory, Griffin, London, 1965.
J. Kevorkian and J.D. Cole, Perturbation Methods in Applied Mathematics, Springer-Verlag, New York, 1981.
J. Kevorkian and J.D. Cole, Multiple Scale and Singular Perturbation Methods, Springer-Verlag, New York, 1996.
H. Kesten and G.C. Papanicolaou, A limit theorem for stochastic acceleration, Comm. Math. Phys. 78 (1980), 19–63.
R.Z. Khasminskii, On diffusion processes with a small parameter, Izv. Akad. Nauk U.S.S.R. Ser. Mat. 27 (1963), 1281–1300.
R.Z. Khasminskii, On stochastic processes defined by differential equations with a small parameter, Theory Probab. Appl. 11 (1966), 211–228.
R.Z. Khasminskii, On an averaging principle for Ito stochastic differential equations, Kybernetika 4 (1968), 260-279.
R.Z. Khasminskii, Stochastic Stability of Differential Equations, 2nd Ed., Springer, New York, 2012.
R.Z. Khasminskii and G. Yin, Asymptotic series for singularly perturbed Kolmogorov-Fokker-Planck equations, SIAM J. Appl. Math. 56 (1996), 1766–1793.
R.Z. Khasminskii and G. Yin, On transition densities of singularly perturbed diffusions with fast and slow components, SIAM J. Appl. Math. 56 (1996), 1794–1819.
R.Z. Khasminskii and G. Yin, On averaging principles: An asymptotic expansion approach, SIAM J. Math. Anal., 35 (2004), 1534–1560.
R.Z. Khasminskii and G. Yin, Limit behavior of two-time-scale diffusions revisited, J. Differential Eqs., 212 (2005) 85–113.
R.Z. Khasminskii, G. Yin, and Q. Zhang, Asymptotic expansions of singularly perturbed systems involving rapidly fluctuating Markov chains, SIAM J. Appl. Math. 56 (1996), 277–293.
R.Z. Khasminskii, G. Yin, and Q. Zhang, Constructing asymptotic series for probability distribution of Markov chains with weak and strong interactions, Quart. Appl. Math. LV (1997), 177–200.
J.G. Kimemia and S.B. Gershwin, An algorithm for the computer control production in flexible manufacturing systems, IIE Trans. 15 (1983), 353–362.
J.F.C. Kingman, Poisson Processes, Oxford Univ. Press, Oxford, 1993.
S. Kirkpatrick, C. Gebatt, and M. Vecchi, Optimization by simulated annealing, Science 220 (1983), 671–680.
C. Knessel, On finite capacity processor-shared queues, SIAM J. Appl. Math. 50 (1990), 264–287.
C. Knessel and J.A. Morrison, Heavy traffic analysis of a data handling system with multiple sources, SIAM J. Appl. Math. 51 (1991), 187–213.
P.V. Kokotovic, Application of singular perturbation techniques to control problems, SIAM Rev. 26 (1984), 501–550.
P.V. Kokotovic, A. Bensoussan, and G. Blankenship (Eds.), Singular Perturbations and Asymptotic Analysis in Control Systems, Lecture Notes in Control and Inform. Sci. 90, Springer-Verlag, Berlin, 1987.
P.V. Kokotovic and H.K. Khalil (Eds.), Singular Perturbations in Systems and Control, IEEE Press, New York, 1986.
P.V. Kokotovic, H.K. Khalil, and J. O’Reilly, Singular Perturbation Methods in Control, Academic Press, London, 1986.
V. Korolykuk and A. Swishchuk, Evolution of Systems in Random Media, CRC Press, Boca Raton, 1995.
V.S. Korolyuk and N. Limnios, Diffusion approximation with equilibrium of evolutionary systems switched by semi-Markov processes, translation in Ukrainian Math. J. 57 (2005), 1466–1476.
V.S. Korolyuk and N. Limnios, Stochastic systems in merging phase space, World Sci., Hackensack, NJ, 2005.
N.M. Krylov and N.N. Bogoliubov, Introduction to Nonlinear Mechanics, Princeton Univ. Press, Princeton, 1947.
H. Kunita and S. Watanabe, On square integrable martingales, Nagoya Math. J. 30 (1967), 209–245.
T.G. Kurtz, A limit theorem for perturbed operator semigroups with applications to random evolutions, J. Functional Anal. 12 (1973), 55–67.
T.G. Kurtz, Approximation of Population Processes, SIAM, Philadelphia, PA, 1981.
T.G. Kurtz, Averaging for martingale problems and stochastic approximation, in Proc. US-French Workshop on Appl. Stochastic Anal., Lecture Notes in Control and Inform. Sci., 177, I. Karatzas and D. Ocone (Eds.), 186–209, Springer-Verlag, New York, 1991.
H.J. Kushner, Probability Methods for Approximation in Stochastic Control and for Elliptic Equations, Academic Press, New York, 1977.
H.J. Kushner, Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory, MIT Press, Cambridge, MA, 1984.
H.J. Kushner, Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems, Birkhäuser, Boston, 1990.
H.J. Kushner and P.G. Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, Springer-Verlag, New York, 1992.
H.J. Kushner and W. Runggaldier, Nearly optimal state feedback controls for stochastic systems with wideband noise disturbances, SIAM J Control Optim. 25 (1987), 289–315.
H.J. Kushner and F.J. Vázquez-Abad, Stochastic approximation algorithms for systems over an infinite horizon, SIAM J. Control Optim. 34 (1996), 712–756.
H.J. Kushner and G. Yin, Asymptotic properties of distributed and communicating stochastic approximation algorithms, SIAM J. Control Optim. 25 (1987), 1266–1290.
H.J. Kushner and G. Yin, Stochastic Approximation and Recursive Algorithms and Applications, 2nd Edition, Springer-Verlag, New York, 2003.
X.R. Li, Hybrid estimation techniques, in Control and Dynamic Systems, Vol. 76, C.T. Leondes (Ed.), Academic Press, New York, 1996.
Yu. V. Linnik, On the theory of nonhomogeneous Markov chains, Izv. Akad. Nauk. USSR Ser. Mat. 13 (1949), 65–94.
Y.J. Liu, G. Yin, and X.Y. Zhou, Near-optimal controls of random-switching LQ problems with indefinite control weight costs, Automatica, 41 (2005) 1063–1070.
P. Lochak and C. Meunier, Multiphase Averaging for Classical Systems, Springer-Verlag, New York, 1988.
J. Lehoczky, S.P. Sethi, H.M. Soner, and M. Taksar, An asymptotic analysis of hierarchical control of manufacturing systems under uncertainty, Math. Oper. Res. 16 (1992), 596–608.
G. Lerman and Z. Schuss, Asymptotic theory of large deviations for Markov chains, SIAM J. Appl. Math., 58 (1998), 1862–1877.
D. Ludwig, Persistence of dynamical systems under random perturbations, SIAM Rev. 17 (1975), 605–640.
X. Mao and C. Yuan, Stochastic Differential Equations with Markovian Switching, Imperial College Press, London, UK, 2006.
M. Mariton, Robust jump linear quadratic control: A mode stabilizing solution, IEEE Trans. Automat. Control, AC-30 (1985), 1145–1147.
M. Mariton, Jump Linear Systems in Automatic Control, Marcel Dekker, Inc., New York, 1990.
L.F. Martins and H.J. Kushner, Routing and singular control for queueing networks in heavy traffic, SIAM J. Control Optim. 28 (1990), 1209–1233.
W.A. Massey and W. Whitt, Uniform acceleration expansions for Markov chains with time-varying rates, Ann. Appl. Probab., 8 (1998), 1130–1155.
B.J. Matkowsky and Z. Schuss, The exit problem for randomly perturbed dynamical systems, SIAM J. Appl. Math. 33 (1977), 365–382.
S.P. Meyn and R.L. Tweedie, Markov Chains and Stochastic Stability, Springer-Verlag, London, 1993.
T. Naeh, M.M. Klosek, B.J. Matkowski, and Z. Schuss, A direct approach to the exit problem, SIAM J. Appl. Math. 50 (1990), 595–627.
A.H. Nayfeh, Introduction to Perturbation Techniques, J. Wiley, New York, 1981.
M.F. Neuts, Matrix-Geometric Solutions in Stochastic Models, Johns Hopkins Univ. Press, Baltimore, 1981.
R.E. O’Malley, Jr., Singular Perturbation Methods for Ordinary Differential Equations, Springer-Verlag, New York, 1991.
Z.G. Pan and T. Başar, \({H}^{\infty }\)-control of Markovian jump linear systems and solutions to associated piecewise-deterministic differential games, in New Trends in Dynamic Games and Applications, G.J. Olsder Ed., 61–94, Birkhäuser, Boston, MA, 1995.
Z.G. Pan and T. Başar, \({H}^{\infty }\) control of large scale jump linear systems via averaging and aggregation, in Proc. 34th IEEE Conf. Decision Control, 2574-2579, New Orleans, LA, 1995.
Z.G. Pan and T. Başar, Random evolutionary time-scale decomposition in robust control of jump linear systems, in Proc. 35th IEEE Conf. Decision Control, Kobe, Japan, 1996.
Z.G. Pan and T. Basar, \({H}^{\infty }\) control of large-scale jump linear systems via averaging and aggregation, Internat. J. Control, 72 (1999), 866–881.
G.C. Papanicolaou, D. Stroock, and S.R.S. Varadhan, Martingale approach to some limit theorems, in Proc. 1976 Duke Univ. Conf. on Turbulence, Durham, NC, 1976.
G.C. Papanicolaou, Introduction to the asymptotic analysis of stochastic equations, in Lectures in Applied Mathematics, Amer. Math. Soc., Vol. 16, 1977, 109-147.
G.C. Papanicolaou, Asymptotic analysis of stochastic equations, Studies in Probability Theory, M. Rosenblatt (Ed.), Vol. 18, MAA, 1978, 111–179.
E. Pardoux and S. Peng, Adapted solution of backward stochastic equation, Syst. Control Lett., 14 (1990), 55–61.
A. Pazy, Semigroups of Linear Operators and Applications to Partial Differential Equations, Springer-Verlag, New York, 1983.
L. Perko, Differential Equations and Dynamical Systems, Springer, 3rd Ed., New York, 2001.
A.A. Pervozvanskii and V.G. Gaitsgori, Theory of Suboptimal Decisions: Decomposition and Aggregation, Kluwer, Dordrecht, 1988.
R.G. Phillips and P.V. Kokotovic, A singular perturbation approach to modelling and control of Markov chains, IEEE Trans. Automat. Control 26 (1981), 1087–1094.
M.A. Pinsky, Differential equations with a small parameter and the central limit theorem for functions defined on a Markov chain, Z. Wahrsch. verw. Gebiete 9 (1968), 101–111.
M.A. Pinsky, Multiplicative operator functionals and their asymptotic properties, in Advances in Probability Vol. 3, P. Ney and S. Port (Eds.), Marcel Dekker, New York, 1974.
L. Prandtl, Über Flüssigkeits – bewegung bei kleiner Reibung, Verhandlungen, in III. Internat. Math. Kongresses, (1905), 484–491.
M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, J. Wiley, New York, 1994.
D. Revuz, Markov Chains, Revised Ed., North-Holland, Amsterdam, 1975.
R. Rishel, Controlled wear process: Modeling optimal control, IEEE Trans. Automat. Control 36 (1991), 1100–1102.
H. Risken, The Fokker-Planck Equation: Methods of Solution and Applications, 2nd Ed., Springer-Verlag, London, 1989.
M. Rosenblatt, Markov Processes: Structure and Asymptotic Behavior, Springer-Verlag, Berlin, 1971.
S. Ross, Introduction to Stochastic Dynamic Programming, Academic Press, New York, 1983.
E. Roxin, The existence of optimal controls, Mich. Math. J. 9 (1962), 109–119.
V.R. Saksena, J. O’Reilly, and P.V. Kokotovic, Singular perturbations and time-scale methods in control theory: Survey 1976-1983, Automatica 20 (1984), 273–293.
Z. Schuss, Singularly perturbation methods in stochastic differential equations of mathematical physics, SIAM Rev. 22 (1980), 119–155.
Z. Schuss, Theory and Applications of Stochastic Differential Equations, J. Wiley, New York, 1980.
E. Seneta, Non-negative Matrices and Markov Chains, Springer-Verlag, New York, 1981.
R. Serfozo, Introduction to Stochastic Networks, Springer, New York, 1999.
S.P. Sethi and G.L. Thompson, Applied Optimal Control: Applications to Management Science, Martinus Nijhoff, Boston, MA, 1981.
S.P. Sethi and Q. Zhang, Hierarchical Decision Making in Stochastic Manufacturing Systems, Birkhäuser, Boston, 1994.
S.P. Sethi and Q. Zhang, Multilevel hierarchical decision making in stochastic marketing-production systems, SIAM J. Control Optim. 33 (1995), 528–553.
O.P. Sharma, Markov Queues, Ellis Horwood, New York, 1990.
H.A. Simon, Models of Discovery and Other Topics in the Methods of Science, D. Reidel Publ. Co., Boston, MA, 1977.
H.A. Simon and A. Ando, Aggregation of variables in dynamic systems, Econometrica 29 (1961), 111–138.
A.V. Skorohod, Studies in the Theory of Random Processes, Dover, New York, 1982.
A.V. Skorohod, Asymptotic Methods of the Theory of Stochastic Differential Equations, Trans. Math. Monographs, Vol. 78, Amer. Math. Soc., Providence, 1989.
D.R. Smith, Singular Perturbation Theory, Cambridge Univ. Press, New York, 1985.
D. Snyder, Random Point Processes, Wiley, New York, 1975.
H.M. Soner, Optimal control with state space constraints II, SIAM J. Control Optim. 24 (1986), 1110–1122.
H.M. Soner, Singular perturbations in manufacturing systems, SIAM J. Control Optim. 31 (1993), 132–146.
D.W. Stroock and S.R.S. Varadhan, Multidimensional Diffusion Processes, Springer-Verlag, Berlin, 1979.
H.M. Taylor and S. Karlin, An Introduction to Stochastic Modeling, Academic Press, Boston, 1994.
W.A. Thompson, Jr., Point Process Models with Applications to Safety and Reliability, Chapman and Hall, New York, 1988.
D.N.C. Tse, R.G. Gallager, and J.N. Tsitsiklis, Statistical multiplexing of multiple time-scale Markov streams, IEEE J. Selected Areas Comm. 13 (1995), 1028–1038.
S.R.S. Varadhan, Large Deviations and Applications, SIAM, Philadelphia, 1984.
N.G. van Kampen, Stochastic Processes in Physics and Chemistry, North-Holland, Amsterdam, 1992.
A.B. Vasil’eava and V.F. Butuzov, Asymptotic Expansions of the Solutions of Singularly Perturbed Equations, Nauka, Moscow, 1973.
A.B. Vasil’eava and V.F. Butuzov, Asymptotic Methods in Singular Perturbations Theory (in Russian), Vysshaya Shkola, Moscow, 1990.
D. Vermes, Optimal control of piecewise deterministic Markov processes, Stochastics, 14 (1985), 165–207.
L.Y. Wang, P.P. Khargonekar, and A. Beydoun, Robust control of hybrid systems: Performance guided strategies, in Hybrid Systems V, P. Antsaklis, W. Kohn, M. Lemmon, A. Nerode, amd S. Sastry Eds., Lecuture Notes in Computer Sci., 1567, 356–389, Berlin, 1999.
Z. Wang and X. Yang, Birth and Death Processes and Markov Chains, Springer-Verlag, Science Press, Beijing, 1992.
J. Warga, Relaxed variational problems, J Math. Anal. Appl. 4 (1962), 111–128.
W. Wasow, Asymptotic Expansions for Ordinary Differential Equations, Interscience, New York, 1965.
W. Wasow, Linear Turning Point Theory, Springer-Verlag, New York, 1985.
A.D. Wentzel, On the asymptotics of eigenvalues of matrices with elements of order \(\exp (-{V }_{ij}/2{\varepsilon }^{2})\), Dokl. Akad. Nauk SSSR 222 (1972), 263–265.
D.J. White, Markov Decision Processes, Wiley, New York, 1992.
J.H. Wilkinson, The Algebraic Eigenvalue Problem, Oxford University Press, New York, 1988.
H. Yan, G. Yin, and S. X. C. Lou, Using stochastic optimization to determine threshold values for control of unreliable manufacturing systems, J. Optim. Theory Appl. 83 (1994), 511–539.
H. Yan and Q. Zhang, A numerical method in optimal production and setup scheduling in stochastic manufacturing systems, IEEE Trans. Automat. Control, 42 (1997), 1452–1455.
G. Yin, Asymptotic properties of an adaptive beam former algorithm, IEEE Trans. Information Theory IT-35 (1989), 859-867.
G. Yin, Asymptotic expansions of option price under regime-switching diffusions with a fast-varying switching process, Asymptotic Anal., 65 (2009), 203–222.
G. Yin and I. Gupta, On a continuous time stochastic approximation problem, Acta Appl. Math. 33 (1993), 3–20.
G. Yin, V. Krishnamurthy, and C. Ion, Regime switching stochastic approximation algorithms with application to adaptive discrete stochastic optimization, SIAM J. Optim., 14 (2004), 1187–1215.
G. Yin and D.T. Nguyen, Asymptotic expansions of backward equations for two-time-scale Markov chains in continuous time, Acta Math. Appl. Sinica, 25 (2009), 457–476.
G. Yin and K.M. Ramachandran, A differential delay equation with wideband noise perturbation, Stochastic Process Appl. 35 (1990), 231–249.
G. Yin, H. Yan, and S.X.C. Lou, On a class of stochastic optimization algorithms with applications to manufacturing models, in Model-Oriented Data Analysis, W.G. Müller, H.P. Wynn and A.A. Zhigljavsky (Eds.), 213–226, Physica-Verlag, Heidelberg, 1993.
G. Yin and H.L. Yang, Two-time-scale jump-diffusion models with Markovian switching regimes, Stochastics Stochastics Rep., 76 (2004), 77–99.
G. Yin and H. Zhang, Two-time-scale markov chains and applications to quasi-birth-death queues, SIAM J. Appl. Math., 65 (2005), 567–586.
G. Yin and H. Zhang, Singularly perturbed markov chains: Limit results and applications, Ann. Appl. Probab., 17 (2007), 207–229.
G. Yin, H. Zhang, and Q. Zhang, Applications of Two-time-scale Markovian Systems, Preprint, 2012.
G. Yin and Q. Zhang, Near optimality of stochastic control in systems with unknown parameter processes, Appl. Math. Optim. 29 (1994), 263–284.
G. Yin and Q. Zhang, Control of dynamic systems under the influence of singularly perturbed Markov chains, J. Math. Anal. Appl., 216 (1997), 343–367.
G. Yin and Q. Zhang (Eds.), Recent Advances in Control and Optimization of Manufacturing Systems, Lecture Notes in Control and Information Sciences (LNCIS) series, Vol. 214, Springer-Verlag, New York, 1996.
G. Yin and Q. Zhang (Eds.), Mathematics of Stochastic Manufacturing Systems, Proc. 1996 AMS-SIAM Summer Seminar in Applied Mathematics, Lectures in Applied Mathematics, Amer. Math. Soc., Providence, RI, 1997.
G. Yin and Q. Zhang, Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, 1st Ed., Springer-Verlag, New York, 1998.
G. Yin and Q. Zhang, Discrete-time Markov Chains: Two-time-scale Methods and Applications, Springer, New York. 2005.
G. Yin, Q. Zhang, and G. Badowski, Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states, Ann. Appl. Probab., 10 (2000), 549–572.
G. Yin, Q. Zhang, and G. Badowski, Singularly perturbed Markov chains: Convergence and aggregation, J. Multivariate Anal., 72 (2000), 208–229.
G. Yin, Q. Zhang, and G. Badowski, Occupation measures of singularly perturbed Markov chains with absorbing states, Acta Math. Sinica, 16 (2000), 161–180.
G. Yin, Q. Zhang, and G. Badowski, Discrete-time singularly perturbed Markov chains: Aggregation, occupation measures, and switching diffusion limit, Adv. in Appl. Probab., 35 (2003), 449–476.
G. Yin and X.Y. Zhou, Markowitz’s mean-variance portfolio selection with regime switching: From discrete-time models to their continuous-time limits, IEEE Trans. Automat. Control, 49 (2004), 349–360.
G. Yin and C. Zhu, Hybrid Switching Diffusions: Properties and Applications, Springer, New York, 2010.
K. Yosida, Functional Analysis, 6th Ed., Springer-Verlag, New York, NY, 1980.
J. Yong and X.Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer, New York, 1999.
Q. Zhang, An asymptotic analysis of controlled diffusions with rapidly oscillating parameters, Stochastics Stochastics Rep. 42 (1993), 67–92.
Q. Zhang, Risk sensitive production planning of stochastic manufacturing systems: A singular perturbation approach, SIAM J. Control Optim. 33 (1995), 498–527.
Q. Zhang, Finite state Markovian decision processes with weak and strong interactions, Stochastics Stochastics Rep. 59 (1996), 283–304.
Q. Zhang, Nonlinear filtering and control of a switching diffusion with small observation noise, SIAM J. Control Optim., 36 (1998), 1738–1768.
Q. Zhang and G. Yin, Turnpike sets in stochastic manufacturing systems with finite time horizon, Stochastics Stochastics Rep. 51 (1994), 11–40.
Q. Zhang and G. Yin, Central limit theorems for singular perturbations of nonstationary finite state Markov chains, Ann. Appl. Probab. 6 (1996), 650–670.
Q. Zhang and G. Yin, Structural properties of Markov chains with weak and strong interactions, Stochastic Process Appl., 70 (1997), 181–197.
Q. Zhang and G. Yin, On nearly optimal controls of hybrid LQG problems, IEEE Trans. Automat. Control, 44 (1999), 2271–2282.
Q. Zhang and G. Yin, Nearly optimal asset allocation in hybrid stock-investment models, J. Optim. Theory Appl., 121 (2004), 197–222.
Q. Zhang, G. Yin, and E.K. Boukas, Controlled Markov chains with weak and strong interactions: Asymptotic optimality and application in manufacturing, J. Optim. Theory Appl., 94 (1997), 169–194.
Q. Zhang, G. Yin, and R.H. Liu, A near-optimal selling rule for a two-time-scale market model, SIAM J. Multiscale Modeling Simulation, 4 (2005), 172–193.
X.Y. Zhou, Verification theorem within the framework of viscosity solutions, J. Math. Anal. Appl. 177 (1993), 208–225.
X.Y. Zhou and G. Yin, Markowitz mean-variance portfolio selection with regime switching: A continuous-time model, SIAM J. Control Optim., 42 (2003), 1466–1482.
C. Zhu, G. Yin, and Q.S. Song, Stability of random-switching systems of differential equations, Quart. Appl. Math., 67 (2009), 201–220.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Yin, G.G., Zhang, Q. (2013). Hybrid LQG Problems. In: Continuous-Time Markov Chains and Applications. Stochastic Modelling and Applied Probability, vol 37. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4346-9_10
Download citation
DOI: https://doi.org/10.1007/978-1-4614-4346-9_10
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-4345-2
Online ISBN: 978-1-4614-4346-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)