Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics pp 3-27 | Cite as

# Introduction and Literature Survey of Statistical Control: Going Beyond the Mean

## Summary

In traditional optimal control, the system is modeled as a stochastic differential equation and an optimal controller is determined such that the expected value of a cost function is minimized. An example is the well-known linear-quadratic-Gaussian problem that has been studied extensively since the 1960s. The mean or the first cumulant is a useful performance metric, however, the mean is only one of the cumulants that describe the distribution of a random variable. It is possible for the operator to optimize the whole distribution of the cost function instead of just the mean. In fact, a denumerable sum of all the cost cumulants has been optimized in risk-sensitive control. The key idea behind statistical control is to optimize other statistical quantities such as the variance, skewness, and kurtosis of the cost function. This leads to the optimal performance shaping concept. Furthermore, we use this statistical concept to generalize \(H_\infty\) and multiple player game theory. In both traditional \(H_\infty\) theory and game theory, the mean of the cost function was the object of optimization, and we can extend this to the optimization of any cumulants if we utilize the statistical control concept. In this chapter, we formulate and provide a literature survey of statistical control. We also review minimal cost variance (second cumulant) control, \(k\)th cost cumulant control, and multiobjective cumulant games. Furthermore, risk-sensitive control is presented as a special case of statistical control. When we view the cost function as a random variable, and optimize any of the cost cumulants, linear-quadratic-Gaussian, minimal cost variance, risk-sensitive, game theoretic, and \(H_\infty\) control all fall under the umbrella of statistical control. Finally, we interpret the cost functions via utility functions.

## Keywords

Cost Function Utility Function Infinite Horizon Cost Cumulants Quadratic Cost Function## References

- [Ast70]K. J. Åstrom,
*Introduction to Stochastic Control Theory*, New York: Academic Press, 1970.Google Scholar - [Ath71]M. Athans, The Role and Use of the Stochastic Linear-Quadratic-Gaussian Problem in Control System Design,
*IEEE Transactions on Automatic Control*, AC-16, Number 6, pp. 529–551, December 1971.Google Scholar - [BB91]T. Başar and P. Bernhard, \(H_\infty\)–
*Optimal Control and Related Minimax Design Problems*, Boston: Birkhäuser, 1991.MATHGoogle Scholar - [BV85]A. Bensoussan and J. H. van Schuppen, Optimal Control of Partially Observable Stochastic Systems with an Exponential-of-Integral Performance Index,
*SIAM Journal on Control and Optimization*, Volume 23, pp. 599–613, 1985.MathSciNetMATHCrossRefGoogle Scholar - [Ben92]A. Bensoussan,
*Stochastic Control of Partially Observable Systems*, London: Cambridge University Press, 1992.MATHCrossRefGoogle Scholar - [Ber76]D. P. Bertsekas,
*Dynamic Programming and Stochastic Control*, London: Academic Press, 1976.MATHGoogle Scholar - [Bor89]V. S. Borkar,
*Optimal Control of Diffusion Processes*, England: Longman Scientific & Technical, 1989.MATHGoogle Scholar - [BH75]A. E. Bryson, Jr., and Y.-C. Ho,
*Applied and Optimal Control, Optimization, Estimation, and Control*, Revised Printing, New York: Hemisphere Publishing Co., 1975.Google Scholar - [CJ93]M. C. Campi and M. R. James, Risk-Sensitive Control: A Bridge Between \(H_2\) and \(H_\infty\) Control,
*Proceedings of the 32nd Conference on Decision and Control*, San Antonio, TX, December 1993.Google Scholar - [Cos69]L. Cosenza, On the Minimum Variance Control of Discrete-Time Systems, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, January 1969.Google Scholar
- [Dav77]M. H. A. Davis,
*Linear Estimation and Stochastic Control,*London: Halsted Press, 1977.MATHGoogle Scholar - [Die06]R. W. Diersing, \(H_\infty\), Cumulants and Games, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, 2006.Google Scholar
- [Die07]R. W. Diersing, M. K. Sain, and C.-H. Won, Bi-Cumulant Games: A Generalization of H-infinity and H2/H-infinity Control,
*IEEE Transactions on Automatic Control*, submitted, 2007. Google Scholar - [Doo53]J. L. Doob,
*Stochastic Processes,*New York: John Wiley & Sons, Inc., 1953.MATHGoogle Scholar - [Doy89]J. Doyle, K. Glover, P. Khargonekar, and B. Francis, State-Space Solutions to Standard \(H_2\) and \(H_\infty\) Control Problems,
*IEEE Transactions on Automatic Control*, Vol. 34, No. 8, pp. 831–847, 1989.MathSciNetMATHCrossRefGoogle Scholar - [FR75]W. H. Fleming and R. W. Rishel,
*Deterministic and Stochastic Optimal Control*, New York: Springer-Verlag, 1975.MATHCrossRefGoogle Scholar - [FS92]W. H. Fleming and H. M. Soner,
*Controlled Markov Processes and Viscosity Solutions*, New York: Springer-Verlag, 1992.Google Scholar - [FM92]W. H. Fleming and W. M. McEneaney, “Risk Sensitive Optimal Control and Differential Games,”
*Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences 184*, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 185–197, 1992.Google Scholar - [FW84]M. I. Freidlin and A. D. Wentzell,
*Random Perturbations of Dynamical Systems,*New York: Springer-Verlag, 1984.MATHCrossRefGoogle Scholar - [Gar85]C. W. Gardiner,
*Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences,*2nd ed., New York: Springer-Verlag, 1985.Google Scholar - [GS69]I. I. Gihman and A. V. Skorohod,
*Introduction to the Theory of Random Processes,*London: W. B. Saunders, 1969.Google Scholar - [GS72]I. I. Gihman and A. V. Skorohod,
*Stochastic Differential Equations,*New York: Springer-Verlag, 1972.MATHCrossRefGoogle Scholar - [GS79]I. I. Gihman and A. V. Skorohod,
*Controlled Stochastic Processes,*New York: Springer-Verlag, 1979.CrossRefGoogle Scholar - [GD88]K. Glover and J. C. Doyle, State-Space Formulae for All Stabilizing Controllers That Satisfy an \(H_\infty\)-Norm Bound and Relations to Risk Sensitivity,
*Systems and Control Letters*, Volume 11, pp. 167–172, 1988.MathSciNetMATHCrossRefGoogle Scholar - [Glo89]K. Glover, Minimum Entropy and Risk-Sensitive Control: The Continuous Time Case,
*Proceedings 28th IEEE Conference on Decision and Control*, pp. 388–391, December 1989.CrossRefGoogle Scholar - [Hop94]W. E. Hopkins, Jr., Exponential Linear Quadratic Optimal Control with Discounting,
*IEEE Transactions on Automatic Control*, AC-39, No. 1, pp. 175–179, 1994.Google Scholar - [Ig101a]P. A. Iglesias, Tradeoffs in Linear Time-Varying Systems: An Analogue of Bode’s sensitivity integral,
*Automatica,*Vol. 37, pp. 1541–1550, 2001.MathSciNetMATHCrossRefGoogle Scholar - [Ig101b]P. A. Iglesias, “An analogue of Bode’s integral for stable non linear systems: Relations to entropy,”
*Proceedings of the 40th IEEE Conference on Decision and Control,*Orlando, FL, pp. 3419–3420, December 2001.Google Scholar - [Jac73]D. H. Jacobson, Optimal Stochastic Linear Systems with Exponential Performance Criteria and Their Relationship to Deterministic Differential Games,
*IEEE Transactions on Automatic Control*, AC-18, pp. 124–131, 1973.Google Scholar - [Jam92]M. R. James, Asymptotic Analysis of Nonlinear Stochastic Risk-Sensitive Control and Differential Games,
*Mathematics of Control, Signals, and Systems*, 5, pp. 401–417, 1992.MathSciNetMATHCrossRefGoogle Scholar - [Jam94]M. R. James, J. S. Baras, and R. J. Elliott, Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems,
*IEEE Transactions on Automatic Control*, AC-39, No. 4, pp. 780–792, 1994.Google Scholar - [Kal60]R. E. Kalman, Contributions to the Theory of Optimal Control,
*Bol. de Soc. Math. Mexicana,*p. 102, 1960.Google Scholar - [Kar87]I. Karatzas and S. E. Shereve,
*Brownian Motion and Stochastic Calculus,*New York: Springer-Verlag, 1987.Google Scholar - [Kv81]P. R. Kumar and J. H. van Schuppen, On the Optimal Control of Stochastic Systems with an Exponential-of-Integral Performance Index,
*Journal of Mathematical Analysis and Applications*, Volume 80, pp. 312–332, 1981.MathSciNetMATHCrossRefGoogle Scholar - [Kus71]H. Kushner,
*Introduction to Stochastic Control*, New York: Holt, Rinehart and Winston, Inc., 1971.MATHGoogle Scholar - [KS72]H. Kwakernaak and R. Sivan,
*Linear Optimal Control Systems*, New York: John Wiley & Sons, Inc., 1972.MATHGoogle Scholar - [Lib71]S. R. Liberty, “Characteristic Functions of LQG Control,” Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, August 1971.Google Scholar
- [LH76]S. R. Liberty and R. C. Hartwig, On the Essential Quadratic Nature of LQG Control-Performance Measure Cumulants,
*Information and Control*, Volume 32, Number 3, pp. 276–305, 1976.MathSciNetMATHCrossRefGoogle Scholar - [LH78]S. R. Liberty and R. C. Hartwig, Design-Performance-Measure Statistics for Stochastic Linear Control Systems,
*IEEE Transactions on Automatic Control*, AC-23, Number 6, pp. 1085–1090, December 1978.Google Scholar - [LAH94]D. J. N. Limebeer, B. D. O. Anderson, and D. Hendel, A Nash Game Approach to Mixed \(H_2\)/\(H_\infty\) Control,
*IEEE Transactions on Automatic Control*, Vol. 39, Number 1, pp. 69–82, January 1994.MathSciNetMATHCrossRefGoogle Scholar - [MG90]
*Minimum entropy*\(H_\infty\)*control,*Vol. 146 of Lecture Notes in Control and Information Sciences, Springer-Verlag, Berlin, 1990.Google Scholar - [Oks89]BØ.ksendal,
*Stochastic Differential Equations, An Introduction with Applications*, Second Edition, New York: Springer-Verlag, 1989.Google Scholar - [PI97]M. A. Peters and P. A. Iglesias,
*Minimum Entropy Control for Time-Varying Systems*, Systems and Control: Foundations & Applications, Birkhäuser, Boston, 1997.CrossRefGoogle Scholar - [PBGM62]L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkriledze, and E. F. Mischenko,
*The Mathematical Theory of Optimal Processes*, New York: Interscience Publishers, 1962.MATHGoogle Scholar - [PSL02]K. D. Pham, M. K. Sain, and S. R. Liberty, Cost Cumulant Control: State-Feedback, Finite-Horizon Paradigm with Applications to Seismic Protection,
*Special Issue of Journal of Optimization Theory and Applications*, Edited by A. Miele, Kluwer Academic/Plenum Publishers, New York, Vol. 115, No. 3, pp. 685–710, December 2002.Google Scholar - [PSL04]K. D. Pham, M. K. Sain, and S. R. Liberty, Infinite Horizon Robustly Stable Seismic Protection of Cable-Stayed Bridges Using Cost Cumulants,
*Proceedings American Control Conference*, Boston, MA, USA, pp. 691–696, June 2004.Google Scholar - [Pha04]K. D. Pham, “Statistical Control Paradigm for Structural Vibration Suppression”, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, May 2004.Google Scholar
- [RS93]L. Ray and R. Stengel, A Monte Carlo Approach to the Analysis of Control System Robustness,
*Automatica*, Vol. 29, No. 1, pp. 229–236, 1993.MathSciNetMATHCrossRefGoogle Scholar - [Run94]T. Runolfsson, The Equivalence Between Infinite-Horizon Optimal Control of Stochastic Systems with Exponential-of-Integral Performance Index and Stochastic Differential Games,
*IEEE Transactions on Automatic Control*, Vol. 39, No. 8, pp. 1551–1563, 1994.MathSciNetMATHCrossRefGoogle Scholar - [Sag68]A. P. Sage,
*Optimum Systems Control*. Englewood Cliffs, NJ: Prentice-Hall Inc., 1968.Google Scholar - [Sai65]M. K. Sain, On Minimal-Variance Control of Linear Systems with Quadratic Loss, Ph.D Thesis, Department of Electrical Engineering and Coordinated Science Laboratory, University of Illinois, Urbana, IL, January 1965.Google Scholar
- [Sai66]M. K. Sain, Control of Linear Systems According to the Minimal Variance Criterion–-A New Approach to the Disturbance Problem,
*IEEE Transactions on Automatic Control*, AC-11, No. 1, pp. 118–122, January 1966.Google Scholar - [Sai67]M. K. Sain, Performance Moment Recursions, with Application to Equalizer Control Laws,
*Proc. 5th Allerton Conference*, pp. 327–336, 1967.Google Scholar - [SS68]M. K. Sain and C. R. Souza, A Theory for Linear Estimators Minimizing the Variance of the Error Squared,
*IEEE Transactions on Information Theory*, IT-14, Number 5, pp. 768–770, September 1968.Google Scholar - [SL71]M. K. Sain and S. R. Liberty, Performance Measure Densities for a Class of LQG Control Systems,
*IEEE Transactions on Automatic Control*, AC-16, Number 5, pp. 431–439, October 1971.Google Scholar - [SWS92]M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulant Minimization and Robust Control,
*Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences 184*, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 411–425, 1992.Google Scholar - [SWS95]M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants in Risk-Sensitive Control: The Full-State Feedback Cost Variance Case,
*Proceedings of the Conference on Decision and Control*, New Orleans, LA, pp. 1036–1041, 1995.Google Scholar - [SWS00]M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants and Risk Sensitive Control: A Cost Mean and Variance Theory with Applications to Seismic Protection of Structures,
*Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games*, Vol. 5, J. A. Filor, V. Gaisgory, K Mizukami (Eds), Birkhäuser, Boston, 2000.Google Scholar - [SSSS92]P. Sain, M., B. F. Spencer, Jr., M. K. Sain, and J. Suhardjo, Structural Control Design in the Presence of Time Delays,
*Proceedings of the ASCE Engineering Mechanics Conference*, College Station, TX, pp. 812–815, 1992.Google Scholar - [SSWKS93]B. F. Spencer, M. K. Sain, C.-H. Won, D. C.Kaspari, and P. M. Sain, Reliability-Based Measures of Structural Control Robustness,
*Structural Safety*, 15, pp. 111–129, 1993.CrossRefGoogle Scholar - [SDJ74]J. L. Speyer, J. Deyst, and D. H. Jacobson, Optimization of Stochastic Linear Systems with Additive Measurement and Process Noise Using Exponential Performance Criteria,
*IEEE Transactions on Automatic Control*, AC-19, No. 4, pp. 358–366, August 1974.Google Scholar - [Spe76]J. L. Speyer, An Adaptive Terminal Guidance Scheme Based on an Exponential Cost Criterion with Application to Homing Missile Guidance,
*IEEE Transactions on Automatic Control*, AC-21, pp. 371–375, 1976.Google Scholar - [SS96]A. A. Stoorvogel and J. H. Van Schuppen, System Identification with Information Theoretic Criteria,
*Identification, Adaptation, Learning: The Science of Learning Models from Data (NATO Asi Series. Series F, Computer and Systems Sciences, Vol. 153*, Sergio Bittanti (Editor), Giorgio Picci (Editor), Springer, Berlin, 1996, pp. 289–338.Google Scholar - [Uch89]K. Uchida and M. Fujita, On the Central Controller: Characterizations via Differential Games and LEQG Control Problems,
*Systems & Control Letters*, Volume 13, pp9–13, 1989.MathSciNetMATHCrossRefGoogle Scholar - [Whi81]P. Whittle, Risk-Sensitive Linear/Quadratic/Gaussian Control,
*Advances in Applied Probability*, Vol. 13, pp. 764–777, 1981.MathSciNetMATHCrossRefGoogle Scholar - [Whi90]P. Whittle,
*Risk Sensitive Optimal Control*, New York: John Wiley & Sons, 1990.Google Scholar - [Whi91]P. Whittle, A Risk-Sensitive Maximum Principle: The Case of Imperfect State Observation,
*IEEE Transactions on Automatic Control*, Vol. 36, No. 7, pp. 793–801, July 1991.MathSciNetMATHCrossRefGoogle Scholar - [Wk86]P. Whittle and J. Kuhn, A Hamiltonian formulation of risk-sensitive linear/ quadratic/Gaussian control,
*International Journal of Control*, Vol. 43, pp. 1–12, 1986.MathSciNetMATHCrossRefGoogle Scholar - [WSS94]C.-H. Won, M. Sain, and B. Spencer, Risk-Sensitive Structural Control Strategies,
*Proceedings of the Second International Conference on Computational Stochastic Mechanics*, Athens, Greece, June 13–15, 1994.Google Scholar - [Won95]C.-H. Won, Cost Cumulants in Risk-Sensitive and Minimal Cost Variance Control, Ph.D. Dissertation, University of Notre Dame, Notre Dame, IN, 1995.Google Scholar
- [WSL03]C.-H. Won, M. K. Sain, and S. Liberty. Infinite Time Minimal Cost Variance Control and Coupled Algebraic Riccati Equations
*Proceedings American Control Conference*, Denver, Co, pp. 5155–5160, June 4–6, 2003.Google Scholar - [Won04]C.-H. Won, Cost Distribution Shaping: The Relations Between Bode Integral, Entropy, Risk-Sensitivity, and Cost Cumulant Control,
*Proceedings American Control Conference*, Boston, MA, pp. 2160–2165, June 2004.Google Scholar - [Won05]C.-H. Won, Nonlinear
*n*-th Cost Cumulant Control and Hamilton-Jacobi-Bellman Equations for Markov Diffusion Process,*Proceedings of 44th IEEE Conference on Decision and Control,*Seville, Spain, pp. 4524–4529, 2005.Google Scholar - [Won63]W. M. Wonham, Stochastic Problems in Optimal Control,
*1963 IEEE Int. Conv. Rec.*, part 2, pp. 114–124, 1963.Google Scholar - [Won74]W. M. Wonham,
*Linear Multivariable Control, A Geometric Approach, Lecture Notes in Economics and Mathematical Systems, 101*, Springer-Verlag, 1974.Google Scholar