Summary
In traditional optimal control, the system is modeled as a stochastic differential equation and an optimal controller is determined such that the expected value of a cost function is minimized. An example is the well-known linear-quadratic-Gaussian problem that has been studied extensively since the 1960s. The mean or the first cumulant is a useful performance metric, however, the mean is only one of the cumulants that describe the distribution of a random variable. It is possible for the operator to optimize the whole distribution of the cost function instead of just the mean. In fact, a denumerable sum of all the cost cumulants has been optimized in risk-sensitive control. The key idea behind statistical control is to optimize other statistical quantities such as the variance, skewness, and kurtosis of the cost function. This leads to the optimal performance shaping concept. Furthermore, we use this statistical concept to generalize \(H_\infty\) and multiple player game theory. In both traditional \(H_\infty\) theory and game theory, the mean of the cost function was the object of optimization, and we can extend this to the optimization of any cumulants if we utilize the statistical control concept. In this chapter, we formulate and provide a literature survey of statistical control. We also review minimal cost variance (second cumulant) control, \(k\)th cost cumulant control, and multiobjective cumulant games. Furthermore, risk-sensitive control is presented as a special case of statistical control. When we view the cost function as a random variable, and optimize any of the cost cumulants, linear-quadratic-Gaussian, minimal cost variance, risk-sensitive, game theoretic, and \(H_\infty\) control all fall under the umbrella of statistical control. Finally, we interpret the cost functions via utility functions.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
K. J. Åstrom, Introduction to Stochastic Control Theory, New York: Academic Press, 1970.
M. Athans, The Role and Use of the Stochastic Linear-Quadratic-Gaussian Problem in Control System Design, IEEE Transactions on Automatic Control, AC-16, Number 6, pp. 529–551, December 1971.
T. Başar and P. Bernhard, \(H_\infty\)–Optimal Control and Related Minimax Design Problems, Boston: Birkhäuser, 1991.
A. Bensoussan and J. H. van Schuppen, Optimal Control of Partially Observable Stochastic Systems with an Exponential-of-Integral Performance Index, SIAM Journal on Control and Optimization, Volume 23, pp. 599–613, 1985.
A. Bensoussan, Stochastic Control of Partially Observable Systems, London: Cambridge University Press, 1992.
D. P. Bertsekas, Dynamic Programming and Stochastic Control, London: Academic Press, 1976.
V. S. Borkar, Optimal Control of Diffusion Processes, England: Longman Scientific & Technical, 1989.
A. E. Bryson, Jr., and Y.-C. Ho, Applied and Optimal Control, Optimization, Estimation, and Control, Revised Printing, New York: Hemisphere Publishing Co., 1975.
M. C. Campi and M. R. James, Risk-Sensitive Control: A Bridge Between \(H_2\) and \(H_\infty\) Control, Proceedings of the 32nd Conference on Decision and Control, San Antonio, TX, December 1993.
L. Cosenza, On the Minimum Variance Control of Discrete-Time Systems, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, January 1969.
M. H. A. Davis, Linear Estimation and Stochastic Control, London: Halsted Press, 1977.
R. W. Diersing, \(H_\infty\), Cumulants and Games, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, 2006.
R. W. Diersing, M. K. Sain, and C.-H. Won, Bi-Cumulant Games: A Generalization of H-infinity and H2/H-infinity Control, IEEE Transactions on Automatic Control, submitted, 2007.
J. L. Doob, Stochastic Processes, New York: John Wiley & Sons, Inc., 1953.
J. Doyle, K. Glover, P. Khargonekar, and B. Francis, State-Space Solutions to Standard \(H_2\) and \(H_\infty\) Control Problems, IEEE Transactions on Automatic Control, Vol. 34, No. 8, pp. 831–847, 1989.
W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, New York: Springer-Verlag, 1975.
W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, New York: Springer-Verlag, 1992.
W. H. Fleming and W. M. McEneaney, “Risk Sensitive Optimal Control and Differential Games,” Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences 184, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 185–197, 1992.
M. I. Freidlin and A. D. Wentzell, Random Perturbations of Dynamical Systems, New York: Springer-Verlag, 1984.
C. W. Gardiner, Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences, 2nd ed., New York: Springer-Verlag, 1985.
I. I. Gihman and A. V. Skorohod, Introduction to the Theory of Random Processes, London: W. B. Saunders, 1969.
I. I. Gihman and A. V. Skorohod, Stochastic Differential Equations, New York: Springer-Verlag, 1972.
I. I. Gihman and A. V. Skorohod, Controlled Stochastic Processes, New York: Springer-Verlag, 1979.
K. Glover and J. C. Doyle, State-Space Formulae for All Stabilizing Controllers That Satisfy an \(H_\infty\)-Norm Bound and Relations to Risk Sensitivity, Systems and Control Letters, Volume 11, pp. 167–172, 1988.
K. Glover, Minimum Entropy and Risk-Sensitive Control: The Continuous Time Case, Proceedings 28th IEEE Conference on Decision and Control, pp. 388–391, December 1989.
W. E. Hopkins, Jr., Exponential Linear Quadratic Optimal Control with Discounting, IEEE Transactions on Automatic Control, AC-39, No. 1, pp. 175–179, 1994.
P. A. Iglesias, Tradeoffs in Linear Time-Varying Systems: An Analogue of Bode’s sensitivity integral, Automatica, Vol. 37, pp. 1541–1550, 2001.
P. A. Iglesias, “An analogue of Bode’s integral for stable non linear systems: Relations to entropy,” Proceedings of the 40th IEEE Conference on Decision and Control, Orlando, FL, pp. 3419–3420, December 2001.
D. H. Jacobson, Optimal Stochastic Linear Systems with Exponential Performance Criteria and Their Relationship to Deterministic Differential Games, IEEE Transactions on Automatic Control, AC-18, pp. 124–131, 1973.
M. R. James, Asymptotic Analysis of Nonlinear Stochastic Risk-Sensitive Control and Differential Games, Mathematics of Control, Signals, and Systems, 5, pp. 401–417, 1992.
M. R. James, J. S. Baras, and R. J. Elliott, Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems, IEEE Transactions on Automatic Control, AC-39, No. 4, pp. 780–792, 1994.
R. E. Kalman, Contributions to the Theory of Optimal Control, Bol. de Soc. Math. Mexicana, p. 102, 1960.
I. Karatzas and S. E. Shereve, Brownian Motion and Stochastic Calculus, New York: Springer-Verlag, 1987.
P. R. Kumar and J. H. van Schuppen, On the Optimal Control of Stochastic Systems with an Exponential-of-Integral Performance Index, Journal of Mathematical Analysis and Applications, Volume 80, pp. 312–332, 1981.
H. Kushner, Introduction to Stochastic Control, New York: Holt, Rinehart and Winston, Inc., 1971.
H. Kwakernaak and R. Sivan, Linear Optimal Control Systems, New York: John Wiley & Sons, Inc., 1972.
S. R. Liberty, “Characteristic Functions of LQG Control,” Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, August 1971.
S. R. Liberty and R. C. Hartwig, On the Essential Quadratic Nature of LQG Control-Performance Measure Cumulants, Information and Control, Volume 32, Number 3, pp. 276–305, 1976.
S. R. Liberty and R. C. Hartwig, Design-Performance-Measure Statistics for Stochastic Linear Control Systems, IEEE Transactions on Automatic Control, AC-23, Number 6, pp. 1085–1090, December 1978.
D. J. N. Limebeer, B. D. O. Anderson, and D. Hendel, A Nash Game Approach to Mixed \(H_2\)/\(H_\infty\) Control, IEEE Transactions on Automatic Control, Vol. 39, Number 1, pp. 69–82, January 1994.
Minimum entropy \(H_\infty\) control, Vol. 146 of Lecture Notes in Control and Information Sciences, Springer-Verlag, Berlin, 1990.
BØ.ksendal, Stochastic Differential Equations, An Introduction with Applications, Second Edition, New York: Springer-Verlag, 1989.
M. A. Peters and P. A. Iglesias, Minimum Entropy Control for Time-Varying Systems, Systems and Control: Foundations & Applications, Birkhäuser, Boston, 1997.
L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkriledze, and E. F. Mischenko, The Mathematical Theory of Optimal Processes, New York: Interscience Publishers, 1962.
K. D. Pham, M. K. Sain, and S. R. Liberty, Cost Cumulant Control: State-Feedback, Finite-Horizon Paradigm with Applications to Seismic Protection, Special Issue of Journal of Optimization Theory and Applications, Edited by A. Miele, Kluwer Academic/Plenum Publishers, New York, Vol. 115, No. 3, pp. 685–710, December 2002.
K. D. Pham, M. K. Sain, and S. R. Liberty, Infinite Horizon Robustly Stable Seismic Protection of Cable-Stayed Bridges Using Cost Cumulants, Proceedings American Control Conference, Boston, MA, USA, pp. 691–696, June 2004.
K. D. Pham, “Statistical Control Paradigm for Structural Vibration Suppression”, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, May 2004.
L. Ray and R. Stengel, A Monte Carlo Approach to the Analysis of Control System Robustness, Automatica, Vol. 29, No. 1, pp. 229–236, 1993.
T. Runolfsson, The Equivalence Between Infinite-Horizon Optimal Control of Stochastic Systems with Exponential-of-Integral Performance Index and Stochastic Differential Games, IEEE Transactions on Automatic Control, Vol. 39, No. 8, pp. 1551–1563, 1994.
A. P. Sage, Optimum Systems Control. Englewood Cliffs, NJ: Prentice-Hall Inc., 1968.
M. K. Sain, On Minimal-Variance Control of Linear Systems with Quadratic Loss, Ph.D Thesis, Department of Electrical Engineering and Coordinated Science Laboratory, University of Illinois, Urbana, IL, January 1965.
M. K. Sain, Control of Linear Systems According to the Minimal Variance Criterion–-A New Approach to the Disturbance Problem, IEEE Transactions on Automatic Control, AC-11, No. 1, pp. 118–122, January 1966.
M. K. Sain, Performance Moment Recursions, with Application to Equalizer Control Laws, Proc. 5th Allerton Conference, pp. 327–336, 1967.
M. K. Sain and C. R. Souza, A Theory for Linear Estimators Minimizing the Variance of the Error Squared, IEEE Transactions on Information Theory, IT-14, Number 5, pp. 768–770, September 1968.
M. K. Sain and S. R. Liberty, Performance Measure Densities for a Class of LQG Control Systems, IEEE Transactions on Automatic Control, AC-16, Number 5, pp. 431–439, October 1971.
M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulant Minimization and Robust Control, Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences 184, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 411–425, 1992.
M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants in Risk-Sensitive Control: The Full-State Feedback Cost Variance Case, Proceedings of the Conference on Decision and Control, New Orleans, LA, pp. 1036–1041, 1995.
M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants and Risk Sensitive Control: A Cost Mean and Variance Theory with Applications to Seismic Protection of Structures, Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games, Vol. 5, J. A. Filor, V. Gaisgory, K Mizukami (Eds), Birkhäuser, Boston, 2000.
P. Sain, M., B. F. Spencer, Jr., M. K. Sain, and J. Suhardjo, Structural Control Design in the Presence of Time Delays, Proceedings of the ASCE Engineering Mechanics Conference, College Station, TX, pp. 812–815, 1992.
B. F. Spencer, M. K. Sain, C.-H. Won, D. C.Kaspari, and P. M. Sain, Reliability-Based Measures of Structural Control Robustness, Structural Safety, 15, pp. 111–129, 1993.
J. L. Speyer, J. Deyst, and D. H. Jacobson, Optimization of Stochastic Linear Systems with Additive Measurement and Process Noise Using Exponential Performance Criteria, IEEE Transactions on Automatic Control, AC-19, No. 4, pp. 358–366, August 1974.
J. L. Speyer, An Adaptive Terminal Guidance Scheme Based on an Exponential Cost Criterion with Application to Homing Missile Guidance, IEEE Transactions on Automatic Control, AC-21, pp. 371–375, 1976.
A. A. Stoorvogel and J. H. Van Schuppen, System Identification with Information Theoretic Criteria, Identification, Adaptation, Learning: The Science of Learning Models from Data (NATO Asi Series. Series F, Computer and Systems Sciences, Vol. 153, Sergio Bittanti (Editor), Giorgio Picci (Editor), Springer, Berlin, 1996, pp. 289–338.
K. Uchida and M. Fujita, On the Central Controller: Characterizations via Differential Games and LEQG Control Problems, Systems & Control Letters, Volume 13, pp9–13, 1989.
P. Whittle, Risk-Sensitive Linear/Quadratic/Gaussian Control, Advances in Applied Probability, Vol. 13, pp. 764–777, 1981.
P. Whittle, Risk Sensitive Optimal Control, New York: John Wiley & Sons, 1990.
P. Whittle, A Risk-Sensitive Maximum Principle: The Case of Imperfect State Observation, IEEE Transactions on Automatic Control, Vol. 36, No. 7, pp. 793–801, July 1991.
P. Whittle and J. Kuhn, A Hamiltonian formulation of risk-sensitive linear/ quadratic/Gaussian control, International Journal of Control, Vol. 43, pp. 1–12, 1986.
C.-H. Won, M. Sain, and B. Spencer, Risk-Sensitive Structural Control Strategies, Proceedings of the Second International Conference on Computational Stochastic Mechanics, Athens, Greece, June 13–15, 1994.
C.-H. Won, Cost Cumulants in Risk-Sensitive and Minimal Cost Variance Control, Ph.D. Dissertation, University of Notre Dame, Notre Dame, IN, 1995.
C.-H. Won, M. K. Sain, and S. Liberty. Infinite Time Minimal Cost Variance Control and Coupled Algebraic Riccati Equations Proceedings American Control Conference, Denver, Co, pp. 5155–5160, June 4–6, 2003.
C.-H. Won, Cost Distribution Shaping: The Relations Between Bode Integral, Entropy, Risk-Sensitivity, and Cost Cumulant Control, Proceedings American Control Conference, Boston, MA, pp. 2160–2165, June 2004.
C.-H. Won, Nonlinear n-th Cost Cumulant Control and Hamilton-Jacobi-Bellman Equations for Markov Diffusion Process, Proceedings of 44th IEEE Conference on Decision and Control, Seville, Spain, pp. 4524–4529, 2005.
W. M. Wonham, Stochastic Problems in Optimal Control, 1963 IEEE Int. Conv. Rec., part 2, pp. 114–124, 1963.
W. M. Wonham, Linear Multivariable Control, A Geometric Approach, Lecture Notes in Economics and Mathematical Systems, 101, Springer-Verlag, 1974.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Birkhäuser Boston, a part of Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Won, CH., Diersing, R.W., Yurkovich, S. (2008). Introduction and Literature Survey of Statistical Control: Going Beyond the Mean. In: Won, CH., Schrader, C., Michel, A. (eds) Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics. Systems & Control: Foundations & Applications. Birkhäuser, Boston, MA. https://doi.org/10.1007/978-0-8176-4795-7_1
Download citation
DOI: https://doi.org/10.1007/978-0-8176-4795-7_1
Published:
Publisher Name: Birkhäuser, Boston, MA
Print ISBN: 978-0-8176-4794-0
Online ISBN: 978-0-8176-4795-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)