# Introduction and Literature Survey of Statistical Control: Going Beyond the Mean

• Chang-Hee Won
• Ronald W. Diersing
• Stephen Yurkovich
Chapter
Part of the Systems & Control: Foundations & Applications book series (SCFA)

## Summary

In traditional optimal control, the system is modeled as a stochastic differential equation and an optimal controller is determined such that the expected value of a cost function is minimized. An example is the well-known linear-quadratic-Gaussian problem that has been studied extensively since the 1960s. The mean or the first cumulant is a useful performance metric, however, the mean is only one of the cumulants that describe the distribution of a random variable. It is possible for the operator to optimize the whole distribution of the cost function instead of just the mean. In fact, a denumerable sum of all the cost cumulants has been optimized in risk-sensitive control. The key idea behind statistical control is to optimize other statistical quantities such as the variance, skewness, and kurtosis of the cost function. This leads to the optimal performance shaping concept. Furthermore, we use this statistical concept to generalize $$H_\infty$$ and multiple player game theory. In both traditional $$H_\infty$$ theory and game theory, the mean of the cost function was the object of optimization, and we can extend this to the optimization of any cumulants if we utilize the statistical control concept. In this chapter, we formulate and provide a literature survey of statistical control. We also review minimal cost variance (second cumulant) control, $$k$$th cost cumulant control, and multiobjective cumulant games. Furthermore, risk-sensitive control is presented as a special case of statistical control. When we view the cost function as a random variable, and optimize any of the cost cumulants, linear-quadratic-Gaussian, minimal cost variance, risk-sensitive, game theoretic, and $$H_\infty$$ control all fall under the umbrella of statistical control. Finally, we interpret the cost functions via utility functions.

## Keywords

Cost Function Utility Function Infinite Horizon Cost Cumulants Quadratic Cost Function

## References

1. [Ast70]
K. J. Åstrom, Introduction to Stochastic Control Theory, New York: Academic Press, 1970.Google Scholar
2. [Ath71]
M. Athans, The Role and Use of the Stochastic Linear-Quadratic-Gaussian Problem in Control System Design, IEEE Transactions on Automatic Control, AC-16, Number 6, pp. 529–551, December 1971.Google Scholar
3. [BB91]
T. Başar and P. Bernhard, $$H_\infty$$Optimal Control and Related Minimax Design Problems, Boston: Birkhäuser, 1991.
4. [BV85]
A. Bensoussan and J. H. van Schuppen, Optimal Control of Partially Observable Stochastic Systems with an Exponential-of-Integral Performance Index, SIAM Journal on Control and Optimization, Volume 23, pp. 599–613, 1985.
5. [Ben92]
A. Bensoussan, Stochastic Control of Partially Observable Systems, London: Cambridge University Press, 1992.
6. [Ber76]
D. P. Bertsekas, Dynamic Programming and Stochastic Control, London: Academic Press, 1976.
7. [Bor89]
V. S. Borkar, Optimal Control of Diffusion Processes, England: Longman Scientific & Technical, 1989.
8. [BH75]
A. E. Bryson, Jr., and Y.-C. Ho, Applied and Optimal Control, Optimization, Estimation, and Control, Revised Printing, New York: Hemisphere Publishing Co., 1975.Google Scholar
9. [CJ93]
M. C. Campi and M. R. James, Risk-Sensitive Control: A Bridge Between $$H_2$$ and $$H_\infty$$ Control, Proceedings of the 32nd Conference on Decision and Control, San Antonio, TX, December 1993.Google Scholar
10. [Cos69]
L. Cosenza, On the Minimum Variance Control of Discrete-Time Systems, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, January 1969.Google Scholar
11. [Dav77]
M. H. A. Davis, Linear Estimation and Stochastic Control, London: Halsted Press, 1977.
12. [Die06]
R. W. Diersing, $$H_\infty$$, Cumulants and Games, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, 2006.Google Scholar
13. [Die07]
R. W. Diersing, M. K. Sain, and C.-H. Won, Bi-Cumulant Games: A Generalization of H-infinity and H2/H-infinity Control, IEEE Transactions on Automatic Control, submitted, 2007. Google Scholar
14. [Doo53]
J. L. Doob, Stochastic Processes, New York: John Wiley & Sons, Inc., 1953.
15. [Doy89]
J. Doyle, K. Glover, P. Khargonekar, and B. Francis, State-Space Solutions to Standard $$H_2$$ and $$H_\infty$$ Control Problems, IEEE Transactions on Automatic Control, Vol. 34, No. 8, pp. 831–847, 1989.
16. [FR75]
W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, New York: Springer-Verlag, 1975.
17. [FS92]
W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, New York: Springer-Verlag, 1992.Google Scholar
18. [FM92]
W. H. Fleming and W. M. McEneaney, “Risk Sensitive Optimal Control and Differential Games,” Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences 184, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 185–197, 1992.Google Scholar
19. [FW84]
M. I. Freidlin and A. D. Wentzell, Random Perturbations of Dynamical Systems, New York: Springer-Verlag, 1984.
20. [Gar85]
C. W. Gardiner, Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences, 2nd ed., New York: Springer-Verlag, 1985.Google Scholar
21. [GS69]
I. I. Gihman and A. V. Skorohod, Introduction to the Theory of Random Processes, London: W. B. Saunders, 1969.Google Scholar
22. [GS72]
I. I. Gihman and A. V. Skorohod, Stochastic Differential Equations, New York: Springer-Verlag, 1972.
23. [GS79]
I. I. Gihman and A. V. Skorohod, Controlled Stochastic Processes, New York: Springer-Verlag, 1979.
24. [GD88]
K. Glover and J. C. Doyle, State-Space Formulae for All Stabilizing Controllers That Satisfy an $$H_\infty$$-Norm Bound and Relations to Risk Sensitivity, Systems and Control Letters, Volume 11, pp. 167–172, 1988.
25. [Glo89]
K. Glover, Minimum Entropy and Risk-Sensitive Control: The Continuous Time Case, Proceedings 28th IEEE Conference on Decision and Control, pp. 388–391, December 1989.
26. [Hop94]
W. E. Hopkins, Jr., Exponential Linear Quadratic Optimal Control with Discounting, IEEE Transactions on Automatic Control, AC-39, No. 1, pp. 175–179, 1994.Google Scholar
27. [Ig101a]
P. A. Iglesias, Tradeoffs in Linear Time-Varying Systems: An Analogue of Bode’s sensitivity integral, Automatica, Vol. 37, pp. 1541–1550, 2001.
28. [Ig101b]
P. A. Iglesias, “An analogue of Bode’s integral for stable non linear systems: Relations to entropy,” Proceedings of the 40th IEEE Conference on Decision and Control, Orlando, FL, pp. 3419–3420, December 2001.Google Scholar
29. [Jac73]
D. H. Jacobson, Optimal Stochastic Linear Systems with Exponential Performance Criteria and Their Relationship to Deterministic Differential Games, IEEE Transactions on Automatic Control, AC-18, pp. 124–131, 1973.Google Scholar
30. [Jam92]
M. R. James, Asymptotic Analysis of Nonlinear Stochastic Risk-Sensitive Control and Differential Games, Mathematics of Control, Signals, and Systems, 5, pp. 401–417, 1992.
31. [Jam94]
M. R. James, J. S. Baras, and R. J. Elliott, Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems, IEEE Transactions on Automatic Control, AC-39, No. 4, pp. 780–792, 1994.Google Scholar
32. [Kal60]
R. E. Kalman, Contributions to the Theory of Optimal Control, Bol. de Soc. Math. Mexicana, p. 102, 1960.Google Scholar
33. [Kar87]
I. Karatzas and S. E. Shereve, Brownian Motion and Stochastic Calculus, New York: Springer-Verlag, 1987.Google Scholar
34. [Kv81]
P. R. Kumar and J. H. van Schuppen, On the Optimal Control of Stochastic Systems with an Exponential-of-Integral Performance Index, Journal of Mathematical Analysis and Applications, Volume 80, pp. 312–332, 1981.
35. [Kus71]
H. Kushner, Introduction to Stochastic Control, New York: Holt, Rinehart and Winston, Inc., 1971.
36. [KS72]
H. Kwakernaak and R. Sivan, Linear Optimal Control Systems, New York: John Wiley & Sons, Inc., 1972.
37. [Lib71]
S. R. Liberty, “Characteristic Functions of LQG Control,” Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, August 1971.Google Scholar
38. [LH76]
S. R. Liberty and R. C. Hartwig, On the Essential Quadratic Nature of LQG Control-Performance Measure Cumulants, Information and Control, Volume 32, Number 3, pp. 276–305, 1976.
39. [LH78]
S. R. Liberty and R. C. Hartwig, Design-Performance-Measure Statistics for Stochastic Linear Control Systems, IEEE Transactions on Automatic Control, AC-23, Number 6, pp. 1085–1090, December 1978.Google Scholar
40. [LAH94]
D. J. N. Limebeer, B. D. O. Anderson, and D. Hendel, A Nash Game Approach to Mixed $$H_2$$/$$H_\infty$$ Control, IEEE Transactions on Automatic Control, Vol. 39, Number 1, pp. 69–82, January 1994.
41. [MG90]
Minimum entropy $$H_\infty$$ control, Vol. 146 of Lecture Notes in Control and Information Sciences, Springer-Verlag, Berlin, 1990.Google Scholar
42. [Oks89]
BØ.ksendal, Stochastic Differential Equations, An Introduction with Applications, Second Edition, New York: Springer-Verlag, 1989.Google Scholar
43. [PI97]
M. A. Peters and P. A. Iglesias, Minimum Entropy Control for Time-Varying Systems, Systems and Control: Foundations & Applications, Birkhäuser, Boston, 1997.
44. [PBGM62]
L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkriledze, and E. F. Mischenko, The Mathematical Theory of Optimal Processes, New York: Interscience Publishers, 1962.
45. [PSL02]
K. D. Pham, M. K. Sain, and S. R. Liberty, Cost Cumulant Control: State-Feedback, Finite-Horizon Paradigm with Applications to Seismic Protection, Special Issue of Journal of Optimization Theory and Applications, Edited by A. Miele, Kluwer Academic/Plenum Publishers, New York, Vol. 115, No. 3, pp. 685–710, December 2002.Google Scholar
46. [PSL04]
K. D. Pham, M. K. Sain, and S. R. Liberty, Infinite Horizon Robustly Stable Seismic Protection of Cable-Stayed Bridges Using Cost Cumulants, Proceedings American Control Conference, Boston, MA, USA, pp. 691–696, June 2004.Google Scholar
47. [Pha04]
K. D. Pham, “Statistical Control Paradigm for Structural Vibration Suppression”, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, May 2004.Google Scholar
48. [RS93]
L. Ray and R. Stengel, A Monte Carlo Approach to the Analysis of Control System Robustness, Automatica, Vol. 29, No. 1, pp. 229–236, 1993.
49. [Run94]
T. Runolfsson, The Equivalence Between Infinite-Horizon Optimal Control of Stochastic Systems with Exponential-of-Integral Performance Index and Stochastic Differential Games, IEEE Transactions on Automatic Control, Vol. 39, No. 8, pp. 1551–1563, 1994.
50. [Sag68]
A. P. Sage, Optimum Systems Control. Englewood Cliffs, NJ: Prentice-Hall Inc., 1968.Google Scholar
51. [Sai65]
M. K. Sain, On Minimal-Variance Control of Linear Systems with Quadratic Loss, Ph.D Thesis, Department of Electrical Engineering and Coordinated Science Laboratory, University of Illinois, Urbana, IL, January 1965.Google Scholar
52. [Sai66]
M. K. Sain, Control of Linear Systems According to the Minimal Variance Criterion–-A New Approach to the Disturbance Problem, IEEE Transactions on Automatic Control, AC-11, No. 1, pp. 118–122, January 1966.Google Scholar
53. [Sai67]
M. K. Sain, Performance Moment Recursions, with Application to Equalizer Control Laws, Proc. 5th Allerton Conference, pp. 327–336, 1967.Google Scholar
54. [SS68]
M. K. Sain and C. R. Souza, A Theory for Linear Estimators Minimizing the Variance of the Error Squared, IEEE Transactions on Information Theory, IT-14, Number 5, pp. 768–770, September 1968.Google Scholar
55. [SL71]
M. K. Sain and S. R. Liberty, Performance Measure Densities for a Class of LQG Control Systems, IEEE Transactions on Automatic Control, AC-16, Number 5, pp. 431–439, October 1971.Google Scholar
56. [SWS92]
M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulant Minimization and Robust Control, Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences 184, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 411–425, 1992.Google Scholar
57. [SWS95]
M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants in Risk-Sensitive Control: The Full-State Feedback Cost Variance Case, Proceedings of the Conference on Decision and Control, New Orleans, LA, pp. 1036–1041, 1995.Google Scholar
58. [SWS00]
M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants and Risk Sensitive Control: A Cost Mean and Variance Theory with Applications to Seismic Protection of Structures, Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games, Vol. 5, J. A. Filor, V. Gaisgory, K Mizukami (Eds), Birkhäuser, Boston, 2000.Google Scholar
59. [SSSS92]
P. Sain, M., B. F. Spencer, Jr., M. K. Sain, and J. Suhardjo, Structural Control Design in the Presence of Time Delays, Proceedings of the ASCE Engineering Mechanics Conference, College Station, TX, pp. 812–815, 1992.Google Scholar
60. [SSWKS93]
B. F. Spencer, M. K. Sain, C.-H. Won, D. C.Kaspari, and P. M. Sain, Reliability-Based Measures of Structural Control Robustness, Structural Safety, 15, pp. 111–129, 1993.
61. [SDJ74]
J. L. Speyer, J. Deyst, and D. H. Jacobson, Optimization of Stochastic Linear Systems with Additive Measurement and Process Noise Using Exponential Performance Criteria, IEEE Transactions on Automatic Control, AC-19, No. 4, pp. 358–366, August 1974.Google Scholar
62. [Spe76]
J. L. Speyer, An Adaptive Terminal Guidance Scheme Based on an Exponential Cost Criterion with Application to Homing Missile Guidance, IEEE Transactions on Automatic Control, AC-21, pp. 371–375, 1976.Google Scholar
63. [SS96]
A. A. Stoorvogel and J. H. Van Schuppen, System Identification with Information Theoretic Criteria, Identification, Adaptation, Learning: The Science of Learning Models from Data (NATO Asi Series. Series F, Computer and Systems Sciences, Vol. 153, Sergio Bittanti (Editor), Giorgio Picci (Editor), Springer, Berlin, 1996, pp. 289–338.Google Scholar
64. [Uch89]
K. Uchida and M. Fujita, On the Central Controller: Characterizations via Differential Games and LEQG Control Problems, Systems & Control Letters, Volume 13, pp9–13, 1989.
65. [Whi81]
P. Whittle, Risk-Sensitive Linear/Quadratic/Gaussian Control, Advances in Applied Probability, Vol. 13, pp. 764–777, 1981.
66. [Whi90]
P. Whittle, Risk Sensitive Optimal Control, New York: John Wiley & Sons, 1990.Google Scholar
67. [Whi91]
P. Whittle, A Risk-Sensitive Maximum Principle: The Case of Imperfect State Observation, IEEE Transactions on Automatic Control, Vol. 36, No. 7, pp. 793–801, July 1991.
68. [Wk86]
P. Whittle and J. Kuhn, A Hamiltonian formulation of risk-sensitive linear/ quadratic/Gaussian control, International Journal of Control, Vol. 43, pp. 1–12, 1986.
69. [WSS94]
C.-H. Won, M. Sain, and B. Spencer, Risk-Sensitive Structural Control Strategies, Proceedings of the Second International Conference on Computational Stochastic Mechanics, Athens, Greece, June 13–15, 1994.Google Scholar
70. [Won95]
C.-H. Won, Cost Cumulants in Risk-Sensitive and Minimal Cost Variance Control, Ph.D. Dissertation, University of Notre Dame, Notre Dame, IN, 1995.Google Scholar
71. [WSL03]
C.-H. Won, M. K. Sain, and S. Liberty. Infinite Time Minimal Cost Variance Control and Coupled Algebraic Riccati Equations Proceedings American Control Conference, Denver, Co, pp. 5155–5160, June 4–6, 2003.Google Scholar
72. [Won04]
C.-H. Won, Cost Distribution Shaping: The Relations Between Bode Integral, Entropy, Risk-Sensitivity, and Cost Cumulant Control, Proceedings American Control Conference, Boston, MA, pp. 2160–2165, June 2004.Google Scholar
73. [Won05]
C.-H. Won, Nonlinear n-th Cost Cumulant Control and Hamilton-Jacobi-Bellman Equations for Markov Diffusion Process, Proceedings of 44th IEEE Conference on Decision and Control, Seville, Spain, pp. 4524–4529, 2005.Google Scholar
74. [Won63]
W. M. Wonham, Stochastic Problems in Optimal Control, 1963 IEEE Int. Conv. Rec., part 2, pp. 114–124, 1963.Google Scholar
75. [Won74]
W. M. Wonham, Linear Multivariable Control, A Geometric Approach, Lecture Notes in Economics and Mathematical Systems, 101, Springer-Verlag, 1974.Google Scholar

## Authors and Affiliations

• Chang-Hee Won
• 1
• Ronald W. Diersing
• 2
• Stephen Yurkovich
• 3
1. 1.Department of Electrical and Computer EngineeringTemple UniversityPhiladelphiaUSA
2. 2.Department of EngineeringUniversity of Southern IndianaEvansvilleUSA
3. 3.Department of Electrical EngineeringThe Ohio State UniversityColumbusUSA