Introduction and Literature Survey of Statistical Control: Going Beyond the Mean

  • Chang-Hee Won
  • Ronald W. Diersing
  • Stephen Yurkovich
Part of the Systems & Control: Foundations & Applications book series (SCFA)


In traditional optimal control, the system is modeled as a stochastic differential equation and an optimal controller is determined such that the expected value of a cost function is minimized. An example is the well-known linear-quadratic-Gaussian problem that has been studied extensively since the 1960s. The mean or the first cumulant is a useful performance metric, however, the mean is only one of the cumulants that describe the distribution of a random variable. It is possible for the operator to optimize the whole distribution of the cost function instead of just the mean. In fact, a denumerable sum of all the cost cumulants has been optimized in risk-sensitive control. The key idea behind statistical control is to optimize other statistical quantities such as the variance, skewness, and kurtosis of the cost function. This leads to the optimal performance shaping concept. Furthermore, we use this statistical concept to generalize \(H_\infty\) and multiple player game theory. In both traditional \(H_\infty\) theory and game theory, the mean of the cost function was the object of optimization, and we can extend this to the optimization of any cumulants if we utilize the statistical control concept. In this chapter, we formulate and provide a literature survey of statistical control. We also review minimal cost variance (second cumulant) control, \(k\)th cost cumulant control, and multiobjective cumulant games. Furthermore, risk-sensitive control is presented as a special case of statistical control. When we view the cost function as a random variable, and optimize any of the cost cumulants, linear-quadratic-Gaussian, minimal cost variance, risk-sensitive, game theoretic, and \(H_\infty\) control all fall under the umbrella of statistical control. Finally, we interpret the cost functions via utility functions.


Cost Function Utility Function Infinite Horizon Cost Cumulants Quadratic Cost Function 


  1. [Ast70]
    K. J. Åstrom, Introduction to Stochastic Control Theory, New York: Academic Press, 1970.Google Scholar
  2. [Ath71]
    M. Athans, The Role and Use of the Stochastic Linear-Quadratic-Gaussian Problem in Control System Design, IEEE Transactions on Automatic Control, AC-16, Number 6, pp. 529–551, December 1971.Google Scholar
  3. [BB91]
    T. Başar and P. Bernhard, \(H_\infty\)Optimal Control and Related Minimax Design Problems, Boston: Birkhäuser, 1991.MATHGoogle Scholar
  4. [BV85]
    A. Bensoussan and J. H. van Schuppen, Optimal Control of Partially Observable Stochastic Systems with an Exponential-of-Integral Performance Index, SIAM Journal on Control and Optimization, Volume 23, pp. 599–613, 1985.MathSciNetMATHCrossRefGoogle Scholar
  5. [Ben92]
    A. Bensoussan, Stochastic Control of Partially Observable Systems, London: Cambridge University Press, 1992.MATHCrossRefGoogle Scholar
  6. [Ber76]
    D. P. Bertsekas, Dynamic Programming and Stochastic Control, London: Academic Press, 1976.MATHGoogle Scholar
  7. [Bor89]
    V. S. Borkar, Optimal Control of Diffusion Processes, England: Longman Scientific & Technical, 1989.MATHGoogle Scholar
  8. [BH75]
    A. E. Bryson, Jr., and Y.-C. Ho, Applied and Optimal Control, Optimization, Estimation, and Control, Revised Printing, New York: Hemisphere Publishing Co., 1975.Google Scholar
  9. [CJ93]
    M. C. Campi and M. R. James, Risk-Sensitive Control: A Bridge Between \(H_2\) and \(H_\infty\) Control, Proceedings of the 32nd Conference on Decision and Control, San Antonio, TX, December 1993.Google Scholar
  10. [Cos69]
    L. Cosenza, On the Minimum Variance Control of Discrete-Time Systems, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, January 1969.Google Scholar
  11. [Dav77]
    M. H. A. Davis, Linear Estimation and Stochastic Control, London: Halsted Press, 1977.MATHGoogle Scholar
  12. [Die06]
    R. W. Diersing, \(H_\infty\), Cumulants and Games, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, 2006.Google Scholar
  13. [Die07]
    R. W. Diersing, M. K. Sain, and C.-H. Won, Bi-Cumulant Games: A Generalization of H-infinity and H2/H-infinity Control, IEEE Transactions on Automatic Control, submitted, 2007. Google Scholar
  14. [Doo53]
    J. L. Doob, Stochastic Processes, New York: John Wiley & Sons, Inc., 1953.MATHGoogle Scholar
  15. [Doy89]
    J. Doyle, K. Glover, P. Khargonekar, and B. Francis, State-Space Solutions to Standard \(H_2\) and \(H_\infty\) Control Problems, IEEE Transactions on Automatic Control, Vol. 34, No. 8, pp. 831–847, 1989.MathSciNetMATHCrossRefGoogle Scholar
  16. [FR75]
    W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, New York: Springer-Verlag, 1975.MATHCrossRefGoogle Scholar
  17. [FS92]
    W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, New York: Springer-Verlag, 1992.Google Scholar
  18. [FM92]
    W. H. Fleming and W. M. McEneaney, “Risk Sensitive Optimal Control and Differential Games,” Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences 184, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 185–197, 1992.Google Scholar
  19. [FW84]
    M. I. Freidlin and A. D. Wentzell, Random Perturbations of Dynamical Systems, New York: Springer-Verlag, 1984.MATHCrossRefGoogle Scholar
  20. [Gar85]
    C. W. Gardiner, Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences, 2nd ed., New York: Springer-Verlag, 1985.Google Scholar
  21. [GS69]
    I. I. Gihman and A. V. Skorohod, Introduction to the Theory of Random Processes, London: W. B. Saunders, 1969.Google Scholar
  22. [GS72]
    I. I. Gihman and A. V. Skorohod, Stochastic Differential Equations, New York: Springer-Verlag, 1972.MATHCrossRefGoogle Scholar
  23. [GS79]
    I. I. Gihman and A. V. Skorohod, Controlled Stochastic Processes, New York: Springer-Verlag, 1979.CrossRefGoogle Scholar
  24. [GD88]
    K. Glover and J. C. Doyle, State-Space Formulae for All Stabilizing Controllers That Satisfy an \(H_\infty\)-Norm Bound and Relations to Risk Sensitivity, Systems and Control Letters, Volume 11, pp. 167–172, 1988.MathSciNetMATHCrossRefGoogle Scholar
  25. [Glo89]
    K. Glover, Minimum Entropy and Risk-Sensitive Control: The Continuous Time Case, Proceedings 28th IEEE Conference on Decision and Control, pp. 388–391, December 1989.CrossRefGoogle Scholar
  26. [Hop94]
    W. E. Hopkins, Jr., Exponential Linear Quadratic Optimal Control with Discounting, IEEE Transactions on Automatic Control, AC-39, No. 1, pp. 175–179, 1994.Google Scholar
  27. [Ig101a]
    P. A. Iglesias, Tradeoffs in Linear Time-Varying Systems: An Analogue of Bode’s sensitivity integral, Automatica, Vol. 37, pp. 1541–1550, 2001.MathSciNetMATHCrossRefGoogle Scholar
  28. [Ig101b]
    P. A. Iglesias, “An analogue of Bode’s integral for stable non linear systems: Relations to entropy,” Proceedings of the 40th IEEE Conference on Decision and Control, Orlando, FL, pp. 3419–3420, December 2001.Google Scholar
  29. [Jac73]
    D. H. Jacobson, Optimal Stochastic Linear Systems with Exponential Performance Criteria and Their Relationship to Deterministic Differential Games, IEEE Transactions on Automatic Control, AC-18, pp. 124–131, 1973.Google Scholar
  30. [Jam92]
    M. R. James, Asymptotic Analysis of Nonlinear Stochastic Risk-Sensitive Control and Differential Games, Mathematics of Control, Signals, and Systems, 5, pp. 401–417, 1992.MathSciNetMATHCrossRefGoogle Scholar
  31. [Jam94]
    M. R. James, J. S. Baras, and R. J. Elliott, Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems, IEEE Transactions on Automatic Control, AC-39, No. 4, pp. 780–792, 1994.Google Scholar
  32. [Kal60]
    R. E. Kalman, Contributions to the Theory of Optimal Control, Bol. de Soc. Math. Mexicana, p. 102, 1960.Google Scholar
  33. [Kar87]
    I. Karatzas and S. E. Shereve, Brownian Motion and Stochastic Calculus, New York: Springer-Verlag, 1987.Google Scholar
  34. [Kv81]
    P. R. Kumar and J. H. van Schuppen, On the Optimal Control of Stochastic Systems with an Exponential-of-Integral Performance Index, Journal of Mathematical Analysis and Applications, Volume 80, pp. 312–332, 1981.MathSciNetMATHCrossRefGoogle Scholar
  35. [Kus71]
    H. Kushner, Introduction to Stochastic Control, New York: Holt, Rinehart and Winston, Inc., 1971.MATHGoogle Scholar
  36. [KS72]
    H. Kwakernaak and R. Sivan, Linear Optimal Control Systems, New York: John Wiley & Sons, Inc., 1972.MATHGoogle Scholar
  37. [Lib71]
    S. R. Liberty, “Characteristic Functions of LQG Control,” Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, August 1971.Google Scholar
  38. [LH76]
    S. R. Liberty and R. C. Hartwig, On the Essential Quadratic Nature of LQG Control-Performance Measure Cumulants, Information and Control, Volume 32, Number 3, pp. 276–305, 1976.MathSciNetMATHCrossRefGoogle Scholar
  39. [LH78]
    S. R. Liberty and R. C. Hartwig, Design-Performance-Measure Statistics for Stochastic Linear Control Systems, IEEE Transactions on Automatic Control, AC-23, Number 6, pp. 1085–1090, December 1978.Google Scholar
  40. [LAH94]
    D. J. N. Limebeer, B. D. O. Anderson, and D. Hendel, A Nash Game Approach to Mixed \(H_2\)/\(H_\infty\) Control, IEEE Transactions on Automatic Control, Vol. 39, Number 1, pp. 69–82, January 1994.MathSciNetMATHCrossRefGoogle Scholar
  41. [MG90]
    Minimum entropy \(H_\infty\) control, Vol. 146 of Lecture Notes in Control and Information Sciences, Springer-Verlag, Berlin, 1990.Google Scholar
  42. [Oks89]
    BØ.ksendal, Stochastic Differential Equations, An Introduction with Applications, Second Edition, New York: Springer-Verlag, 1989.Google Scholar
  43. [PI97]
    M. A. Peters and P. A. Iglesias, Minimum Entropy Control for Time-Varying Systems, Systems and Control: Foundations & Applications, Birkhäuser, Boston, 1997.CrossRefGoogle Scholar
  44. [PBGM62]
    L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkriledze, and E. F. Mischenko, The Mathematical Theory of Optimal Processes, New York: Interscience Publishers, 1962.MATHGoogle Scholar
  45. [PSL02]
    K. D. Pham, M. K. Sain, and S. R. Liberty, Cost Cumulant Control: State-Feedback, Finite-Horizon Paradigm with Applications to Seismic Protection, Special Issue of Journal of Optimization Theory and Applications, Edited by A. Miele, Kluwer Academic/Plenum Publishers, New York, Vol. 115, No. 3, pp. 685–710, December 2002.Google Scholar
  46. [PSL04]
    K. D. Pham, M. K. Sain, and S. R. Liberty, Infinite Horizon Robustly Stable Seismic Protection of Cable-Stayed Bridges Using Cost Cumulants, Proceedings American Control Conference, Boston, MA, USA, pp. 691–696, June 2004.Google Scholar
  47. [Pha04]
    K. D. Pham, “Statistical Control Paradigm for Structural Vibration Suppression”, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, May 2004.Google Scholar
  48. [RS93]
    L. Ray and R. Stengel, A Monte Carlo Approach to the Analysis of Control System Robustness, Automatica, Vol. 29, No. 1, pp. 229–236, 1993.MathSciNetMATHCrossRefGoogle Scholar
  49. [Run94]
    T. Runolfsson, The Equivalence Between Infinite-Horizon Optimal Control of Stochastic Systems with Exponential-of-Integral Performance Index and Stochastic Differential Games, IEEE Transactions on Automatic Control, Vol. 39, No. 8, pp. 1551–1563, 1994.MathSciNetMATHCrossRefGoogle Scholar
  50. [Sag68]
    A. P. Sage, Optimum Systems Control. Englewood Cliffs, NJ: Prentice-Hall Inc., 1968.Google Scholar
  51. [Sai65]
    M. K. Sain, On Minimal-Variance Control of Linear Systems with Quadratic Loss, Ph.D Thesis, Department of Electrical Engineering and Coordinated Science Laboratory, University of Illinois, Urbana, IL, January 1965.Google Scholar
  52. [Sai66]
    M. K. Sain, Control of Linear Systems According to the Minimal Variance Criterion–-A New Approach to the Disturbance Problem, IEEE Transactions on Automatic Control, AC-11, No. 1, pp. 118–122, January 1966.Google Scholar
  53. [Sai67]
    M. K. Sain, Performance Moment Recursions, with Application to Equalizer Control Laws, Proc. 5th Allerton Conference, pp. 327–336, 1967.Google Scholar
  54. [SS68]
    M. K. Sain and C. R. Souza, A Theory for Linear Estimators Minimizing the Variance of the Error Squared, IEEE Transactions on Information Theory, IT-14, Number 5, pp. 768–770, September 1968.Google Scholar
  55. [SL71]
    M. K. Sain and S. R. Liberty, Performance Measure Densities for a Class of LQG Control Systems, IEEE Transactions on Automatic Control, AC-16, Number 5, pp. 431–439, October 1971.Google Scholar
  56. [SWS92]
    M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulant Minimization and Robust Control, Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences 184, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 411–425, 1992.Google Scholar
  57. [SWS95]
    M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants in Risk-Sensitive Control: The Full-State Feedback Cost Variance Case, Proceedings of the Conference on Decision and Control, New Orleans, LA, pp. 1036–1041, 1995.Google Scholar
  58. [SWS00]
    M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants and Risk Sensitive Control: A Cost Mean and Variance Theory with Applications to Seismic Protection of Structures, Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games, Vol. 5, J. A. Filor, V. Gaisgory, K Mizukami (Eds), Birkhäuser, Boston, 2000.Google Scholar
  59. [SSSS92]
    P. Sain, M., B. F. Spencer, Jr., M. K. Sain, and J. Suhardjo, Structural Control Design in the Presence of Time Delays, Proceedings of the ASCE Engineering Mechanics Conference, College Station, TX, pp. 812–815, 1992.Google Scholar
  60. [SSWKS93]
    B. F. Spencer, M. K. Sain, C.-H. Won, D. C.Kaspari, and P. M. Sain, Reliability-Based Measures of Structural Control Robustness, Structural Safety, 15, pp. 111–129, 1993.CrossRefGoogle Scholar
  61. [SDJ74]
    J. L. Speyer, J. Deyst, and D. H. Jacobson, Optimization of Stochastic Linear Systems with Additive Measurement and Process Noise Using Exponential Performance Criteria, IEEE Transactions on Automatic Control, AC-19, No. 4, pp. 358–366, August 1974.Google Scholar
  62. [Spe76]
    J. L. Speyer, An Adaptive Terminal Guidance Scheme Based on an Exponential Cost Criterion with Application to Homing Missile Guidance, IEEE Transactions on Automatic Control, AC-21, pp. 371–375, 1976.Google Scholar
  63. [SS96]
    A. A. Stoorvogel and J. H. Van Schuppen, System Identification with Information Theoretic Criteria, Identification, Adaptation, Learning: The Science of Learning Models from Data (NATO Asi Series. Series F, Computer and Systems Sciences, Vol. 153, Sergio Bittanti (Editor), Giorgio Picci (Editor), Springer, Berlin, 1996, pp. 289–338.Google Scholar
  64. [Uch89]
    K. Uchida and M. Fujita, On the Central Controller: Characterizations via Differential Games and LEQG Control Problems, Systems & Control Letters, Volume 13, pp9–13, 1989.MathSciNetMATHCrossRefGoogle Scholar
  65. [Whi81]
    P. Whittle, Risk-Sensitive Linear/Quadratic/Gaussian Control, Advances in Applied Probability, Vol. 13, pp. 764–777, 1981.MathSciNetMATHCrossRefGoogle Scholar
  66. [Whi90]
    P. Whittle, Risk Sensitive Optimal Control, New York: John Wiley & Sons, 1990.Google Scholar
  67. [Whi91]
    P. Whittle, A Risk-Sensitive Maximum Principle: The Case of Imperfect State Observation, IEEE Transactions on Automatic Control, Vol. 36, No. 7, pp. 793–801, July 1991.MathSciNetMATHCrossRefGoogle Scholar
  68. [Wk86]
    P. Whittle and J. Kuhn, A Hamiltonian formulation of risk-sensitive linear/ quadratic/Gaussian control, International Journal of Control, Vol. 43, pp. 1–12, 1986.MathSciNetMATHCrossRefGoogle Scholar
  69. [WSS94]
    C.-H. Won, M. Sain, and B. Spencer, Risk-Sensitive Structural Control Strategies, Proceedings of the Second International Conference on Computational Stochastic Mechanics, Athens, Greece, June 13–15, 1994.Google Scholar
  70. [Won95]
    C.-H. Won, Cost Cumulants in Risk-Sensitive and Minimal Cost Variance Control, Ph.D. Dissertation, University of Notre Dame, Notre Dame, IN, 1995.Google Scholar
  71. [WSL03]
    C.-H. Won, M. K. Sain, and S. Liberty. Infinite Time Minimal Cost Variance Control and Coupled Algebraic Riccati Equations Proceedings American Control Conference, Denver, Co, pp. 5155–5160, June 4–6, 2003.Google Scholar
  72. [Won04]
    C.-H. Won, Cost Distribution Shaping: The Relations Between Bode Integral, Entropy, Risk-Sensitivity, and Cost Cumulant Control, Proceedings American Control Conference, Boston, MA, pp. 2160–2165, June 2004.Google Scholar
  73. [Won05]
    C.-H. Won, Nonlinear n-th Cost Cumulant Control and Hamilton-Jacobi-Bellman Equations for Markov Diffusion Process, Proceedings of 44th IEEE Conference on Decision and Control, Seville, Spain, pp. 4524–4529, 2005.Google Scholar
  74. [Won63]
    W. M. Wonham, Stochastic Problems in Optimal Control, 1963 IEEE Int. Conv. Rec., part 2, pp. 114–124, 1963.Google Scholar
  75. [Won74]
    W. M. Wonham, Linear Multivariable Control, A Geometric Approach, Lecture Notes in Economics and Mathematical Systems, 101, Springer-Verlag, 1974.Google Scholar

Copyright information

© Birkhäuser Boston, a part of Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Chang-Hee Won
    • 1
  • Ronald W. Diersing
    • 2
  • Stephen Yurkovich
    • 3
  1. 1.Department of Electrical and Computer EngineeringTemple UniversityPhiladelphiaUSA
  2. 2.Department of EngineeringUniversity of Southern IndianaEvansvilleUSA
  3. 3.Department of Electrical EngineeringThe Ohio State UniversityColumbusUSA

Personalised recommendations