Journal of Theoretical Probability

, Volume 29, Issue 4, pp 1458–1484 | Cite as

Risk-Sensitive Control and an Abstract Collatz–Wielandt Formula

  • Ari Arapostathis
  • Vivek S. Borkar
  • K. Suresh Kumar


The ‘value’ of infinite horizon risk-sensitive control is the principal eigenvalue of a certain positive operator. For the case of compact domain, Chang has built upon a nonlinear version of the Krein–Rutman theorem to give a ‘min–max’ characterization of this eigenvalue which may be viewed as a generalization of the classical Collatz–Wielandt formula for the Perron–Frobenius eigenvalue of a nonnegative irreducible matrix. We apply this formula to the Nisio semigroup associated with risk-sensitive control and derive a variational characterization of the optimal risk-sensitive cost. For the linear, i.e., uncontrolled case, this is seen to reduce to the celebrated Donsker–Varadhan formula for principal eigenvalue of a second-order elliptic operator.


Risk-sensitive control Collatz–Wielandt formula Nisio semigroup Variational formulation Principal eigenvalue Donsker–Varadhan functional 

Mathematics Subject Classification (2010)

Primary 60J60 Secondary 60F10 93E20 


  1. 1.
    Arapostathis, A.: A Correction to a Nonlinear Version of the Kreĭn–Rutman Theorem by Rajesh Mahadevan (preprint)Google Scholar
  2. 2.
    Biswas, A.: An eigenvalue approach to the risk sensitive control problem in near monotone case. Syst. Control Lett. 60, 181–184 (2011)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Borkar, V.S.: Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series, vol. 203. Longman Scientific & Technical, Harlow (1989)Google Scholar
  4. 4.
    Chang, K.C.: A nonlinear Krein Rutman theorem. J. Syst. Sci. Complex. 22, 542–554 (2009)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Collatz, L.: Einschließungssatz für die charakteristischen Zahlen von Matrizen. Math. Z. 48, 221–226 (1942)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Donsker, M.D., Varadhan, S.R.S.: On a variational formula for the principal eigenvalue for operators with maximum principle. Proc. Nat. Acad. Sci. USA 72, 780–783 (1975)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Fan, Ky: Fixed-point and minimax theorems in locally convex topological linear spaces. Proc. Nat. Acad. Sci. USA 38, 121–126 (1952)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Fleming, W.H., McEneaney, W.M.: Risk-sensitive control on an infinite time horizon. SIAM J. Control Optim. 33, 1881–1915 (1995)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Friedman, A.: Partial Differential Equations of Parabolic Type. Prentice-Hall, Englewood Cliffs (1964)MATHGoogle Scholar
  10. 10.
    Krylov, N.V.: Lectures on Elliptic and Parabolic Equations in Hölder Spaces. Graduate Studies in Mathematics, vol. 12. American Mathematical Society, Providence (1996)Google Scholar
  11. 11.
    Ladyženskaja, O. A., Solonnikov, V. A., Ural’ceva, N. N.: Linear and Quasilinear Equations of Parabolic Type, Translated from the Russian by S. Smith. Translations of Mathematical Monographs, vol. 23. American Mathematical Society, Providence (1968)Google Scholar
  12. 12.
    Mahadevan, R.: A note on a non-linear Krein–Rutman theorem. Nonlinear Anal. 67, 3084–3090 (2007)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Meyer, C.: Matrix Analysis and Applied Linear Algebra. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (2000)Google Scholar
  14. 14.
    Nisio, M.: On stochastic optimal controls and envelope of Markovian semigroups. In: Proceedings of the International Symposium on Stochastic Differential Equations (Res. Inst. Math. Sci., Kyoto Univ., Kyoto, 1976), pp. 297–325. New York, Wiley (1978)Google Scholar
  15. 15.
    Smith, H.L.: Monotone Dynamical Systems. Mathematical Surveys and Monographs, vol. 41. American Mathematical Society, Providence (1995)Google Scholar
  16. 16.
    Whittle, P.: Risk-Sensitive Optimal Control, Wiley-Interscience Series in Systems and Optimization. Wiley, Chichester (1990)Google Scholar
  17. 17.
    Wielandt, H.: Unzerlegbare, nicht negative Matrizen. Math. Z. 52, 642–648 (1950)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Ari Arapostathis
    • 1
  • Vivek S. Borkar
    • 2
  • K. Suresh Kumar
    • 3
  1. 1.Department of Electrical and Computer EngineeringThe University of Texas at AustinAustinUSA
  2. 2.Department of Electrical EngineeringIndian Institute of TechnologyPowai, MumbaiIndia
  3. 3.Department of MathematicsIndian Institute of TechnologyPowai, MumbaiIndia

Personalised recommendations