A Class of Markovian Decision Processes

Conference paper
Part of the Proceedings in Operations Research book series (ORP, volume 1973)


A class of Markovian decision processes is characterized using a weak row sum criterion. The criterion is shown to be necessary and sufficient for the absolute convergence of present values. A modified successive value iteration procedure is obtained.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Collatz, L.: Funktionalanalysis und Numerische Mathematik, Springer-Verlag, Berlin 1964.Google Scholar
  2. Denardo, E.: Contraction mappings in the theory underlying dynamic programming. SIAM Rev., 9, 165–177, 1967.CrossRefGoogle Scholar
  3. Derman, C.:Finite State Markovian Decision Processes, Academic Press, New York, 1970.Google Scholar
  4. Maitra, A.: Dynamic programming for countable state systems. Sankya 27A 241–248, 1965.Google Scholar
  5. Reetz, D.:Solution of a Markovian Decision Problem by successive overrelaxation. Z.O.R. 17, 29–32, 1973.CrossRefGoogle Scholar
  6. Veinott, A.F.: Discrete dynamic programming with sensitive discount optimality criteria, Ann. Math. Statist. 40, 1635–1660, 1969.CrossRefGoogle Scholar

Copyright information

© Physica-Verlag, Rudolf Liebing KG, Würzburg 1974

Authors and Affiliations

  1. 1.BerlinGermany

Personalised recommendations