An extrapolation method for Bayesian control of Markov chains

  • K.-H. Waldmann
Conference paper
Part of the Proceedings in Operations Research 8 book series (ORP, volume 1978)


We consider a system with state space S and action space A(s), sεS, in which state sn+lεS at stage n+1, nε IN :={1,2,...} is partially determined by the outcome of an i.i.d. random variable Xn not controllable by the decision maker. Let Xn have a distribution q(.) depending on an unknown parameter ε θ. Let X denote the state space of the so-called external process Xn.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. van Hee, K.M. (1978) Bayesian Control of Markov Chains, dissertation, Eindhoven University of TechnologyGoogle Scholar
  2. MacQueen, J. (1966) A modified dynamic programming method for Markovian decision processes, J.Math.Anal.App1. 14, 38–43CrossRefGoogle Scholar
  3. Porteus, E.L. (1971) Some bounds for discounted sequential decision processes, Management Science 18, 7–11CrossRefGoogle Scholar
  4. Rieder, U. (1975) Bayesian dynamic programming. Adv.Appl.Prob. 7, 330–348CrossRefGoogle Scholar
  5. Waldmann, K.-H. (1976) Stationäre Bayessche Entscheidungsmodelle mit Anwendungen in der Lagerhaltungstheorie, Dissertation TH Darmstadt, GermanyGoogle Scholar
  6. Waldmann, K.-H. (1977) On the optimality of (z,Z)-policies in Bayesian dynamic inventory models.TH Darmstadt, Preprint 327, GermanyGoogle Scholar
  7. Waldmann, K.-H. (1978) A natural extension of the MacQueen extrapolation. TH Darmstadt, Preprint 436, GermanyGoogle Scholar
  8. Waldmann, K.-H. (1978a) On approximations of dynamic programs, TH Darmstadt, Preprint 439, GermanyGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1979

Authors and Affiliations

  • K.-H. Waldmann
    • 1
  1. 1.DarmstadtGermany

Personalised recommendations