Discrete-Time Markov Control Processes

Basic Optimality Criteria

  • Onésimo Hernández-Lerma
  • Jean Bernard Lasserre

Part of the Applications of Mathematics book series (SMAP, volume 30)

Table of contents

  1. Front Matter
    Pages i-xiv
  2. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 1-12
  3. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 13-21
  4. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 23-42
  5. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 43-73
  6. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 75-124
  7. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 125-167
  8. Back Matter
    Pages 169-216

About this book


This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro­ grams; sometimes, particularly when the state space is a countable set, they are also called Markov decision (or controlled Markov) chains. Regardless of the name used, MCPs appear in many fields, for example, engineering, economics, operations research, statistics, renewable and nonrenewable re­ source management, (control of) epidemics, etc. However, most of the lit­ erature (say, at least 90%) is concentrated on MCPs for which (a) the state space is a countable set, and/or (b) the costs-per-stage are bounded, and/or (c) the control constraint sets are compact. But curiously enough, the most widely used control model in engineering and economics--namely the LQ (Linear system/Quadratic cost) model-satisfies none of these conditions. Moreover, when dealing with "partially observable" systems) a standard approach is to transform them into equivalent "completely observable" sys­ tems in a larger state space (in fact, a space of probability measures), which is uncountable even if the original state process is finite-valued.


Markov property linear optimization management model operations research production programming quality science

Authors and affiliations

  • Onésimo Hernández-Lerma
    • 1
  • Jean Bernard Lasserre
    • 2
  1. 1.Departamento de MatemáticasCINVESTAV-IPNMéxico DFMéxico
  2. 2.LAAS-CNRSToulouse CédexFrance

Bibliographic information

  • DOI
  • Copyright Information Springer-Verlag New York, Inc. 1996
  • Publisher Name Springer, New York, NY
  • eBook Packages Springer Book Archive
  • Print ISBN 978-1-4612-6884-0
  • Online ISBN 978-1-4612-0729-0
  • Series Print ISSN 0172-4568
  • Buy this book on publisher's site
Industry Sectors
Finance, Business & Banking
IT & Software
Energy, Utilities & Environment
Oil, Gas & Geosciences