© 1996

Discrete-Time Markov Control Processes

Basic Optimality Criteria


Part of the Applications of Mathematics book series (SMAP, volume 30)

Table of contents

  1. Front Matter
    Pages i-xiv
  2. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 1-12
  3. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 13-21
  4. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 23-42
  5. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 43-73
  6. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 75-124
  7. Onésimo Hernández-Lerma, Jean Bernard Lasserre
    Pages 125-167
  8. Back Matter
    Pages 169-216

About this book


This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro­ grams; sometimes, particularly when the state space is a countable set, they are also called Markov decision (or controlled Markov) chains. Regardless of the name used, MCPs appear in many fields, for example, engineering, economics, operations research, statistics, renewable and nonrenewable re­ source management, (control of) epidemics, etc. However, most of the lit­ erature (say, at least 90%) is concentrated on MCPs for which (a) the state space is a countable set, and/or (b) the costs-per-stage are bounded, and/or (c) the control constraint sets are compact. But curiously enough, the most widely used control model in engineering and economics--namely the LQ (Linear system/Quadratic cost) model-satisfies none of these conditions. Moreover, when dealing with "partially observable" systems) a standard approach is to transform them into equivalent "completely observable" sys­ tems in a larger state space (in fact, a space of probability measures), which is uncountable even if the original state process is finite-valued.


Markov property linear optimization management model operations research production programming quality science

Authors and affiliations

  1. 1.Departamento de MatemáticasCINVESTAV-IPNMéxico DFMéxico
  2. 2.LAAS-CNRSToulouse CédexFrance

Bibliographic information

  • Book Title Discrete-Time Markov Control Processes
  • Book Subtitle Basic Optimality Criteria
  • Authors Onesimo Hernandez-Lerma
    Jean B. Lasserre
  • Series Title Applications of Mathematics
  • DOI
  • Copyright Information Springer-Verlag New York, Inc. 1996
  • Publisher Name Springer, New York, NY
  • eBook Packages Springer Book Archive
  • Hardcover ISBN 978-0-387-94579-8
  • Softcover ISBN 978-1-4612-6884-0
  • eBook ISBN 978-1-4612-0729-0
  • Series ISSN 0172-4568
  • Edition Number 1
  • Number of Pages XIV, 216
  • Number of Illustrations 0 b/w illustrations, 0 illustrations in colour
  • Topics Probability Theory and Stochastic Processes
    Applications of Mathematics
    Engineering, general
  • Buy this book on publisher's site
Industry Sectors
IT & Software
Finance, Business & Banking
Energy, Utilities & Environment
Oil, Gas & Geosciences