Continuous-Time Markov Chains and Applications

A Two-Time-Scale Approach

  • G. George Yin
  • Qing Zhang

Part of the Stochastic Modelling and Applied Probability book series (SMAP, volume 37)

Table of contents

  1. Front Matter
    Pages i-xxi
  2. Prologue and Preliminaries

    1. Front Matter
      Pages 1-1
    2. G. George Yin, Qing Zhang
      Pages 3-16
    3. G. George Yin, Qing Zhang
      Pages 17-29
    4. G. George Yin, Qing Zhang
      Pages 31-56
  3. Two-Time-Scale Markov Chains

    1. Front Matter
      Pages 57-57
    2. G. George Yin, Qing Zhang
      Pages 59-140
    3. G. George Yin, Qing Zhang
      Pages 141-234
    4. G. George Yin, Qing Zhang
      Pages 235-257
  4. Applications: MDPs, Near-optimal Controls, Numerical Methods, and LQG with Switching

    1. Front Matter
      Pages 259-259
    2. G. George Yin, Qing Zhang
      Pages 261-283
    3. G. George Yin, Qing Zhang
      Pages 285-318
    4. G. George Yin, Qing Zhang
      Pages 319-339
    5. G. George Yin, Qing Zhang
      Pages 341-372
  5. Back Matter
    Pages 373-427

About this book


This book gives a systematic treatment of singularly perturbed systems that naturally arise in control and optimization, queueing networks, manufacturing systems, and financial engineering. It presents results on asymptotic expansions of solutions of Komogorov forward and backward equations, properties of functional occupation measures, exponential upper bounds, and functional limit results for Markov chains with weak and strong interactions. To bridge the gap between theory and applications, a large portion of the book is devoted to  applications in controlled dynamic systems, production planning, and numerical methods for controlled Markovian systems with large-scale and complex structures in the real-world problems. This second edition  has been updated throughout and includes two new chapters on asymptotic expansions of solutions for backward equations and hybrid LQG problems. The chapters on analytic and probabilistic properties of two-time-scale Markov chains have been almost completely rewritten and the notation has been streamlined and simplified.

This book is written for applied mathematicians, engineers, operations researchers, and applied scientists. Selected material from the book can also be used for a one semester advanced graduate-level course in applied probability and stochastic processes.


Markov chains continuous-time systems probability vectors stochastic processes

Authors and affiliations

  • G. George Yin
    • 1
  • Qing Zhang
    • 2
  1. 1.Department of MathematicsWayne State UniversityDetroitUSA
  2. 2.Dept. MathematicsUniversity of GeorgiaAthensUSA

Bibliographic information

  • DOI
  • Copyright Information Springer Science+Business Media, LLC 2013
  • Publisher Name Springer, New York, NY
  • eBook Packages Mathematics and Statistics
  • Print ISBN 978-1-4614-4345-2
  • Online ISBN 978-1-4614-4346-9
  • Series Print ISSN 0172-4568
  • Series Online ISSN 2197-439X
  • Buy this book on publisher's site
Industry Sectors
Finance, Business & Banking
IT & Software
Energy, Utilities & Environment
Oil, Gas & Geosciences