© 2020

Markov Chains

Gibbs Fields, Monte Carlo Simulation and Queues


Part of the Texts in Applied Mathematics book series (TAM, volume 31)

Table of contents

  1. Front Matter
    Pages i-xvi
  2. Pierre Brémaud
    Pages 1-62
  3. Pierre Brémaud
    Pages 63-109
  4. Pierre Brémaud
    Pages 111-144
  5. Pierre Brémaud
    Pages 145-169
  6. Pierre Brémaud
    Pages 171-186
  7. Pierre Brémaud
    Pages 187-225
  8. Pierre Brémaud
    Pages 227-254
  9. Pierre Brémaud
    Pages 255-287
  10. Pierre Brémaud
    Pages 289-329
  11. Pierre Brémaud
    Pages 331-367
  12. Pierre Brémaud
    Pages 369-398
  13. Pierre Brémaud
    Pages 399-422
  14. Pierre Brémaud
    Pages 423-489
  15. Pierre Brémaud
    Pages 491-526
  16. Back Matter
    Pages 527-557

About this book


This 2nd edition is a thoroughly revised and augmented version of the book with the same title published in 1999. The author begins with the elementary theory of Markov chains and very progressively brings the reader to more advanced topics. He gives a useful review of probability, making the book self-contained, and provides an appendix with detailed proofs of all the prerequisites from calculus, algebra, and number theory. A number of carefully chosen problems of varying difficulty are proposed at the close of each chapter, and the mathematics is slowly and carefully developed, in order to make self-study easier. The book treats the classical topics of Markov chain theory, both in discrete time and continuous time, as well as connected topics such as finite Gibbs fields, nonhomogeneous Markov chains, discrete-time regenerative processes, Monte Carlo simulation, simulated annealing, and queuing theory.

The main additions of the 2nd edition are the exact sampling algorithm of Propp and Wilson, the electrical network analogy of symmetric random walks on graphs, mixing times and additional details on the branching process. The structure of the book has been modified in order to smoothly incorporate this new material. Among the features that should improve reader-friendliness, the three main ones are: a shared numbering system for the definitions, theorems and examples; the attribution of titles to the examples and exercises; and the blue highlighting of important terms. The result is an up-to-date textbook on stochastic processes.

Students and researchers in operations research and electrical engineering, as well as in physics and biology, will find it very accessible and relevant.


60-XX, 68-XX, 82-XX, 90-XX, 92-XX, 94-XX Gibbs Fields Markov Chains Markov Fields Martingale Monte Carlo Simulation stochastic model Queueing theory electrical engineering ergodicity linear optimization Renewal theory operations research regenerative process Monte Carlo simulation

Authors and affiliations

  1. 1.ParisFrance

About the authors

Pierre Brémaud graduated from the École Polytechnique and obtained his Doctorate in Mathematics from the University of Paris VI and his PhD from the department of Electrical Engineering and Computer Science at the University of California, Berkeley. He is a major contributor to the theory of stochastic processes and their applications, and has authored or co-authored several reference books and textbooks. 

Bibliographic information

Industry Sectors
IT & Software
Finance, Business & Banking
Energy, Utilities & Environment
Oil, Gas & Geosciences