Generalized Markov Models

Abstract

The main focus of this book is to study systems that evolve randomly in time. We encountered several applications in Chapter 2 where the system is observed at time n = 0, 1, 2, 3,.... In such cases, we define X _n as the state of the system at time n and study the discrete-time stochastic process {X _n, n ≥ 0}. In Chapter 2, we studied the systems that have the Markov property at each time n = 0, 1, 2, 3,...; i.e., the future of the system from any time nonward depends on its history up to time nonly through the state of the system at time n. We found this property to be immensely helpful in studying the behavior of these systems.