Probability Matrices

Synonyms

Markov chain; Markov matrix; Stochastic matrix; Substitution matrix; Transition matrix

Glossary

Aperiodic Markov Chain:

A Markov chain in which the period of every state is one

Discrete Time Markov Chain:

A sequence of random variables $X={\left\{{X}_n\right\}}_{n=1}^{\infty }$ taking values in some state space S such that the probability of X _n moving to any state only depends upon its current state and the time n

Irreducible Markov Chain:

A Markov chain in which there is a positive probability of moving from any state to any other state in a finite amount of time

Positive Recurrent Markov Chain:

A Markov chain is called positive recurrent is for every state $i\in S, {\sum}_{m=1}^{\infty }{nf}_{ii}^{(n)}<\infty$ where ${f}_{ii}^{(n)}= P\left( \inf \right\{ m\ge 1$: X _m  = i|X ₀ = i} = n)

Probability Matrix:

A matrix p describing the probability of traveling between states in a Discrete Time Markov Chain

Definition

This section provides the definition of a...