Learning Continuous-Time Hidden Markov Models for Event Data

Abstract

The Continuous-Time Hidden Markov Model (CT-HMM) is an attractive modeling tool for mHealth data that takes the form of events occurring at irregularly-distributed continuous time points. However, the lack of an efficient parameter learning algorithm for CT-HMM has prevented its widespread use, necessitating the use of very small models or unrealistic constraints on the state transitions. In this paper, we describe recent advances in the development of efficient EM-based learning methods for CT-HMM models. We first review the structure of the learning problem, demonstrating that it consists of two challenges: (1) the estimation of posterior state probabilities and (2) the computation of end-state conditioned expectations. The first challenge can be addressed by reformulating the estimation problem in terms of an equivalent discrete time-inhomogeneous hidden Markov model. The second challenge is addressed by exploiting computational methods traditionally used for continuous-time Markov chains and adapting them to the CT-HMM domain. We describe three computational approaches and analyze the tradeoffs between them. We evaluate the resulting parameter learning methods in simulation and demonstrate the use of models with more than 100 states to analyze disease progression using glaucoma and Alzheimer’s Disease datasets.

Appendix: Derivation of Vectorized Eigen

In [20, 21], it is stated without proof that the naïve Eigen is equivalent to Vectorized Eigen. Here we present the derivation. Let

$$\displaystyle{ \tau _{k,l}^{i,j}(t) =\sum _{ p=1}^{n}U_{ kp}U_{pi}^{-1}\sum _{ q=1}^{n}U_{ jq}U_{ql}^{-1}\varPsi _{ pq}(t) }$$

(40)

where the symmetric matrix Ψ(t) = [Ψ _pq (t)] _{p, q ∈ S} is defined as:

$$\displaystyle{ \varPsi _{pq}(t) = \left \{\begin{array}{@{}l@{\quad }l@{}} te^{t\lambda _{p}}\text{if }\lambda _{p} =\lambda _{q} \quad \\ \frac{e^{t\lambda _{p}}-e^{t\lambda _{q}}} {\lambda _{p}-\lambda _{q}} \text{if }\lambda _{p}\neq \lambda _{q}\quad \end{array} \right. }$$

(41)

Letting V = U ⁻¹, this is equivalent to

$$\displaystyle{ \tau _{k,l}^{i,j}(t) = [U[V _{ i}^{T}U_{ j}\circ \varPsi ]V ]_{kl} }$$

(42)

To see why, first, note that for the outer product,

$$\displaystyle{ V _{i}^{T}U_{ j}\circ \varPsi = \left (\begin{array}{ccc} U_{1,i}^{-1}U_{j,1}\varPsi _{1,1} & \cdots & U_{1,i}^{-1}U_{j,n}\varPsi _{1,n}\\ \vdots & & \vdots \\ U_{n,i}^{-1}U_{j,1}\varPsi _{n,1} & \cdots &U_{n,i}^{-1}U_{j,n}\varPsi _{n,n}\\ \end{array} \right ) }$$

(43)

Then

$$\displaystyle{ U[V _{i}^{T}U_{ j}\circ \varPsi ] = \left (\begin{array}{ccc} U_{1,1} & \cdots & U_{1,n}\\ \vdots & & \vdots \\ U_{n,1} & \cdots &U_{n,n} \end{array} \right )\left (\begin{array}{ccc} U_{1,i}^{-1}U_{j,1}\varPsi _{1,1} & \cdots & U_{1,i}^{-1}U_{j,n}\varPsi _{1,n}\\ \vdots & & \vdots \\ U_{n,i}^{-1}U_{j,1}\varPsi _{n,1} & \cdots &U_{n,i}^{-1}U_{j,n}\varPsi _{n,n}\\ \end{array} \right ) }$$

(44)

$$\displaystyle{ = \left (\begin{array}{ccc} \sum _{p=1}^{n}U_{1,p}U_{p,i}^{-1}U_{j,1}\psi _{p,1} & \cdots & \sum _{p=1}^{n}U_{1,p}U_{p,i}^{-1}U_{j,n}\psi _{p,n}\\ \vdots & & \vdots \\ \sum _{p=1}^{n}U_{n,p}U_{p,i}^{-1}U_{j,1}\psi _{p,1} & \cdots &\sum _{p=1}^{n}U_{n,p}U_{p,i}^{-1}U_{j,n}\psi _{p,n} \end{array} \right ) }$$

(45)

$$\displaystyle\begin{array}{rcl} & & U[V _{i}^{T}U_{ j}\circ \varPsi ]U^{-1} = \left (\begin{array}{ccc} \sum _{p=1}^{n}U_{ 1,p}U_{p,i}^{-1}U_{ j,1}\psi _{p,1} & \cdots & \sum _{p=1}^{n}U_{ 1,p}U_{p,i}^{-1}U_{ j,n}\psi _{p,n}\\ \vdots & & \vdots \\ \sum _{p=1}^{n}U_{n,p}U_{p,i}^{-1}U_{j,1}\psi _{p,1} & \cdots &\sum _{p=1}^{n}U_{n,p}U_{p,i}^{-1}U_{j,n}\psi _{p,n} \end{array} \right ) \cdot \\ & &\qquad \qquad \left (\begin{array}{ccc} U_{1,1}^{-1} & \cdots & U_{1,n}^{-1}\\ \vdots & & \vdots \\ U_{n,1}^{-1} & \cdots &U_{n,n}^{-1} \end{array} \right ) {}\end{array}$$

(46)

$$\displaystyle{ = \left (\begin{array}{ccc} \sum _{q=1}^{n}\sum _{p=1}^{n}U_{1,p}U_{p,i}^{-1}U_{j,q}U_{q,1}^{-1}\varPsi _{p,q} &\cdots & \sum _{q=1}^{n}\sum _{p=1}^{n}U_{1,p}U_{p,i}^{-1}U_{j,q}U_{q,n}^{-1}\varPsi _{p,q}\\ \vdots & & \vdots \\ \sum _{q=1}^{n}\sum _{p=1}^{n}U_{n,p}U_{p,i}^{-1}U_{j,q}U_{q,1}^{-1}\varPsi _{p,q}&\cdots &\sum _{q=1}^{n}\sum _{p=1}^{n}U_{n,p}U_{p,i}^{-1}U_{j,q}U_{q,n}^{-1}\varPsi _{p,q}\\ \end{array} \right ) }$$

(47)

$$\displaystyle{ = \left (\begin{array}{ccc} \sum _{p=1}^{n}U_{1,p}U_{p,i}^{-1}\sum _{q=1}^{n}U_{j,q}U_{q,1}^{-1}\varPsi _{p,q} &\cdots & \sum _{p=1}^{n}U_{1,p}U_{p,i}^{-1}\sum _{q=1}^{n}U_{j,q}U_{q,n}^{-1}\varPsi _{p,q}\\ \vdots & & \vdots \\ \sum _{p=1}^{n}U_{n,p}U_{p,i}^{-1}\sum _{q=1}^{n}U_{j,q}U_{q,1}^{-1}\varPsi _{p,q}&\cdots &\sum _{p=1}^{n}U_{n,p}U_{p,i}^{-1}\sum _{q=1}^{n}U_{j,q}U_{q,n}^{-1}\varPsi _{p,q}\\ \end{array} \right ) }$$

(48)

So that

$$\displaystyle{ [U[V _{i}^{T}U_{ j} \circ \varPsi (t)]U^{-1}]_{ kl} =\sum _{ p=1}^{n}U_{ k,p}U_{p,i}^{-1}\sum _{ q=1}^{n}U_{ j,q}U_{q,l}^{-1}\varPsi _{ p,q}(t) }$$

(49)

as desired.