Predicting viewer-perceived activity/dominance in soccer games with stick-breaking HMM using data from a fixed set of cameras

Abstract

We attempted to predict activity/dominance for soccer games, where activity is defined as the degree of activity of the game as perceived by the viewer, whereas dominance is the degree at which the viewer perceives a particular team to dominate over the other team. Such activity/dominance information would help a layman viewer understand the game. It would also enable construction of an automatic digest creation system that extracts scenes having high activity/dominance. There are two facets of this study: 1. The main part of the underlying prediction model consists of a Stick-Breaking Hidden Markov Model, where the data automatically estimates the number of states of the Markov process behind the data. 2. The data used in this paper is vector time-series data consisting of player, referee, and ball positions, together with team information, acquired by a set of fixed cameras. The problem was approached with a Bayesian framework where learning and prediction were implemented by three different methods: Markov Chain Monte Carlo, Expectation Maximization, and Variational Bayes. The proposed method was tested using a dataset consisting of 10 professional soccer games and was compared against standard regression methods.

Appendices

Appendix I: Details of Gibbs Samling

For simplicity, we present the case where the training data is a single sequence. One can easily obtain a case where the training data is multiple sequences. First, let us recall the the function Ψ _t (⋅) used in the (42). This function can be set as

$$\begin{array}{@{}rcl@{}} &&{\Psi}_{t}(y, z_{-\{t\}}, \phi)={{\Psi}^{1}_{t}}(y_{-\{t\}},z_{-\{t\}};\beta) {{\Psi}^{2}_{t}}(z_{-\{t\}}; \alpha){{\Psi}^{3}_{t}}(z_{1};\gamma), \end{array} $$

(58)

$$\begin{array}{@{}rcl@{}} &&{{\Psi}^{1}_{t}}(y_{-\{t\}},z_{-\{t\}};\beta)=P(y_{-\{t\}}|z_{-\{t\}}, \beta), \end{array} $$

(59)

$$\begin{array}{@{}rcl@{}} &&{{\Psi}^{2}_{t}}(z_{-\{t\}};\alpha)= \left\{ \begin{array}{ll} P(z_{-\{1,2\}}|z_{2},\alpha)&(t=1)\\ P(z_{-\{1,t, t+1\}}|z_{1}, z_{t+1},\alpha)&(1 < t<T-1)\\ P(z_{-\{1,T-1,T\}}|z_{1},\alpha)&(t=T-1)\\ P(z_{-\{1,T\}}|z_{1},\alpha)&(t=T)\\ \end{array} \right., \end{array} $$

(60)

$$\begin{array}{@{}rcl@{}} &&{{\Psi}^{3}_{t}}(z_{1};\gamma)= \left\{ \begin{array}{ll} 1&(t=1)\\ P(z_{1}|\gamma)&(1 < t \le T)\\ \end{array} \right., \end{array} $$

(61)

where y _−{t}=(y _1:t−1,y _t+2:T ), z _−{t}=(z _1:t−1,z _t+2:T ), and z _−{1,t,t+1}=(z _2:t−1,z _t+2:T ). By using this setting, we can decrease (drastically) computational cost for the full conditional distribution P(z _t |z _−{t},y,ϕ).

More concretely, we can write (42)

$$ \frac{P(y,z|\phi)}{{\Psi}_{t}(z_{-\{t\}},y; \phi)} =\frac{P(y|z, \beta)}{{{\Psi}^{1}_{t}}(y_{-\{t\}},z_{-\{t\}};\beta)} \frac{P(z_{-\{1\}}|z_{1},\alpha)}{{{\Psi}^{2}_{t}}(z_{-\{t\}}; \alpha)} \frac{P(z_{1}|\gamma)}{{{\Psi}_{t}^{3}}(z_{1};\gamma)}. $$

(62)

The computations of these three terms $\frac {P(y|z, \beta )}{{{\Psi }^{1}_{t}}(y_{-\{t\}},z_{-\{t\}};\beta )}$, $\frac {P(z_{-\{1\}}|z_{1},\alpha )}{{{\Psi }^{2}_{t}}(z_{-\{t\}}; \alpha )}$, and $\frac {P(z_{1}|\gamma )}{{{\Psi }^{3}_{t}}(z_{1};\gamma )}$ are much cheaper than those of the terms P(y|z,β), P(z _−{1}|z ₁,α), and P(z ₁|γ) for an evaluation of P(y,z|ϕ).

For the later arguments, let the function n _i (⋅) be defined as

$$ n_{i}(\zeta)= \sum\limits_{\tau \in {\mathcal T}} I(\zeta_{\tau}=i) $$

(63)

for a discrete sequence $\zeta =(\zeta _{\tau })_{\tau \in {\mathcal T}}$. The function n _{i k} (⋅) is defined as

$$ n_{ik}\left(\zeta^{1}, \zeta^{2}\right)= \sum\limits_{\tau \in {\mathcal T}} I\left(\zeta^{1}_{\tau}=i\right)I\left(\zeta^{2}_{\tau}=k\right) $$

(64)

for two discrete sequences $\zeta ^{1}=(\zeta ^{1}_{\tau })_{\tau \in \mathcal T}$ and $\zeta ^{2}=(\zeta ^{2}_{\tau })_{\tau \in \mathcal T}$. The function ν _{i j} (⋅) is defined by

$$ \nu_{ij}(\zeta)= \sum\limits_{\tau \in \{{\mathcal T}| (\tau -1) \in \mathcal T\}} I(\zeta_{\tau-1}=i)I(\zeta_{\tau}=j) $$

(65)

for a discrete sequence $\zeta =(\zeta _{\tau })_{\tau \in \mathcal T}$.

• 1. When t=1

  $$\begin{array}{llll} &P\left(z_{t}|z_{-\{t\}},y, \phi \right) \propto \frac{\gamma^{1}_{z_{1}}}{\gamma^{1}_{z_{1}} + \gamma^{2}_{z_{1}}} \cdot \left( \prod\limits_{i=1}^{z_{1}-1} \frac{{\gamma_{i}^{2}} }{{\gamma_{i}^{1}} + {\gamma_{i}^{2}} }\right) \\ & \quad\times \frac{\nu_{z_{1}z_{2}}(z_{-\{1\}})+\alpha_{z_{1}z_{2}}^{1}}{{\sum}_{j^{\prime}=z_{2}}^{U} \nu_{z_{1}j^{\prime}}(z_{-\{1\}})+\alpha_{z_{1}z_{2}}^{1} + \alpha_{z_{1}z_{2}}^{2}} \cdot \left(\prod\limits_{j=1}^{z_{2}-1} \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{1}j^{\prime}}(z_{-\{1\}})+\alpha_{z_{1}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{1}j^{\prime}}(z_{-\{1\}})+\alpha_{z_{1}j}^{1} + \alpha_{z_{1}j}^{2} } \right)\\ & \quad\times \frac{n_{z_{1}e_{1}}(z_{-\{1\}}, e_{-\{1\}})+\beta_{e,z_{1}e_{1}}} {n_{z_{1}}(z_{-\{1\}})+{\sum}_{m=1}^{M+1} \beta_{e,z_{1}k}} \cdot \left(\prod\limits_{l=1}^{L} \frac{n_{z_{1}f_{l,1}}(z_{-\{1\}},f_{l,-\{1\}})+\beta_{f_{l},z_{1}f_{l,1}}} { n_{z_{1}}(z_{-\{1\}})+ {\sum}_{k=1}^{K_{l}} \beta_{f_{l},z_{1}k} }\right). \end{array} $$

• 2. When 1<t<T and z _t ≠z _t−1,

  $$\begin{array}{lllll} &P\left( z_{t}|z_{-\{t\}},y, \phi \right) \\ &\quad\propto \frac{\nu_{z_{t-1}z_{t}}(z_{-\{t\}})+\alpha_{z_{t-1}z_{t}}^{1}} {{\sum}_{j^{\prime}=z_{t}}^{U} \nu_{z_{t-1}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t-1}z_{t}}^{1} + \alpha_{z_{t-1}z_{t}}^{2}} \cdot \left( \prod\limits_{j=1}^{z_{t}-1} \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t-1}k}(z_{-\{t\}})+\alpha_{z_{t-1}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{t-1}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t-1}j}^{1} + \alpha_{z_{t-1}j}^{2}} \right) \\ & \quad\times \frac{\nu_{z_{t}z_{t+1}}(z_{-\{t\}})+\alpha_{z_{t}z_{t+1}}^{1}} {{\sum}_{j^{\prime}=z_{t+1}}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}z_{t+1}}^{1} + \alpha_{z_{t}z_{t+1}}^{2}} \cdot \left( \prod\limits_{j=1}^{z_{t+1}-1} \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}j}^{1} + \alpha_{z_{t}j}^{2}} \right)\\ & \quad\times \frac{n_{z_{t}e_{t}}(z_{-\{t\}},e_{-\{t\}})+\beta_{e,z_{t}e_{t}}} {n_{z_{t}}(z_{-\{t\}})+{\sum}_{m=1}^{M+1} \beta_{e,z_{t}m}} \cdot \left( \prod\limits_{l=1}^{L} \frac{n_{z_{t}f_{l,t}}(z_{-\{t\}},f_{l,-\{t\}})+\beta_{f_{l},z_{t}f_{l,t}}} {n_{z_{t}}(z_{-\{t\}})+{\sum}_{k=1}^{K_{l}} \beta_{f_{l},z_{t}k}} \right). \end{array} $$

• 3. When 1<t<T, z _t =z _t−1, and z _t+1<z _t ,

  $$\begin{array}{lllll} &P\left( z_{t}|z_{-\{t\}},y, \phi \right) \\ &\quad\propto\left(\prod\limits_{j=1}^{z_{t+1}-1} \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+ \alpha_{z_{t}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+ \alpha_{z_{t}j}^{1} + \alpha_{z_{t}j}^{2}} \cdot \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+ 1 +\alpha_{z_{t}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+1+\alpha_{z_{t}j}^{1} + \alpha_{z_{t}j}^{2} } \right)\\ & \quad\times \frac{\nu_{z_{t}z_{t+1}}(z_{-\{t\}})+\alpha_{z_{t}z_{t+1}}^{1}} {{\sum}_{j^{\prime}=z_{t+1}}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}z_{t+1}}^{1} + \alpha_{z_{t}z_{t+1}}^{2}} \cdot \frac{{\sum}_{j^{\prime}=z_{t+1}+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}z_{t+1}}^{2}} {{\sum}_{j^{\prime}=z_{t+1}}^{U} \nu_{z_{t}j^{\prime}}.(z_{-\{t\}})+ 1+\alpha_{z_{t}z_{t+1}}^{1} + \alpha_{z_{t}z_{t+1}}^{2}}\\ & \quad\times \left(\prod\limits_{j=z_{t+1}+1}^{z_{t}-1} \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+ \alpha_{z_{t}j}^{1} + \alpha_{z_{t}j}^{2}} \right) \cdot \frac{\nu_{z_{t}z_{t}}(z_{-\{t\}})+\alpha_{z_{t}z_{t}}^{1}} {{\sum}_{j^{\prime}=z_{t}}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}z_{t}}^{1} + \alpha_{z_{t}z_{t}}^{2}} \\ & \quad\times \frac{n_{z_{t}e_{t}}(z_{-\{t\}},e_{-\{t\}})+\beta_{e,z_{t}e_{t}}} {n_{z_{t}}(z_{-\{t\}})+{\sum}_{m=1}^{M+1} \beta_{e,z_{t}m}} \cdot \left( \prod\limits_{l=1}^{L} \frac{n_{z_{t}f_{l,t}}(z_{-\{t\}},f_{l,-\{t\}})+\beta_{f_{l},z_{t}f_{l,t}}} {n_{z_{t}}(z_{-\{t\}})+{\sum}_{k=1}^{K_{l}} \beta_{f_{l},z_{t}k}} \right). \end{array} $$

• 4. When 1<t<T, z _t =z _t−1, and z _t+1>z _t ,

  $$\begin{array}{lll} &P\left( z_{t}|z_{-\{t\}},y, \phi \right) \\ &\quad\propto\left( \prod\limits_{j=1}^{z_{t}-1} \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}}) + \alpha_{z_{t}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+ \alpha_{z_{t}j}^{1} + \alpha_{z_{t}j}^{2}} \cdot \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+1 +\alpha_{z_{t}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+1+\alpha_{z_{t}j}^{1} + \alpha_{z_{t}j}^{2}} \right) \\ & \quad\times \frac{\nu_{z_{t}z_{t}}(z_{-\{t\}})+\alpha_{z_{t}z_{t}}^{1}} {{\sum}_{j^{\prime}=z_{t}}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}z_{t}}^{1} + \alpha_{z_{t}z_{t}}^{2}} \cdot \frac{{\sum}_{j^{\prime}=z_{t}+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}z_{t}}^{2}} {{\sum}_{j^{\prime}=z_{t}}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+1+\alpha_{z_{t}z_{t}}^{1} + \alpha_{z_{t}z_{t}}^{2}} \\ & \quad\times \left( \prod\limits_{j=z_{t}+1}^{z_{t+1}-1} \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}j}^{1} + \alpha_{z_{t}j}^{2}}\right) \cdot \frac{\nu_{z_{t}z_{t+1}}(z_{-\{t\}})+\alpha_{z_{t}z_{t+1}}^{1}} {{\sum}_{j^{\prime}=z_{t}}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}z_{t+1}}^{1} + \alpha_{z_{t}z_{t+1}}^{2}} \\ & \quad\times \frac{n_{z_{t}e_{t}}(z_{-\{t\}},e_{-\{t\}})+\beta_{e,z_{t}e_{t}}} {n_{z_{t}}(z_{-\{t\}})+{\sum}_{m=1}^{M+1} \beta_{e,z_{t}m}} \cdot \left( \prod\limits_{l=1}^{L} \frac{n_{z_{t}f_{l,t}}(z_{-\{t\}},f_{l,-\{t\}})+\beta_{f_{l},z_{t}f_{l,t}}} {n_{z_{t}}(z_{-\{t\}})+{\sum}_{k=1}^{K_{l}} \beta_{f_{l},z_{t}k}} \right). \end{array} $$

• 5. When 1<t<T, z _t =z _t−1, and z _t+1=z _t ,

  $$\begin{array}{lllll} &P\left( z_{t}|z_{-\{t\}},y, \phi \right) \\ & \quad\propto\left(\prod\limits_{j=1}^{z_{t}-1} \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t}j^{\prime}}\left(z_{-\{t\}}\right)+\alpha_{z_{t}j}^{2}} {{\sum}_{j^{\prime}=j}^{U}\nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}j}^{1} + \alpha_{z_{t}j}^{2}} \right.\\ &\qquad\left.\times \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+1+\alpha_{z_{t}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+1+\alpha_{z_{t}j}^{1} + \alpha_{z_{t}j}^{2}} \right) \\ &\qquad \times \frac{\nu_{z_{t}z_{t}}(z_{-\{t\}})+\alpha_{z_{t}z_{t}}^{1}} { {\sum}_{j^{\prime}=z_{t}}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+\alpha_{z_{t}z_{t}}^{1} + \alpha_{z_{t}z_{t}}^{2}} \cdot \frac{\nu_{z_{t}z_{t}}(z_{-\{t\}})+1+\alpha_{z_{t}z_{t}}^{1}} {{\sum}_{j^{\prime}=z_{t}}^{U} \nu_{z_{t}j^{\prime}}(z_{-\{t\}})+1+\alpha_{z_{t}z_{t}}^{1} + \alpha_{z_{t}z_{t}}^{2}} \\ & \qquad\times \frac{n_{z_{t}e_{t}}(z_{-\{t\}},e_{-\{t\}})+\beta_{e,z_{t}e_{t}}} {n_{z_{t}}(z_{-\{t\}})+{\sum}_{m=1}^{M+1} \beta_{e,z_{t}m}} \cdot \left( \prod\limits_{l=1}^{L} \frac{n_{z_{t}f_{l,t}}(z_{-\{t\}},f_{l,-\{t\}})+\beta_{f_{l},z_{t}f_{l,t}}} {n_{z_{t}}(z_{-\{t\}})+{\sum}_{k=1}^{K_{l}} \beta_{f_{l},z_{t}k}} \right). \end{array} $$

• 6. When t=T

  $$\begin{array}{lllll} &P\left( z_{t}|z_{-\{t\}},y, \phi \right) \\ &\propto \left( \prod\limits_{j=1}^{z_{t}-1} \frac{{\sum}_{j^{\prime}=j+1}^{U} \nu_{z_{T-1}j^{\prime}}(z_{-\{T\}})+\alpha_{z_{T-1}j}^{2}} {{\sum}_{j^{\prime}=j}^{U} \nu_{z_{T-1}j^{\prime}}(z_{-\{T\}})+\alpha_{z_{T-1}j}^{1} + \alpha_{z_{T-1}j}^{2}} \cdot \frac{\nu_{z_{T-1}z_{T}}(z_{-\{T\}})+\alpha_{z_{T-1}z_{T}}^{1}} {{\sum}_{j^{\prime}=z_{T}}^{U} \nu_{z_{T-1}j^{\prime}}(z_{-\{T\}})+\alpha_{z_{T-1}z_{T}}^{1} + \alpha_{z_{T-1}z_{T}}^{2}} \right) \\ & \quad\times \frac{n_{z_{T}e_{T}}(z_{-\{T\}},e_{-\{T\}})+\beta_{e,z_{T}e_{T}}} {n_{z_{T}}(z_{-\{T\}})+{\sum}_{m=1}^{M+1} \beta_{e,z_{T}m}} \cdot \left( \prod\limits_{l=1}^{L} \frac{n_{z_{T}f_{l,T}}(z_{-\{T\}},f_{l,-\{T\}})+\beta_{f_{l},z_{T}f_{l,T}}} {n_{z_{T}}(z_{-\{T\}})+{\sum}_{k=1}^{K_{l}} \beta_{f_{l},z_{T}k}} \right). \end{array} $$

Appendix II: Derivation of MAP EM for SB-HMM

For simplicity, we present the case where the training data is a single sequence. We first note that if the state is truncated at U, the Q-function for the Stick-Breaking HMM is given by

$$\begin{array}{@{}rcl@{}} &&Q\left( \theta| \theta^{(r)}\right) = \sum\limits_{{ z}}P\left({ z}|{ y}, \theta^{(r)}\right)\log P({ y},{ z}| \theta)+\log P(\theta), \end{array} $$

where

$$\begin{array}{@{}rcl@{}} &&\sum\limits_{{ z}}P\left({ z}|{ y}, \theta^{(r)}\right)\log P({ y},{ z}| \theta)= \sum\limits_{i=1}^{U}\mu_{1}^{(r)}(i)\log \pi_{i}+ \sum\limits_{t=1}^{T-1}\sum\limits_{i=1}^{U}\sum\limits_{j=1}^{U}\epsilon_{t}^{(r)}(i,j)\log a_{ij} \\ && + \sum\limits_{l=1}^{L}\sum\limits_{t=1}^{T} \sum\limits_{i=1}^{U} \sum\limits_{k=1}^{K_{l}} \mu_{t}^{(r)}(i)I(f_{l,t}=k) \log b_{f_{l}, ik} + \sum\limits_{t=1}^{T} \sum\limits_{i=1}^{U} \sum\limits_{m=1}^{M+1} \mu_{t}^{(r)}(i)I(e_{t}=m ) \log b_{e, im}. \end{array} $$

We will discuss the state transition parameter a output probability b and initial state probability π, separately.

• Parameters π

  In the Q-function, those terms that depend on π reads:

  $$\begin{array}{@{}rcl@{}} F(\pi) = \sum\limits_{i=1}^{U}\mu_{1}^{(r)}(i)\log \pi_{i} + \log P(\pi). \end{array} $$

  (66)

  We rewrite (66) in terms of V _π . By noting that V _π,U =1, we have

  $$\begin{array}{@{}rcl@{}} F(V_{\pi}) &=& \mu^{(r)}_{1}(1) \log V_{\pi,1} + \sum\limits_{i=2}^{U-1} \mu^{(r)}_{1}(i) \left( \log V_{\pi,i} + \sum\limits_{i^{\prime}=1}^{i-1} \log (1-V_{\pi,i^{\prime}})\right) \\ && \hspace{2cm} + \sum\limits_{i=1}^{U-1} \left({\gamma^{1}_{i}} -1\right)\log V_{\pi,i} + \sum\limits_{i=1}^{U-1} \left({\gamma^{2}_{i}} -1\right) \log(1-V_{\pi,i}) \\ &=& \sum\limits_{i=1}^{U-1} \left(\mu^{(r)}_{1}(i)+{\gamma^{1}_{i}}-1 \right)\log V_{\pi,i} \\ && \hspace{2cm} + \sum\limits_{i=1}^{U-1} \left( \sum\limits_{i^{\prime}=i+1}^{U}\mu^{(r)}_{1}(i^{\prime}) + {\gamma^{2}_{i}}-1 \right) \log(1-V_{\pi,i}). \end{array} $$

  Let

  $$\begin{array}{@{}rcl@{}} A_{i}^{(r)}=\mu_{1}^{(r)}(i)+{\gamma^{1}_{i}}-1,\hspace{0.5cm} B_{i}^{(r)}=\sum\limits_{i^{\prime}=i+1}^{U}\mu_{1}^{(r)}(i^{\prime})+{\gamma^{2}_{i}}-1 \end{array} $$

  for i=1, ⋯, U−1. Then we have

  $$\begin{array}{@{}rcl@{}} \frac{\partial}{\partial V_{\pi,i} } F(V_{\pi}) &=& \sum\limits_{i^{\prime}=1}^{U-1} \frac{\partial}{\partial V_{\pi,i}} \left(A_{i^{\prime}}^{(r)}\log V_{\pi,i^{\prime}}+B_{i^{\prime}}^{(r)} \log (1- V_{\pi,i^{\prime}}) \right)\\ &=&\frac{A_{i}^{(r)}}{V_{\pi,i}}-\frac{B_{i}^{(r)}}{1-V_{\pi,i}} =\frac{A_{i}^{(r)}-\left(A_{i}^{(r)}+B_{i}^{(r)}\right)V_{\pi,i}}{V_{\pi,i}(1-V_{\pi,i})} \end{array} $$

  (67)

  for i=1, ⋯, U−1. By setting $\frac {\partial }{\partial V_{\pi ,i} } F(V_{\pi })=0$, we have

  $$\begin{array}{@{}rcl@{}} V_{\pi,i}&=&\frac{A_{i}^{(r)}}{A_{i}^{(r)}+B_{i}^{(r)}}, \end{array} $$

  which implies

  $$\begin{array}{@{}rcl@{}} V_{\pi,i} = \frac{\mu_{1}^{(r)}(i)+{\gamma^{1}_{i}}-1}{{\sum}_{i^{\prime}=i}^{U}\mu^{(r)}_{1}(i^{\prime})+{\gamma^{1}_{i}}+{\gamma^{2}_{i}}-2} \end{array} $$

  (68)

  for i=1, ⋯, U−1. Note V _π,i satisfies 0<V _π,i <1, since we selected hyperparameters ${\gamma ^{1}_{i}}$ and ${\gamma ^{2}_{i}}$ to satisfy ${\gamma ^{1}_{i}}+{\gamma ^{2}_{i}} > 2$ for i=1, ⋯, U−1.

• Parameters

  a An update formula for state transition parameter a can be derived in a manner similar to that for π:

  $$\begin{array}{@{}rcl@{}} V_{a,ij} = \frac{{\sum}_{t=1}^{T-1}\epsilon_{t}^{(r)}(i,j)+\alpha^{1}_{ij}-1}{ {\sum}_{t=1}^{T-1}{\sum}_{j^{\prime}=j}^{U}\epsilon_{t}^{(r)}(i,j^{\prime})+\alpha^{1}_{ij}+\alpha^{2}_{ij}-2} \end{array} $$

  (69)

  for i=1, ⋯, U and j=1, ⋯, U−1. In this paper, we selected hyperparameters $\alpha ^{1}_{ij}$ and $\alpha ^{2}_{ij}$ to satisfy $\alpha ^{1}_{ij}+\alpha ^{2}_{ij} > 2$ for i=1, ⋯, U and j=1, ⋯, U−1.

• Parameters b

  Noticing that the prior for b is a finite-dimensional Dirichlet, we consider the terms containing $b_{f_{l}}$:

  $$\begin{array}{@{}rcl@{}} \begin{array}{lll} F(b_{f_{l}}, \lambda) &=& \sum\limits_{t=1}^{T} \sum\limits_{i=1}^{U} \sum\limits_{k=1}^{K_{l}} \mu_{t}^{(r)}(i) I(f_{l,t}=k) \log b_{f_{l}, ik} \\ &&+ \sum\limits_{i=1}^{U}\sum\limits_{k=1}^{K_{l}} (\beta_{f_{l}, ik}-1) \log b_{f_{l}, ik} - \lambda \left( \sum\limits_{k=1}^{K_{l}} b_{f_{l}, ik}-1\right) , \end{array} \end{array} $$

  (70)

  where λ is a Lagrange multiplier.⁶

Taking the derivative of this function with respect to $b_{f_{l}, ik}$ yields

$$\begin{array}{@{}rcl@{}} \frac{\partial}{\partial b_{f_{l},ik}} F(b_{f_{l}}, \lambda) &=& \sum\limits_{t=1}^{T} \frac{\mu_{t}^{(r)}(i)I(f_{l,t}=k) + \beta_{f_{l}, ik}-1 }{b_{f_{l}, ik}} - \lambda \\ \end{array} $$

for i=1, ⋯, U, k=1, ⋯, K _l , and l=1⋯L. By setting $\frac {\partial }{\partial b_{f_{l},ik}} F(b_{f_{l}}, \lambda )=0$, we have

$$\begin{array}{@{}rcl@{}} b_{f_{l}, ik} &=&\frac{1}{\lambda} \left(\sum\limits_{t=1}^{T} \mu_{t}^{(r)}(i)I(f_{l,t}=k) + \beta_{f_{l},ik}-1 \right) \end{array} $$

for i=1, ⋯, U, k=1, ⋯, K _l , and l=1⋯L. It follows from the constraints on $b_{f_{l}, ik}$ that

$$\begin{array}{@{}rcl@{}} \sum\limits_{k=1}^{K_{l}}b_{f_{l}, ik} &=& \frac{1}{\lambda} \left( \sum\limits_{t=1}^{T}\sum\limits_{k=1}^{K_{l}} \mu_{t}^{(r)}(i)I(f_{l,t}=k) + \sum\limits_{k=1}^{K_{l}}(\beta_{f_{l},ik}-1) \right) = 1 \end{array} $$

for i=1, ⋯, U and l=1⋯L. Then we have

$$\begin{array}{@{}rcl@{}} \lambda &=& \sum\limits_{t=1}^{T} \mu_{t}^{(r)}(i) + \sum\limits_{k=1}^{K_{l}}\beta_{f_{l},ik}-K_{l} \end{array} $$

for i=1, ⋯, U and l=1⋯L. Therefore, the update formula for $b_{f_{l},ik}$ is given by

$$\begin{array}{@{}rcl@{}} b_{f_{l},ik} = \frac{{\sum}_{t=1}^{T} \mu_{t}^{(r)}(i)I(f_{l,t}=k) + \beta_{f_{l},ik}-1}{{\sum}_{t=1}^{T} \mu_{t}^{(r)}(i) + {\sum}_{k^{\prime}=1}^{K_{l}}\beta_{f_{l}, ik^{\prime}}- K_{l}} \end{array} $$

(71)

for i=1, ⋯, U, k=1, ⋯, K _l , and l=1⋯L.

An update formula for b _e is derived in a similar manner as

$$\begin{array}{@{}rcl@{}} b_{e,im} = \frac{{\sum}_{t=1}^{T} \mu_{t}^{(r)}(i)I(e_{t}=m) + \beta_{e,im}-1}{{\sum}_{t=1}^{T} \mu_{t}^{(r)}(i) + {\sum}_{m^{\prime}=1}^{M+1}\beta_{e,im^{\prime}}- M-1}. \end{array} $$

(72)

for i=1, ⋯, U and m=1, ⋯, M+1.