Interactive Deep Metric Learning for Healthcare Cohort Discovery

Abstract

Given the continuous growth of large-scale complex electronic healthcare data, a data-driven healthcare cohort discovery facilitated by machine learning tools with domain expert knowledge is required to gain further insights of the healthcare system. Specifically, clustering plays a crucial role in healthcare cohort discovery, and metric learning is able to incorporate expert feedback to generate more fit-for-purpose clustering outputs. However, most of the existing metric learning methods assume all labelled instances already pre-exists, which is not always true in real-world applications. In addition, big data in healthcare also brings new challenges to metric learning on handling complex structured data. In this paper, we propose a novel systematic method, namely Interactive Deep Metric Learning (IDML), which uses an interactive process to iteratively incorporate feedback from domain experts to identify cohorts that are more relevant to a particular pre-defined purpose. Moreover, the proposed method leverages powerful deep learning-based embedding techniques to incrementally gain effective representations for the complex structures inherit in patient journey data. We experimentally evaluate the effectiveness of the proposed IDML using two public healthcare datasets. The proposed method has also been implemented into an interactive cohort discovery tool for a real-world application in healthcare.

Appendices

A LSTM Formulation

$$\begin{aligned} \begin{array}{l} i_t = \sigma (W^{i}v_t + U^{i}h_{t-1})\\ f_t = \sigma (W^{f}v_t + U^{f}h_{t-1})\\ o_t = \sigma (W^{o}v_t + U^{o}h_{t-1})\\ \widetilde{c}_{t} = tanh(W^{c}v_t + U^{c}h_{t-1})\\ c_{t} = f_t * {c}_{t-1} + i_{t} * \widetilde{c}_{t}\\ h_t = tanh(c_{t})*o_t \end{array} \end{aligned}$$

(6)

where $i_t,f_t, o_t$ are input, forget and output gates respectively. The gates are different neural networks that decide which information is allowed on the cell state. The gates can learn what information is relevant to keep or forget during training. $v_t$ is an input to a network at time t, $h_{t-1}$ is an output at time $t-1$ and ${c}_{t-1}$ is an internal cell state at $t-1$.

B Clustering Evaluation Measures

• The Normalized Mutual Information (NMI) is defined as:

  $$\begin{aligned} NMI (\widehat{K};K) = \dfrac{2\times I(\widehat{K};K)}{\left[ H( \widehat{K}) +H(K) \right] } \end{aligned}$$

  (7)

  where $I(\widehat{K};K)$ is the mutual information and the entropies $H(\widehat{K})$ and H(K) are used for normalizing the mutual information to be in the range of [0, 1]. The higher the NMI is, the better the corresponding clustering is.

• The Adjusted Rand Index (ARI) of clustering is defined as:

  $$\begin{aligned} ARI = \dfrac{RI - E[RI]}{max(RI) - E[RI]} \end{aligned}$$

  (8)

  where $RI = \dfrac{a+b}{C^{N}_{2}}$, a is the number of patient pairs coming from the same cohort and also grouped into same cluster, b is the number of patient pairs belonging to different cohorts and grouped into different clusters. N is the total number of patients. The range of ARI values is [−1, 1]. The higher the ARI is, the better the corresponding clustering is.

• The Purity of clustering is defined as:

  $$\begin{aligned} Purity (\widehat{K},K) = \dfrac{1}{N}\sum ^{|\widehat{K}|}_{i=1}\max _{j}\left| \widehat{k_{i}}\cap k_{j}\right| , \end{aligned}$$

  (9)

  where $\widehat{K}=\{\widehat{k_{1}}, \widehat{k_{2}},\dots , \widehat{k_{|\widehat{K}|}}\}$ is the set of clusters produced by the chosen clustering algorithms. $|\widehat{K}|$ is the total number of clusters. $K=\{k_{1}, k_{2},\dots , k_{|K|}\}$ is the group of patient cohorts (i.e. ground truth). $\max _{j}\left| \widehat{k_{i}}\cap k_{j}\right|$ is the size of the intersection of cluster $\widehat{k_{i}}$ and patient cohort $k_{j}$ which is most frequent inside. The range of Purity values is [0, 1]. The higher the Purity is, the better the corresponding clustering is.