Statistical Shape Model of Nested Structures Based on the Level Set

Abstract

We propose a method for constructing a multi-shape statistical shape model (SSM) for nested structures such that each is a subset or superset of another. The proposed method has potential application to any pair of shapes with an inclusive relationship. These types of shapes are often found in anatomy such as the brain and ventricle. Most existing multi-shape SSMs can be used to describe these nested shapes; however, none of them guarantees a correct inclusive relationship. The basic concept of the proposed method is to describe nested shapes by applying different thresholds to a single continuous real-valued function in an image space. We demonstrate that there exists a one-to-one mapping from an arbitrary pair of nested shapes to this type of function. We also demonstrate that this method can be easily extended to represent three or more nested structures. We demonstrate the effectiveness of proposed SSM using brain and ventricle volumes obtained from particular stages of human embryos. The performance of the SSM was evaluated in terms of generalization and specificity ability. Additionally, we measured leakage criteria to assess the ability to preserve inclusive relationships. A quantitative comparison of our SSM with conventional multi-shape SSMs demonstrates the superiority of the proposed method.

1 Introduction

Statistical shape models (SSMs) have played an important role in medical image segmentation. One crucial issue in construction of a SSM is the choice of shape representation method. Many researchers have attempted to describe shapes, as shown in a comprehensive review by Heimann and Meinzer [1], in both explicit and implicit ways. The explicit method includes a point distribution model (PDM) [2]. Implicit models include the level set distribution model (LSDM); the applications of this model to segmentation were reviewed by [3].

SSMs for multiple shapes have grown quickly as a research topic in recent years. One advantage of using multi-shape SSMs is that they can aid in segmentation by considering the relationships between neighboring structures. A number of researchers have proposed multi-shape SSMs using both explicit [4] and implicit [5, 6] shape representations. Okada et al. [4] developed PDM for eight abdominal organs, which considers the relationships between neighboring organs via an organ correlation graph. Tsai et al. [5] proposed a simple extension of LSDM for three brain structures by applying statistical analysis to the concatenated level set functions. Yang et al. [6] modeled the differences in the level set functions for neighboring shapes and applied them to a neighbor-constrained segmentation algorithm. However, these SSMs have a potential risk of creating anatomically incorrect overlap between neighboring organs.

Several authors researched probabilistic shape representations [7,8,9] in which the class with highest probability is assigned to each voxel. Such one-label-per-voxel descriptions can intrinsically rule out organ overlaps. Pohl et al. [7] proposed a logarithm of odds (LogOdds) based shape representation that embeds multiple level set functions into a vector space and relates them to probabilistic atlases (PA). Malcolm et al. [8] presented a label space that is a mapping of class labels to vertices on a regular simplex, which forms the basis of a convex linear structure. Changizi et al. [9] proposed an isometric log ratio (ILR) transformation, which is an isometric and bijective mapping between the simplex and the vector space.

One interesting subject in the field of statistical multi-shape modeling is the nesting of structures such that each shape is a subset or superset of another. This type of inclusive relationship is often found in anatomy, such as the brain and ventricle or bones and their medullary cavities. Most existing SSMs can be applied to nested shapes. For example, Frangi et al. [10] extended PDM to model multiple structures, including the outer surface of the left myocardium and the surface of the blood pool inside it. However, none of the existing SSMs for nested shapes guarantees a correct inclusive relationship for the generated shapes. Several authors [11, 12] proposed level set based active contour models for nested shapes, called multilayer level set methods, in which shapes are represented by several distinct level contours from the same level set function. However, it is difficult to define such function from an arbitrary training shapes, and construction of SSM is outside the scope of these studies.

The work presented in this paper focuses on a level set based SSM for nested structures. The basic concept of the proposed method is to embed nested shapes into multiple level contours from a single continuous real-valued function. In order to perform statistical shape analysis on the nested structures, we introduce a one-to-one mapping from an arbitrary pair of nested shapes to such a real-valued function. We also introduce extensions of this method to work with three or more nested structures. In contrast to conventional multi-shape representation techniques, the proposed method inherently preserves the inclusive relationship of nested structures. Another important aspect of our method is that the dimension of the shape representation is independent on the number of nested objects.

The proposed method was applied to the statistical shape modeling of the brain and ventricle of the human embryo. A quantitative comparison of the performance of the proposed SSM to conventional SSMs demonstrated the effectiveness of the proposed method.

2 Methods

2.1 Level-set Based Shape Representation

First, we introduce the basic concept of a level-set based shape representation. We define a shape as a closed set of points $S\subset \varOmega$ inside an object where $\varOmega \subset \mathbb {R}^d$ is a d-dimensional image domain. Suppose $\phi _S: \varOmega \mapsto \mathbb {R}$ is a signed distance function that maps a point $\mathbf {x}\in \varOmega$ to its signed distance from the surface $\partial S$, which has a negative sign inside the shape and a positive sign outside the shape:

$$\begin{aligned} \phi _S(\mathbf {x}) = {\left\{ \begin{array}{ll} -\min _{\mathbf {y} \in \partial S} {\left\| \mathbf {x} - \mathbf {y} \right\| } &{}\text {if } \mathbf {x} \in S\\ +\min _{\mathbf {y} \in \partial S} { \left\| \mathbf {x} - \mathbf {y} \right\| } &{}\text {if } \mathbf {x} \notin S\end{array}\right. }. \end{aligned}$$

(1)

In practice, the domain $\varOmega$ is a set of finite image grid points, and Eq. (1) is efficiently calculated for all points $\mathbf {x}\in \varOmega$ by using a Euclidean distance transform [13]. The shape S is embedded as the zero sublevel set of the function $\phi _S(\mathbf {x})$, i. e.,

$$\begin{aligned} S = \left\{ \mathbf {x}\in \varOmega \mid \phi _S(\mathbf {x}) < 0\right\} . \end{aligned}$$

(2)

We refer to this function $\phi _S(\mathbf {x})$ as the level set function (LSF) in this study.

2.2 Level Set Function for Two Nested Objects

This section extends the level-set based shape representation method for a pair of shapes that satisfy following properties: (i) one is a subset of the other, and (ii) their boundaries do not come in contact with one another. We denote the class of shape pairs to be modeled in this study as $\mathcal {X}$, which can be expressed as

$$\begin{aligned} \mathcal {X} = \{ (A, B) \mid \emptyset \ne A^{-} \subset B\subset \varOmega \}. \end{aligned}$$

(3)

Here, A and B are the closed sets, and $A^-=A\cup \partial {A}$ is the closure of A which is introduced to guarantee property (ii) above.

The basic concept of the proposed method is to embed a pair of nested shapes $(A, B)\in \mathcal {X}$ as sublevel sets of a continuous real-valued function $\psi _{A,B}:\mathbb {R}^3\rightarrow \mathbb {R}$, defined using two distinct levels, $t_1$ and $t_2$ ($t_1 < t_2$). Thus, we are required to design $\psi _{A,B}$, $t_1$, and $t_2$ such that the following equations are satisfied for any $(A, B)\in \mathcal {X}$:

$$\begin{aligned} A= \{ \mathbf {x} \in \varOmega \mid \psi _{A,B}(\mathbf {x})< t_1 \}, \ B= \{ \mathbf {x} \in \varOmega \mid \psi _{A,B}(\mathbf {x}) < t_2 \}. \end{aligned}$$

(4)

There are innumerable examples of the functions $\psi _{A,B}$, $t_1$, and $t_2$ that will satisfy Eq. (4), from which we choose one explicit solution. Let $(\phi _{A}, \phi _{B})$ be a pair of signed distance functions associated with $(A, B)\in \mathcal {X}$. The relationships between $\psi _{A,B}(\mathbf {x})$, $\phi _{A}(\mathbf {x})$, and $\phi _{B}(\mathbf {x})$, required to satisfy Eq. (4), are written as

$$\begin{aligned} \psi _{A,B}(\mathbf {x}) = t_1 \Leftrightarrow \phi _{A}(\mathbf {x}) = 0 , \quad \psi _{A,B}(\mathbf {x}) = t_2 \Leftrightarrow \phi _{B}(\mathbf {x}) = 0. \end{aligned}$$

(5)

Considering the polar factorization $(\phi _{A}(\mathbf {x}), \, \phi _{B}(\mathbf {x})) = \big (r(\mathbf {x})\cos (\theta (\mathbf {x})+\pi /4),\big .$ $\big . r(\mathbf {x})\sin (\theta (\mathbf {x})+\pi /4)\big )$, where $\theta (\mathbf {x}) = \arctan \frac{\phi _{A}(\mathbf {x})+\phi _{B}(\mathbf {x})}{\phi _{A}(\mathbf {x}) - \phi _{B}(\mathbf {x})}$ because $\phi _{A}(\mathbf {x})>\phi _{B}(\mathbf {x})$ for all $\mathbf {x}\in \varOmega$, the conditions of Eq. (5) can be rewritten as

$$\begin{aligned} \psi _{A,B}(\mathbf {x}) = t_1 \Leftrightarrow \theta (\mathbf {x}) = -\frac{\pi }{4} , \quad \psi _{A,B}(\mathbf {x}) = t_2 \Leftrightarrow \theta (\mathbf {x}) = +\frac{\pi }{4}. \end{aligned}$$

(6)

Thus, we can state that $(\psi _{A,B},t_1,t_2) = \left( \theta ,-\pi /4,+\pi /4\right)$ is one possible solution of Eq. (4). This type of transformation is analogous to the calculation of shape index from principal curvatures (see [14] for details). Finally, by generalizing this solution with an additional parameter $\lambda > 0$, we see the following:

$$\begin{aligned} \psi _{A,B}(\mathbf {x})&= \arctan \left( \lambda \cdot \frac{\phi _{A}(\mathbf {x})+\phi _{B}(\mathbf {x})}{\phi _{A}(\mathbf {x})-\phi _{B}(\mathbf {x})} \right) , \end{aligned}$$

(7)

$$\begin{aligned} (t_1, t_2)&= \left( -\arctan {\lambda }, \ +\arctan {\lambda }\right) . \end{aligned}$$

(8)

Fig. 1.

Relationship of $\psi _{A,B}(\mathbf {x})$ with $\phi _{A}(\mathbf {x})$ and $\phi _{B}(\mathbf {x})$ when $\lambda =1$. The value $\psi _{A,B}(\mathbf {x})$ is related to the counterclockwise angle on a coordinate system composed of ($\phi _{A}, \phi _{B})$.

The relationship between $\phi _{A}(\mathbf {x})$, $\phi _{B}(\mathbf {x})$, and $\psi _{A,B}(\mathbf {x})$, when $\lambda = 1$, is illustrated in Fig. 1. Figure 2 explains the influence of the parameter $\lambda$ on the function $\psi _{A,B}(\mathbf {x})$. The function $\psi _{A,B}(\mathbf {x})$ has two horizontal asymptotes at $\pm \pi /2$, and the parameter $\lambda$ defines the convergence speed. The parameter $\lambda$ should be chosen according to the amount of shape variation in the object to be modeled. $\lambda$ should be made smaller for objects with larger variation in order to relax the slope of the function $\psi _{A,B}(\mathbf {x})$, to aid in the correct capture of intersubject variability.

Fig. 2.

One-dimensional example of the function $\phi _{A}(\mathbf {x})$, $\phi _{B}(\mathbf {x})$ and $\psi _{A,B}(\mathbf {x})$. (a) Level set functions $\phi _{A}(\mathbf {x})$, $\phi _{B}(\mathbf {x})$. (b) Combined level set functions $\psi _{A,B}(\mathbf {x})$ for different parameters $\lambda \in \{0.2,0.5,1.0\}$. Horizontal lines indicate the thresholds for the functions.

2.3 Level Set Function for $k$ Nested Objects

The proposed shape representation can be extended to an arbitrary number of nested objects. Suppose we have a sequence of $k$ shapes $(S_{1}, \ldots , S_{k})$, where $(S_{i},S_{i+1})\in \mathcal {X}\ \forall i\in \left\{ 1,\ldots ,k-1\right\}$, and with thresholds $t_1< \cdots < t_k$. For convenience, in order to assign an arbitrary threshold $(t_i, t_{i+1})$ rather than $\left( -\arctan {\lambda }, \ +\arctan {\lambda }\right)$ to the function $\psi _{S_i,S_{i+1}}$, we apply a linear transformation to $\psi _{S_i,S_{i+1}}$ as

$$\begin{aligned} f_{i,{i+1}}(\mathbf {x}) = t_i + \frac{t_{i+1} - t_i}{2\arctan {\lambda }} \cdot \arctan \left( \lambda \cdot \frac{\phi _{S_i}(\mathbf {x})+\phi _{S_{i+1}}(\mathbf {x})}{\phi _{S_i}(\mathbf {x})-\phi _{S_{i+1}}(\mathbf {x})} \right) . \end{aligned}$$

(9)

Then, the level set function for $k$ nested shapes is defined by combining Eq. (9):

$$\begin{aligned} \psi _{S_{1},\ldots ,S_{k}}(\mathbf {x}) = {\left\{ \begin{array}{ll} f_{1,2}(\mathbf {x}) &{} \mathbf {x} \in S_{1}\\ f_{i,i+1}(\mathbf {x}) &{} \mathbf {x} \in S_{i+1} \setminus S_{i} \ (i\in \{1,\ldots ,k-1\}) \\ f_{k-1,k}(\mathbf {x}) &{} \mathbf {x} \notin {S_{k}} \\ \end{array}\right. }. \end{aligned}$$

(10)

3 Experiments

The proposed method was demonstrated in the context of statistical shape modeling of the brain and ventricle of a human embryo. This study has been approved by the Ethics Committee, Graduate School and Faculty of Medicine at Kyoto University (R0316). Two-fold cross-validation study was carried out on 60 sets of brain and ventricle labels, delineated on magnetic resonance (MR) microscope volumes derived from the Kyoto Collection of Human Embryos [15]. The MR volumes were acquired using T1-weighted spin echo sequences with a repetition time of 100 ms and an echo time of 10–16 ms in a system equipped with a 2.35-T/40-cm bore magnet [16]. We focused on the data with Carnegie stages of 15–20 that were selected based on the criteria from [17].

Prior to statistical shape modeling, shape labels were aligned by generalized Procrustes analysis involving translation, rotation, and scaling using 21 semi-automatically defined landmarks. The image size after alignment was $200\times 250\times 250$ with a 0.1 mm isotropic voxel size. The SSM was built through applying principal component analysis (PCA) on the discretized level set functions. The  parameter for the proposed shape representation was $\lambda = 0.5$, and the number of principal components was set to 3 throughout the experiment.

We refer to the proposed shape representation method as a single level set function (single LSF), which was compared to the conventional methods, “multiple LSF” and “LogOdds”. Multiple LSF is a simple extension of the level set for multiple shapes. It serially concatenates the vector of level set functions for the brain and ventricle in the same manner as [5]. LogOdds is the probabilistic shape representation method introduced by [7]. Note that the dimensionality of the shape representation of multiple LSF and LogOdds is k-times larger than that of our single LSF, where k is the number of nested objects.

The performance of the SSMs was measured using three criteria: generalization, specificity, and leakage. Generalization is the ability to reconstruct unknown shapes, and specificity is the ability to exclude unnatural shapes [18]. The similarity measure between nested objects $X = (X_\textit{int},X_\textit{ext})$ and $Y = (Y_\textit{int},Y_\textit{ext})$, which is required to calculate generalization and specificity, is defined as:

$$\begin{aligned} s(X,Y) = \textit{JI}(X_\textit{int},Y_\textit{int}) + \textit{JI}(X_\textit{ext},Y_\textit{ext}) \end{aligned}$$

(11)

where $\textit{JI}(\cdot ,\cdot )$ is the Jaccard index between two sets. Leakage is introduced to determine if the inclusive relationship holds for both the reconstructed shapes $\mathcal {R}$ and the randomly generated shapes $\mathcal {S}$, which is defined as $\frac{1}{|\mathcal {R}|}\sum _{X\in \mathcal {R}}{} \textit{Leak}(X)+\frac{1}{|\mathcal {S}|}\sum _{X\in \mathcal {S}}{} \textit{Leak}(X)$ and is desired to be zero. The value $\textit{Leak}(X)$ is a rough estimate of the area of the surface of the ventricle $X_\textit{int}$ that protrudes out of the brain $X_\textit{ext}$:

$$\begin{aligned} \textit{Leak}(X) = \left| \left\{ \mathbf {x} \in X_\textit{int} \mid \exists {\mathbf {y} \in \mathcal {N}_\mathbf {x}}\,(\mathbf {y} \notin X_\textit{ext})\right\} \right| \end{aligned}$$

(12)

where $\mathcal {N}_\mathbf {x}$ is the set of 6-neighborhood voxels of $\mathbf {x}$.

Fig. 3.

Comparison of SSM performance between single LSF (the proposed method), multiple LSF, and LogOdds using different criteria: (a) generalization, (b) specificity, and (c) leakage.

Figure 3 displays the comparative results of the proposed method (Single LSF) with the conventional methods for three criteria. The best performing SSM was our single LSF, which was superior to (or at least comparable to) multiple LSF and LogOdds for all criteria. The most notable feature of the proposed method is that it is free of leakage, which was verified by these results. In order to visualize the generalization capability, we display an example of reconstruction of the test brain and ventricle shapes in Fig. 4. The proposed method achieves superior shape reconstruction, which can be accounted for by the elimination of leakage observed around the area indicated by the red arrows in Fig. 4.

Fig. 4.

Example of the reconstruction of the brain (blue) and ventricle (yellow). From left to right, an input label volume (a), the reconstruction results of SSMs based on single LSF (b), multiple LSF (c), and LogOdds (d). The numerals indicate similarity between the reconstructed shape and the original one (cf. Eq. (11)), and the leakage of the reconstructed shape (cf. Eq. (12)).

4 Conclusion

We proposed a multi-shape SSM for nested structures. The effectiveness of our method was demonstrated in the context of statistical shape modeling of the brain and ventricle of a human embryo. Unlike traditional SSMs, the proposed SSM has the ability to preserve inclusive relationships and displayed superior performance with respect to generalization and specificity. In the future, we plan to incorporate the proposed SSM into the segmentation algorithms of various nested structures.