Sparse Representation-Based Deformation Model for Atlas-Based Segmentation of Liver CT Images

Abstract

Liver segmentation in computed tomography (CT) images is a fundamental step for various computer-assisted clinical applications. However, automatic liver segmentation from CT images is still a challenging task. In this paper, we propose a novel non-parametric sparse representation-based deformation model (SRDM) for atlas-based liver segmentation framework using nonrigid registration based on free-form deformations (FFDs) model. Specifically, during atlas-based segmentation procedure, our proposed SRDM provides a regularization for the resulting deformation that maps the atlas to the space of the target image, constraining it to be a sparse linear combination of existing training deformations in a deformation repository. We evaluated our proposed method based on a set of 30 contrast-enhanced abdominal CT images, resulting in superior performance when compared to state-of-the-art atlas-based segmentation methods.

1 Introduction

Segmentation of the liver tissue in computed tomography (CT) images is a fundamental step for various clinical applications, such as computer-assisted liver cancer diagnosis and surgical planning. However, automatic liver segmentation from CT images is still a challenging task, because of large shape variability, adjacent organs with similar intensity values, and the presence of tumors.

In the literature, a variety of methods have been proposed for liver segmentation [3, 4]. Of these, one promising method is the atlas-based segmentation [7], where the prior knowledge about the liver tissue existence probability is incorporated. An atlas is typically consists of two parts: atlas intensity image and its corresponding atlas label image. After the atlas intensity image is warped to the space of the target image, the atlas label image is propagated to the target image using the resulting transformation and defined as the final segmentation result. Therefore, the main source of error for atlas-based segmentation comes from the employed registration method. To largely avoid these registration errors caused by the large variability of soft tissue (e.g., liver), statistical deformation models (SDMs) [8] is proposed to regularize the resulting transformation that maps the atlas to the space of the target image, constraining it to be within principal subspace of plausible transformation learned from the training images. However, SDMs are parametric models assuming normal distribution of data and only account for small dense Gaussian noise.

In this paper, inspired by the recently proposed Sparse Shape Composition (SSC) shape prior modeling method [13], we propose a novel non-parametric sparse representation-based deformation model (SRDM) for atlas-based liver segmentation framework using nonrigid registration based on free-form deformations (FFDs) model [9]. Specifically, during atlas-based segmentation procedure, our proposed SRDM provides a regularization for the resulting deformation that maps the atlas to the space of the target image, constraining it to be a sparse linear combination of existing training deformations in a deformation repository. Because the sparse gross errors are explicitly modeled in our method, it is robust against both small dense Gaussian noise and sparse gross registration errors. We applied our proposed method to segment the liver tissue based on a set of 30 contrast-enhanced abdominal CT images.

2 Background

In this section, we briefly describe the main idea of nonrigid registration using free-form deformations (FFDs) and statistical deformation models (SDMs), we refer the readers to [8, 9] for more details.

2.1 Nonrigid Registration Using Free-Form Deformations

Free-form deformations (FFDs) model based on B-splines [9] is a popular spline-based transformation model for nonrigid image registration. The main idea of FFDs is to deform a rectangular grid placed on the reference image by manipulating an underlying B-spline control points. The optimal transformation $\mathbf {T}$ is found by minimizing the following cost function:

$$\begin{aligned} \mathcal {C} = \mathcal {C}_{similarity}( I_{target}, \mathbf {T}(I_{source}) ) + \lambda \mathcal {C}_{smooth}(\mathbf {T}), \end{aligned}$$

(1)

which consists of two competing terms: image similarity (e.g., normalized mutual information (NMI) [10]) and transformation smoothness. The minimization problem can be iteratively solved by using gradient descent technique.

2.2 Statistical Deformation Models

Unlike the concept of statistical shape models (SSMs), in statistical deformation models (SDMs) [8], principal component analysis (PCA) is applied to the control points which form compact parameterization of the B-spline based FFDs rather than to the anatomical landmarks. Assume that we have a reference image $I_r$ and a set of k training data $\{I_i~|~i=1,2,...,k\}$. We can derive a set of k corresponding B-spline based FFDs $\{FFD_i~|~i=1,2,...,k\}$, which map the liver structure of the reference image $I_r$ to that of all other training data $I_i$, and each $FFD_i$ is represented by a control point vector $\mathbf {C}_i = (x_1, y_1, z_1, ..., x_{n_p}, y_{n_p}, z_{n_p})^T$ with $n_p$ underlying mesh of 3-D control points that parameterize the FFDs. We define the corresponding covariance matrix as: $\mathbf {S} = \frac{1}{k-1} \sum _{i=1}^{k} (\mathbf {C}_i - \mathbf {\bar{C}})(\mathbf {C}_i - \mathbf {\bar{C}})^T$, where $\mathbf {\bar{C}}$ is the mean control point vector (i.e., average deformation field) of all subjects: $\mathbf {\bar{C}} = \frac{1}{k} { \sum _{i=1}^{k} { \mathbf {C}_i } }$. Then the statistical deformation model (SDM) can be built by an eigen-decomposition on the covariance matrix $\mathbf {S}$:

$$\begin{aligned} \mathbf {S} = \mathbf {U}\mathbf {D}\mathbf {U}^T, \end{aligned}$$

(2)

where columns of matrix of $\mathbf {U}$ form the principal modes of variation $\phi _m$ (eigenvectors), and diagonal entries of $\mathbf {D}$ are their respective variances $\lambda _m$ (eigenvalues). Any valid deformation field of liver structure thus can be approximated by a linear combination of the first c modes of variation:

$$\begin{aligned} \mathbf {C} = \mathbf {\bar{C}} + \sum _{m=1}^{c} { b_m\phi _m }, \end{aligned}$$

(3)

where $c = \min \{ t ~ | ~ { \sum _{i=1}^{t} \lambda _t }/ { \sum _{i=1}^{k-1} \lambda _t } > 0.98 \}$, and $b_m$ is the deformation parameter constrained to the interval $b_m \in \left[ -3\sqrt{\lambda _m}, 3\sqrt{\lambda _m}\right]$.

3 Methods

In this section, we describe our proposed non-parametric sparse representation-based deformation model (SRDM) for atlas-based liver segmentation framework. Figure 1 shows the proposed segmentation framework, including training and testing phases.

Fig. 1.

The main workflow of our proposed liver segmentation framework, which consists of training and testing phases.

In training phase, the liver atlas and our proposed SRDM which is employed as the deformation prior model are learned (Sect. 3.1). In testing phase, the atlas-based method is used to segment the liver tissue in the target image, where the learned deformation prior model (i.e., SRDM) is employed to regularize the deformation that maps the atlas to the space of the target image (Sect. 3.2).

3.1 Sparse Representation-Based Deformation Model

Before we can actually learn our proposed SRDM from training images, we need to construct the liver atlas first.

Construction of Liver Atlas. For the construction of liver atlas, we employ an iterative procedure to avoid bias towards particular anatomy of the selected reference image.

Let $\{I_i~|~i = 1, . . ., K \}$ be the $K = 20$ preprocessed training images with corresponding label image $\{L_i~|~i = 1, . . ., K \}$. One image with a mean liver shape is selected from the training data ($I_1$ in our study) as the initial template image. All other images are affinely mapped and resampled to this image. To avoid bias towards the chosen template image, we used the Minimal Deformation Target (MDT) strategy [11] to derive the initial average template. We define the mean deformation of $I_1$ when nonrigidly mapped to all other training images as: $\bar{T}_1 = \frac{1}{K-1} \sum _{j \ne 1} T_{1j}$. Then the MDT template for image $I_1$ is defined as: $MDT_1 = \bar{T}_1 ( I_1 )$.

Ideally, $MDT_1$ can be directly employed as the final liver atlas. However, because the registration algorithm is topology preserving, $MDT_1$ is still biased towards the shape of the selected initial template image $I_1$ [11]. Therefore, we define $MDT_1$ as the initial average template, and all other training images are nonrigidly mapped and resampled to this image in an iterative fashion with a maximum iteration of 3. The FFD model based on B-splines is employed to perform all the nonrigid registration in this step.

Using this procedure, we obtain a set of K aligned training images $\{I_i^{'}~|~i = 1, . . ., K \}$ with corresponding label image $\{L_i^{'}~|~i = 1, . . ., K \}$. The liver atlas $( \bar{I}, \bar{L} )$ can thus be defined as the average of the aligned data:

$$\begin{aligned} \bar{I} \leftarrow \frac{ 1 }{ K } \sum _{i=1}^{K} { I_i^{'} },\quad \bar{L} \leftarrow \frac{ 1 }{ K } \sum _{i=1}^{K} { L_i^{'} }. \end{aligned}$$

(4)

Sparse Representation-Based Deformation Model. Once the liver atlas $( \bar{I}, \bar{L} )$ is derived, we can build our proposed non-parametric sparse representation-based deformation model (SRDM).

We first derive the training deformation data by nonrigidly registering the liver atlas to all the training images $\{I_i~|~i = 1, . . ., K \}$, which are preprocessed by affinely registering and resampling to $I_1$. The FFD model based on B-splines is also employed to perform the nonrigid registration. Through this procedure, we derive a set of K training deformations, which are represented by the control point vectors of the B-splines that parameterize the FFDs: $\{\mathbf {C}_i~|~i=1,2,...,K\}$.

Sparse Shape Composition (SSC) [13] is a sparse representation-based shape prior modeling method that implicitly incorporates the shape priors on-the-fly to overcome the main issues of traditional shape prior modeling method. Inspired by this method, we propose a novel non-parametric sparse representation-based deformation model (SRDM) to deal with registration errors caused by the large variability of liver tissue. Based on two sparsity properties of the input deformation ( i.e., the input deformation can be approximately represented by a sparse linear combination of existing training deformations in a deformation repository; the input deformation may include sparse gross errors), we can cast the deformation modeling as the following sparse optimization problem:

$$\begin{aligned} (\hat{\mathbf {x}}, \hat{\mathbf {e}}) = \mathop {\text {arg min}}\limits _{\mathbf {x}, \mathbf {e}} \Vert \mathbf {c} - \mathbf {D}\mathbf {x}-\mathbf {e}\Vert _2^2 +\lambda _1\Vert \mathbf {x}\Vert _1+\lambda _2\Vert \mathbf {e}\Vert _1, \end{aligned}$$

(5)

where $\mathbf {c} \in \mathbb {R}^{3N}$ (N is the number of B-spline control points) is an input deformation to be refined, $\mathbf {D} = [\mathbf {C}_1, \mathbf {C}_2, ..., \mathbf {C}_K ] \in \mathbb {R}^{3N \times K}$ is the deformation repository (i.e., the dictionary) that includes K training deformations $\mathbf {C}_i$, $\mathbf {x}\in \mathbb {R}^K$ denotes the coefficients of linear combination, $\mathbf {e} \in \mathbb {R}^{3N}$ represents the sparse gross errors, $\Vert \mathbf {v}\Vert _1 = \sum _{i} { |v_i| }$ is the $\ell _1$-norm of $\mathbf {v}$ that induces sparsity, $\Vert \mathbf {v}\Vert _2$ is the Euclidean norm of $\mathbf {v}$, $\lambda _1$ and $\lambda _2$ are positive parameters that control the sparsity of $\mathbf {x}$ and $\mathbf {e}$ respectively. Then the input deformation $\mathbf {c}$ is refined as $\mathbf {\hat{c}} = \mathbf {D} \hat{\mathbf {x}}$.

Equation 5 is the sparse minimization problem of our proposed SRDM. Because of the sparsity-inducing property of the $\ell _1$-norm, the solution of this optimization problem will be the sparsest, meaning that most entries in $\hat{\mathbf {x}}$ and $\hat{\mathbf {e}}$ will be zero. We employ $\mathbf {e}$ and the $\ell _2$-norm to explicitly model the sparse gross errors and the small dense Gaussian noise, respectively. Therefore, SRDM is robust against both large sparse registration errors and small dense Gaussian noise.

Among numerous optimization algorithms proposed in the literature [1], we employ the Fast Iterative Shrinkage Thresholding Algorithm (FISTA) [2] to solve the sparse optimization problem in Eq. 5, due to its efficiency and robustness. The global rate of convergence of FISTA is $O(1/k^2)$, where k is the iteration counter [2]. For a more detailed description of the FISTA method, we refer the reader to [2]. Also it has been shown that stable recovery of both $\mathbf {x}$ and $\mathbf {e}$ in Eq. 5 can be guaranteed [12].

During atlas-based segmentation procedure, our proposed SRDM provides a regularization for the resulting deformation $\mathbf {c}$ that maps the atlas to the space of the target image, constraining the resulting deformation to be a sparse linear combination of existing training deformations learned from the training data in a deformation repository.

3.2 Atlas-Based Liver Segmentation Framework

After the liver atlas and SRDM are learned, we employ atlas-based method to segment the liver tissue, which is mainly based on nonrigid registration from the atlas to the target image.

Given a target image $I_{new}$ to be segmented, which is preprocessed by affinely registering and resampling to $I_1$, the atlas intensity image $\bar{I}$ is nonrigidly warped to the space of preprocessed $I_{new}$ using FFD model based on B-splines. Then the found deformation $\mathbf {c}$ that maps the atlas intensity image to the space of the target image is regularized by the learned SRDM deformation prior model according to Eq. 5. Finally, the atlas label image $\bar{L}$ is deformed by using the regularized deformation $\mathbf {\hat{c}} = \mathbf {D} \hat{\mathbf {x}}$ instead of the original deformation $\mathbf {c}$, and we derive the finally propagated atlas label image $\bar{L}^{'}$. The final liver segmentation $L_{new}$ is then extracted by thresholding the propagated atlas label image $\bar{L}^{'}$ using Otsu’s method, followed by removing unconnected components and filling small holes via the closing operator.

4 Results

To quantify the accuracy and performance of our proposed method, we have tested it on a set of 30 contrast-enhanced abdominal CT images. 20 CT images with corresponding ground truth, coming from the public database SLIVER07-Train (http://www.sliver07.org), were used for training models. These 20 images have varied in-plane resolution between 0.58 mm and 0.81 mm and slice thickness between 0.7 mm and 5.0 mm. The remaining 10 CT images from our clinical partner with corresponding expert segmentations were used for testing the accuracy of our proposed method. These 10 images have varied in-plane resolution between 0.58 mm and 0.67 mm, and a slice thickness of 1.0 mm. Most of the datasets in this study are pathological, including tumors, metastases and cysts.

We compared the segmentation results with the ground truth by using the average symmetric surface distance (ASD) metric, which is defined as the average distance between the surfaces of two segmentation results. The ASD is given in millimeters, and smaller value means more accurate segmentation result. We refer the readers to [4] for the detailed definition of the ASD metric.

Our method were implemented in C++ on Linux platform, and tested on a PC with a 2.5 GHz Intel quad-core processor. The source code is available at http://github.com/ivanshih/SRDM. We used the elastix package (http://elastix.isi.uu.nl) [5] to perform both affine and FFD model-based nonrigid registration between atlas and all the input images by maximization of the normalized mutual information (NMI) [10]. In the nonrigid registration procedure, we employed three resolution levels and the spacing of the underlying B-spline control points in the finest resolution level is set to 10 mm. We employed the FISTA implemented in the open source optimization toolbox SPAMS (http://spams-devel.gforge.inria.fr) [6] to solve the sparse optimization problems in Eq. 5.

In our implementation, the parameters used in the liver segmentation framework were the same for all the test data. We set $\lambda _1 = 60$ (sparsity parameter for $\mathbf {x}$ in Eq. 5) and $\lambda _2 = 0.2$ (sparsity parameter for $\mathbf {e}$ in Eq. 5). We also compared our proposed SRDM model with the closely related statistical deformation model (SDM) [8]. In our implementation of SDM, 98 % of the total deformation variance in the datasets is captured by the used modes of variation.

4.1 Generalization Ability of the SRDM

We first evaluate the performance of our proposed SRDM based on the training data using the generalization ability measure.

The generalization ability quantifies the capability of the model to represent unknown deformations of the same object class. It can be measured by performing a series of leave-one-out tests on the training set, where the difference between the omitted training deformation $\mathbf {C}_i$ and the reconstructed deformation $\mathbf {C}_i^*$ of $\mathbf {C}_i$ is measured. The final result is defined as the average over the complete set of tests. To quantify the difference between two corresponding deformations, we employed the root-mean-square deviation (RMSD) based metrics. The generalization ability can be defined as:

$$\begin{aligned} \mathcal {G} = \frac{1}{K} \sum _{i=1}^{K} { \sqrt{ \frac{ \Vert \mathbf {C}_i^* - \mathbf {C}_i \Vert ^2}{3N} } }, \end{aligned}$$

(6)

where N is the number of B-spline control points. And smaller value of the generalization ability indicates better deformation modeling method.

We also compared our model with the SDM deformation prior model. Table 1 shows the quantitative comparisons of the generalization ability based on the SLIVER07-Train database. Compared with SDM, our proposed SRDM achieves a better generalization ability with a smaller variance.

Table 1. The mean and standard deviation of the generalization ability of the two different deformation prior modeling methods based on the SLIVER07-Train database.

4.2 Liver Segmentation from CT Images

To demonstrate the effectiveness of our proposed SRDM, we applied it to the application of segmenting the liver tissue in CT Images. Specifically, SRDM was used to regularize the resulting deformation $\mathbf {c}$ that mapped the atlas to the space of the target image during the atlas-based segmentation procedure. We also compared our model with two closely related methods: traditional unregularized FFD model and SDM regularized model. To make a fair comparison, the same atlas-based liver segmentation framework is employed in all compared methods. Only the deformation prior modeling method used to refine the resulting deformation $\mathbf {c}$ is different.

Fig. 2.

Visual comparisons of liver segmentation results between unregularized FFD model and our proposed SRDM deformation prior method in two difficult cases: adjacent organ (i.e., the heart) has similar intensity values as the live tissue (first row); and the livers contain large tumors (second row). Each figure shows one case. The red contour shows the ground truth, the yellow and blue contours show the results of the unregularized FFD model and our method, respectively.

Qualitative Results: Figures 2a and b compare the segmentation results between unregularized FFD model and our method in difficult cases where adjacent organ (i.e., the heart) has similar intensity values as the live tissue. The unregularized FFD model based method over-segments the live tissue and includes part of the heart with similar intensity values, while our method successfully excludes these regions. Figures 2c and d compare the segmentation results between unregularized FFD model and our method in difficult cases where the livers contain large tumors. The unregularized FFD model cannot restore regions containing large tumors, while our method successfully preserves and reconstructs these regions. The reason is that in our method, these segmentation errors of the unregularized FFD model are explicitly modeled and reconstructed as the sparse gross errors. Therefore, these experimental results demonstrate that our method is robust in overcoming both over- and under-segmentation.

Table 2. Quantitative comparisons of liver segmentation results between the unregularized FFD model and two different deformation prior modeling methods based on the testing database. The measure is represented as mean and standard deviation of the overall datasets.

Quantitative Results: Table 2 shows the quantitative comparisons of liver segmentation results between the unregularized FFD model and two different deformation prior modeling methods based on the testing database. The measure is represented as mean and standard deviation of the overall datasets. Of these compared methods, our method achieves the best accuracy and the smallest variances according to the ASD metric, indicating the robustness of our method against registration errors caused by the large variability of liver tissue. In particular, the mean ASD of our method is 1.72 mm. The mean ASD of the unregularized FFD model and SDM based method are 0.45 mm and nearly 0.2 mm higher than that of our method, respectively.

These experiments demonstrate that our proposed method is more accurate and robust than the other two methods in segmenting the liver tissue, especially for difficult cases where adjacent organs have similar intensity values as the live tissue and the livers contain large tumors.

5 Conclusion

In this paper, we have presented a novel non-parametric sparse representation-based deformation model (SRDM) for atlas-based liver segmentation framework. To deal with registration errors caused by the large variability of liver tissue, our proposed SRDM provides a regularization for the resulting deformation that maps the atlas to the space of the target image. We evaluated our proposed method for liver segmentation based on a set of 30 contrast-enhanced abdominal CT images and achieved promising results, especially for difficult cases where adjacent organs have similar intensity values as the live tissue and the livers contain large tumors. Comparison with unregularized FFD model and SDM shows the superior performance of our proposed method.

In the future, we plan to employ more training data to learn the atlas and SRDM, which we believe will further improve the segmentation accuracy.