1 Introduction

Glioblastoma is the most common and aggressive malignant brain tumor that heavily and heterogeneously infiltrates surrounding tissue. This infiltration complicates treatment [1], as it is difficult to precisely localize the extent of infiltration. Considering that more than 80% of patients have a local tumor recurrence close to the initial resection cavity [2] (hence to the infiltrated brain tissue), we identify the need to accurately map correspondences between pre-operative (pre) and post-recurrence (post) brain tumor scans. Such registrations would support research into the early detection of tumor recurrence, e.g., enable the identification of subtle imaging phenotypic characteristics of tumor recurrence. Even though correspondences are established between longitudinal image-pairs, where scans are expected to be comparable (as they are of the same patient), registration is challenging due to there being two sources of image appearance changes: first, the pre scans contain tumors and mass effect deformations; second, the post scans typically contain tumor resection cavities (where the tumor used to be in the pre scan) and show signs of tumor infiltration and recurrence.

Registration in the presence of pathologies may employ cost function masking [3] to exclude regions without clear correspondences and hence avoid influencing the registration’s image similarity term. Alternatively, one could combine cost function masking with a model of infiltration and mass effect [4]. A joint segmentation-registration method has also been proposed [5, 6], that incorporates a tumor growth model and that estimates a patient-specific atlas to guide the segmentation and registration, while using 4 MRI modalities. Building upon this and considering that both pre and post scans are of the same patient, Kwon et al. [7] proposed a framework to jointly segment and register the post to the pre scans. However, it may be challenging to estimate the deformations of pre scans with large tumors and post scans with large mass effect relaxation. In addition, these segmentation-registration approaches require manual interaction in the form of seed-points to initialize the growth model and to model the intensity distribution of each brain tissue across modalities. This complicates the use for large-scale studies and hampers the clinical translatability of these methods.

To account for missing correspondences in pathological regions, an alternative strategy is to estimate a quasi-normal image by learning from population data and use it for registration. Quasi-normal images can, for example, be estimated by a low-rank/sparse (LRS) decomposition [8] or via deep variational encoder-decoder architectures [9]. However, these methods either blur the normal tissue appearance and compromise the registration results or require a large number of training images. Inspired by the LRS decomposition, Han et al. [10] proposed a joint PCA/image-reconstruction model, which also decomposes the pathological image into two parts: (1) normal tissue appearance is captured by a statistical (PCA) model; and (2) large pathologies are captured via a total-variation (TV) term, which avoids blurring of the normal tissue and retains fine details in the quasi-normal image. The reconstructed quasi-normal image is then used for atlas registration. One could directly apply this method independently to the pre and the post scans, and then register the resulting quasi-normal images. However, this strategy would ignore the fact that these scans come from the same patient and the statistical model in the atlas space may not adequately capture the normal appearance for a specific patient; consequently, the registration quality may be impaired. Similarly, Kwon et al. [11] extended their framework [7] by incorporating an inpainting strategy to account for pathological regions. However, prior knowledge about the tumor of each scan, comprising of seed-points with associated radii and initial intensity modeling of each brain tissue type, is required for the algorithm. This manual interactive step, in addition to introducing an extra burden to the method’s usability and increasing the time footprint of the method, also affects the objectivity and repeatability of the obtained results. All these together have a direct impact on the consideration of the method for potential clinical translation, as well as for large research studies. Therefore, a method combining the benefits of pathology modeling with patient-specificity, while eliminating manual interactions, is highly desirable.

Contributions. In this work, we present an automatic, repeatable, patient-specific registration approach for pre and post brain MRI scans that requires only a single modality. This is accomplished through careful adjustment of the PCA/image-reconstruction model [10]. In particular, we show how to (1) model each patient separately to improve the registration results and (2) leverage the decomposition’s TV term to intrinsically exclude the estimated pathology in case the image is not well-aligned to the target space.

Organization. Section 2 describes our patient-specific registration framework, Sect. 3 presents the qualitative and quantitative evaluations of our approach, compared against other state-of-the-art methods, and Sect. 4 concludes the paper with a discussion and an outlook on future work.

2 Methodology

We first present an overview of the low-rank/sparse approach [8] and the PCA-TV model [10]. We then propose modifications for patient-specific registration.

2.1 Low-Rank/Sparse (LRS) Decomposition

In the LRS approach, images \(\{I_i:i=1,2,...,n\}\) are first arranged as columns of a matrix \(I = [I_1, ..., I_n]\), where n describes the number of images. This matrix is then decomposed into a low-rank matrix \(L = [L_1, ..., L_n]\) and a sparse matrix \(S=[S_1, ..., S_n]\) by solving the problem:

$$\begin{aligned} \{L,S\} = \mathop {\text {arg min}}_{L,S} (\Vert L\Vert _* + \lambda \Vert S\Vert _1),\quad \text {s.t.}\quad I = L + S, \end{aligned}$$
(1)

where \(\Vert \cdot \Vert _*\) is the nuclear norm (i.e., a convex approximation of the rank), \(\Vert \cdot \Vert _1\) denotes the \(\ell _1\) norm, and \(\lambda \) weighs the penalty on sparse term. Liu et al. [8] proposed a low-rank-based registration method by alternating the LRS decomposition and registering the low-rank image to an atlas. Upon convergence, the low-rank matrix contains the normal information from all images, while the sparse matrix obtains the estimated pathology. The low-rank images are then used for registration. While effective, the approach suffers two shortcomings: First, it requires optimization over the entire population, which is ineffective and computationally expensive. Second, while it recovers normal appearance in pathological regions, normal tissue areas are blurred which may negatively affect registration results.

2.2 PCA-TV Model

Inspired by the LRS framework and the Rudin-Osher-Fatemi (ROF) image denoising model [12], Han et al. proposed a PCA-TV registration framework [10]. It registers all the “normal” images, i.e., images from healthy controls, to an atlas space only once, followed by a PCA on the warped normal images. The PCA basis is kept fixed, and the PCA-TV model decomposes the image by solving the following problem:

$$\begin{aligned} \{\hat{L}, T, \varvec{\alpha }\} = \mathop {\text {arg min}}_{\hat{L}, T, \varvec{\alpha }}\frac{1}{2}\Vert \hat{L}-B\varvec{\alpha }\Vert _2^2 + \gamma \Vert \nabla T\Vert _{2,1},\quad \text {s.t.}\quad \hat{I} = \hat{L} + T, \end{aligned}$$
(2)

where \(\hat{I}\) denotes the “pathological image”, i.e., an image with at least a pathology/tumor, after we subtract the mean. \(\Vert \nabla T\Vert _{2,1} = \sum _i\Vert \nabla T_i\Vert _2\), i is the spatial location, \(\{\varvec{\alpha }\}\) are the PCA coefficients and B is the PCA basis. The model consists of (1) a quasi-low-rank part \(\hat{L}\) that is close to the PCA space and retains image detail, and (2) a TV term, which captures pathologies that are large, spatially contiguous, and not expressed by the PCA basis. The quasi-normal image is obtained by adding the mean image to the quasi-low-rank image. Overall, this model is more effective than the LRS decomposition, as it works on just one image and explicitly leverages spatial information.

Additionally, an iterative regularization strategy can be used after the decomposition, just as for the ROF model [13]. In particular, after solving (2) and obtaining \(\tilde{L}_0=\hat{L}\) and \(\varvec{\alpha }_0\), for \(k\ge 1\), one can iteratively solve

$$\begin{aligned} \{\tilde{L}_k, T_k, \varvec{\alpha }_k\} = \mathop {\text {arg min}}_{\tilde{L_k}, T_k, \varvec{\alpha }_k}\frac{1}{2}\Vert \tilde{L}_{k}-B\varvec{\alpha }_{k}\Vert _2^2 + \gamma \Vert \nabla T_{k}\Vert _{2,1}\quad \text {s.t.}\quad \hat{I}_{k}=\tilde{L}_{k}+T_{k}, \end{aligned}$$
(3)

where \(\hat{I}_k = \hat{I} + \tilde{L}_{k-1} - B\varvec{\alpha }_{k-1}\). After N regularization steps, the TV term \(T_N\) captures the pathology and the quasi-low-rank term can be obtained by subtracting the TV term from the input image, i.e., \(\hat{L}_N = \hat{I} - \hat{T}_N\).

The entire framework iteratively alternates between the image decomposition and atlas registration. Each iteration includes the registration of the quasi-normal image to the atlas, the transformation of the input image to the atlas space, and the decomposition of the warped image in the atlas space. In addition, to avoid accumulating deformation errors, the quasi-normal image is always transformed back to the original image space prior to registration.

2.3 Patient-Specific Registration

When registering the pre to the post scan, one could simply apply the PCA-TV model (Sect. 2.2) independently on each scan and then register the corresponding quasi-normal images. However, this ignores that both scans are of the same patient. In addition, both the LRS and the PCA-TV approaches register quasi-normal images to the atlas during each iteration, but never use the sparse/TV information. In case an image contains tumors with large mass effect, it is drastically misaligned with the population images. Hence, the decomposition may not work sufficiently, unless the image is well-aligned with the atlas. This is especially true during the first iteration of registration and decomposition. To overcome these shortcomings and improve the registration of pre and post scans, we propose the following key adjustments to the PCA-TV model.

PCA-TV-Mask Model. When we compute the decomposition in the first iteration, the image is only affinely aligned to the atlas. We apply Otsu thresholding to the TV image, to obtain a coarse mask of the pathological region (TV-mask). This mask is then used during the registration, i.e., we register the quasi-normal image to the atlas, but use the TV-mask for cost-function masking of the tumor. Once the image is better aligned to the atlas via a deformable registration, we remove the TV-mask and use the entire quasi-normal image for registration. We refer to this improvement as the PCA-TV-mask model.

Patient-Specific PCA. Considering (i) that the post scan is relatively free from mass effects (e.g., except for scarring) and (ii) that the tumor resection cavity is easily modeled via the TV term, we propose the following two-step strategy. In the first step, we apply the PCA-TV-mask model to the post scan, resulting in a quasi-normal reconstructed image, in addition to registering the post scan to the atlas space. In the second step, we use the inverse transformation of the first step to map all normal images into the post scan space and then construct a new PCA basis from this data. Importantly, we can now use this new PCA basis together with the quasi-normal post image (now warped back to the patient space and used as atlas) to run the PCA-TV-mask model on the pre scan. Overall, this strategy allows direct registration between the pre and the post scans. Another advantage of using this patient-specific strategy is that by running PCA in the patient-specific space, the normal space spanned by the PCA basis is expected to be more consistent with the pre scan, which in turn improves the decomposition and registration results, when compared with the original framework.

Fig. 1.
figure 1

Boxplots of the mean landmark errors. For each method, the landmark errors are computed against the gold-standard. On each box, the red line is the median and the green star is the mean. The bottom and top edges of the box denote the 25th and 75th percentiles, respectively, the whiskers extend to the most extreme datas that are not considered outliers and the outliers are plotted in circle. (A) AFFINE; (B) GREEDY; (C) DRAMMS; (D) ANTs; (E) NiftyReg; (F) PCA-TV; (G) PCA-PS; (H) RATER. Our result is plotted in red. (Color figure online)

Full size image

3 Experiments

We evaluate our framework on 10 pairs of pre and post clinically-acquired scans of patients diagnosed with de novo (primary) glioblastoma. Each timepoint contains native (T1) and contrast-enhanced T1-weighted (T1-CE), T2-weighted and FLAIR MRI. All modalities of each patient are skull-stripped, bias-field corrected, and affinely co-registered to the pre T1-CE scan of this patient that describes a \(192\times 256\times 192\) volume with voxel size of \(0.977\times 0.977\times 1.0\) [mm\(^3]\). For quantitative evaluation, we use manually seeded landmarks from two clinical experts. The first expert placed 20 landmarks within 30[mm] from the tumor region and 30 landmarks outside the 30[mm] region in each pre scan. Then, both experts independently placed matching landmarks in the post scans. The landmarks placed by the first expert are considered the gold-standard and the ones placed by the second expert serve as a baseline comparison, referred to as RATER. In our experiments, we only use the T1 volumes from each patient and run 6 iterations of registration and decomposition. The remaining 3 modalities were only used by the experts for seeding the landmarks. We pick 100 normal images from OASIS [14] and select 50 as PCA basis. For registration, we use NiftyReg [15] as B-spline registration with the default settings and local normalized cross correlation as similarity measure (--lncc 40). The TV-mask is used in the first iteration when the image is only affinely aligned to the target image. After B-spline registration, we remove the TV-mask for subsequent iterations. We also apply the regularization steps in the last three iterations. \(\gamma \) in (2) and (3) for the decomposition model is chosen as 1 if no regularization step is used and 2 if regularization steps are used. We compare with AFFINE [16], GREEDY [17], DRAMMS [18], ANTS [19], NiftyReg [15] and PCA-TV [10]. Although PORTR [7] was specifically designed for this task, we did not include it in our analysis, as our intent is to compare methods that do not require multiple modalities or manual interaction and hence more easily translate to clinical use.

Table 1. Statistic results for all paired landmark errors in both regions. For each landmark, we calculate the paired error; i.e., we subtract the landmark error of compared method from the landmark error of our method. This is to calculate the improvement obtained by our method. For each compared method, we rank the paired landmark errors and show the statistics in the table. The green boxes indicate results where errors from our framework are smaller.
Full size table

We compute the mean landmark error for each region of each patient (Fig. 1) and we note that all deformable methods are better than affine registration, but worse than RATER. Compared to other deformable methods, our patient-specific approach improves the results in the close-to-tumor region. We also improve results in the region far away from the tumor, except when comparing with NiftyReg and the original PCA-TV model. In fact, as shown in Table 2, the improvements in the close-to-tumor region are statistically significant, assessed via a one-tailed paired Wilcoxon signed-rank test with a Benjamini-Hochberg procedure to control the false discovery rate at level \(\alpha =0.05\). For far-from-tumor regions, the results are only significant when compared to AFFINE and GREEDY. We also calculate the effect sizes with each paired rank test. Most of the tests result in large or medium effect sizes.

We also evaluate the statistics of the paired landmark errors in both regions (Table 1). For each landmark, we calculate the differences of the errors between our framework and competing methods. Compared to RATER, our method shows worse performance on more than 50% of the landmarks. However, when comparing to other automatic registration methods, although at some landmarks our method performs worse than others by less than 1.5[mm] near the tumor and 1[mm] far away from the tumor, as shown at 5% statistics, it shows better performance on more than 50% of the landmarks. In fact, the improvement, especially near the tumor, can be larger than 5[mm], as shown at 95% statistics in the table. Furthermore, on average, we perform better than other registration methods by 0.5[mm] near the tumor and by less than 0.2[mm] far away from the tumor. This is consistent with the green stars shown in Fig. 1. Our patient-specific method also improves over the PCA-TV model near the tumor which illustrates its utility and the benefit of the patient-specific model.

Table 2. p-values and effect sizes for one-tailed paired Wilcoxon signed-rank test. We compare all methods (except for RATER) with our patient specific framework. Green boxes indicate statistically significant results after false discovery rate correction or effect sizes that are at least medium (>0.3).
Full size table

Finally, Fig. 2 shows example results from three patients, where we register the pre to the post scans. For the PCA-TV model and our patient-specific PCA-PS model, we reconstruct the quasi-normal images from each patient which are used to guide the registrations. Although the visual differences between our method and the PCA-TV model are subtle, other results show that by modeling the pathologies registrations are qualitatively more accurate. Note that, Fig. 2(c) illustrates the T2-FLAIR scans for the post images, only for visualization purposes, to better depict the surgically-imposed cavities of these illustrated examples. All the applied registration methods use only the T1 volumes.

Fig. 2.
figure 2

Example registration results from three patients. (a) and (b) show the pre and the post T1 scans. (c) shows the post T2-FLAIR scans, only for visualization purposes. (d)-(i) show registration results of pre to post from GREEDY, DRAMMS, ANTs, NiftyReg, PCA-TV, and our patient-specific model, PCA-PS. In addition, (j) and (k) show the quasi-normal reconstructions of the pre and post scans, respectively. The red box highlights major differences. (Color figure online)

Full size image

4 Conclusion

We proposed a patient-specific registration framework based on a PCA-TV-mask model, which registers pre-operative and post-recurrence scans of the same patient. The framework uses the post scan, which is relatively free from mass effects, to build a patient-specific PCA basis, and directly registers the pre scan to the patient space. The validation results show that our framework is more effective than the PCA-TV model, as well as other registration methods that do not explicitly model pathologies. In addition, our framework does not require any manual interaction, neither in the form of segmentation nor as tumor seeding, and only requires a single modality. In future work we will explore our method for different diseases, for example, to register acute and chronic image pairs from patients with traumatic brain injuries.