A novel framework for evaluating the image accuracy of dynamic MRI and the application on accelerated breast DCE MRI

  • Yuan Le
  • Marcel Dominik Nickel
  • Stephan Kannengiesser
  • Berthold Kiefer
  • Bruce Spottiswoode
  • Brian Dale
  • Victor Soon
  • Chen Lin
Research Article

Abstract

Objective

To develop a novel framework for evaluating the accuracy of quantitative analysis on dynamic contrast-enhanced (DCE) MRI with a specific combination of imaging technique, scanning parameters, and scanner and software performance and to test this framework with breast DCE MRI with Time-resolved angiography WIth Stochastic Trajectories (TWIST).

Materials and methods

Realistic breast tumor phantoms were 3D printed as cavities and filled with solutions of MR contrast agent. Full k-space raw data of individual tumor phantoms and a uniform background phantom were acquired. DCE raw data were simulated by sorting the raw data according to TWIST view order and scaling the raw data according to the enhancement based on pharmaco-kinetic (PK) models. The measured spatial and temporal characteristics from the images reconstructed using the scanner software were compared with the original PK model (ground truth).

Results

Images could be reconstructed using the manufacturer’s platform with the modified ‘raw data.’ Compared with the ‘ground truth,’ the RMS error in all images was <10% in most cases. With increasing view-sharing acceleration, the error of the initial uptake slope decreased while the error of peak enhancement increased. Deviations of PK parameters varied with the type of enhancement.

Conclusion

A new framework has been developed and tested to more realistically evaluate the quantitative measurement errors caused by a combination of the imaging technique, parameters and scanner and software performance in DCE-MRI.

Keywords

View-sharing acceleration DCE-MRI Simulation Breast imaging Tumor model 

Introduction

Recently, there is an increasing interest in the quantitative analysis of dynamic contrast-enhanced (DCE) MRI to reduce the variability in diagnosis [1, 2, 3]. Such biomarkers include pharmacokinetic (PK) parameters (K trans, V e, etc.) and morphologic features (shape, volume, texture, etc.). For this purpose, the time course of the signal intensity needs to be measured as accurately as possible. One of the factors that impacts the accuracy of DCE-MRI is that accelerated MR image acquisition techniques with k-space undersampling are usually required to better balance the requirement of high temporal and spatial resolution. Examples of such techniques include the k-space view-sharing techniques such as Time-resolved angiography WIth Stochastic Trajectories (TWIST) [4], Time Resolved Imaging of Contrast KineticS (TRICKS) [5], and more advanced nonlinear reconstruction techniques such as HYPR [6, 7] and compressed sensing [8, 9]. These techniques, originally developed for time-resolved Contrast-Enhanced MR Angiography (CE-MRA), have been increasingly adapted for the acceleration of dynamic contrast-enhanced (DCE) MRI for the breast, liver and kidney to better balance the temporal and spatial resolution [10, 11, 12, 13, 14].

To ensure that the acquired images are accurate enough for quantitative analysis of certain temporal and spatial features, these imaging protocols using k-space undersampling techniques should be routinely evaluated based on the requirement of each specific clinical application and for each scanner. However, currently such image quality evaluations are rarely performed. This is partially due to the difficulty of performing repeated DCE-MRI acquisitions to compare the results with different view-sharing parameters. Such experiments would require infusion of contrast agent multiple times into human subjects or animals, which is both unethical and logistically challenging [14]. Moreover, the ‘ground truth’ of the enhancement is usually unknown in animal or human studies. On the other hand, no currently available phantom can accurately and reliably reproduce in vivo tissue perfusion and contrast kinetics. Another approach is to simulate tissue contrast enhancement using digital phantoms and generic image acquisition and/or reconstruction algorithms [14, 15]. They represent ideal situations without taking into account factors such as proprietary vender-specific data acquisition or reconstruction techniques or the scanner conditions such as SNR.

To provide scanner-specific image quality/measurement accuracy evaluation, a novel framework has been developed for DCE-MRI applications. Within the scope of this work, this framework was used to evaluate the errors in clinical breast DCE-MRI due to TWIST view-sharing acceleration as an example. However, this framework may be adapted to other imaging techniques or applications and to include additional error estimations.

This system includes:
  1. 1.

    A set of 3D printed tumor phantoms with geometries based on the digital diffusion limited aggregation (DLA) tumor model [16] and a dynamic contrast enhancement time course based on the Tofts model.

     
  2. 2.

    A software package that generates simulated dynamic contrast-enhanced DICOM images using the raw data acquired on, the view order generated by, and the image reconstruction pipeline from a clinical MRI system.

     
  3. 3.

    An evaluation package that compares the measured tumor model characteristics in the produced images with the “ground truth.”

     

We expect that this novel approach provides a more accurate evaluation of the scanner-specific error in spatial and temporal characteristics of the enhancement with respect to certain imaging techniques and protocol parameters, could serve as a self-check image quality control module for scanners, and in turn aids the standardization of the clinical DCE-MRI protocols.

Materials and methods

The phantom we used in this framework consists of two separate parts: the dynamic tumor phantom and static background phantom. This can be regarded as a synthetic or hybrid scan [1] to mimic the existence of both normal background tissue that shows very little or no enhancement, and tumors that enhance, in breast DCE-MRI, and with a known ‘ground truth’ enhancement. These tumor models and background phantom were scanned separately so that different enhancements can be applied to each of them.

Tumor model

The DLA model was first proposed by Witten and Sander [17] to describe the process of crystal formation. It describes the process of particles undergoing a random walk and aggregates into a cluster with an uneven and spiculated surface. Digital tumor phantoms based on the DLA model were first developed and tested for the evaluation of digital mammography and produced realistic mass appearances [16]. Compared with tumor models mapped from patient images, this approach avoided the impact of the imaging system on the ‘ground truth’ and provided more realistic estimation of tumor shape deviation in the final results.

The DLA growth started with a center particle (voxel) as the initial ‘sticky’ cluster. A pre-defined number of ‘particles (voxels)’ are launched, one by one, from launching spheres (a series of isocenter spheres with the center located at the center particle with predefined radii). Each of these particles moves with a random walk process with a predefined step size. If the particle finally arrives somewhere adjacent to the center cluster, with a certain predefined possibility (sticking possibility), it becomes aggregated to the cluster. If, instead of getting ‘stuck’ to the center cluster, the particle moves beyond a certain distance from the cluster, or it fails in the predefined sticking possibility, this particle is discarded. Figure 1a–c demonstrates the formation of a DLA model. According to [16], the size of the final cluster increases with an increasing number of particles and decreases with increasing step size between the launching spheres. An increase of the random walk step size results in a more concentrated cluster, and the sticking possibility impacts the density of the final cluster.
Fig. 1

ac The growth of the DLA model (a 2D illustration). a The first particle started out from the first launching sphere and finally stuck to the center after a random walk movement; b the second particle set out from the second launching sphere and stuck to the cluster (on the first particle); c the third particle moved outside the distance range and was discarded. d Contour of the final cluster with N = 8000, step size = 2, and sticking possibility = 0.9; e the convolution of the DLA cluster and a 1-mm-diameter sphere (final tumor model to 3D printer)

The resolution of the MR images is usually much lower than that of digital mammography (1–1.5 mm vs. 35 µm) [16]; therefore, our tumor models were generated with fewer particles (N = 8000), smaller step size (15 for tumor phantom 1 and 2 for tumor phantom 2), respectively, and sticking possibility = 0.9. The size of the tumor models were limited to <2 × 2 × 2 cm3. The radii of the launching spheres were defined as:
$$R\; = \; 2 {\text{ cm}}\; \times \;\left( { 1\; + \; 4 9\; \times \;\left( {n - 1} \right)/N} \right)/ 100);{\text{ for the }}n{\text{th}}\;{\text{particle}}.$$
(1)
In this study, the voxel (particle) size was 0.25 × 0.25 × 0.68 mm3. Figure 1d shows the DLA model for tumor 1 (step size = 15). To facilitate the 3D printing, these models were convolved with a sphere of 1 mm diameter as shown in Fig. 1e. The generated tumor model was then exported to STL (STereoLithography) files and modified according to the requirements for 3D printing using Meshlab (meshlab.sourceforge.net). The modified tumor models were then printed with Poly-Lactic Acid (PLA) plastic using a Prusa i3 3D printer [18] (Fig. 2a) as cavities inside small cubes (Fig. 2b, c). The cavities were subsequently filled with 0.2 mM Gd-BOPTA solution, forming two tumor phantoms for this study. At the same time, a uniform background phantom was created by filling a container with the same Gd-BOPTA solution.
Fig. 2

a Photo of the 3D printer; b, c tumor phantoms generated with DLA model; b tumor model 1 generated with a step of 15 and c tumor model 2 with a step of 2 and convolved with a 1 mm sphere; d, e 3D reconstructed high-resolution MR images of the physical tumor phantoms 1 (c) and 2 (d). The white rectangles at the upper left corner showed the cross-section high-resolution MR images that are used to reconstruct these 3D figures

To verify the tumor phantom design and construction, a set of high-resolution T 1-weighted gradient echo MR images (voxel size 0.18 × 0.18 × 0.68 mm3) was acquired for each phantom and the reconstructed 3D maximum intensity projections. The size of the phantoms measured from these images was 0.54 × 0.49 × 0.78 cm3 for tumor phantom 1 and 0.71 × 0.74 × 1.28 cm3 for tumor phantom 2 (Fig. 2d, e).

Simulation of dynamic acquisition

To simulate dynamic acquisitions, the acquired tumor phantom raw data were view-by-view multiplied with the enhancement at the corresponding time point according to selected enhancement curves.

The ‘ground truth’ enhancement due to the change of contrast concentration was determined according to Eqs. (1) and (2):

$$\frac{1}{{T_{1} \left( t \right)}}\; = \;\frac{1}{{T_{10} }}\; + \;r_{1} \; \times \;C_{t} \left( t \right),$$
(2)
$$S\left( t \right)\; \propto \;\frac{{{ \sin }\alpha (1 - {\text{e}}^{{ - {\text{TR}}/T_{1} )}} }}{{\left( {1 - { \cos }\alpha \; \times \;{\text{e}}^{{ - {\text{TR}}/T_{1} }} } \right)}}{\text{e}}^{{ - \frac{\text{TE}}{{T_{2}^{*} }}}} ,$$
(3)
where S(t) is the signal intensity, α is the flip angle, and r 1 is the relaxivity of the contrast agent. The enhancement was defined as:
$$E\left( t \right)\; = \;\frac{S \left( t \right)\; - \;S(t\; < \;0)}{S(t\; < \;0)}\; \times \;100\% ,$$
(4)
where S(t < 0) is the pre-contrast baseline signal intensity.
As shown in Fig. 3, raw data of multiple tumor phantoms can each be multiplied with different enhancement curves and finally added together for the final simulated dynamic raw data. To calculate the signal intensity change for each k-space view during a dynamic TWIST acquisition, the timing of each k-space view relative to the contrast enhancement, C t (t), needs to be determined. This was accomplished by executing the same pulse sequence in the Siemens Integrated Development Environment for Applications (IDEA). Simulated raw data sets were then generated for the protocol prescribed by the user based on the phantom raw data, and these data sets can be reconstructed into dynamic images on scanner. The final step of the image evaluation was then performed by comparing the enhancement at each voxel in the reconstructed images to the ‘ground truth.’ The image error indices were calculated as prescribed by the users (Fig. 4).
Fig. 3

Diagram of dynamic raw data generation. For each k-space view, the raw data of the corresponding tumor phantoms were multiplied by the signal intensity curves, after which the modified raw data were added to the background data to generate the final simulated dynamic raw data

Fig. 4

Diagram of the entire simulation and evaluation process, where tumor images are reconstructed from simulated dynamic raw data with different imaging parameters and the measured spatial and temporal characteristics are compared with the 'ground truth' in the pre-defined tumor models to determine the errors from the imaging system and the their dependence on imaging techniques and parameters

Application to TWIST view sharing

The tumor phantoms and background phantom were scanned separately on a 3-T clinical MR scanner (MAGNETOM Skyra, Siemens Healthcare, Erlangen, Germany) with a 20-channel head coil. A prototypical pulse sequence was used, which is a spoiled gradient echo sequence with Cartesian readout and TWIST view sharing [4] (Fig. 5a). The imaging parameters for static raw data acquisitions are TR/TE = 9/1.55 ms, FOV = 160 × 160 × 92 mm3; acquisition matrix = 192 × 192 × 88; partial phase resolution = 100%; partial slice resolution = 80%; flip angle (α) = 10°. Full k-space data acquisition was used so no TWIST undersampling was actually carried out. The TR was selected to keep the time of acquisition similar to the clinical breast DCE-MRI applications (about 1 min and 40 s for full k-space acquisition). The background phantom was scanned with one measurement and one average, while the tumor phantom’s raw data were an average of five measurements. This is to prevent the amplification of noise when the tumor phantom raw data are later scaled by the enhancement model and combined with background phantom raw data. By doing so, the noise level in the whole series of images will be kept consistent, which is true when the dynamic scan is actually performed. The tumor phantom raw data were then normalized according to the background phantom images in both phase and magnitude. The phase map of the tumor images was corrected to be the same as the corresponding background phantom images. The average magnitude in the tumor phantom was normalized to the average magnitude at the corresponding location in the background phantom image. In tumor regions with uniform and close-to-average signal intensity, this step ensured that the background phantom signal intensity provided a correct baseline signal for PK analysis for most of the voxels.
Fig. 5

a k-space views (represented as dots in kykz plane) are divided into central (A) and peripheral (B1 and B2) regions in TWIST; b view order of the simulated data ‘acquisition’ with TWIST sequence. The first and last image set were acquired with full k-space sampling (no undersampling/view sharing); for all other image sets in between, partial acquisition were used and view sharing was applied. Dots on the contrast enhancement curve represent k-space views; solid dots represent the k-space views in the central region (A) of kykz plane

In general, any type of enhancement can be used. In this study, the enhancement curves were generated according to the Tofts model [19]:
$$\begin{aligned} C_{t} \left( t \right)\; =\, & K^{\text{trans}} \mathop \int\limits_{0}^{t} C_{\text{p}} \left( \tau \right){\text{e}}^{{ - \left( {t - \tau } \right)k_{\text{ep}} }} {\text{d}}\tau , \\ & \;{\text{with}} \;K^{\text{trans}} \; = \;k_{\text{ep}} v_{\text{e}} . \\ \end{aligned}$$
(5)

The parameters we selected were (1) K trans = 0.3 min−1, v e = 0.3 and K trans = 0.03 min−1, v e = 0.1; these two parameter sets generated typical washout and persistent types of enhancement observed in breast DCE MRI [20]. The contrast concentration in the blood, C p(t), was generated according to the ‘intermediate’ arterial input function preset provided by Tissue4D [21].

For pre-contrast tissue T 1 (T 10) = 1666 ms [22, 23, 24] and contrast relaxivity (r 1) = 3.9 (s mM)−1 were used. The T 2* effect is assumed to be negligible in the scope of this work.

The imaging parameters such as the acquisition matrix size and TR were the same as used in the static tumor and background phantom raw data acquisition. The view order of the entire dynamic acquisition was generated with a series of view-sharing acceleration parameter combinations: the chosen percentage of the central k-space region size (pA) was 10, 15, 20, 33, 50, and 99% (no view sharing), and the peripheral k-space region update rate (pB) was 10, 20, 25, and 50%. The TWIST undersampling and the timing of simulated dynamic TWIST acquisition relative to contrast enhancement curves are illustrated in Fig. 5b.

For each tumor phantom and each enhancement curve, 20 dynamic raw data sets were generated with different combinations of pA and pB values as mentioned. The temporal resolution of each view-shared data set was summarized as in Table 1. The temporal resolution for pA = 99% is 101.6 s per frame.
Table 1

Temporal resolution of the generated DCE-MRI with various pA and pB (unit: s/frame)

pA

pB

10%

20%

25%

50%

10%

19.4

28.7

33.3

56.3

15%

24.2

32.8

37.1

58.9

20%

28.7

36.9

41.0

61.5

33%

40.7

47.6

51.0

68.1

50%

56.4

61.5

64.0

76.8

An ‘Ideal’ dynamic raw data set was also created by (1) generating a raw data set with 28 measurements (one every 16 s [25]) and (2) multiplying the tumor phantom raw data of each measurement with the enhancement at the corresponding time points of 0, 16, 32 s… so the enhancement was the same for all k-space views of each measurement. Since there is no signal variation due to the contrast wash-in/-out between the k-space views within one measurement, this approach allowed studying data processing effects caused by the image reconstruction such as conventional k-space filtering. Here it should be noted that the physical k-space footprint is typically larger than acquired matrix so that edge filtering conventionally needs to be applied for artifact suppression.

In breast DCE-MRI, the morphologic features of the enhanced tumor, especially at peak enhancement, provide important information for diagnosis. To quantify the error of tumor enhancement in the images produced from the simulation, the root mean square error of all tumor voxels during the entire dynamic series (RMSall) and the root mean square error of all tumor voxels at peak enhancement (or at 60 s post-contrast for persistent enhancement) (RMSpeak) were calculated.

RMSall was calculated as:

$${\text{RMS}}_{\text{all}} \; = \;\sqrt {\frac{1}{N}\mathop \sum \limits_{n\; = \;1}^{N} \frac{1}{T}\mathop \sum \limits_{t\; \in \;T} \left( {E_{\text{measured}} \left( t \right) - E_{\text{ground truth}} \left( t \right)} \right)^{2} ,}$$
(6)
where N is the number of voxels inside the tumor phantom volume, and T refers to the time points when k-space center views were acquired. E measured(t) is the measured enhancement from the simulated images at a specific time point, and E ground truth(t) is the ‘ground truth’ enhancement at the same time point.
RMSpeak was evaluated as [26, 27]:
$${\text{RMS}}_{\text{peak}} \; = \;\sqrt {\frac{1}{N}\mathop \sum \limits_{n\, = \;1}^{N} \left( {E_{\text{measured}} \left( {t\; = \;{\text{peak contrast}}} \right) - E_{\text{ground truth}} \left( {t\; = \;{\text{peak contrast}}} \right)} \right)^{2} .}$$
(7)
The initial slope of contrast uptake is associated with tumor vasculature and has been used as a criterion to differentiate between malignant versus benign tumors [28]. To evaluate the deviation of measured temporal characteristics of dynamic contrast enhancement, the RMS error of the initial uptake slope over the entire tumor phantom volume (RMSslope) was calculated as:
$${\text{RMS}}_{\text{slope}} \; = \;\sqrt {\frac{1}{N}\sum\limits_{n\; = \;1}^{N} {\left( {{\text{Slope}}_{\text{measured}} - {\text{Slope}}_{{{\text{ground}}{\kern 1pt} {\kern 1pt} {\text{truth}}}} } \right)^{2} } } .$$
(8)

The initial uptakes slopes (Slopemeasured and Slopeground truth) were determined as the maximum slopes from the linear regression of any three consecutive time points during the initial uptake period.

The impact of TWIST view sharing on tumor morphologic features at peak enhancement was evaluated by comparing the irregularity measured from the peak enhancement images and the ‘ground truth.’ While there are different ways to define irregularity, within the scope of this work, the irregularity was calculated as [29]:

$$\begin{aligned} {\text{Irregularity}}\;{ = }\; & 1- \frac{{\pi \; \times \;{\text{Effective diameter}}^{ 2} }}{\text{Surface area of the tumor}} \\ {\text{With effective diameter}}\;{ = }\; & 2\; \times \;\sqrt[ 3]{{\frac{{ 3\; \times \;{\text{Tumor volume}}}}{ 4\pi }}}, \\ \end{aligned}$$
(9)
where tumor volume was calculated as the voxel numbers in the tumor model, and the surface was calculated by applying Heron’s formula on the surface object generated by the function ‘isosurface’ in MATLAB. All the error indices were evaluated offline using custom MATLAB (Mathworks, Natick, MA, USA) programs. The results were plotted against pA and pB values for each enhancement curve type and each tumor phantom.

Quantitative PK analysis is not routinely performed in clinical breast MRI, but recent studies suggested that such analysis could be beneficial [22]. Therefore, besides the qualitative analyses that are routinely performed in clinical practice, quantitative PK analysis was also performed using the Tissue 4D software package (Siemens Healthcare, Erlangen, Germany) on some of the data sets with sufficient temporal resolution, such as the raw data sets with pA = 10% and pB = 10–25%; pA = 15% and pB = 10–20%; and pA = 20% and pB = 10% as well as the images reconstructed from the ‘Ideal’ raw data. In tumor phantom 1, the relatively large inhomogeneity in the small tumor phantom causes bias when analyzing the error in PK parameters. This variation was taken into consideration in our curve shape evaluation. With commercial software, we are not able to take this into consideration, which caused inaccurate results. Therefore, for qualitative evaluation we only performed the PK analysis in tumor phantom 2. In each case, an elliptical ROI was defined manually in the tumor phantom for each data set; the mean values of K trans and v e within the ROI were calculated.

Results

Simulated dynamic images

DICOM images were successfully reconstructed from the simulated raw data sets using the same reconstruction software on the scanner. An image of background phantom (Fig. 6a) and images reconstructed from the combined background and tumor phantom raw data (Fig. 6b, c for tumor phantom 1 and tumor phantom 2, respectively) showed tumor phantoms with similar overall shapes as those in Fig. 2. Figure 6d, e shows that in tumor phantom 2, with persistent enhancement, the deviation from ‘ground truth’ in average enhancement caused by TWIST view sharing was almost negligible compared with the inherent error in the ‘Ideal’ data. With wash-out enhancement, however, the deviation caused by TWIST view sharing was more prominent near the peak enhancement.
Fig. 6

a Image of the background phantom; b image of the background embedded with tumor phantom 1; c image of the background embedded with tumor phantom 2; d the signal intensity and tumor phantom 2 changes along the persistent curve; e the signal intensity of tumor phantom 2 changes along the wash-out curve. In (d) and (e), the solid lines represent the ‘real’ enhancement, and the dashed lines show the enhancement measured in the ‘Ideal’ data set. The marks represent the measured enhancement in images with various pA and pB values inside the volume of the tumor phantoms at the respective time to center

Quantitative results of error indices

Figure 7 shows the error estimation for the typical persistent curve with K trans = 0.03, v e = 0.1 and wash-out curve with K trans = 0.3, v e = 0.3. Figure 7a–h shows that for both tumor phantoms, the RMSall and RMSpeak with the selected pA and pB values for persistent enhancement were 1–2%, while error was larger in general for wash-out enhancement (1–9.0%). The minimal RMSall for persistent enhancement was achieved at pA ~10% and pB ~10% (0.86% for phantom 1 and 1.0% for phantom 2) and for wash-out enhancement at pA ~10% and pB ~10% (2.4% for phantom 1 and 3.1% for phantom 2). This was the same for both phantom 1 and 2. Local minima were found for RMSpeak at either pA ~10% and pB ~50% or pA = 99% (no view sharing), which is 0.98% for phantom 1 and persistent at both pA ~10% and pB ~50%, 1.0% for phantom 2 and persistent for pA ~10% and pB ~50%, 0.24% for phantom 1 and wash-out at pA = 99%, 0.43% for phantom 2 and wash-out at pA = 99%.
Fig. 7

Errors in TWIST images for tumor phantoms 1 and 2, with persistent (K trans = 0.03, v e = 0.1) and wash-out (K trans = 0.3, v e = 0.3) type of enhancement, respectively. The measurement error in the ‘Ideal’ data set was labeled on the plots. RMSall (ad) and RMSpeak (eh) for both tumor phantoms and all selected pA and pB were small in general and less in general for the persistent enhancement. RMSslope (il) was less for higher temporal resolution protocols, having lower pA and pB. Irregularity (mp) was underestimated (negative error) for lower pB values in tumor phantoms of wash-out enhancement and overestimated (positive error) in tumor phantoms of persistent enhancement for most of the combinations of pA and pB values

Figure 7i–l shows RMSslope as a function of pA and pB. Since RMSslope can only be calculated when there are at least three data sets acquired before the peak of contrast enhancement, RMSslope is not available for higher pA/pB. RMSslope was lower for higher temporal resolution (lower pA and pB) in general. The lowest RMSslope error occurred at pA = 10% and pB = 10%, which is 8.6% for phantom 1 and persistent, 8.1% for phantom 2 and persistent, 12% for phantom 1 and wash-out, and 12% for phantom 2 and wash-out. For this error index, the error is lower in phantom 2 than phantom 1.

Figure 7m–p shows irregularity as a function of pA and pB. The measured irregularity was less than the irregularity of the original tumor phantom for both ‘Ideal’ data and TWIST view-shared data. The minimum irregularity error value was 0.47% with pA = 15% and pB = 50% for phantom 1 and persistent enhancement and −2.01% with pA = 10% and pB = 50% for phantom 2 and persistent enhancement; this error index was −2.8% with pA = 99% for phantom 1 and wash-out enhancement and 0.1% with pA = 15% and pB = 50% for phantom 2 and wash-out enhancement. In general, the irregularity error was less for the wash-out type of enhancement, and the errors from the two phantoms were in the same range. The irregularity error was less predictable compared with the RMS error indices.

For all the error indices evaluated except irregularity, with each type of enhancement, the minimal and maximal error occurred with similar pA and pB for both tumor phantoms. However, the magnitude of error strongly depended on the tumor phantom morphology. Comparing the two tumor phantoms used in this study, tumor phantom 1 is smaller but smoother; the measured error indices are in general smaller than those measured for tumor phantom 2.

For the PK analysis of persistent curve with K trans = 0.03, v e = 0.1 and wash-out curve with K trans = 0.3, v e = 0.3 (Fig. 8; Table 2), even in the ‘Ideal’ data set, the measured K trans deviated from the ‘ground truth’ by 6–10%, and the same effect was found for v e. The PK parameters were sensitive to the error in the signal intensity, and this error was highly dependent on the shape of the phantoms. The error was also found to be the lowest with the highest temporal resolution (pA = 10%, pB = 10%) in general. With pA = 10% and pB = 25%, the curve fitting failed for the curve with K trans = 0.3, v e = 0.3.
Fig. 8

Estimated K trans maps with various pA and pB values in tumor 2 using the Tissue 4D package. The calculated PK parameters were zoomed from the original images. ac K trans = 0.03 min−1, v e = 0.1; df K trans = 0.3 min−1, v e = 0.3; (a, d) ‘Ideal’ (b, e) pA = 10%, pB = 10%; (c, f) pA = 15%, pB = 20%

Table 2

Estimated K trans and v e with different values of pA and pB (phantom 2)

Input K trans = 0.03, unit min−1

Input v e = 0.1

Measurement from ‘Ideal’ 0.027 ± 0.001

Measurement from ‘Ideal’ 0.090 ± 0.005

pA

pB

pA

pB

10%

20%

25%

10%

20%

25%

10%

0.026 ± 0.002

0.029 ± 0.002

0.027 ± 0.001

10%

0.089 ± 0.005

0.084 ± 0.007

0.088 ± 0.004

15%

0.027 ± 0.001

0.027 ± 0.001

15%

0.092 ± 0.005

0.088 ± 0.005

20%

0.027 ± 0.001

20%

0.085 ± 0.006

Input K trans = 0.3, unit min−1

Input v e = 0.3

Measurement from ‘Ideal’ 0.28 ± 0.02

Measurement from ‘Ideal’ 0.27 ± 0.02

pA

pB

pA

pB

10%

20%

25%

10%

20%

25%

10%

0.28 ± 0.03

0.27 ± 0.02

10%

0.25 ± 0.02

0.24 ± 0.02

15%

0.27 ± 0.02

0.26 ± 0.02

15%

0.25 ± 0.02

0.25 ± 0.01

20%

0.25 ± 0.02

20%

0.25 ± 0.01

Discussion

A novel image quality evaluation system has been developed for evaluating the accuracy of dynamic contrast-enhanced images acquired with view-sharing acceleration. The tumor phantoms constructed based on DLA models have spiculated margins and looked realistic in MRI images, mimicking the shape and texture of malignant breast tumors better than spherical phantoms [30, 31]. These data sets were reconstructed using the actual raw data and actual image reconstruction system of the scanner. Such a strategy has several unique advantages over previous works. More realistic tumor phantoms reveal the temporal and spatial behavior of the imaging technique in real clinical applications better than spherical or elliptical phantoms. The raw data used for simulation were acquired on a clinical scanner, which was impacted by ‘real’ limitations of the actual scanner situation. Using a replicate of the same reconstruction pipeline on the scanner ensured that the images produced in the simulation were as close to the ‘real’ images from a specific scanner as possible and revealed the overall error and impact of the image reconstruction algorithm on that scanner. From our results, we found that even in the ‘Ideal’ images, there was a detectable deviation in the curve RMS estimation. This may be caused by k-space filters that naturally act more strongly on smaller objects compared to a homogeneous background and the nonlinear effect in the calculation of the magnitude image. This type of deviation will not be detected with other generic simulation studies; therefore, we believe such simulation provides realistic evaluation of the errors in the measured morphologic and dynamic features in DCE-MRI by comparing them with the ground truth in the model.

For the purpose of image quality evaluation, tumor phantoms can be generated through one of these three approaches: (1) digital reference objects inserted in acquired or simulated images [16]; (2) physical phantoms of known geometry constructed to mimic the shape and property of tumors [32]. Either type can be generated using mathematical models or from patient data [1]. To generate a tumor model using patient images, one would then need to independently measure its geometry to know the ground truth. There are pros and cons of each approach. We choose the DLA tumor model because with breast DCE-MRI, the details of tumor shapes may be lost because of the temporal-spatial resolution limit, while to test image quality, we need a tumor phantom with these details preserved. With 3D printed tumor models, static images instead of DCE-MRI images are used as ‘ground truth;’ therefore, the loss of details due to the limited temporal resolution can be avoided.

As a test of this image quality evaluation system, this method was applied to TWIST breast DCE-MRI. For this purpose, both the tumor phantoms and the imaging protocols were chosen to be appropriate for breast DCE-MRI. The results suggested that with the scanner we selected, the TWIST view sharing can provide contrast enhancement measurement with low overall RMS error even in those tumors with irregular shape [14]. The impact of TWIST view sharing on shape irregularity was found to be less predictable and depends on both the tumor shape and enhancement type. Such information could be very helpful in sequence and parameter selection when similar morphologic analysis is to be performed [29, 33, 34, 35, 36]. Our results also showed that even in ‘Ideal’ images, the irregularity errors were larger and less predictable compared with RMS errors.

With lower K trans, the RMS error of the signal intensity in the TWIST view-sharing data set was very close to the error in the ‘Ideal’ data set, indicating that the additional deviation caused by TWIST view sharing was less than the deviation caused by additional processing in the image reconstruction. This finding suggests that, when optimizing DCE-MRI, both the error due to undersampling and the error from image reconstruction should be taken into consideration in a balanced approach.

Errors in PK parameter estimation were less predictable than the RMS errors of the signal intensity. Besides the nonlinear curve fitting process in PK analysis, there is another source of error in that the baseline signal intensity of the tumor phantom cannot be matched voxel-by-voxel to the background phantom, which causes an error in pre-contrast intensity and, consequently, the estimation of relative enhancement and contrast agent concentration. The shape of phantoms caused a complex partial volume effect and susceptibility artifact, making the signal level highly heterogeneous inside the phantom. Therefore, it can be challenging to correct the signal magnitude in the physical tumor phantom to exactly match the signal magnitude in the corresponding location in the background phantom for each voxel or even for most of the voxels. While such discrepancy will not affect qualitative curve shape evaluation, it did caused over- and underestimation of the enhancement in different tumor regions. In general, the Tofts model parameters appear to be rather sensitive to signal variations. This is one of the limitations of this study; a digital phantom set instead of physical phantoms may solve this problem. Another option is to use larger phantoms with sizes similar to typical breast tumors.

There are several other limitations to this work. The susceptibility difference between the saline-filled cavity and poly-lactic acid (PLA) material in the 3D printed tumor phantoms may cause artifacts at the tumor phantom boundary. Even with the phase correction, such artifacts may contribute to the deviation of the measured tumor phantom geometry from the tumor model. However, the contribution from susceptibility artifacts is expected to be the same for different view-sharing parameters. The susceptibility artifacts can be reduced by increasing the receiver bandwidth or shortening TE. Another alternative would be to use material with less susceptibility difference in 3D printing. Another possible improvement is in the design of the tumor phantoms. A ‘tumor’ model could be constructed with several ‘sub-regions’ separately, each with different enhancement features, to simulate the heterogeneity and texture of the real breast tumor [16, 37]. The enhancement curve sets can also be increased to cover the possible enhancement curve types of various breast tumor types.

We used population-averaged AIF when performing breast DCE-MRI image evaluation. This is because breast DCE-MRI typically does not have sufficient temporal resolution to support individual AIF measurement. Due to the challenges in obtaining patient-specific AIF, it is often not used in clinical practice and not an option in some DCE-MRI analysis software including Tissue4D. Potential errors in the AIF will be taken into account in the future improvement by adding an AIF phantom into the framework.

Slope measurement and quantitative analysis were only performed on data sets with adequate temporal resolution. If the temporal resolution further decreases, the results can become unstable as the uncertainty of the contrast arrival time relative to sampling points may affect the results dramatically. Performing additional analysis by shifting sampling points may offer useful information regarding this problem. However, such study is beyond the scope of this work, but will be considered in the future.

When evaluating the impact of the TWIST view sharing on the quality of breast DCE-MRI using the proposed framework, the simulations were performed with one set of raw data acquired with a specific SNR. Although we believe that the SNR was sufficient and the impact of noise would not affect the final results and conclusions, the exact uncertainty caused by noise is unknown. A future study with simulations of varying degrees of noise is needed to evaluate the impact of SNR.

For future studies, raw data acquired from healthy volunteers instead of a uniform background phantom could be used [16]. Furthermore, the appearance of tumor phantoms could be evaluated by radiologists, as done in [16]. Similarly, the simulated dynamic images, including those at ‘peak contrast,’ could also be reviewed by radiologists to evaluate how the errors would impact the diagnoses.

The parameters for the enhancement curves can be optimized to represent the most ‘demanding’ curves in breast DCE-MRI so that people can be sure that if the errors we get with such curves and phantoms are acceptable, with other possible enhancement curves the results will most probably be better. In our study, we found that with higher K trans and v e, the RMS of the whole curve or at the peak contrast were the highest, but with high K trans and low v e the slope and irregularity errors were higher. Based on such results, an optimized enhancement curve ‘group’ can be selected to represent most of the ‘difficult’ cases so that the results of the image quality evaluation can be more conservative.

We selected the RMS error of all measurements, peak enhancement measurement, and the initial enhancement slope as the indices to characterize the fidelity of DCE MRI measurement. This is because in the current clinical applications, qualitative evaluation was performed according to BI-RADS. The accuracy of the curve shape, peak contrast tumor shape, and speed of initial contrast uptake are therefore important indices. In this study, the imaging protocol was set up as typically used in current clinical breast DCE-MRI, so PK analysis was only performed with the seven high-temporal-resolution data sets. Even so, the temporal resolution was not optimal for the PK analysis; therefore, the estimated K trans and v e may be affected.

On the other hand, quantitative analysis of DCE-MRI using PK models such as the Tofts model, or a more advanced Shutter-Speed model, has been shown to provide valuable diagnostic information [22, 24], and one of the motivations for applying acceleration techniques is to allow for quantitative PK analysis in the future. Therefore, in future studies, the imaging protocol can be further optimized for the PK analysis, and PK analysis can be improved with ROIs defined automatically based on the segmentation of the phantoms.

Enhancement, K trans, and v e variations inside the tumor model in post-contrast images were observed in both tumor models, demonstrating that the ‘filtering’ effect of the undersampling technique was detected by this framework. Such variation, unfortunately, was difficult to quantify in the Tissue4D software because it does not provide irregular ROI selection or tumor area segmentation. Therefore, estimating the variation within tumor area was not possible. Other than this undersampling effect, we consider the detection of the error in the ‘Ideal’ image set to be very helpful in evaluating the performance of the scanner, the data acquisition, and the accuracy of the image reconstruction. With generic computer simulation, we will usually not be able to get this information.

Conclusion

In summary, a new approach has been demonstrated to evaluate the image fidelity for the application of breast DCE-MRI acquired with fast imaging techniques using physical tumor phantoms and image reconstruction pipelines on the commercial MR scanners. Initial work with the TWIST sequence showed that in typical clinical breast DCE-MRI applications, the error introduced by the TWIST view sharing was relatively low in general compared with the error induced by the image reconstruction, while the error in morphologic features should be further investigated. We believe this framework has the potential of providing realistic evaluation and comparison of other imaging techniques and applications as well.

Notes

Compliance with ethical standards

Conflict of interest

This study was sponsored by Siemens Healthcare.

References

  1. 1.
    Sullivan D (2016) The quantitative imaging biomarkers aliance (QIBA). In: Proceeding of the 24th scientific meeting, International Society for Magnetic Resonance in Medicine, Singapore, p 6875Google Scholar
  2. 2.
    Jackson EF (2016) Reproducibility and standardisation of MR biomarkers. In: Proceeding of the 24th scientific meeting, International Society for Magnetic Resonance in Medicine, Singapore, p 6876Google Scholar
  3. 3.
    Sullivan DC, Obuchowski NA, Kessler LG, Raunig DL, Gatsonis C, Huang EP, Kondratovich M, McShane LM, Reeves AP, Barboriak DP, Guimaraes AR, Wahl RL, Group R-QMW (2015) Metrology standards for quantitative imaging biomarkers. Radiology 277(3):813–825CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Lim RP, Shapiro M, Wang EY, Law M, Babb JS, Rueff LE, Jacob JS, Kim S, Carson RH, Mulholland TP, Laub G, Hecht EM (2008) 3D time-resolved MR angiography (MRA) of the carotid arteries with time-resolved imaging with stochastic trajectories: comparison with 3D contrast-enhanced Bolus-Chase MRA and 3D time-of-flight MRA. AJNR Am J Neuroradiol 29(10):1847–1854CrossRefPubMedGoogle Scholar
  5. 5.
    Korosec FR, Frayne R, Grist TM, Mistretta CA (1996) Time-resolved contrast-enhanced 3D MR angiography. Magn Reson Med 36(3):345–351CrossRefPubMedGoogle Scholar
  6. 6.
    Mistretta CA (2009) Undersampled radial MR acquisition and highly constrained back projection (HYPR) reconstruction: potential medical imaging applications in the post-Nyquist era. J Magn Reson Imaging 29(3):501–516CrossRefPubMedGoogle Scholar
  7. 7.
    Mistretta CA, Wieben O, Velikina J, Block W, Perry J, Wu Y, Johnson K (2006) Highly constrained backprojection for time-resolved MRI. Magn Reson Med 55(1):30–40CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Lustig M, Donoho D, Pauly JM (2007) Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn Reson Med 58(6):1182–1195CrossRefPubMedGoogle Scholar
  9. 9.
    Trzasko JD, Haider CR, Borisch EA, Campeau NG, Glockner JF, Riederer SJ, Manduca A (2011) Sparse-CAPR: highly accelerated 4D CE-MRA with parallel imaging and nonconvex compressive sensing. Magn Reson Med 66(4):1019–1032CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Herrmann KH, Baltzer PA, Dietzel M, Krumbein I, Geppert C, Kaiser WA, Reichenbach JR (2011) Resolving arterial phase and temporal enhancement characteristics in DCE MRM at high spatial resolution with TWIST acquisition. J Magn Reson Imaging 34(4):973–982CrossRefPubMedGoogle Scholar
  11. 11.
    Tudorica LA, Oh KY, Roy N, Kettler MD, Chen Y, Hemmingson SL, Afzal A, Grinstead JW, Laub G, Li X, Huang W (2012) A feasible high spatiotemporal resolution breast DCE-MRI protocol for clinical settings. Magn Reson Imaging 30(9):1257–1267CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Platel B, Mus R, Welte T, Karssemeijer N, Mann R (2014) Automated characterization of breast lesions imaged with an ultrafast DCE-MR protocol. IEEE Trans Med Imaging 33(2):225–232CrossRefPubMedGoogle Scholar
  13. 13.
    Janka R, Wenkel E, Geppert C, Kiefer B, Uder M (2010) Breast MRI using TWIST: Doubling the spatial resolution “for free”. In: Proceeding of the 18th scientific meeting, International Society for Magnetic Resonance in Medicine, Stockholm, Sweden, p 4553Google Scholar
  14. 14.
    Song T, Laine AF, Chen Q, Rusinek H, Bokacheva L, Lim RP, Laub G, Kroeker R, Lee VS (2009) Optimal k-space sampling for dynamic contrast-enhanced MRI with an application to MR renography. Magn Reson Med 61(5):1242–1248CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Le Y, Kroeker R, Geppert C, Dale B, Kipfer HD, Lin C (2013) Impact of TWIST View Sharing on Lesion Enhancement Profile in Dynamic Breast MRI. In: Proceeding of the 21th scientific meeting, International Society for Magnetic Resonance in Medicine, Salt Lake City, p 3375Google Scholar
  16. 16.
    Rashidnasab A, Elangovan P, Yip M, Diaz O, Dance DR, Young KC, Wells K (2013) Simulation and assessment of realistic breast lesions using fractal growth models. Phys Med Biol 58(16):5613–5627CrossRefPubMedGoogle Scholar
  17. 17.
    Witten TA, Sander LM (1981) Diffusion-limited aggregation, a kinetic critical phenomenon. Phys Rev Lett 47(19):1400–1403CrossRefGoogle Scholar
  18. 18.
    RepRap.org (2015) Prusa i3 Build Manual. http://reprap.org/wiki/Prusa_i3_Build_Manual. Accessed 6 March 2015
  19. 19.
    Tofts PS, Brix G, Buckley DL, Evelhoch JL, Henderson E, Knopp MV, Larsson HB, Lee TY, Mayr NA, Parker GJ, Port RE, Taylor J, Weisskoff RM (1999) Estimating kinetic parameters from dynamic contrast-enhanced T(1)-weighted MRI of a diffusable tracer: standardized quantities and symbols. J Magn Reson Imaging 10(3):223–232CrossRefPubMedGoogle Scholar
  20. 20.
    Lopata RG, Backes WH, van den Bosch PP, van Riel NA (2007) On the identifiability of pharmacokinetic parameters in dynamic contrast-enhanced imaging. Magn Reson Med 58(2):425–429CrossRefPubMedGoogle Scholar
  21. 21.
    Parker GJ, Roberts C, Macdonald A, Buonaccorsi GA, Cheung S, Buckley DL, Jackson A, Watson Y, Davies K, Jayson GC (2006) Experimentally-derived functional form for a population-averaged high-temporal-resolution arterial input function for dynamic contrast-enhanced MRI. Magn Reson Med 56(5):993–1000CrossRefPubMedGoogle Scholar
  22. 22.
    Huang W, Li X, Chen Y, Chang MC, Oborski MJ, Malyarenko DI, Muzi M, Jajamovich GH, Fedorov A, Tudorica A, Gupta SN, Laymon CM, Marro KI, Dyvorne HA, Miller JV, Barbodiak DP, Chenevert TL, Yankeelov TE, Mountz JM, Kinahan PE, Kikinis R, Taouli B, Fennessy F, Kalpathy-Cramer J (2014) Variations of dynamic contrast-enhanced magnetic resonance imaging in evaluation of breast cancer therapy response: a multicenter data analysis challenge. Transl Oncol 7(1):153–166CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Rakow-Penner R, Daniel B, Yu H, Sawyer-Glover A, Glover GH (2006) Relaxation times of breast tissue at 1.5T and 3T measured using IDEAL. J Magn Reson Imaging 23(1):87–91CrossRefPubMedGoogle Scholar
  24. 24.
    Li X, Huang W, Yankeelov TE, Tudorica A, Rooney WD, Springer CS Jr (2005) Shutter-speed analysis of contrast reagent bolus-tracking data: preliminary observations in benign and malignant breast disease. Magn Reson Med 53(3):724–729CrossRefPubMedGoogle Scholar
  25. 25.
    Henderson E, Rutt BK, Lee TY (1998) Temporal sampling requirements for the tracer kinetics modeling of breast disease. Magn Reson Imaging 16(9):1057–1073CrossRefPubMedGoogle Scholar
  26. 26.
    Le Y, Geppert C, Kroeker R, Dale B, Lin C (2012) Simulations of the impact of TWIST view sharing on the measured enhancement in breast DCE MRI. In: Proceeding of the 20th scientific meeting, International Society for Magnetic Resonance in Medicine, Melbourne, p 2404Google Scholar
  27. 27.
    Le Y, Nickel D, Kroeker R, Geppert C, Dale B, Kipfer H, Lin C (2014) A simulation study of the flexible TWIST view sharing impact on the breast DCE MRI. In: Proceeding of the 22th scientific meeting, International Society for Magnetic Resonance in Medicine, Milan, Italy, p 6436Google Scholar
  28. 28.
    Mann RM, Mus RD, van Zelst J, Geppert C, Karssemeijer N, Platel B (2014) A novel approach to contrast-enhanced breast magnetic resonance imaging for screening: high-resolution ultrafast dynamic imaging. Invest Radiol 49(9):579–585CrossRefPubMedGoogle Scholar
  29. 29.
    Agliozzo S, De Luca M, Bracco C, Vignati A, Giannini V, Martincich L, Carbonaro LA, Bert A, Sardanelli F, Regge D (2012) Computer-aided diagnosis for dynamic contrast-enhanced breast MRI of mass-like lesions using a multiparametric model combining a selection of morphological, kinetic, and spatiotemporal features. Med Phys 39(4):1704–1715CrossRefPubMedGoogle Scholar
  30. 30.
    Carton AK, Bakic P, Ullberg C, Derand H, Maidment ADA (2011) Development of a physical 3D anthropomorphic breast phantom. Med Phys 38(2):891–896CrossRefPubMedPubMedCentralGoogle Scholar
  31. 31.
    Liney GP, Tozer DJ, Turnbull LW (1999) A simple and realistic tissue-equivalent breast phantom for MRI. J Magn Reson Imaging 10(6):968–971CrossRefPubMedGoogle Scholar
  32. 32.
    Tiemann D, Wilbig J, Ezzat R, Terzenback D, Kompan IN, Laue H, Rascher-Friesenhausen R (2011) Perfusion mamma phantom adapted to physical MRI characteristics of breast tissue including tumorlike structure. In: Paper presented at the European Society for Magnetic Resonance in Medicine and Biology, Leipzig, Germany, October 6–8Google Scholar
  33. 33.
    Ahmed A, Gibbs P, Pickles M, Turnbull L (2013) Texture analysis in assessment and prediction of chemotherapy response in breast cancer. J Magn Reson Imaging 38(1):89–101CrossRefPubMedGoogle Scholar
  34. 34.
    Kale MC, Fleig JD, Imal N (2013) Assessment of feasibility to use computer aided texture analysis based tool for parametric images of suspicious lesions in DCE-MR mammography. Comput Math Methods Med 2013:872676CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Wang TC, Huang YH, Huang CS, Chen JH, Huang GY, Chang YC, Chang RF (2014) Computer-aided diagnosis of breast DCE-MRI using pharmacokinetic model and 3-D morphology analysis. Magn Reson Imaging 32(3):197–205CrossRefPubMedGoogle Scholar
  36. 36.
    Teruel JR, Heldahl MG, Goa PE, Pickles M, Lundgren S, Bathen TF, Gibbs P (2014) Dynamic contrast-enhanced MRI texture analysis for pretreatment prediction of clinical and pathological response to neoadjuvant chemotherapy in patients with locally advanced breast cancer. NMR Biomed 27(8):887–896CrossRefPubMedGoogle Scholar
  37. 37.
    Baish JW, Jain RK (2000) Fractals and cancer. Can Res 60(14):3683–3688Google Scholar

Copyright information

© ESMRMB 2017

Authors and Affiliations

  1. 1.Department of Radiology and Imaging ScienceIndiana University School of MedicineIndianapolisUSA
  2. 2.Siemens HealthcareErlangenGermany
  3. 3.Siemens Medical Solutions USA IncMalvenUSA

Personalised recommendations