Fill the K-Space and Refine the Image: Prompting for Dynamic and Multi-Contrast MRI Reconstruction

Abstract

The key to dynamic or multi-contrast magnetic resonance imaging (MRI) reconstruction lies in exploring inter-frame or inter-contrast information. Currently, the unrolled model, an approach combining iterative MRI reconstruction steps with learnable neural network layers, stands as the best-performing method for MRI reconstruction. However, there are two main limitations to overcome: firstly, the unrolled model structure and GPU memory constraints restrict the capacity of each denoising block in the network, impeding the effective extraction of detailed features for reconstruction; secondly, the existing model lacks the flexibility to adapt to variations in the input, such as different contrasts, resolutions or views, necessitating the training of separate models for each input type, which is inefficient and may lead to insufficient reconstruction. In this paper, we propose a two-stage MRI reconstruction pipeline to address these limitations. The first stage involves filling the missing k-space data, which we approach as a physics-based reconstruction problem. We first propose a simple yet efficient baseline model, which utilizes adjacent frames/contrasts and channel attention to capture the inherent inter-frame/-contrast correlation. Then, we extend the baseline model to a prompt-based learning approach, PromptMR, for all-in-one MRI reconstruction from different views, contrasts, adjacent types, and acceleration factors. The second stage is to refine the reconstruction from the first stage, which we treat as a general video restoration problem to further fuse features from neighboring frames/contrasts in the image domain. Extensive experiments show that our proposed method significantly outperforms previous state-of-the-art accelerated MRI reconstruction methods.

Appendices

Appendix

A Details of the Baseline Model

1.1 A.1 CAUnet

(See Fig. 6).

Fig. 6.

Overview of the CAUnet architecture in the proposed baseline model.

1.2 A.2 Unrolled Model Architecture

(See Fig. 7).

Fig. 7.

Overview of the unrolled model architecture for both our baseline model and PromptMR. The primary distinction is in the denoiser D and sensitivity map estimation (SME) networks: the baseline employs CAUnet, whereas PromptMR utilizes PromptUnet. Each cascade represents an updating step in Eq. 5 in the main text. The red module indicates the learnable part in the unrolled model.

B Experiments on FastMRI Multi-Coil Knee Dataset

Benchmark on FastMRI Multi-Coil Knee Dataset. To assess the performance of our proposed method across different anatomies, we benchmarked it on another large-scale MRI reconstruction dataset, the fastMRI multi-coil knee dataset [19]. Since the online evaluation platform for the fastMRI test set is unavailable¹, we divided the original 199 validation cases into 99 for validation and 100 for testing. The results of other methods are reported using their officially pretrained models. As presented in Table 4, our models outperform all previous state-of-the-art methods, without significantly increasing the number of network parameters compared to E2E-Varnet.

Table 4. Performance of state-of-the-art accelerated MRI reconstruction techniques on the fastMRI knee multi-coil $\times 8$ test dataset. The best and second best results are highlighted in and colors, respectively.

Effectiveness of Two-Stage Pipeline. We employed ShiftNet to refine the images reconstructed by the pretrained E2E-Varnet on the fastMRI multi-coil knee test dataset with $\times 8$ undersampling. Table 5 shows that the second-stage refinement substantially improves the reconstruction quality, which implies that the multi-slice information in the fastMRI dataset might not be comprehensively utilized by the single-stage unrolled model.

Table 5. Effectiveness of the second-stage image refinement on the fastMRI knee multi-coil $\times 8$ test dataset.

C Additional Qualitative Results on CMRxRecon Dataset

(See Fig. 8).

Fig. 8.

Visual comparison of reconstructions from the CMRxRecon dataset with $\times 10$ acceleration. PromptMR can recover fine details (highlighted in red box) on reconstructed images that other state-of-the-art methods may miss. (Color figure online)