Abstract
MRI reconstruction techniques based on deep learning have led to unprecedented reconstruction quality especially in highly accelerated settings. However, deep learning techniques are also known to fail unexpectedly and hallucinate structures. This is particularly problematic if reconstructions are directly used for downstream tasks such as real-time treatment guidance or automated extraction of clinical parameters (e.g. via segmentation). Well-calibrated uncertainty quantification will be a key ingredient for safe use of this technology in clinical practice. In this paper we propose a novel probabilistic reconstruction technique (PHiRec) building on the idea of conditional hierarchical variational autoencoders. We demonstrate that our proposed method produces high-quality reconstructions as well as uncertainty quantification that is substantially better calibrated than several strong baselines. We furthermore demonstrate how uncertainties arising in the MR reconstruction can be propagated to a downstream segmentation task, and show that PHiRec also allows well-calibrated estimation of segmentation uncertainties that originated in the MR reconstruction process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The code for PHiRec is available at https://github.com/paulkogni/MR-Recon-UQ.
References
Angelopoulos, A.N., et al.: Image-to-image regression with distribution-free uncertainty quantification and applications in imaging, February 2022. arXiv arXiv:2202.05265 [cs, eess, q-bio, stat]
Baumgartner, C.F., et al.: Phiseg: Capturing uncertainty in medical image segmentation (2019). https://doi.org/10.48550/ARXIV.1906.04045, arXiv:1906.04045
Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight uncertainty in neural network. In: International Conference on Machine Learning, pp. 1613–1622. PMLR (2015)
Calivá, F., et al.: Breaking speed limits with simultaneous ultra-fast MRI reconstruction and tissue segmentation. In: Medical Imaging with Deep Learning, pp. 94–110. PMLR (2020)
Chung, H., Ye, J.C.: Score-based diffusion models for accelerated MRI. Med. Image Anal. 80, 102479 (2022)
Desai, A.D., et al.: SKM-TEA: a dataset for accelerated MRI reconstruction with dense image labels for quantitative clinical evaluation (2022)
Gottschling, N.M., Antun, V., Adcock, B., Hansen, A.C.: The troublesome kernel: why deep learning for inverse problems is typically unstable. arXiv preprint arXiv:2001.01258 (2020)
Hauptmann, A., Arridge, S., Lucka, F., Muthurangu, V., Steeden, J.A.: Real-time cardiovascular MR with spatio-temporal artifact suppression using deep learning-proof of concept in congenital heart disease. Magn. Reson. Med. 81(2), 1143–1156 (2019)
Hepp, T., Gatidis, S., Hammernik, K., Küstner, T.: Uncertainty estimation via ensembling for deep learning-based MR image reconstruction. In: ISMRM, vol. 685 (2022)
Hyun, C.M., Kim, H.P., Lee, S.M., Lee, S., Seo, J.K.: Deep learning for undersampled MRI reconstruction. Phys. Med. Biol. 63(13), 135007 (2018)
Jalal, A., Arvinte, M., Daras, G., Price, E., Dimakis, A.G., Tamir, J.I.: Robust compressed sensing MRI with deep generative priors, December 2021. arXiv arXiv:2108.01368 [cs, math, stat]
Jin, K.H., McCann, M.T., Froustey, E., Unser, M.: Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Image Process. 26(9), 4509–4522 (2017). https://doi.org/10.1109/TIP.2017.2713099, http://ieeexplore.ieee.org/document/7949028/
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)
Kobler, E., Effland, A., Kunisch, K., Pock, T.: Total deep variation for linear inverse problems. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7549–7558 (2020)
Kohl, S.A.A., et al.: A hierarchical probabilistic u-net for modeling multi-scale ambiguities (2019)
Kohl, S.A.A., et al.: A probabilistic u-net for segmentation of ambiguous images (2018). https://doi.org/10.48550/ARXIV.1806.05034, https://arxiv.org/abs/1806.05034
Laves, M.H., Ihler, S., Fast, J.F., Kahrs, L.A., Ortmaier, T.: Recalibration of aleatoric and epistemic regression uncertainty in medical imaging. arXiv preprint arXiv:2104.12376 (2021)
Morshuis, J.N., Gatidis, S., Hein, M., Baumgartner, C.F.: Adversarial robustness of MR image reconstruction under realistic perturbations. In: Haq, N., Johnson, P., Maier, A., Qin, C., Würfl, T., Yoo, J. (eds.) Machine Learning for Medical Image Reconstruction, vol. 13587, pp. 24–33. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-17247-2_3
Narnhofer, D., Effland, A., Kobler, E., Hammernik, K., Knoll, F., Pock, T.: Bayesian uncertainty estimation of learned variational MRI reconstruction. IEEE Trans. Med. Imaging 41(2), 279–291 (2022). https://doi.org/10.1109/TMI.2021.3112040
Ongie, G., Jalal, A., Metzler, C.A., Baraniuk, R.G., Dimakis, A.G., Willett, R.: Deep learning techniques for inverse problems in imaging, May 2020. arXiv arXiv:2005.06001 [cs, eess, stat]
Paszke, A., et al.: An imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Inc. (2019). http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
Peng, C., Guo, P., Zhou, S.K., Patel, V., Chellappa, R.: Towards performant and reliable undersampled MR reconstruction via diffusion model sampling (2022). https://doi.org/10.48550/ARXIV.2203.04292, arXiv:2203.04292
Pruessmann, K.P., Weiger, M., Scheidegger, M.B., Boesiger, P.: SENSE: sensitivity encoding for fast MRI. Magn. Reson. Med. Off. J. Int. Soci. Magn. Reson. Med. 42(5), 952–962 (1999)
Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 37(2), 491–503 (2017)
Schlemper, J., et al.: Bayesian deep learning for accelerated MR image reconstruction. In: Knoll, F., Maier, A., Rueckert, D. (eds.) MLMIR 2018. LNCS, vol. 11074, pp. 64–71. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00129-2_8
Schlemper, J., et al.: Cardiac MR segmentation from undersampled k-space using deep latent representation learning. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 259–267. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_30
Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Sriram, A., et al.: End-to-end variational networks for accelerated MRI reconstruction. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 64–73. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_7
Tezcan, K.C., Karani, N., Baumgartner, C.F., Konukoglu, E.: Sampling possible reconstructions of undersampled acquisitions in MR imaging with a deep learned prior. IEEE Trans. Med. Imaging 41(7), 1885–1896 (2022)
Tolpadi, A.A., et al.: K2S challenge: from undersampled k-space to automatic segmentation. Bioengineering 10(2), 267 (2023)
Waddington, D.E.J., et al.: On real-time image reconstruction with neural networks for MRI-guided radiotherapy, May 2022. arXiv:2202.05267 [physics]
Xie, Y., Li, Q.: Measurement-conditioned denoising diffusion probabilistic model for under-sampled medical image reconstruction. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention, vol. 13436, pp. 655–664. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16446-0_62
Zeng, G., et al.: A review on deep learning MRI reconstruction without fully sampled k-space. BMC Med. Imaging 21(1), 195 (2021). https://doi.org/10.1186/s12880-021-00727-9
Zhang, C., Barbano, R., Jin, B.: Conditional variational autoencoder for learned image reconstruction. Comput. 9(11), 114 (2021)
Zhou, Z., et al.: Parallel imaging and convolutional neural network combined fast MR image reconstruction: applications in low-latency accelerated real-time imaging. Med. Phys. 46(8), 3399–3413 (2019)
Acknowledgments
Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy - EXC number 2064/1 - Project number 390727645. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Paul Fischer.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fischer, P., Thomas, K., Baumgartner, C.F. (2023). Uncertainty Estimation and Propagation in Accelerated MRI Reconstruction. In: Sudre, C.H., Baumgartner, C.F., Dalca, A., Mehta, R., Qin, C., Wells, W.M. (eds) Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. UNSURE 2023. Lecture Notes in Computer Science, vol 14291. Springer, Cham. https://doi.org/10.1007/978-3-031-44336-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-44336-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44335-0
Online ISBN: 978-3-031-44336-7
eBook Packages: Computer ScienceComputer Science (R0)
