Skip to main content

Advertisement

SpringerLink
Log in

Novel adversarial semantic structure deep learning for MRI-guided attenuation correction in brain PET/MRI

  • Original Article
  • Published:
European Journal of Nuclear Medicine and Molecular Imaging Aims and scope Submit manuscript

Abstract

Objective

Quantitative PET/MR imaging is challenged by the accuracy of synthetic CT (sCT) generation from MR images. Deep learning-based algorithms have recently gained momentum for a number of medical image analysis applications. In this work, a novel sCT generation algorithm based on deep learning adversarial semantic structure (DL-AdvSS) is proposed for MRI-guided attenuation correction in brain PET/MRI.

Materials and methods

The proposed DL-AdvSS algorithm exploits the ASS learning framework to constrain the synthetic CT generation process to comply with the extracted structural features from CT images. The proposed technique was evaluated through comparison to an atlas-based sCT generation method (Atlas), previously developed for MRI-only or PET/MRI-guided radiation planning. Moreover, the commercial segmentation-based approach (Segm) implemented on the Philips TF PET/MRI system was included in the evaluation. Clinical brain studies of 40 patients who underwent PET/CT and MR imaging were used for the evaluation of the proposed method under a two-fold cross validation scheme.

Results

The accuracy of cortical bone extraction and CT value estimation were investigated for the three different methods. Atlas and DL-AdvSS exhibited similar cortical bone extraction accuracy resulting in a Dice coefficient of 0.78 ± 0.07 and 0.77 ± 0.07, respectively. Likewise, DL-AdvSS and Atlas techniques performed similarly in terms of CT value estimation in the cortical bone region where a mean error (ME) of less than −11 HU was obtained. The Segm approach led to a ME of −1025 HU. Furthermore, the quantitative analysis of corresponding PET images using the three approaches assuming the CT-based attenuation corrected PET (PETCTAC) as reference demonstrated comparative performance of DL-AdvSS and Atlas techniques with a mean standardized uptake value (SUV) bias less than 4% in 63 brain regions. In addition, less that 2% SUV bias was observed in the cortical bone when using Atlas and DL-AdvSS approaches. However, Segm resulted in 14.7 ± 8.9% SUV underestimation in the cortical bone.

Conclusion

The proposed DL-AdvSS approach demonstrated competitive performance with respect to the state-of-the-art atlas-based technique achieving clinically tolerable errors, thus outperforming the commercial segmentation approach used in the clinic.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Zaidi H, Becker M. The promise of hybrid PET/MRI: technical advances and clinical applications. IEEE Signal Process Mag. 2016;33:67–85.

    Article  Google Scholar 

  2. Mehranian A, Arabi H, Zaidi H. Vision 20/20: magnetic resonance imaging-guided attenuation correction in PET/MRI: challenges, solutions, and opportunities. Med Phys. 2016;43:1130–55.

    Article  Google Scholar 

  3. Erlandsson K, Dickson J, Arridge S, Atkinson D, Ourselin S, Hutton BF. MR imaging-guided partial volume correction of PET data in PET/MR imaging. PET Clinics. 2016;11:161–77.

    Article  Google Scholar 

  4. Catana C. Motion correction options in PET/MRI. Semin Nucl Med. 2015;45:212–23.

    Article  Google Scholar 

  5. Mehranian A, Belzunce M, Prieto C, Hammers A, Reader A. Synergistic PET and SENSE MR image reconstruction using joint sparsity regularization. IEEE Trans Med Imaging. 2018;37:20–34.

    Article  Google Scholar 

  6. Gong K, Cheng-Liao J, Wang G, Chen KT, Catana C, Qi J. Direct patlak reconstruction from dynamic PET data using the kernel method with MRI information based on structural similarity. IEEE Trans Med Imaging. 2018;37:955–65.

    Article  Google Scholar 

  7. Zaidi H, Montandon M-L, Slosman DO. Magnetic resonance imaging-guided attenuation and scatter corrections in three-dimensional brain positron emission tomography. Med Phys. 2003;30:937–48.

    Article  Google Scholar 

  8. Martinez-Moller A, Souvatzoglou M, Delso G, Bundschuh RA, Chefd’hotel C, Ziegler SI, et al. Tissue classification as a potential approach for attenuation correction in whole-body PET/MRI: evaluation with PET/CT data. J Nucl Med. 2009;50:520–6.

    Article  Google Scholar 

  9. Keereman V, Fierens Y, Broux T, De Deene Y, Lonneux M, Vandenberghe S. MRI-based attenuation correction for PET/MRI using ultrashort echo time sequences. J Nucl Med. 2010;51:812–8.

    Article  Google Scholar 

  10. Arabi H, Rager O, Alem A, Varoquaux A, Becker M, Zaidi H. Clinical assessment of MR-guided 3-class and 4-class attenuation correction in PET/MR. Mol Imaging Biol. 2015;17:264–76.

    Article  CAS  Google Scholar 

  11. Wollenweber SD, Ambwani S, Delso G, Lonn AHR, Mullick R, Wiesinger F, et al. Evaluation of an atlas-based PET head attenuation correction using PET/CT & MR patient data. IEEE Trans Nucl Sci. 2013;60:3383–90.

    Article  Google Scholar 

  12. Arabi H, Zaidi H. One registration multi-atlas-based pseudo-CT generation for attenuation correction in PET/MRI. Eur J Nucl Med Mol Imaging. 2016;43:2021–35.

    Article  CAS  Google Scholar 

  13. Burgos N, Cardoso MJ, Guerreiro F, Veiga C, Modat M, McClelland J, et al. Robust CT synthesis for radiotherapy planning: application to the head and neck region. In: International conference on medical image computing and computer-assisted intervention: Springer; 2015. p. 476–84.

  14. Defrise M, Rezaei A, Nuyts J. Time-of-flight PET data determine the attenuation sinogram up to a constant. Phys Med Biol. 2012;57:885–99.

    Article  Google Scholar 

  15. Mehranian A, Zaidi H, Reader AJ. MR-guided joint reconstruction of activity and attenuation in brain PET-MR. NeuroImage. 2017;162:276–88.

    Article  Google Scholar 

  16. Rezaei A, Deroose CM, Vahle T, Boada F, Nuyts J. Joint reconstruction of activity and attenuation in fime-of-flight PET: a quantitative analysis. J Nucl Med. 2018;59:1624–9.

    Article  Google Scholar 

  17. Huynh T, Gao Y, Kang J, Wang L, Zhang P, Lian J, et al. Estimating CT image from MRI data using structured random forest and auto-context model. IEEE Trans Med Imaging. 2016;35:174–83.

    Article  Google Scholar 

  18. Han X. MR-based synthetic CT generation using a deep convolutional neural network method. Med Phys. 2017;44:1408–19.

    Article  CAS  Google Scholar 

  19. Leynes AP, Yang J, Wiesinger F, Kaushik SS, Shanbhag DD, Seo Y, et al. Zero-echo-time and Dixon deep pseudo-CT (ZeDD CT): direct generation of pseudo-CT images for pelvic PET/MRI attenuation correction using deep convolutional neural networks with multiparametric MRI. J Nucl Med. 2018;59:852–8.

    Article  Google Scholar 

  20. Nie D, Trullo R, Lian J, Petitjean C, Ruan S, Wang Q, et al. Medical image synthesis with context-aware generative adversarial network. In: Medical image computing and computer-assisted intervention − MICCAI 2017, Quebec, Canada; Springer, 2017. pp 417–25.

  21. Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology. 2018;286:676–84.

    Article  Google Scholar 

  22. Greenspan H, van Ginneken B, Summers RM. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging. 2016;35:1153–9.

    Article  Google Scholar 

  23. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.

    Article  Google Scholar 

  24. Emami H, Dong M, Nejad-Davarani SP, Glide-Hurst CK. Generating synthetic CTs from magnetic resonance images using generative adversarial networks. Med Phys. 2018;45:3627–36.

    Article  Google Scholar 

  25. Dinkla AM, Wolterink JM, Maspero M, Savenije MHF, Verhoeff JJC, Seravalli E, et al. MR-only brain radiation therapy: dosimetric evaluation of synthetic CTs generated by a dilated convolutional neural network. Int J Radiat Oncol Biol Phys. 2018;102:801–12.

    Article  Google Scholar 

  26. Hwang D, Kim KY, Kang SK, Seo S, Paeng JC, Lee DS, et al. Improving the accuracy of simultaneously reconstructed activity and attenuation maps using deep learning. J Nucl Med. 2018;59:1624–9.

    Article  CAS  Google Scholar 

  27. Chen S, Qin A, Zhou D, Yan D. Technical note: U-net-generated synthetic CT images for magnetic resonance imaging-only prostate intensity-modulated radiation therapy treatment planning. Med Phys. 2018;45:5659–65.

    Article  Google Scholar 

  28. Xiang L, Wang Q, Nie D, Zhang L, Jin X, Qiao Y, et al. Deep embedding convolutional neural network for synthesizing CT image from T1-weighted MR image. Med Image Anal. 2018;47:31–44.

    Article  Google Scholar 

  29. Arabi H, Koutsouvelis N, Rouzaud M, Miralbell R, Zaidi H. Atlas-guided generation of pseudo-CT images for MRI-only and hybrid PET–MRI-guided radiotherapy treatment planning. Phys Med Biol. 2016;61:6531–52.

    Article  CAS  Google Scholar 

  30. Arabi H, Dowling JA, Burgos N, Han X, Greer PB, Koutsouvelis N, et al. Comparative study of algorithms for synthetic CT generation from MRI: consequences for MRI-guided radiation planning in the pelvic region. Med Phys. 2018;45:5218–33.

    Article  Google Scholar 

  31. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on machine learning, Lille, France, vol. 37; 2015. pp 448–56.

  32. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60:84–90.

    Article  Google Scholar 

  33. Zaidi H, Ojha N, Morich M, Griesmer J, Hu Z, Maniawski P, et al. Design and performance evaluation of a whole-body ingenuity TF PET-MRI system. Phys Med Biol. 2011;56:3091–106.

    Article  CAS  Google Scholar 

  34. Zaidi H, Montandon M-L, Meikle S. Strategies for attenuation compensation in neurological PET studies. Neuroimage. 2007;34:518–41.

    Article  Google Scholar 

  35. Mehranian A, Arabi H, Zaidi H. Quantitative analysis of MRI-guided attenuation correction techniques in time-of-flight brain PET/MRI. NeuroImage. 2016;130:123–33.

    Article  Google Scholar 

  36. Chartsias A, Joyce T, Dharmakumar R, Tsaftaris SA. Adversarial image synthesis for unpaired nulti-modal cardiac data. In: International workshop on simulation and synthesis in medical imaging, SASHIMI; 2017. pp 3–13.

  37. Huo Y, Xu Z, Bao S, Assad A, Abramson RG, Landman BA. Adversarial synthesis learning enables segmentation without target modality ground truth. In: IEEE 15th International Symposium on Biomedical Imaging (ISBI); 2018. pp 1217–20.

  38. Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. IEEE international conference on computer vision. 2017. pp. 2223-2232.

  39. Su KH, Hu L, Stehning C, Helle M, Qian P, Thompson CL, et al. Generation of brain pseudo-CTs using an undersampled, single-acquisition UTE-mDixon pulse sequence and unsupervised clustering. Med Phys. 2015;42:4974–86.

    Article  Google Scholar 

  40. Gong K, Yang J, Kim K, El Fakhri G, Seo Y, Li Q. Attenuation correction for brain PET imaging using deep neural network based on Dixon and ZTE MR images. Phys Med Biol. 2018;63:125011.

    Article  Google Scholar 

  41. Liu F, Jang H, Kijowski R, Zhao G, Bradshaw T, McMillan AB. A deep learning approach for 18 F-FDG PET attenuation correction. EJNMMI Phys. 2018;5:24.

    Article  Google Scholar 

  42. Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging–based attenuation correction for PET/MR imaging. Radiology. 2017;286:676–84.

    Article  Google Scholar 

  43. Gong K, Yang J, Kim K, El Fakhri G, Seo Y, Li Q. Attenuation correction for brain PET imaging using deep neural network based on Dixon and ZTE MR images. Phys Med Biol. 2018;63(12):125011.

    Article  Google Scholar 

  44. Spuhler KD, Gardus J, Gao Y, DeLorenzo C, Parsey R, Huang C. Synthesis of patient-specific transmission image for PET attenuation correction for PET/MR imaging of the brain using a convolutional neural network. J Nucl Med. 2019;60:555–60.

    Article  CAS  Google Scholar 

Download references

Acknowledgments

This work was supported by the Swiss National Science Foundation under grant SNFN 320030_176052 and the Swiss Cancer Research Foundation under Grant KFS-3855-02-2016.

Author information

Authors and Affiliations

  1. Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, CH-1211, Geneva 4, Switzerland

    Hossein Arabi & Habib Zaidi

  2. Institute for Surgical Technology and Biomechanics, University of Bern, CH-3014, Bern, Switzerland

    Guodong Zeng & Guoyan Zheng

  3. School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China

    Guoyan Zheng

  4. Geneva University Neurocenter, Geneva University, CH-1205, Geneva, Switzerland

    Habib Zaidi

  5. Department of Nuclear Medicine and Molecular Imaging, University of Groningen, University Medical Center Groningen, 9700 RB, Groningen, Netherlands

    Habib Zaidi

  6. Department of Nuclear Medicine, University of Southern Denmark, 500, Odense, Denmark

    Habib Zaidi

Authors
  1. Hossein Arabi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guodong Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoyan Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Habib Zaidi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Habib Zaidi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

None of the authors have affiliations that present financial or non-financial competing interests for this work.

Research involving human participants

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Advanced Image Analyses (Radiomics and Artificial Intelligence)

Electronic supplementary material

ESM 1

(DOCX 717 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arabi, H., Zeng, G., Zheng, G. et al. Novel adversarial semantic structure deep learning for MRI-guided attenuation correction in brain PET/MRI. Eur J Nucl Med Mol Imaging 46, 2746–2759 (2019). https://doi.org/10.1007/s00259-019-04380-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00259-019-04380-x

Keywords

Search

Navigation