Abstract
In this work, we investigate approaches to generating synthetic Computed Tomography (CT) images from the real Magnetic Resonance Imaging (MRI) data. Generating the radiological scans has grown in popularity in the recent years due to its promise to enable single-modality radiotherapy planning in clinical oncology, where the co-registration of the radiological modalities is cumbersome. We rely on the Generative Adversarial Network (GAN) models with cycle consistency which permit unpaired image-to-image translation between the modalities. We also introduce the perceptual loss function term and the coordinate convolutional layer to further enhance the quality of translated images. The Unsharp masking and the Super-Resolution GAN (SRGAN) were considered to improve the quality of synthetic images. The proposed architectures were trained on the unpaired MRI-CT data and then evaluated on the paired brain dataset. The resulting CT scans were generated with the mean absolute error (MAE), the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) scores of 60.83 HU, 17.21 dB, and 0.8, respectively. DualGAN with perceptual loss function term and coordinate convolutional layer proved to perform best. The MRI-CT translation approach holds potential to eliminate the need for the patients to undergo both examinations and to be clinically accepted as a new tool for radiotherapy planning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Battista, J.J., Rider, W.D., Van Dyk, J.: Computed tomography for radiotherapy planning. Int. J. Radiat. Oncol.* Biol.* Phys. 6(1), 99–107 (1980)
Chen, L., et al.: MRI-based treatment planning for radiotherapy: dosimetric verification for prostate IMRT. Int. J. Radiat. Oncol.* Biol.* Phys. 60(2), 636–647 (2004)
Clark, K., et al.: The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26(6), 1045–1057 (2013)
Coy, P., Kennelly, G.: The role of curative radiotherapy in the treatment of lung cancer. Cancer 45(4), 698–702 (1980)
(CPTAC), N.C.I.C.P.T.A.C.: Radiology Data from the Clinical Proteomic Tumor Analysis Consortium Glioblastoma Multiforme [CPTAC-GBM] collection [Data set]. The Cancer Imaging Archive (2018). https://doi.org/10.7937/k9/tcia.2018.3rje41q1
Gelband, H., Jha, P., Sankaranarayanan, R., Horton, S.: Disease Control Priorities: Cancer, vol. 3. The World Bank (2015)
Hofmann, M., et al.: MRI-based attenuation correction for PET/MRI: a novel approach combining pattern recognition and atlas registration. J. Nucl. Med. 49(11), 1875–1883 (2008)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Kapanen, M., Collan, J., Beule, A., Seppälä, T., Saarilahti, K., Tenhunen, M.: Commissioning of MRI-only based treatment planning procedure for external beam radiotherapy of prostate. Magn. Reson. Med. 70(1), 127–135 (2013)
Karlsson, M., Karlsson, M.G., Nyholm, T., Amies, C., Zackrisson, B.: Dedicated magnetic resonance imaging in the radiotherapy clinic. Int. J. Radiat. Oncol.* Biol.* Phys. 74(2), 644–651 (2009)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. CoRR abs/1609.04802 (2016)
Lei, Y., et al.: MRI-based pseudo CT synthesis using anatomical signature and alternating random forest with iterative refinement model. J. Med. Imaging 5(4), 043504 (2018)
Liu, M., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. CoRR abs/1703.00848 (2017). http://arxiv.org/abs/1703.00848
Liu, R., et al.: An intriguing failing of convolutional neural networks and the coordconv solution. arXiv preprint arXiv:1807.03247 (2018)
Mph, R.L.S., Kimberly, D.: Cancer statistics, 2017. CA Cancer J. Clin. 67(1), 7–30 (2017)
Polesel, A., Ramponi, G., Mathews, V.J.: Image enhancement via adaptive unsharp masking. IEEE Trans. Image Process. 9(3), 505–510 (2000)
Prokopenko, D., Stadelmann, J.V., Schulz, H., Renisch, S., Dylov, D.V.: Synthetic CT generation from MRI using improved DualGAN (2019)
Ren, H.: SRGAN: A PyTorch implementation of SRGAN based on CVPR 2017 paper photo-realistic single image super-resolution using a generative adversarial network
Vallières, M., et al.: Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Sci. Rep. 7(1), 10117 (2017)
Vallières, M., et al.: Data from Head-Neck-PET-CT. The Cancer Imaging Archive (2017). https://doi.org/10.7937/K9/TCIA.2017.8oje5q00
Wolterink, J.M., Dinkla, A.M., Savenije, M.H.F., Seevinck, P.R., van den Berg, C.A.T., Isgum, I.: Deep MR to CT synthesis using unpaired data. CoRR abs/1708.01155 (2017). http://arxiv.org/abs/1708.01155
Yi, Z., Zhang, H., Tan, P., Gong, M.: DualGAN: unsupervised dual learning for image-to-image translation. CoRR abs/1704.02510 (2017)
Zhang, R., Pfister, T., Li, J.: Harmonic unpaired image-to-image translation. In: ICLR (2019)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint (2017)
Acknowledgement
Data used in this publication were generated by the National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Prokopenko, D., Stadelmann, J.V., Schulz, H., Renisch, S., Dylov, D.V. (2019). Unpaired Synthetic Image Generation in Radiology Using GANs. In: Nguyen, D., Xing, L., Jiang, S. (eds) Artificial Intelligence in Radiation Therapy. AIRT 2019. Lecture Notes in Computer Science(), vol 11850. Springer, Cham. https://doi.org/10.1007/978-3-030-32486-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-32486-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32485-8
Online ISBN: 978-3-030-32486-5
eBook Packages: Computer ScienceComputer Science (R0)
