Advertisement

TPSDicyc: Improved Deformation Invariant Cross-domain Medical Image Synthesis

  • Chengjia WangEmail author
  • Giorgos Papanastasiou
  • Sotirios Tsaftaris
  • Guang Yang
  • Calum Gray
  • David Newby
  • Gillian Macnaught
  • Tom MacGillivray
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11905)

Abstract

Cycle-consistent generative adversarial network (CycleGAN) has been widely used for cross-domain medical image systhesis tasks particularly due to its ability to deal with unpaired data. However, most CycleGAN-based synthesis methods can not achieve good alignment between the synthesized images and data from the source domain, even with additional image alignment losses. This is because the CycleGAN generator network can encode the relative deformations and noises associated to different domains. This can be detrimental for the downstream applications that rely on the synthesized images, such as generating pseudo-CT for PET-MR attenuation correction. In this paper, we present a deformation invariant model based on the deformation-invariant CycleGAN (DicycleGAN) architecture and the spatial transformation network (STN) using thin-plate-spline (TPS). The proposed method can be trained with unpaired and unaligned data, and generate synthesised images aligned with the source data. Robustness to the presence of relative deformations between data from the source and target domain has been evaluated through experiments on multi-sequence brain MR data and multi-modality abdominal CT and MR data. Experiment results demonstrated that our method can achieve better alignment between the source and target data while maintaining superior image quality of signal compared to several state-of-the-art CycleGAN-based methods.

Notes

Acknowledgments

This work is funded by British Heart Fundation (no. RG/16/10/32375). S.A. Tsaftaris and G. Papanastasiou acknowledge support from the EPSRC Grant (EP/P022928/1). Support from NHS Lothian R&D, and Edinburgh Imaging and the Edinburgh Clinical Research Facility at the University of Edinburgh is gratefully acknowledged.

References

  1. 1.
    Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593 (2017)
  2. 2.
    Chartsias, A., Joyce, T., Dharmakumar, R., Tsaftaris, S.A.: Adversarial image synthesis for unpaired multi-modal cardiac data. In: Tsaftaris, S.A., Gooya, A., Frangi, A.F., Prince, J.L. (eds.) SASHIMI 2017. LNCS, vol. 10557, pp. 3–13. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-68127-6_1CrossRefGoogle Scholar
  3. 3.
    Wang, C., Macnaught, G., Papanastasiou, G., MacGillivray, T., Newby, D.: Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks. In: Gooya, A., Goksel, O., Oguz, I., Burgos, N. (eds.) SASHIMI 2018. LNCS, vol. 11037, pp. 52–60. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00536-8_6CrossRefGoogle Scholar
  4. 4.
    Hiasa, Y., et al.: Cross-modality image synthesis from unpaired data using cyclegan: effects of gradient consistency loss and training data size. arXiv preprint arXiv:1803.06629 (2018)
  5. 5.
    Yang, H., et al.: Unpaired brain MR-to-CT synthesis using a structure-constrained cyclegan. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 174–182. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00889-5_20CrossRefGoogle Scholar
  6. 6.
    Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y.: Deformable convolutional networks. CoRR, abs/1703.06211, vol. 1, no. 2 (2017). 3Google Scholar
  7. 7.
    Wolterink, J.M., Dinkla, A.M., Savenije, M.H.F., Seevinck, P.R., van den Berg, C.A.T., Išgum, I.: Deep MR to CT synthesis using unpaired data. In: Tsaftaris, S.A., Gooya, A., Frangi, A.F., Prince, J.L. (eds.) SASHIMI 2017. LNCS, vol. 10557, pp. 14–23. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-68127-6_2CrossRefGoogle Scholar
  8. 8.
    Heinrich, M.P., et al.: Mind: modality independent neighbourhood descriptor for multi-modal deformable registration. Med. Image Anal. 16(7), 1423–1435 (2012)CrossRefGoogle Scholar
  9. 9.
    Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11(Oct), 2837–2854 (2010)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  11. 11.
    Papanastasiou, G., et al.: Multidimensional assessments of abdominal aortic aneurysms by magnetic resonance against ultrasound diameter measurements. In: Valdés Hernández, M., González-Castro, V. (eds.) MIUA 2017. CCIS, vol. 723, pp. 133–143. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-60964-5_12CrossRefGoogle Scholar
  12. 12.
    Hore, A., Ziou, D.: Image quality metrics: PSNR vs. SSIM. In: 2010 20th international conference on Pattern recognition (ICPR), pp. 2366–2369. IEEE (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Chengjia Wang
    • 1
    Email author
  • Giorgos Papanastasiou
    • 2
  • Sotirios Tsaftaris
    • 3
  • Guang Yang
    • 4
  • Calum Gray
    • 2
  • David Newby
    • 1
  • Gillian Macnaught
    • 2
  • Tom MacGillivray
    • 2
  1. 1.BHF Centre for Cardiovascular ScienceUniversity of EdinburghEdinburghUK
  2. 2.Edinburgh Imaging Facility QMRIUniversity of EdinburghEdinburghUK
  3. 3.Institute for Digital Communications, School of EngineeringUniversity of EdinburghEdinburghUK
  4. 4.National Heart and Lung InstituteImperial College LondonLondonUK

Personalised recommendations