Abstract
Joint image registration and segmentation has long been an active area of research in medical imaging. Here, we reformulate this problem in a deep learning setting using adversarial learning. We consider the case in which fixed and moving images as well as their segmentations are available for training, while segmentations are not available during testing; a common scenario in radiotherapy. The proposed framework consists of a 3D end-to-end generator network that estimates the deformation vector field (DVF) between fixed and moving images in an unsupervised fashion and applies this DVF to the moving image and its segmentation. A discriminator network is trained to evaluate how well the moving image and segmentation align with the fixed image and segmentation. The proposed network was trained and evaluated on follow-up prostate CT scans for image-guided radiotherapy, where the planning CT contours are propagated to the daily CT images using the estimated DVF. A quantitative comparison with conventional registration using elastix showed that the proposed method improved performance and substantially reduced computation time, thus enabling real-time contour propagation necessary for online-adaptive radiotherapy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Lu, C., et al.: An integrated approach to segmentation and nonrigid registration for application in image-guided pelvic radiotherapy. Med. Image Anal. 15(5), 772–785 (2011)
Yezzi, A., et al.: A variational framework for integrating segmentation and registration through active contours. Med. Image Anal. 7(2), 171–185 (2003)
Unal, G., Slabaugh, G.: Coupled PDEs for non-rigid registration and segmentation. In: IEEE CVPR (2005)
Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680 (2014)
Kazeminia, S., et al.: GANs for Medical Image Analysis. arXiv:1809.06222v2 (2018)
Haskins, G., et al.: Deep Learning in Medical Image Registration: A Survey. arXiv:1903.02026v1 (2019)
Mahapatra, D., Ge, Z., Sedai, S., Chakravorty, R.: Joint registration and segmentation of Xray images using generative adversarial networks. In: Shi, Y., Suk, H.-I., Liu, M. (eds.) MLMI 2018. LNCS, vol. 11046, pp. 73–80. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_9
de Vos, B.D., et al.: A deep learning framework for unsupervised affine and deformable image registration. In: Medical Image Analysis, pp. 204–212. Springer, Heidelberg (2019)
Klein, S., et al.: Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging. 29(1), 196–205 (2010)
Arjovsky, M., et al.: Wasserstein GAN. arXiv:1701.07875v3 (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Gibson, E., et al.: NiftyNet: a deep-learning platform for medical imaging. Comput. Methods Programs Biomed. 158, 113–122 (2018)
Muren, L.P., et al.: Intensity-modulated radiotherapy of pelvic lymph nodes in locally advanced prostate cancer: planning procedures and early experiences. Int. J. Radiat. Oncol. Biol. Phys. 71, 4, 1034–1041 (2008)
Matin, et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. arXiv:1603.04467 (2017)
Isola, et al.: Image-to-Image Translation with Conditional Adversarial Networks. arXiv:1611.07004v3 (2016)
Qiao, Y.: Fast optimization methods for image registration in adaptive radiation therapy. Ph.D. thesis, Chapter 5, Leiden University Medical Center (2017). http://elastix.isi.uu.nl/marius/downloads/2017_t_Qiao.pdf
Acknowledgements
This study was financially supported by Varian Medical Systems and ZonMw, the Netherlands Organization for Health Research and Development, grant number 104003012. The dataset with contours were collected at Haukeland University Hospital, Bergen, Norway and were provided to us by responsible oncologist Svein Inge Helle and physicist Liv Bolstad Hysing; they are gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Elmahdy, M.S., Wolterink, J.M., Sokooti, H., Išgum, I., Staring, M. (2019). Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11769. Springer, Cham. https://doi.org/10.1007/978-3-030-32226-7_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-32226-7_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32225-0
Online ISBN: 978-3-030-32226-7
eBook Packages: Computer ScienceComputer Science (R0)
