Advertisement

Intraoperative Organ Motion Models with an Ensemble of Conditional Generative Adversarial Networks

  • Yipeng HuEmail author
  • Eli Gibson
  • Tom Vercauteren
  • Hashim U. Ahmed
  • Mark Emberton
  • Caroline M. Moore
  • J. Alison Noble
  • Dean C. Barratt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)

Abstract

In this paper, we describe how a patient-specific, ultrasound-probe-induced prostate motion model can be directly generated from a single preoperative MR image. Our motion model allows for sampling from the conditional distribution of dense displacement fields, is encoded by a generative neural network conditioned on a medical image, and accepts random noise as additional input. The generative network is trained by a minimax optimisation with a second discriminative neural network, tasked to distinguish generated samples from training motion data. In this work, we propose that (1) jointly optimising a third conditioning neural network that pre-processes the input image, can effectively extract patient-specific features for conditioning; and (2) combining multiple generative models trained separately with heuristically pre-disjointed training data sets can adequately mitigate the problem of mode collapse. Trained with diagnostic T2-weighted MR images from 143 real patients and 73,216 3D dense displacement fields from finite element simulations of intraoperative prostate motion due to transrectal ultrasound probe pressure, the proposed models produced physically-plausible patient-specific motion of prostate glands. The ability to capture biomechanically simulated motion was evaluated using two errors representing generalisability and specificity of the model. The median values, calculated from a 10-fold cross-validation, were 2.8 ± 0.3 mm and 1.7 ± 0.1 mm, respectively. We conclude that the introduced approach demonstrates the feasibility of applying state-of-the-art machine learning algorithms to generate organ motion models from patient images, and shows significant promise for future research.

References

  1. 1.
    Wang, Y., et al.: Towards personalized statistical deformable model and hybrid point matching for robust MR-TRUS registration. IEEE-TMI 35(2), 589–604 (2016)Google Scholar
  2. 2.
    Hu, Y., et al.: MR to ultrasound registration for image-guided prostate interventions. Med. Image Anal. 16(3), 687–703 (2012)CrossRefGoogle Scholar
  3. 3.
    Rueckert, D., et al.: Automatic construction of 3-D statistical deformation models of the brain using nonrigid registration. IEEE-TMI 22(8), 1014–1025 (2003)Google Scholar
  4. 4.
    Goodfellow, I., et al.: Generative adversarial nets. NIPS 2014, 2672–2680 (2014)Google Scholar
  5. 5.
    Zhu, Y., et al.: Segmentation of the left ventricle from cardiac MR images using a subject-specific dynamical model. IEEE-TMI 29(3), 669–687 (2010)Google Scholar
  6. 6.
    Hu, Y., et al.: Population-based prediction of subject-specific prostate deformation for MR-to-ultrasound image registration. Med. Image Anal. 26(1), 332–344 (2015)CrossRefGoogle Scholar
  7. 7.
    Pathak, D., et al.: Context encoders: feature learning by inpainting. In: CVPR 2016, pp. 2536–2544 (2016)Google Scholar
  8. 8.
    Gauthier, J.: Conditional generative adversarial nets for convolutional face generation. Stanford University Class Project Report (2014). CS231 N: CNNVR 2014, 5Google Scholar
  9. 9.
    Goodfellow, I.: NIPS 2016 Tutorial: Generative Adversarial Networks. arXiv preprint arXiv:1701.00160. (2016)
  10. 10.
    Salimans, T., et al.: Improved techniques for training gans. In: NIPS, pp. 2226–2234 (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yipeng Hu
    • 1
    • 2
    Email author
  • Eli Gibson
    • 1
  • Tom Vercauteren
    • 1
  • Hashim U. Ahmed
    • 3
  • Mark Emberton
    • 3
  • Caroline M. Moore
    • 3
  • J. Alison Noble
    • 2
  • Dean C. Barratt
    • 1
  1. 1.Centre for Medical Image ComputingUniversity College LondonLondonUK
  2. 2.Institute of Biomedical EngineeringUniversity of OxfordOxfordUK
  3. 3.Division of Surgery and Interventional ScienceUniversity College LondonLondonUK

Personalised recommendations