Resolve Intraoperative Brain Shift as Imitation Game

  • Xia ZhongEmail author
  • Siming Bayer
  • Nishant Ravikumar
  • Norbert Strobel
  • Annette Birkhold
  • Markus Kowarschik
  • Rebecca Fahrig
  • Andreas Maier
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11042)


Soft tissue deformation induced by craniotomy and tissue manipulation (brain shift) limits the use of preoperative image overlay in an image-guided neurosurgery, and therefore reduces the accuracy of the surgery as a consequence. An inexpensive modality to compensate for the brain shift in real-time is Ultrasound (US). The core subject of research in this context is the non-rigid registration of preoperative MR and intraoperative US images. In this work, we propose a learning based approach to address this challenge. Resolving intraoperative brain shift is considered as an imitation game, where the optimal action (displacement) for each landmark on MR is trained with a multi-task network. The result shows a mean target error of 1.21 ± 0.55 mm.


  1. 1.
    Bayer, S., Maier, A., Ostermeier, M., Fahrig, R.: Intraoperative imaging modalities and compensation for brain shift in tumor resection surgery. Int. J. Biomed. Imaging 2017 (2017)CrossRefGoogle Scholar
  2. 2.
    Reinertsen, I., Descoteaux, M., Drouin, S., Siddiqi, K., Collins, D.L.: Vessel driven correction of brain shift. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004. LNCS, vol. 3217, pp. 208–216. Springer, Heidelberg (2004). Scholar
  3. 3.
    Reinertsen, I., Lindseth, F., Unsgaard, G., Collins, D.L.: Clinical validation of vessel-based registration for correction of brain-shift. Med. Image Anal. 11(6), 673–684 (2007)CrossRefGoogle Scholar
  4. 4.
    Farnia, P., Ahmadian, A., Khoshnevisan, A., Jaberzadeh, A., Serej, N.D., Kazerooni, A.F.: An efficient point based registration of intra-operative ultrasound images with MR images for computation of brain shift; a phantom study. In: IEEE EMBC 2011, pp. 8074–8077, August 2011Google Scholar
  5. 5.
    Pennec, X., Cachier, P., Ayache, N.: Tracking brain deformations in time-sequences of 3D US images. Pattern Recognit. Lett. 24(4–5), 801–813 (2003)CrossRefGoogle Scholar
  6. 6.
    Letteboer, M.M.J., Willems, P.W.A., Viergever, M.A., Niessen, W.J.: Non-rigid registration of 3D ultrasound images of brain tumours acquired during neurosurgery. In: Ellis, R.E., Peters, T.M. (eds.) MICCAI 2003. LNCS, vol. 2879, pp. 408–415. Springer, Heidelberg (2003). Scholar
  7. 7.
    Arbel, T., Morandi, X., Comeau, R.M., Collins, D.L.: Automatic non-linear MRI-ultrasound registration for the correction of intra-operative brain deformations. Comput. Aided Surg. 9, 123–136 (2004)CrossRefGoogle Scholar
  8. 8.
    Rivaz, H., Chen, S.S., Collins, D.L.: Automatic deformable MR-ultrasound registration for image-guided neurosurgery. IEEE Trans. Med. Imaging 34(2), 366–380 (2015)CrossRefGoogle Scholar
  9. 9.
    Krebs, J., et al.: Robust non-rigid registration through agent-based action learning. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 344–352. Springer, Cham (2017). Scholar
  10. 10.
    Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robot. Auton. Syst. 57(5), 469–483 (2009)CrossRefGoogle Scholar
  11. 11.
    Bojarski, M., et al.: End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016)
  12. 12.
    Zhong, X., et al.: Action learning for 3D point cloud based organ segmentation. arXiv preprint arXiv:1806.05724 (2018)
  13. 13.
    Simard, P.Y., Steinkraus, D., Platt, J.C., et al.: Best practices for convolutional neural networks applied to visual document analysis. In: ICDAR, vol. 3, pp. 958–962 (2003)Google Scholar
  14. 14.
    Xiao, Y., Fortin, M., Unsgård, G., Rivaz, H., Reinertsen, I.: REtroSpective Evaluation of Cerebral Tumors (RESECT): a clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries. Med. Phys. 44, 3875–3882 (2017)CrossRefGoogle Scholar
  15. 15.
    Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3D classification and segmentation. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 1(2), 4 (2017)Google Scholar
  16. 16.
    Reddi, S.J., Kale, S., Kumar, S.: On the convergence of adam and beyond. In: International Conference on Learning Representations (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Xia Zhong
    • 1
    Email author
  • Siming Bayer
    • 1
  • Nishant Ravikumar
    • 1
  • Norbert Strobel
    • 4
  • Annette Birkhold
    • 2
  • Markus Kowarschik
    • 2
  • Rebecca Fahrig
    • 2
  • Andreas Maier
    • 1
    • 3
  1. 1.Pattern Recognition LabFriedrich-Alexander University Erlangen-NürnbergErlangenGermany
  2. 2.Siemens Healthcare GmbHForchheimGermany
  3. 3.Erlangen Graduate School in Advanced Optical Technologies (SAOT)ErlangenGermany
  4. 4.Fakultät für ElektrotechnikHochschule für angewandte Wissenschaften Würzburg-SchweinfurtWürzburg and SchweinfurtGermany

Personalised recommendations