Advertisement

LSRC: A Long-Short Range Context-Fusing Framework for Automatic 3D Vertebra Localization

  • Jintai Chen
  • Yanjie Wang
  • Ruoqian Guo
  • Bohan Yu
  • Tingting Chen
  • Wenzhe Wang
  • Ruiwei Feng
  • Danny Z. Chen
  • Jian WuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11769)

Abstract

Automatic localization and identification of vertebrae in computed tomography (CT) images is a challenging task, due to the specific spine structure, complex pathological conditions, and limited field-of-view in 3D CT images. The local and long-range contextual information is especially useful for solving this problem. To explore both the local and long-range contextual information of vertebrae, in this paper, we propose a new framework called Long-Short Range Context-fusing framework (LSRC), combining a 3D local semantic network and a 2D long-range contextual network. The 3D local semantic network, using 3D CT images, produces 3D heat maps corresponding to the locations of all vertebrae. The 3D heat maps and CT images are respectively projected onto the sagittal plane and coronal plane, and fed to the 2D long-range contextual network, in which a convolutional encoder-decoder module integrates the long-range contextual information of these two views. Two refined heat maps in the sagittal and coronal planes are generated by a globally-refining module, adjusting vertebra locations using the global location information in an attention manner. Experiments on a public dataset of 302 3D spine CT scans with various pathological conditions show that our new framework outperforms state-of-the-art methods.

Notes

Acknowledgment

The research of the Real Doctor AI Research Centre was partially supported by the Subject of the Major Commissioned Project “Research on China’s Image in the Big Data” of Zhejiang Province’s Social Science Planning Advantage Discipline “Evaluation and Research on the Present Situation of China’s Image” No. 16YSXK01ZD-2YB, Ministry of Education of China under grant No. 2017PT18, the Zhejiang University Education Foundation under grants No. K18-511120-004, No. K17-511120-017, and No. K17-518051-021, the Major Scientific Project of Zhejiang Lab under grant No. 2018DG0ZX01, the National Natural Science Foundation of China under grant No. 61672453, and the Key Laboratory of Medical Neurobiology of Zhejiang Province. The research of D.Z. Chen was supported in part by NSF Grant CCF-1617735. We like to thank three anonymous reviewers for their helpful suggestions.

References

  1. 1.
    Chen, H., et al.: Automatic localization and identification of vertebrae in spine CT via a joint learning model with deep neural networks. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 515–522. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24553-9_63CrossRefGoogle Scholar
  2. 2.
    Girshick, R.: Fast R-CNN. In: ICCV, pp. 1440–1448 (2015)Google Scholar
  3. 3.
    Glocker, B., Feulner, J., Criminisi, A., Haynor, D.R., Konukoglu, E.: Automatic localization and identification of vertebrae in arbitrary field-of-view CT scans. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7512, pp. 590–598. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33454-2_73CrossRefGoogle Scholar
  4. 4.
    Glocker, B., Zikic, D., Konukoglu, E., Haynor, D.R., Criminisi, A.: Vertebrae localization in pathological spine CT via dense classification from sparse annotations. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 262–270. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-40763-5_33CrossRefGoogle Scholar
  5. 5.
    Luvizon, D., et al.: Human pose regression by combining indirect part detection and contextual information. arXiv preprint arXiv:1710.02322 (2017)
  6. 6.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  7. 7.
    Sekuboyina, A., et al.: Btrfly net: vertebrae labelling with energy-based adversarial learning of local spine prior. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 649–657. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00937-3_74CrossRefGoogle Scholar
  8. 8.
    Suzani, A., Seitel, A., Liu, Y., Fels, S., Rohling, R.N., Abolmaesumi, P.: Fast automatic vertebrae detection and localization in pathological CT scans - a deep learning approach. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 678–686. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_81CrossRefGoogle Scholar
  9. 9.
    Yang, D., et al.: Automatic vertebra labeling in large-scale 3D CT using deep image-to-image network with message passing and sparsity regularization. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 633–644. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-59050-9_50CrossRefGoogle Scholar
  10. 10.
    Yang, D., et al.: Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 498–506. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-66179-7_57CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Jintai Chen
    • 1
    • 2
  • Yanjie Wang
    • 1
    • 2
  • Ruoqian Guo
    • 1
    • 2
  • Bohan Yu
    • 1
    • 2
  • Tingting Chen
    • 1
    • 2
  • Wenzhe Wang
    • 1
    • 2
  • Ruiwei Feng
    • 1
    • 2
  • Danny Z. Chen
    • 3
  • Jian Wu
    • 1
    • 2
    Email author
  1. 1.College of Computer Science and TechnologyZhejiang UniversityHangzhouChina
  2. 2.Real Doctor AI Research CentreZhejiang UniversityHangzhouChina
  3. 3.Department of Computer Science and EngineeringUniversity of Notre DameNotre DameUSA

Personalised recommendations