Abstract
Delineation of anatomical structures is often the first step of many medical image analysis workflows. While convolutional neural networks achieve high performance, these do not incorporate anatomical shape information. We introduce a novel segmentation algorithm that uses Deep Conditional Shape models (DCSMs) as a core component. Using deep implicit shape representations, the algorithm learns a modality-agnostic shape model that can generate the signed distance functions for any anatomy of interest. To fit the generated shape to the image, the shape model is conditioned on anatomic landmarks that can be automatically detected or provided by the user. Finally, we add a modality dependent, lightweight refinement network to capture any fine details not represented by the implicit function. The proposed DCSM framework is evaluated on the problem of cardiac left ventricle (LV) segmentation from multiple 3D modalities (contrast-enhanced CT, non-contrasted CT, 3D echocardiography-3DE). We demonstrate that the automatic DCSM outperforms the baseline for non-contrasted CT without the local refinement, and with the refinement for contrasted CT and 3DE, especially with significant improvement in the Hausdorff distance. The semi-automatic DCSM with user-input landmarks, while only trained on contrasted CT, achieves greater than 92% Dice for all modalities. Both automatic DCSM with refinement, and semi-automatic DCSM achieve equivalent or better performance compared to inter-user variability for these modalities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, C., et al.: Deep learning for cardiac image segmentation: a review. Front. Cardiovasc. Med. 7, 25 (2020)
Yousef, R., Gupta, G., Yousef, N., Khari, M.: A holistic overview of deep learning approach in medical imaging. Multimed. Syst. 28, 881–914 (2022)
Zhang, P., Zhong, Y., Deng, Y., Tang, X., Li, X.: A survey on deep learning of small sample in biomedical image analysis (2019)
Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation (2019). ArXiv190105103 Cs
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3D reconstruction in function space (2019). ArXiv181203828 Cs
Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., Geiger, A.: Convolutional occupancy networks (2020). ArXiv200304618 Cs
Chou, G., Chugunov, I., Heide, F.: GenSDF: two-stage learning of generalizable signed distance functions (2022)
Heimann, T., Meinzer, H.-P.: Statistical shape models for 3D medical image segmentation: a review. Med. Image Anal. 13, 543–563 (2009)
Oktay, O., Ferrante, E., Kamnitsas, K., et al.: Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation. IEEE Trans. Med. Imaging 37, 384–395 (2018)
Li, S., Zhang, C., He, X.: Shape-aware semi-supervised 3D semantic segmentation for medical images. In: Martel, A.L., et al. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. LNCS, vol. 12261, pp 552–561. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_54
Painchaud, N., Skandarani, Y., Judge, T., Bernard, O., Lalande, A., Jodoin, P.-M.: Cardiac segmentation with strong anatomical guarantees. IEEE Trans. Med. Imaging 39, 3703–3713 (2020)
Yang, D., et al.: Automatic liver segmentation using an adversarial image-to-image network. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D., Duchesne, S. (eds.) Medical Image Computing and Computer Assisted Intervention − MICCAI 2017. MICCAI 2017. LNCS, vol. 10435, pp. 507–515. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_58
Ding, K., et al.: Graph convolutional networks for multi-modality medical imaging: methods, architectures, and clinical applications (2022)
Chen, Z., Zhang, H.: Learning Implicit Fields for Generative Shape Modeling (2019). ArXiv181202822 Cs
Michalkiewicz, M., Pontes, J.K., Jack, D., Baktashmotlagh, M., Eriksson, A.: Deep level sets: implicit surface representations for 3d shape inference (2019)
Xu, Q., Wang, W., Ceylan, D., Mech, R., Neumann, U.: DISN: deep implicit surface network for high-quality single-view 3D reconstruction (2021)
Amiranashvili, T., Menze, B., Zachow, S.: Learning shape reconstruction from sparse measurements with neural implicit functions. In: Proceedings of the 5th International Conference on Medical Imaging with Deep Learning
Chen, Y., Liu, S., Wang, X.: Learning continuous image representation with local implicit image function. In: 2021 IEEECVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8624–8634. IEEE, Nashville, TN, USA (2021)
Raju, A., Miao, S., Jin, D., Lu, L., Huang, J., Harrison, A.P.: Deep implicit statistical shape models for 3D medical image delineation (2022). ArXiv210402847 Cs
Zhou, Z., Shin, J., Zhang, L., Gurudu, S., Gotway, M., Liang, J.: Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4761–4772. IEEE, Honolulu, HI (2017)
Tajbakhsh, N., et al.: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans. Med. Imaging 35, 1299–1312 (2016)
Samala, R.K., Chan, H.-P., Hadjiiski, L.M., Helvie, M.A., Cha, K.H., Richter, C.D.: Multi-task transfer learning deep convolutional neural network: application to computer-aided diagnosis of breast cancer on mammograms. Phys. Med. Biol. 62, 8894–8908 (2017)
Moeskops, P., et al.: Deep learning for multi-task medical image segmentation in multiple modalities, pp 478–486 (2016)
Moeskops, P., Veta, M., Lafarge, M.W., Eppenhof, K.A.J., Pluim, J.P.W.: Adversarial training and dilated convolutions for brain MRI segmentation (2017)
Javanmardi, M., Tasdizen, T.: Domain adaptation for biomedical image segmentation using adversarial training. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 554–558. IEEE, Washington, DC (2018)
Yang, Q., Li, N., Zhao, Z., Fan, X., Chang, E.I.-C., Xu, Y.: MRI cross-modality image-to-image translation. Sci. Rep. 10, 3753 (2020)
Hoffman, J., et al.: CyCADA: cycle-consistent adversarial domain adaptation (2017). https://doi.org/10.48550/ARXIV.1711.03213
Snell, J., Swersky, K., Zemel, R.S.: Prototypical networks for few-shot learning (2017)
Bian, C., Yuan, C., Ma, K., Yu, S., Wei, D., Zheng, Y.: Domain adaptation meets zero-shot learning: an annotation-efficient approach to multi-modality medical image segmentation. IEEE Trans. Med. Imaging 41, 1043–1056 (2022)
Tancik, M., et al.: Fourier features let networks learn high frequency functions in low dimensional domains (2020)
Desai, A., Parikh, S., Kumari, S., Raman, S.: PointResNet: residual network for 3d point cloud segmentation and classification (2022)
Ghesu, F.-C., et al.: Multi-scale deep reinforcement learning for real-time 3D-landmark detection in CT scans. IEEE Trans. Pattern Anal. Mach. Intell. 41, 176–189 (2019)
Zheng, Y., Barbu, A., Georgescu, B., Scheuering, M., Comaniciu, D.: Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features. IEEE Trans. Med. Imaging 27, 1668–1681 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jacob, A.J., Sharma, P., Ruckert, D. (2024). Deep Conditional Shape Models for 3D Cardiac Image Segmentation. In: Camara, O., et al. Statistical Atlases and Computational Models of the Heart. Regular and CMRxRecon Challenge Papers. STACOM 2023. Lecture Notes in Computer Science, vol 14507. Springer, Cham. https://doi.org/10.1007/978-3-031-52448-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-52448-6_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-52447-9
Online ISBN: 978-3-031-52448-6
eBook Packages: Computer ScienceComputer Science (R0)