Abstract
This chapter aims at providing an introduction to deep learning-based medical image segmentation. First, the reader is guided through the inherent challenges of medical image segmentation, for which actual approaches to overcome those limitations are discussed. Secondly, supervised and semi-supervised architectures are described, where encoder-decoder type networks are the most widely employed ones. Nonetheless, generative adversarial network-based semi-supervised approaches have recently gained the attention of the scientific community. The shift from traditional 2D to 3D architectures is also discussed, as well as the most common loss functions to improve the performance of medical image segmentation approaches. Finally, some future trends and conclusion are described.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Albarqouni, Shadi, Baur, Christoph, Achilles, Felix, Belagiannis, Vasileios, Demirci, Stefanie, Navab, Nassir: AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans. Med. Imaging 35(5), 1313–1321 (2016)
Alom, Z., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on u-net (R2U-Net) for medical image segmentation, p. 12
Anthimopoulos, M., Christodoulidis, S., Ebner, L., Geiser, T., Christe, A., Mougiakakou, S.: Semantic segmentation of pathological lung tissue with dilated fully convolutional networks. IEEE J. Biomed. Health Inform. pp. 1 (2018). arXiv:1803.06167
Apostolopoulos, S., De Zanet, S., Ciller, C., Wolf, S., Sznitman, R.: Pathological OCT retinal layer segmentation using branch residual u-shape networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017. Lecture Notes in Computer Science, pp. 294–301. Springer International Publishing (2017)
Bai, W., Sinclair, M., Tarroni, G., Oktay, O., Rajchl, M., Vaillant, G., Lee, A.M., Aung, N., Lukaschuk, E., Sanghvi, M.M., Zemrak, F., Fung, K., Paiva, J.M., Carapella, V., Kim, Y.J., Suzuki, H., Kainz, B., Matthews, P.M., Petersen, S.E., Piechnik, S.K., Neubauer, S., Glocker, B., Rueckert, D.: Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J. Cardiovasc. Magn. Reson. 20 (2018)
Ben-Cohen, A., Diamant, I., Klang, E., Amitai, M., Greenspan, H.: fully convolutional network for liver segmentation and lesions detection. In: Carneiro, G., Mateus, D., Peter, L., Bradley, A., Tavares, J.M.R.S., Belagiannis, V., Papa, J.P., Nascimento, J.C., Loog, M., Lu, Z., Cardoso, J.S., Cornebise, J. (eds.) Deep Learning and Data Labeling for Medical Applications, vol. 10008, pp. 77–85. Springer International Publishing, Cham (2016)
Bortsova, G., van Tulder, G., Dubost, F., Peng, T., Navab, N., van der Lugt, A., Bos, D., De Bruijne, M.: Segmentation of intracranial arterial calcification with deeply supervised residual dropout networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L. and Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 356–364. Springer International Publishing (2017)
Bui, T.D., Shin, J., Moon, T.: 3D densely convolutional networks for volumetric segmentation (2017)
Cai, J., Lu, L., Zhang, Z., Xing, F., Yang, L., Yin, Q.: Pancreas segmentation in MRI using graph-based decision fusion on convolutional neural networks. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G. and Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 442–450. Springer International Publishing (2016)
Carneiro, G., Zheng, Y., Xing, F., Yang, L.: Review of deep learning methods in mammography, cardiovascular, and microscopy image analysis. In: Lu, L., Zheng, Y., Carneiro, G., Yang, L. (eds.) Deep Learning and Convolutional Neural Networks for Medical Image Computing: Precision Medicine. High Performance and Large-Scale Datasets, Advances in Computer Vision and Pattern Recognition, pp. 11–32. Springer International Publishing, Cham (2017)
Chudzik, P., Majumdar, S., Caliva, F., Al-Diri, B., Hunter, A.: Exudate segmentation using fully convolutional neural networks and inception modules. In: Medical Imaging 2018: Image Processing, vol. 10574, pp. 1057430. International Society for Optics and Photonics (2018)
Çiçed, Ö., Abdulkadir, A., Lienkamp, S.S. Brox, T., Ronneberger, O.: 3D U-net: learning dense volumetric segmentation from sparse annotation (2016). arXiv:1606.06650
Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 2843–2851. Curran Associates, Inc. (2012)
Dolz, J., Gopinath, K., Yuan, J., Lombaert, H., Desrosiers, C., Ayed, I.B.: HyperDense-Net: a hyper-densely connected CNN for multi-modal image segmentation (2018)
Dou, Q., Chen, H., Jin, Y., Yu, L., Qin, J., Heng, P.A.: 3D deeply supervised network for automatic liver segmentation from CT volumes. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.), Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 149–157. Springer International Publishing (2016)
Fritscher, K., Raudaschl, P., Zaffino, P., Spadea, M.F., Sharp, G.C., Schubert, R.: Deep neural networks for fast segmentation of 3D medical images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 158–165. Springer International Publishing (2016)
Gibson, E., Giganti, F., Hu, Y., Bonmati, E., Bandula, S., Gurusamy, K., Davidson, B.R., Pereira, S.P., Clarkson, M.J. and Barratt, D.C.: towards image-guided pancreas and biliary endoscopy: automatic multi-organ segmentation on abdominal CT with dense dilated networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.), Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 728–736. Springer International Publishing (2017)
Guerrero, R., Qin, C., Oktay, O., Bowles, C., Chen, L., Joules, R., Wolz, R., Valds-Hernndez, M.C., Dickie, D.A., Wardlaw, J., Rueckert, D.: White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks. NeuroImage: Clinical 17, 918–934 (2017)
Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017). arXiv:1505.03540
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)
Heinrich, L., Funke, J., Pape, C., Nunez-Iglesias, J., Saalfeld, S.: Synaptic cleft segmentation in non-isotropic volume electron microscopy of the complete drosophila brain. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lpez, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted InterventionMICCAI 2018, Lecture Notes in Computer Science, pp. 317–325. Springer International Publishing (2018)
Heinrich, M.P., Oktay, O.: BRIEFnet: deep pancreas segmentation using binary sparse convolutions. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 329–337. Springer International Publishing (2017)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks (2016). arXiv:1608.06993
Hussain, S., Anwar, S.M., Majid, M.: Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 248–261 (2018). arXiv:1708.00377
Iqbal, Talha, Ali, Hazrat: Generative adversarial network for medical images (MI-GAN). J. Med. Syst. 42(11), 231 (2018)
Jog, A., Fischl, B. (2018) Pulse sequence resilient fast brain segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lpez, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2018, Lecture Notes in Computer Science, pp. 654–662. Springer International Publishing (2018)
Keshwani, D., Kitamura, Y., Li, Y.: Computation of total kidney volume from CT images in autosomal dominant polycystic kidney disease using multi-task 3D convolutional neural networks, p. 8
Koziski, M., Mosinska, A., Salzmann, M., Fua, P.: Learning to segment 3D linear structures using only 2D annotations. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lpez, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2018, Lecture Notes in Computer Science, pp. 283–291. Springer International Publishing (2018)
Kumar, A., Agarwala, S., Dhara, A.K., Nandi, D., Thakur, S.B., Bhadra, A.K., Sadhu, A.: Segmentation of lung field in HRCT images using u-net based fully convolutional networks, p. 10
Li, H., Chen, D., Nailon, W.H., Davies, M.E., Laurenson, D.: Improved breast mass segmentation in mammograms with conditional residual u-net (2018). arXiv:1808.08885
Li, J., Sarma, K.V., Ho, K.C., Gertych, A., Knudsen, B.S., Arnold, C.W.: A multi-scale u-net for semantic segmentation of histological images from radical prostatectomies. In: AMIA Annual Symposium Proceedings, 2017, pp. 1140–1148 (2018)
Li, Rongjian, Zeng, Tao, Peng, Hanchuan, Ji, Shuiwang: Deep learning segmentation of optical microscopy images improves 3-D neuron reconstruction. IEEE Trans. Med. Imaging 36(7), 1533–1541 (2017)
Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., Van Der Laak, J.A., Van Ginneken, B., Sanchez, C.I.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017). arXiv:1702.05747
Liu, X., Deng, Z., Yang, Y.: Recent progress in semantic image segmentation. Artif. Intell. Rev. (2018). arXiv:1809.10198
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation, p. 10
Lopez-Linares, K., Aranjuelo, N., Kabongo, L., Maclair, G., Lete, N., Ceresa, M., Garcia-Familiar, A., Macia, I., Ballester, M.A.G.: Fully automatic detection and segmentation of abdominal aortic thrombus in post-operative CTA images using deep convolutional neural networks. Med. Image Anal. 46, 202–214 (2018)
Luc, P., Couprie, C., Chintala, S. and Verbeek, J.: Semantic segmentation using adversarial networks (2016). arXiv:1611.08408
Meng, Q., Roth, H.R., Kitasaka, T., Oda, M., Ueno, J., Mori, K.: Tracking and segmentation of the airways in chest CT using a fully convolutional network. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 198–207. Springer International Publishing (2017)
Milletari, F., Ahmadi, S.A., Kroll, C., Plate, A., Rozanski, V., Maiostre, J., Levin, J., Dietrich, O., Ertl-Wagner, B., Botzel, K., Navab, N.: Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound (2016). arXiv:1601.07014
Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: Fully convolutional neural networks for volumetric medical image segmentation (2016). arXiv:1606.04797
Moeskops, P., Wolterink, J.M., van der Velden, B.H., Gilhuijs, K.G., Leiner, T., Viergever, M.A., Isgum, I.: Deep learning for multi-task medical image segmentation in multiple modalities. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 478–486. Springer International Publishing (2016)
Novikov, A.A., Major, D., Wimmer, M., Lenis, D., Buhler, K.: Deep sequential segmentation of organs in volumetric medical scans (2018). arXiv:1807.02437
Novikov, A.A., Lenis, D., Major, D., Hladuvka, J., Wimmer, M. and Buhler, K.: Fully convolutional architectures for multi-class segmentation in chest radiographs (2017). arXiv:1701.08816
Onieva, J., Andresen, L., Holsting, J.Q., Rahaghi, F.N., Ballester, M.A.G., Estepar, R.S.J., Roman, K.L.L., de La Bruere, I.: 3D pulmonary artery segmentation from CTA scans using deep learning with realistic data augmentation
Perone, Christian S., Calabrese, Evan, Cohen-Adad, Julien: Spinal cord gray matter segmentation using deep dilated convolutions. Sci. Rep. 8(1), 5966 (2018)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI), vol. 9351 of LNCS, pp. 234–241. Springer, Berlin (2015). arXiv:1505.04597
Roth, H.R., Shen, C., Oda, H., Oda, M., Hayashi, Y., Misawa, K., Mori, K.: Deep learning and its application to medical image segmentation (2018). arXiv:1803.08691
Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3D fully convolutional deep networks (2017). arXiv:1706.05721
Shen, C., Roth, H.R., Oda, H., Oda, M., Hayashi, Y., Misawa, K., Mori, K.: On the influence of Dice loss function in multi-class organ segmentation of abdominal CT using 3D fully convolutional networks (2018). arXiv:1801.05912
Shen, Dinggang, Guorong, Wu, Suk, Heung-Il: Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017)
Shin, H.C., Tenenholtz, N.A., Rogers, J.K., Schwarz, C.G., Senjem, M.L., Gunter, J.L., Andriole, K.P., Michalski, M.: Medical image synthesis for data augmentation and anonymization using generative adversarial networks (2018). arXiv:1807.10225
Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training (2016). arXiv:1612.07828
Stollenga, M.F., Byeon, W., Liwicki, M., Schmidhuber, J.: Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation (2015). arXiv:1506.07452
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going Deeper with Convolutions (2014). arXiv:1409.4842
Tai, L., Ye, H., Ye, Q., Liu, M.: PCA-aided fully convolutional networks for semantic segmentation of multi-channel fMRI (2016). arXiv:1610.01732
Vesal, S., Ravikumar, N., Maier, A.: Dilated convolutions in neural networks for left atrial segmentation in 3D gadolinium enhanced-MRI (2018). arXiv:1808.01673
Wolterink, J.M., Kamnitsas, K., Ledig, C. and Isgum, I.: Generative adversarial networks and adversarial methods in biomedical image analysis (2018). arXiv:1810.10352
Yang, D., Xu, D., Zhou, S.K., Georgescu, B., Chen, M., Grbic, S., Metaxas, D., Comaniciu, D.: Automatic liver segmentation using an adversarial image-to-image network. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L. Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 507–515. Springer International Publishing (2017)
Yang, L., Zhang, Y., Guldner, I.H., Zhang, S., Chen, D.Z.: 3D segmentation of glial cells using fully convolutional networks and k-terminal cut. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 658–666. Springer International Publishing (2016)
Yang, X., Yu, L., Li, S., Wang, X., Wang, N., Qin, J., Ni, D., Heng, P.A.: Towards automatic semantic segmentation in volumetric ultrasound. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 711–719. Springer International Publishing (2017)
Yi, X., Walia, E., Babyn, P.: Generative adversarial network in medical imaging: a review (2018). arXiv:1809.07294
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions (2015). arXiv:1511.07122
Yu, L., Cheng, J.Z., Dou, Q., Yang, X., Chen, H., Qin, J., Heng, P.A.: Automatic 3D cardiovascular MR segmentation with densely-connected volumetric convnets. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 287–295. Springer International Publishing (2017)
Zhou, X., Ito, T., Takayama, R., Wang, S., Hara, T., Fujita, H.: Three-dimensional CT image segmentation by combining 2D fully convolutional network with 3D majority voting. In Carneiro, G., Mateus, D., Peter, L., Bradley, A., Tavares, J.M.R.S, Belagiannis, V., Papa, J.P., Nascimento, J.C., Loog, M., Lu, Z., Cardoso, J.S., Cornebise, J. (eds.) Deep Learning and Data Labeling for Medical Applications, Lecture Notes in Computer Science, pp. 111–120. Springer International Publishing (2016)
Zhou, Y., Xie, L., Fishman, E.K., Yuille, A.L.: Deep supervision for pancreatic cyst segmentation in abdominal CT scans. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 222–230. Springer International Publishing (2017)
Zhou, Y., Xie, L., Shen, W., Wang, Y., Fishman, E.K., Yuille, A.L.: A fixed-point model for pancreas segmentation in abdominal CT scans. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 693–701. Springer International Publishing (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
López-Linares Román, K., García Ocaña, M.I., Lete Urzelai, N., González Ballester, M.Á., Macía Oliver, I. (2020). Medical Image Segmentation Using Deep Learning. In: Chen, YW., Jain, L. (eds) Deep Learning in Healthcare. Intelligent Systems Reference Library, vol 171. Springer, Cham. https://doi.org/10.1007/978-3-030-32606-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-32606-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32605-0
Online ISBN: 978-3-030-32606-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)