Skip to main content

Medical Image Segmentation Using Deep Learning

  • Chapter
  • First Online:
Deep Learning in Healthcare

Abstract

This chapter aims at providing an introduction to deep learning-based medical image segmentation. First, the reader is guided through the inherent challenges of medical image segmentation, for which actual approaches to overcome those limitations are discussed. Secondly, supervised and semi-supervised architectures are described, where encoder-decoder type networks are the most widely employed ones. Nonetheless, generative adversarial network-based semi-supervised approaches have recently gained the attention of the scientific community. The shift from traditional 2D to 3D architectures is also discussed, as well as the most common loss functions to improve the performance of medical image segmentation approaches. Finally, some future trends and conclusion are described.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Albarqouni, Shadi, Baur, Christoph, Achilles, Felix, Belagiannis, Vasileios, Demirci, Stefanie, Navab, Nassir: AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans. Med. Imaging 35(5), 1313–1321 (2016)

    Article  Google Scholar 

  2. Alom, Z., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on u-net (R2U-Net) for medical image segmentation, p. 12

    Google Scholar 

  3. Anthimopoulos, M., Christodoulidis, S., Ebner, L., Geiser, T., Christe, A., Mougiakakou, S.: Semantic segmentation of pathological lung tissue with dilated fully convolutional networks. IEEE J. Biomed. Health Inform. pp. 1 (2018). arXiv:1803.06167

  4. Apostolopoulos, S., De Zanet, S., Ciller, C., Wolf, S., Sznitman, R.: Pathological OCT retinal layer segmentation using branch residual u-shape networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017. Lecture Notes in Computer Science, pp. 294–301. Springer International Publishing (2017)

    Google Scholar 

  5. Bai, W., Sinclair, M., Tarroni, G., Oktay, O., Rajchl, M., Vaillant, G., Lee, A.M., Aung, N., Lukaschuk, E., Sanghvi, M.M., Zemrak, F., Fung, K., Paiva, J.M., Carapella, V., Kim, Y.J., Suzuki, H., Kainz, B., Matthews, P.M., Petersen, S.E., Piechnik, S.K., Neubauer, S., Glocker, B., Rueckert, D.: Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J. Cardiovasc. Magn. Reson. 20 (2018)

    Google Scholar 

  6. Ben-Cohen, A., Diamant, I., Klang, E., Amitai, M., Greenspan, H.: fully convolutional network for liver segmentation and lesions detection. In: Carneiro, G., Mateus, D., Peter, L., Bradley, A., Tavares, J.M.R.S., Belagiannis, V., Papa, J.P., Nascimento, J.C., Loog, M., Lu, Z., Cardoso, J.S., Cornebise, J. (eds.) Deep Learning and Data Labeling for Medical Applications, vol. 10008, pp. 77–85. Springer International Publishing, Cham (2016)

    Google Scholar 

  7. Bortsova, G., van Tulder, G., Dubost, F., Peng, T., Navab, N., van der Lugt, A., Bos, D., De Bruijne, M.: Segmentation of intracranial arterial calcification with deeply supervised residual dropout networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L. and Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 356–364. Springer International Publishing (2017)

    Google Scholar 

  8. Bui, T.D., Shin, J., Moon, T.: 3D densely convolutional networks for volumetric segmentation (2017)

    Google Scholar 

  9. Cai, J., Lu, L., Zhang, Z., Xing, F., Yang, L., Yin, Q.: Pancreas segmentation in MRI using graph-based decision fusion on convolutional neural networks. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G. and Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 442–450. Springer International Publishing (2016)

    Google Scholar 

  10. Carneiro, G., Zheng, Y., Xing, F., Yang, L.: Review of deep learning methods in mammography, cardiovascular, and microscopy image analysis. In: Lu, L., Zheng, Y., Carneiro, G., Yang, L. (eds.) Deep Learning and Convolutional Neural Networks for Medical Image Computing: Precision Medicine. High Performance and Large-Scale Datasets, Advances in Computer Vision and Pattern Recognition, pp. 11–32. Springer International Publishing, Cham (2017)

    Chapter  Google Scholar 

  11. Chudzik, P., Majumdar, S., Caliva, F., Al-Diri, B., Hunter, A.: Exudate segmentation using fully convolutional neural networks and inception modules. In: Medical Imaging 2018: Image Processing, vol. 10574, pp. 1057430. International Society for Optics and Photonics (2018)

    Google Scholar 

  12. Çiçed, Ö., Abdulkadir, A., Lienkamp, S.S. Brox, T., Ronneberger, O.: 3D U-net: learning dense volumetric segmentation from sparse annotation (2016). arXiv:1606.06650

  13. Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 2843–2851. Curran Associates, Inc. (2012)

    Google Scholar 

  14. Dolz, J., Gopinath, K., Yuan, J., Lombaert, H., Desrosiers, C., Ayed, I.B.: HyperDense-Net: a hyper-densely connected CNN for multi-modal image segmentation (2018)

    Google Scholar 

  15. Dou, Q., Chen, H., Jin, Y., Yu, L., Qin, J., Heng, P.A.: 3D deeply supervised network for automatic liver segmentation from CT volumes. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.), Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 149–157. Springer International Publishing (2016)

    Google Scholar 

  16. Fritscher, K., Raudaschl, P., Zaffino, P., Spadea, M.F., Sharp, G.C., Schubert, R.: Deep neural networks for fast segmentation of 3D medical images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 158–165. Springer International Publishing (2016)

    Google Scholar 

  17. Gibson, E., Giganti, F., Hu, Y., Bonmati, E., Bandula, S., Gurusamy, K., Davidson, B.R., Pereira, S.P., Clarkson, M.J. and Barratt, D.C.: towards image-guided pancreas and biliary endoscopy: automatic multi-organ segmentation on abdominal CT with dense dilated networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.), Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 728–736. Springer International Publishing (2017)

    Chapter  Google Scholar 

  18. Guerrero, R., Qin, C., Oktay, O., Bowles, C., Chen, L., Joules, R., Wolz, R., Valds-Hernndez, M.C., Dickie, D.A., Wardlaw, J., Rueckert, D.: White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks. NeuroImage: Clinical 17, 918–934 (2017)

    Article  Google Scholar 

  19. Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017). arXiv:1505.03540

    Article  Google Scholar 

  20. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)

    Google Scholar 

  21. Heinrich, L., Funke, J., Pape, C., Nunez-Iglesias, J., Saalfeld, S.: Synaptic cleft segmentation in non-isotropic volume electron microscopy of the complete drosophila brain. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lpez, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted InterventionMICCAI 2018, Lecture Notes in Computer Science, pp. 317–325. Springer International Publishing (2018)

    Google Scholar 

  22. Heinrich, M.P., Oktay, O.: BRIEFnet: deep pancreas segmentation using binary sparse convolutions. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 329–337. Springer International Publishing (2017)

    Google Scholar 

  23. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks (2016). arXiv:1608.06993

  24. Hussain, S., Anwar, S.M., Majid, M.: Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 248–261 (2018). arXiv:1708.00377

    Article  Google Scholar 

  25. Iqbal, Talha, Ali, Hazrat: Generative adversarial network for medical images (MI-GAN). J. Med. Syst. 42(11), 231 (2018)

    Article  Google Scholar 

  26. Jog, A., Fischl, B. (2018) Pulse sequence resilient fast brain segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lpez, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2018, Lecture Notes in Computer Science, pp. 654–662. Springer International Publishing (2018)

    Google Scholar 

  27. Keshwani, D., Kitamura, Y., Li, Y.: Computation of total kidney volume from CT images in autosomal dominant polycystic kidney disease using multi-task 3D convolutional neural networks, p. 8

    Google Scholar 

  28. Koziski, M., Mosinska, A., Salzmann, M., Fua, P.: Learning to segment 3D linear structures using only 2D annotations. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lpez, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2018, Lecture Notes in Computer Science, pp. 283–291. Springer International Publishing (2018)

    Google Scholar 

  29. Kumar, A., Agarwala, S., Dhara, A.K., Nandi, D., Thakur, S.B., Bhadra, A.K., Sadhu, A.: Segmentation of lung field in HRCT images using u-net based fully convolutional networks, p. 10

    Google Scholar 

  30. Li, H., Chen, D., Nailon, W.H., Davies, M.E., Laurenson, D.: Improved breast mass segmentation in mammograms with conditional residual u-net (2018). arXiv:1808.08885

  31. Li, J., Sarma, K.V., Ho, K.C., Gertych, A., Knudsen, B.S., Arnold, C.W.: A multi-scale u-net for semantic segmentation of histological images from radical prostatectomies. In: AMIA Annual Symposium Proceedings, 2017, pp. 1140–1148 (2018)

    Google Scholar 

  32. Li, Rongjian, Zeng, Tao, Peng, Hanchuan, Ji, Shuiwang: Deep learning segmentation of optical microscopy images improves 3-D neuron reconstruction. IEEE Trans. Med. Imaging 36(7), 1533–1541 (2017)

    Article  Google Scholar 

  33. Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., Van Der Laak, J.A., Van Ginneken, B., Sanchez, C.I.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017). arXiv:1702.05747

    Article  Google Scholar 

  34. Liu, X., Deng, Z., Yang, Y.: Recent progress in semantic image segmentation. Artif. Intell. Rev. (2018). arXiv:1809.10198

  35. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation, p. 10

    Google Scholar 

  36. Lopez-Linares, K., Aranjuelo, N., Kabongo, L., Maclair, G., Lete, N., Ceresa, M., Garcia-Familiar, A., Macia, I., Ballester, M.A.G.: Fully automatic detection and segmentation of abdominal aortic thrombus in post-operative CTA images using deep convolutional neural networks. Med. Image Anal. 46, 202–214 (2018)

    Article  Google Scholar 

  37. Luc, P., Couprie, C., Chintala, S. and Verbeek, J.: Semantic segmentation using adversarial networks (2016). arXiv:1611.08408

  38. Meng, Q., Roth, H.R., Kitasaka, T., Oda, M., Ueno, J., Mori, K.: Tracking and segmentation of the airways in chest CT using a fully convolutional network. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 198–207. Springer International Publishing (2017)

    Google Scholar 

  39. Milletari, F., Ahmadi, S.A., Kroll, C., Plate, A., Rozanski, V., Maiostre, J., Levin, J., Dietrich, O., Ertl-Wagner, B., Botzel, K., Navab, N.: Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound (2016). arXiv:1601.07014

  40. Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: Fully convolutional neural networks for volumetric medical image segmentation (2016). arXiv:1606.04797

  41. Moeskops, P., Wolterink, J.M., van der Velden, B.H., Gilhuijs, K.G., Leiner, T., Viergever, M.A., Isgum, I.: Deep learning for multi-task medical image segmentation in multiple modalities. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 478–486. Springer International Publishing (2016)

    Google Scholar 

  42. Novikov, A.A., Major, D., Wimmer, M., Lenis, D., Buhler, K.: Deep sequential segmentation of organs in volumetric medical scans (2018). arXiv:1807.02437

  43. Novikov, A.A., Lenis, D., Major, D., Hladuvka, J., Wimmer, M. and Buhler, K.: Fully convolutional architectures for multi-class segmentation in chest radiographs (2017). arXiv:1701.08816

  44. Onieva, J., Andresen, L., Holsting, J.Q., Rahaghi, F.N., Ballester, M.A.G., Estepar, R.S.J., Roman, K.L.L., de La Bruere, I.: 3D pulmonary artery segmentation from CTA scans using deep learning with realistic data augmentation

    Google Scholar 

  45. Perone, Christian S., Calabrese, Evan, Cohen-Adad, Julien: Spinal cord gray matter segmentation using deep dilated convolutions. Sci. Rep. 8(1), 5966 (2018)

    Article  Google Scholar 

  46. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI), vol. 9351 of LNCS, pp. 234–241. Springer, Berlin (2015). arXiv:1505.04597

    Google Scholar 

  47. Roth, H.R., Shen, C., Oda, H., Oda, M., Hayashi, Y., Misawa, K., Mori, K.: Deep learning and its application to medical image segmentation (2018). arXiv:1803.08691

  48. Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3D fully convolutional deep networks (2017). arXiv:1706.05721

  49. Shen, C., Roth, H.R., Oda, H., Oda, M., Hayashi, Y., Misawa, K., Mori, K.: On the influence of Dice loss function in multi-class organ segmentation of abdominal CT using 3D fully convolutional networks (2018). arXiv:1801.05912

  50. Shen, Dinggang, Guorong, Wu, Suk, Heung-Il: Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017)

    Article  Google Scholar 

  51. Shin, H.C., Tenenholtz, N.A., Rogers, J.K., Schwarz, C.G., Senjem, M.L., Gunter, J.L., Andriole, K.P., Michalski, M.: Medical image synthesis for data augmentation and anonymization using generative adversarial networks (2018). arXiv:1807.10225

  52. Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training (2016). arXiv:1612.07828

  53. Stollenga, M.F., Byeon, W., Liwicki, M., Schmidhuber, J.: Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation (2015). arXiv:1506.07452

  54. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going Deeper with Convolutions (2014). arXiv:1409.4842

  55. Tai, L., Ye, H., Ye, Q., Liu, M.: PCA-aided fully convolutional networks for semantic segmentation of multi-channel fMRI (2016). arXiv:1610.01732

  56. Vesal, S., Ravikumar, N., Maier, A.: Dilated convolutions in neural networks for left atrial segmentation in 3D gadolinium enhanced-MRI (2018). arXiv:1808.01673

  57. Wolterink, J.M., Kamnitsas, K., Ledig, C. and Isgum, I.: Generative adversarial networks and adversarial methods in biomedical image analysis (2018). arXiv:1810.10352

  58. Yang, D., Xu, D., Zhou, S.K., Georgescu, B., Chen, M., Grbic, S., Metaxas, D., Comaniciu, D.: Automatic liver segmentation using an adversarial image-to-image network. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L. Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 507–515. Springer International Publishing (2017)

    Google Scholar 

  59. Yang, L., Zhang, Y., Guldner, I.H., Zhang, S., Chen, D.Z.: 3D segmentation of glial cells using fully convolutional networks and k-terminal cut. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2016, Lecture Notes in Computer Science, pp. 658–666. Springer International Publishing (2016)

    Google Scholar 

  60. Yang, X., Yu, L., Li, S., Wang, X., Wang, N., Qin, J., Ni, D., Heng, P.A.: Towards automatic semantic segmentation in volumetric ultrasound. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 711–719. Springer International Publishing (2017)

    Google Scholar 

  61. Yi, X., Walia, E., Babyn, P.: Generative adversarial network in medical imaging: a review (2018). arXiv:1809.07294

  62. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions (2015). arXiv:1511.07122

  63. Yu, L., Cheng, J.Z., Dou, Q., Yang, X., Chen, H., Qin, J., Heng, P.A.: Automatic 3D cardiovascular MR segmentation with densely-connected volumetric convnets. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 287–295. Springer International Publishing (2017)

    Google Scholar 

  64. Zhou, X., Ito, T., Takayama, R., Wang, S., Hara, T., Fujita, H.: Three-dimensional CT image segmentation by combining 2D fully convolutional network with 3D majority voting. In Carneiro, G., Mateus, D., Peter, L., Bradley, A., Tavares, J.M.R.S, Belagiannis, V., Papa, J.P., Nascimento, J.C., Loog, M., Lu, Z., Cardoso, J.S., Cornebise, J. (eds.) Deep Learning and Data Labeling for Medical Applications, Lecture Notes in Computer Science, pp. 111–120. Springer International Publishing (2016)

    Google Scholar 

  65. Zhou, Y., Xie, L., Fishman, E.K., Yuille, A.L.: Deep supervision for pancreatic cyst segmentation in abdominal CT scans. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 222–230. Springer International Publishing (2017)

    Google Scholar 

  66. Zhou, Y., Xie, L., Shen, W., Wang, Y., Fishman, E.K., Yuille, A.L.: A fixed-point model for pancreas segmentation in abdominal CT scans. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Lecture Notes in Computer Science, pp. 693–701. Springer International Publishing (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Karen López-Linares Román .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

López-Linares Román, K., García Ocaña, M.I., Lete Urzelai, N., González Ballester, M.Á., Macía Oliver, I. (2020). Medical Image Segmentation Using Deep Learning. In: Chen, YW., Jain, L. (eds) Deep Learning in Healthcare. Intelligent Systems Reference Library, vol 171. Springer, Cham. https://doi.org/10.1007/978-3-030-32606-7_2

Download citation

Publish with us

Policies and ethics