Skip to main content
SpringerLink
Log in

Deep Learning with Convolutional Neural Networks for Histopathology Image Analysis

  • Chapter
  • First Online:
Automated Reasoning for Systems Biology and Medicine

Part of the book series: Computational Biology ((COBO,volume 30))

Abstract

In the recent years, deep learning based methods and, in particular, convolutional neural networks, have been dominating the arena of medical image analysis. This has been made possible both with the advent of new parallel hardware and the development of efficient algorithms. It is expected that future advances in both of these directions will increase this domination. The application of deep learning methods to medical image analysis has been shown to significantly improve the accuracy and efficiency of the diagnoses. In this chapter, we focus on applications of deep learning in microscopy image analysis and digital pathology, in particular. We provide an overview of the state-of-the-art methods in this area and exemplify some of the main techniques. Finally, we discuss some open challenges and avenues for future work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Actually b can be considered as a special weight \(w_0\) associated with a special input \(x_0\) which has a constant value 1. In this way the transfer function becomes slightly simpler \(\sigma (\mathbf {w}^T \mathbf {x})\). However, for the sake of clarity here we keep these two parameters separately.

  2. 2.

    Actually this is a definition of a cross-correlation which is slightly different than the usual mathematical notion of convolution, but in the machine learning practice this is how the convolution operation is implemented [23].

  3. 3.

    In principle, one can unfold the \(m \times n\) covered rectangular patch of the input and the filter into l-dimensional vectors, where \(l = m \times n\). In this way, “*” becomes real a dot product between the vectors. Also, a bias element can be added, like in the traditional neural networks.

  4. 4.

    In recent years, there is growing a trend to use fully convolutional networks in which the fully connected layers are implemented by means of convolutional layers.

References

  1. 15th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2015, Seoul, South Korea, 3–5 November 2015. IEEE (2015). http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7349033

  2. IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015. IEEE Computer Society (2015). http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7293313

  3. Arbelle A, Raviv TR (2018) Microscopy cell segmentation via adversarial neural networks. In: 15th IEEE International symposium on biomedical imaging, ISBI 2018, Washington, DC, USA, 4–7 April 2018. IEEE, pp 645–648. https://doi.org/10.1109/ISBI.2018.8363657

  4. Arbelle A, Raviv TR (2018) Microscopy cell segmentation via convolutional LSTM networks. CoRR. arXiv:1895.11247

  5. Ehteshami Bejnordi B, Veta M, Johannes van Diest P et al (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22):2199–2210. https://doi.org/10.1001/jama.2017.14585

    Article  Google Scholar 

  6. Bejnordi BE, Litjens GJS, Timofeeva N, Otte-Holler I, Homeyer A, Karssemeijer N, van der Laak JAWM (2016) Stain specific standardization of whole-slide histopathological images. IEEE Trans Med Imag 35(2):404–415. https://doi.org/10.1109/TMI.2015.2476509

    Article  Google Scholar 

  7. Bejnordi BE, Zuidhof G, Maschenka Balkenhol MH, Bult P, van Ginneken B, Karssemeijer N, Litjens G, van der Laak J (2017) Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images. J Med Imag 4:4–8. https://doi.org/10.1117/1.JMI.4.4.044504

    Article  Google Scholar 

  8. Bekkers EJ, Lafarge MW, Veta M, Eppenhof KAJ, Pluim JPW, Duits R (2018) Roto-translation covariant convolutional networks for medical image analysis. CoRR. arXiv:1804.03393

  9. Christ PF, Ettlinger F, Grün F, Elshaer MEA, Lipková J, Schlecht S, Ahmaddy F, Tatavarty S, Bickel M, Bilic P, Rempfler M, Hofmann F, D’Anastasi M, Ahmadi S, Kaissis G, Holch J, Sommer WH, Braren R, Heinemann V, Menze BH (2017) Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. CoRR arXiv:1702.05970

  10. Christiansen EM, Yang SJ, Ando DM, Javaherian A, Skibinski G, Lipnick S, Mount E, O’Neil A, Shah K, Lee AK, Goyal P, Fedus W, Poplin R, Esteva A, Berndl M, Rubin LL, Nelson P, Finkbeiner S (2018) In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173(3):792–803.e19. https://doi.org/10.1016/j.cell.2018.03.040, http://www.sciencedirect.com/science/article/pii/S0092867418303647

    Article  Google Scholar 

  11. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3d u-net: learning dense volumetric segmentation from sparse annotation. CoRR. arXiv:1606.06650

  12. Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images. In: Bartlett PL, Pereira FCN, Burges CJC, Bottou L, Weinberger KO (eds.) Advances in Neural information processing systems 25: 26th annual conference on neural information processing systems 2012. Proceedings of a meeting held 3–6 December 2012, Lake Tahoe, Nevada, United States, pp 2852–2860. http://papers.nips.cc/paper/4741-deep-neural-networks-segment-neuronal-membranes-in-electron-microscopy-images

  13. Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. In: Mori K, Sakuma I, Sato Y, Barillot C, Navab N (eds.) Medical image computing and computer-assisted intervention - MICCAI 2013 - 16th international conference, Nagoya, Japan, September 22-26, 2013, Proceedings, Part II, Lecture Notes in Computer Science, vol 8150. Springer, pp 411–418. https://doi.org/10.1007/978-3-642-40763-5_51

    Chapter  Google Scholar 

  14. Codella NCF, Anderson D, Philips T, Porto A, Massey K, Snowdon J, Feris RS, Smith JR (2018) Segmentation of both diseased and healthy skin from clinical photographs in a primary care setting. CoRR. arXiv:1804.05944

  15. Cruz-Roa A, Gilmore H, Basavanhally A, Feldman M, Ganesan S, Shih NNC, Tomaszewski J, González FA, Madabhushi A (2017) Accurate and reproducible invasive breast cancer detection in whole-slide images: a deep learning approach for quantifying tumor extent. Scientif Rep 7:46450 EP. https://doi.org/10.1038/srep46450

  16. Donahue J, Hendricks LA, Guadarrama S, Rohrbach M, Venugopalan S, Darrell T, Saenko K (2015) Long-term recurrent convolutional networks for visual recognition and description. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015, vol 2, pp 2625–2634. https://doi.org/10.1109/CVPR.2015.7298878

  17. Dozat T (2015) Incorporating nesterov momentum into adam

    Google Scholar 

  18. Drozdzal M, Vorontsov E, Chartrand G, Kadoury S, Pal C (2016) The importance of skip connections in biomedical image segmentation. CoRR. arXiv:1608.04117

  19. Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybernet 36:193–202

    Article  Google Scholar 

  20. Fukushima K, Miyake S (1982) Neocognitron: a new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recogn 15(6):455–469. http://www.sciencedirect.com/science/article/B6V14-48MPJ6Y-F7/2/2588c38bc16488ae94fe2334068ed166

    Article  Google Scholar 

  21. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky VS (2016) Domain-adversarial training of neural networks. J Mach Learn Res 17, 59:1–59:35. http://jmlr.org/papers/v17/15-239.html

  22. Goldsborough P, Pawlowski N, Caicedo JC, Singh S, Carpenter A (2017) Cytogan: generative modeling of cell images. bioRxiv. https://doi.org/10.1101/227645, https://www.biorxiv.org/content/early/2017/12/02/227645

  23. Goodfellow IJ, Bengio Y, Courville AC (2016) Deep learning: adaptive computation and machine learning. MIT Press. http://www.deeplearningbook.org/

  24. He K, Gkioxari G, Dollár P, Girshick RB (2017) Mask R-CNN. In: IEEE international conference on computer vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pp 2980–2988. IEEE Computer Society. https://doi.org/10.1109/ICCV.2017.322

  25. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. CoRR. arXiv:1512.03385

  26. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  27. Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017. IEEE Computer Society, pp 2261–2269. https://doi.org/10.1109/CVPR.2017.243

  28. Johnson JW (2018) Adapting mask-rcnn for automatic nucleus segmentation. CoRR. arXiv:1805.00500

  29. Katz G, Barrett C, Dill DL, Julian K, Kochenderfer MJ (2017) Towards proving the adversarial robustness of deep neural networks. In: Bulwahn L, Kamali M, Linker S (eds.) Proceedings first workshop on formal verification of autonomous vehicles, FVAV@iFM 2017, Turin, Italy, 19th September 2017. EPTCS, vol 257, pp 19–26. https://doi.org/10.4204/EPTCS.257.3

    Article  Google Scholar 

  30. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. CoRR arXiv:1412.6980

  31. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th international conference on neural information processing systems, vol 1, NIPS’12, pp 1097–1105. Curran Associates Inc., USA. http://dl.acm.org/citation.cfm?id=2999134.2999257

  32. Lafarge MW, Pluim JPW, Eppenhof KAJ, Moeskops P, Veta M (2017) Domain-adversarial neural networks to address the appearance variability of histopathology images. CoRR. arXiv:1707.06183

  33. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.138.1115, http://www.cs.berkeley.edu/daf/appsem/Handwriting/papers/00726791.pdf

  34. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision - ECCV 2014. Springer International Publishing, Cham, pp 740–755

    Google Scholar 

  35. Litjens GJS, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Imag Anal 42:60–88

    Article  Google Scholar 

  36. Lo SCB, Lou SLA, Lin JS, Freedman MT, Chien MV, Mun SK (1995) Artificial convolution neural network techniques and applications for lung nodule detection. IEEE Trans Med Imag 14(4):711–718. https://doi.org/10.1109/42.476112

    Article  Google Scholar 

  37. Milletari F, Navab N, Ahmadi S (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. CoRR arXiv:1606.04797

  38. Mobadersany P, Yousefi S, Amgad M, Gutman DA, Barnholtz-Sloan JS, Velázquez Vega JE, Brat DJ, Cooper LAD (2018) Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Nat Acad Sci 115(13):E2970–E2979. https://doi.org/10.1073/pnas.1717139115, http://www.pnas.org/content/115/13/E2970

    Article  Google Scholar 

  39. Paeng K, Hwang S, Park S, Kim M (2017) A unified framework for tumor proliferation score prediction in breast histopathology. In: Cardoso MJ, Arbel T, Carneiro G, Syeda-Mahmood TF, Tavares JMRS, Moradi M, Bradley AP, Greenspan H, Papa JP, Madabhushi A, Nascimento JC, Cardoso JS, Belagiannis V, Lu Z (eds.) Deep learning in medical image analysis and multimodal learning for clinical decision support - Third international workshop, DLMIA 2017, and 7th international workshop, ML-CDS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 14, 2017, Proceedings, Lecture Notes in Computer Science, vol 10553. Springer, pp 231–239. https://doi.org/10.1007/978-3-319-67558-9_27

    Chapter  Google Scholar 

  40. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. CoRR arXiv:1505.04597

  41. Shen D, Wu G, Suk HI (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248. https://doi.org/10.1146/annurev-bioeng-071516-044442. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5479722/28301734 [pmid]

    Article  Google Scholar 

  42. Shi X, Chen Z, Wang H, Yeung D, Wong W, Woo W (2015) Convolutional LSTM network: a machine learning approach for precipitation nowcasting. CoRR. arXiv:1506.04214

  43. Sundermann B, Feder S, Wersching H, Teuber A, Schwindt W, Kugel H, Heindel W, Arolt V, Berger K, Pfleiderer B (2017) Diagnostic classification of unipolar depression based on resting-state functional connectivity MRI: effects of generalization to a diverse sample. J Neural Trans 124(5):589–605. https://doi.org/10.1007/s00702-016-1673-8

    Article  Google Scholar 

  44. Szegedy C, Liu W, Jia Y, Sermanet P, Reed SE, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, June 7–12 2015, pp. 1–9. https://doi.org/10.1109/CVPR.2015.7298594

  45. Trajanovski S, Mavroeidis D, Swisher CL, Gebre BG, Veeling B, Wiemker R, Klinder T, Tahmasebi A, Regis SM, Wald C, McKee BJ, MacMahon H, Pien H (2018) Towards radiologist-level cancer risk assessment in CT lung screening using deep learning. CoRR. arXiv:1804.01901

  46. Vanschoren J, van Rijn JN, Bischl B (2015) Taking machine learning research online with openml. In: Proceedings of the 4th international workshop on big data, streams and heterogeneous source mining: algorithms, systems, programming models and applications, BigMine 2015, Sydney, Australia, August 10 2015. JMLR Workshop and Conference Proceedings, vol 41, pp 1–4. JMLR.org. http://jmlr.org/proceedings/papers/v41/vanschoren15.html

  47. Veličković P, Wang D, Lane ND, Liò P (2016) X-cnn: cross-modal convolutional neural networks for sparse datasets. In: 2016 IEEE symposium series on computational intelligence (SSCI), pp 1–8. https://doi.org/10.1109/SSCI.2016.7849978

  48. Wainberg M, Merico D, Delong A, Frey BJ (2018) Deep learning in biomedicine. Nat Biotechnol 36:829 EP. https://doi.org/10.1038/nbt.4233

    Article  Google Scholar 

  49. Wang D, Khosla A, Gargeya R, Irshad H, Beck AH (2016) Deep learning for identifying metastatic breast cancer. CoRR. arXiv:1606.05718

  50. Xu Y, Li Y, Liu M, Wang Y, Lai M, Chang EI (2016) Gland instance segmentation by deep multichannel side supervision. CoRR. arXiv:1607.03222

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers as well as Stojan Trajanovski for their comments and suggestions that contributed to the final version of this paper.

Author information

Authors and Affiliations

  1. Biomedical Engineering Department, Eindhoven University of Technology, Eindhoven, The Netherlands

    Dragan Bošnački, Natal van Riel & Mitko Veta

Authors
  1. Dragan Bošnački
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Natal van Riel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mitko Veta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dragan Bošnački .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Technology, University of Cambridge, Cambridge, UK

    Pietro Liò

  2. School of Computing, Newcastle University, Newcastle, UK

    Paolo Zuliani

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Bošnački, D., van Riel, N., Veta, M. (2019). Deep Learning with Convolutional Neural Networks for Histopathology Image Analysis. In: Liò, P., Zuliani, P. (eds) Automated Reasoning for Systems Biology and Medicine. Computational Biology, vol 30. Springer, Cham. https://doi.org/10.1007/978-3-030-17297-8_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-17297-8_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-17296-1

  • Online ISBN: 978-3-030-17297-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation