Machine learning approaches for pathologic diagnosis

  • Daisuke Komura
  • Shumpei IshikawaEmail author
Review and Perspectives


Machine learning techniques, especially deep learning techniques such as convolutional neural networks, have been successfully applied to general image recognitions since their overwhelming performance at the 2012 ImageNet Large Scale Visual Recognition Challenge. Recently, such techniques have also been applied to various medical, including histopathological, images to assist the process of medical diagnosis. In some cases, deep learning–based algorithms have already outperformed experienced pathologists for recognition of histopathological images. However, pathological images differ from general images in some aspects, and thus, machine learning of histopathological images requires specialized learning methods. Moreover, many pathologists are skeptical about the ability of deep learning technology to accurately recognize histopathological images because what the learned neural network recognizes is often indecipherable to humans. In this review, we first introduce various applications incorporating machine learning developed to assist the process of pathologic diagnosis, and then describe machine learning problems related to histopathological image analysis, and review potential ways to solve these problems.


Machine learning Deep learning Digital pathology WSI (whole slide image) 



This study was supported by the Practical Research for Innovative Cancer Control from the Japan Agency for Medical Research and Development (AMED) (S.I.).


Ishikawa S and Komura D wrote and reviewed the manuscript.


This research was supported by AMED under the Practical Research for Innovative Cancer Control, grant number JP19ck0106400 (S.I.).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Supplementary material

428_2019_2594_MOESM1_ESM.docx (33.6 mb)
ESM 1 (DOCX 34402 kb)


  1. 1.
    Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115:211–252. CrossRefGoogle Scholar
  2. 2.
    Bejnordi BE, Veta M, van DPJ et al (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318:2199–2210. CrossRefGoogle Scholar
  3. 3.
    An augmented reality microscope for cancer detection. In: Google AI Blog. Accessed 13 Aug 2018
  4. 4.
    Ciompi F, Geessink O, Bejnordi BE, et al (2017) The importance of stain normalization in colorectal tissue classification with convolutional networks. arXiv:170205931 [cs]CrossRefGoogle Scholar
  5. 5.
    Bejnordi BE, Litjens G, Timofeeva N et al (2016) Stain specific standardization of whole-slide histopathological images. IEEE Trans Med Imaging 35:404–415. CrossRefGoogle Scholar
  6. 6.
    Khan AM, Rajpoot N, Treanor D, Magee D (2014) A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE Trans Biomed Eng 61:1729–1738. CrossRefGoogle Scholar
  7. 7.
    Li X, Plataniotis KN (2015) A complete color normalization approach to histopathology images using color cues computed from saturation-weighted statistics. IEEE Trans Biomed Eng 62:1862–1873. CrossRefGoogle Scholar
  8. 8.
    Sethi A, Sha L, Vahadane AR, Deaton RJ, Kumar N, Macias V, Gann PH (2016) Empirical comparison of color normalization methods for epithelial-stromal classification in H and E images. J Pathol Inform 7:17. CrossRefGoogle Scholar
  9. 9.
    Selvaraju RR, Cogswell M, Das A, et al (2016) Grad-CAM: visual explanations from deep networks via gradient-based localization. arXiv:161002391 [cs]Google Scholar
  10. 10.
    Koh PW, Liang P (2017) Understanding black-box predictions via influence functions. arXiv:170304730 [cs, stat]Google Scholar
  11. 11.
    Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M (2014) Striving for simplicity: the all convolutional net. arXiv:14126806 [cs]Google Scholar
  12. 12.
    Bayramoglu N, Kannala J, Heikkilä J (2016) Deep learning for magnification independent breast cancer histopathology image classification. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp 2440–2445CrossRefGoogle Scholar
  13. 13.
    Hou L, Nguyen V, Samaras D, et al (2017) Sparse autoencoder for unsupervised nucleus detection and representation in histopathology images. arXiv:170400406 [cs]Google Scholar
  14. 14.
    Transfer learning for cell nuclei classification in histopathology images | SpringerLink. Accessed 22 Nov 2017
  15. 15.
    Xing F, Yang L (2016) Robust nucleus/cell detection and segmentation in digital pathology and microscopy images: a comprehensive review. IEEE Rev Biomed Eng 9:234–263. CrossRefGoogle Scholar
  16. 16.
    Roux L, Racoceanu D, Loménie N, Kulikova M, Irshad H, Klossa J, Capron F, Genestie C, Naour GL, Gurcan MN (2013) Mitosis detection in breast cancer histological images an ICPR 2012 contest. J Pathol Inform 4:8. CrossRefGoogle Scholar
  17. 17.
    Mitosis detection in breast cancer histological images. Accessed 29 Nov 2017
  18. 18.
    Veta M, van Diest PJ, Willems SM, Wang H, Madabhushi A, Cruz-Roa A, Gonzalez F, Larsen ABL, Vestergaard JS, Dahl AB, Cireşan DC, Schmidhuber J, Giusti A, Gambardella LM, Tek FB, Walter T, Wang CW, Kondo S, Matuszewski BJ, Precioso F, Snell V, Kittler J, de Campos TE, Khan AM, Rajpoot NM, Arkoumani E, Lacle MM, Viergever MA, Pluim JPW (2015) Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal 20:237–248. CrossRefGoogle Scholar
  19. 19.
    Chen H, Qi X, Yu L, Heng PA (2016) DCAN: deep contour-aware networks for accurate gland segmentation. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2487–2496CrossRefGoogle Scholar
  20. 20.
    Sirinukunwattana K, Pluim JPW, Chen H, Qi X, Heng PA, Guo YB, Wang LY, Matuszewski BJ, Bruni E, Sanchez U, Böhm A, Ronneberger O, Cheikh BB, Racoceanu D, Kainz P, Pfeiffer M, Urschler M, Snead DRJ, Rajpoot NM (2017) Gland segmentation in colon histology images: the glas challenge contest. Med Image Anal 35:489–502. CrossRefGoogle Scholar
  21. 21.
    Kather JN, Weis C-A (2016) Validation data set for automatic blood vessel segmentation in colorectal cancer histology (IHC)Google Scholar
  22. 22.
    Gupta V, Bhavsar A (2017) Breast cancer histopathological image classification: is magnification important? In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 769–776CrossRefGoogle Scholar
  23. 23.
    Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A (2017) Classification of breast cancer histology images using convolutional neural networks. PLoS One 12:e0177544. CrossRefGoogle Scholar
  24. 24.
    CAMELYON17. Accessed 21 Aug 2017
  25. 25.
    Liu J, Xu B, Zheng C, et al (2018) An end-to-end deep learning histochemical scoring system for breast cancer tissue microarray. arXiv:180106288 [cs]Google Scholar
  26. 26.
    HALO AI – Indica Labs. Accessed 20 Dec 2018
  27. 27.
    PAIGE. Accessed 20 Dec 2018
  28. 28.
    PathAI. Accessed 20 Dec 2018
  29. 29.
    Proscia. Accessed 20 Dec 2018
  30. 30.
    Contextvision. Accessed 20 Dec 2018
  31. 31.
    Luigi: large-scale histopathological image retrieval system using deep texture representations | bioRxiv. Accessed 13 Aug 2018
  32. 32.
    Caicedo JC, González FA, Romero E (2011) Content-based histopathology image retrieval using a kernel-based semantic annotation framework. J Biomed Inform 44:519–528. CrossRefGoogle Scholar
  33. 33.
    Mehta N, Raja’S A, Chaudhary V (2009) Content based sub-image retrieval system for high resolution pathology images using salient interest points. In: Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE. IEEE, pp 3719–3722Google Scholar
  34. 34.
    Qi X, Wang D, Rodero I, Diaz-Montes J, Gensure RH, Xing F, Zhong H, Goodell L, Parashar M, Foran DJ, Yang L (2014) Content-based histopathology image retrieval using CometCloud. BMC Bioinform 15:287CrossRefGoogle Scholar
  35. 35.
    Sridhar A, Doyle S, Madabhushi A (2015) Content-based image retrieval of digitized histopathology in boosted spectrally embedded spaces. J Pathol Inform 6:41. CrossRefGoogle Scholar
  36. 36.
    Lafarge MW, Pluim JPW, Eppenhof KAJ, et al (2017) Domain-adversarial neural networks to address the appearance variability of histopathology images. arXiv:170706183 [cs]Google Scholar
  37. 37.
    ScanNet: a fast and dense scanning framework for metastatic breast cancer detection from whole-slide images - semantic scholar. /paper/ScanNet-A-Fast-and-Dense-Scanning-Framework-for-Me-Lin-Chen/9484287f4d5d52d10b5d362c462d4d6955655f8e. Accessed 22 Nov 2017Google Scholar
  38. 38.
    Vahadane A, Peng T, Sethi A, Albarqouni S, Wang L, Baust M, Steiger K, Schlitter AM, Esposito I, Navab N (2016) Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans Med Imaging 35:1962–1971. CrossRefGoogle Scholar
  39. 39.
    Goodfellow IJ, Pouget-Abadie J, Mirza M, et al (2014) Generative adversarial networksGoogle Scholar
  40. 40.
    Shaban MT, Baur C, Navab N, Albarqouni S (2018) StainGAN: stain style transfer for digital histological images. arXiv:180401601 [cs]Google Scholar
  41. 41.
    Zanjani FG, Zinger S, Bejnordi BE, et al (2018) Histopathology stain-color normalization using deep generative modelsGoogle Scholar
  42. 42.
    Mariani G, Scheidegger F, Istrate R, et al (2018) BAGAN: data augmentation with balancing GANGoogle Scholar
  43. 43.
    Komura D, Ishikawa S (2018) Machine learning methods for histopathological image analysis. Comput Struct Biotechnol J 16:34–42. CrossRefGoogle Scholar
  44. 44.
    grand-challenges - Home. Accessed 17 Dec 2018
  45. 45.
    Peikari M, Zubovits J, Clarke G, Martel AL (2015) Clustering analysis for semi-supervised learning improves classification performance of digital pathology. In: Machine learning in medical imaging. Springer, Cham, pp 263–270CrossRefGoogle Scholar
  46. 46.
    Doyle S, Monaco J, Feldman M, Tomaszewski J, Madabhushi A (2011) An active learning based classification strategy for the minority class problem: application to histopathology annotation. BMC Bioinform 12(424).
  47. 47.
    Padmanabhan RK, Somasundar VH, Griffith SD, Zhu J, Samoyedny D, Tan KS, Hu J, Liao X, Carin L, Yoon SS, Flaherty KT, DiPaola RS, Heitjan DF, Lal P, Feldman MD, Roysam B, Lee WMF (2014) An active learning approach for rapid characterization of endothelial cells in human tumors. PLoS One 9:e90495. CrossRefGoogle Scholar
  48. 48.
    [1805.06983] Terabyte-scale deep multiple instance learning for classification and localization in pathology. Accessed 13 Aug 2018
  49. 49.
    Li Z, Wang C, Han M, et al (2017) Thoracic disease identification and localization with limited supervision. arXiv:171106373 [cs, stat]Google Scholar
  50. 50.
    Gal Y, Ghahramani Z (2015) Bayesian convolutional neural networks with Bernoulli approximate variational inference. arXiv:150602158 [cs, stat]Google Scholar
  51. 51.
    Novak R, Xiao L, Lee J, et al (2018) Bayesian convolutional neural networks with many channels are Gaussian processes. arXiv:181005148 [cs, stat]Google Scholar
  52. 52.
    Shridhar K, Laumann F, Maurin AL, et al (2018) Bayesian convolutional neural networks with variational inference. arXiv:180605978 [cs, stat]Google Scholar
  53. 53.
    Zhao G, Liu F, Oler JA, Meyerand ME, Kalin NH, Birn RM (2018) Bayesian convolutional neural network based MRI brain extraction on nonhuman primates. NeuroImage 175:32–44. CrossRefGoogle Scholar
  54. 54.
    Buda M, Maki A, Mazurowski MA (2017) A systematic study of the class imbalance problem in convolutional neural networks. Google Scholar
  55. 55.
    Haixiang G, Yijing L, Shang J, Mingyun G, Yuanyue H, Bing G (2017) Learning from class-imbalanced data: review of methods and applications. Expert Syst Appl 73:220–239. CrossRefGoogle Scholar
  56. 56.
    Mobadersany P, Yousefi S, Amgad M, Gutman DA, Barnholtz-Sloan JS, Velázquez Vega JE, Brat DJ, Cooper LAD (2018) Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci U S A 115:E2970–E2979. CrossRefGoogle Scholar
  57. 57.
    Krause J, Gulshan V, Rahimy E, Karth P, Widner K, Corrado GS, Peng L, Webster DR (2018) Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy. Ophthalmology 125:1264–1272. CrossRefGoogle Scholar
  58. 58.
    Hou L, Agarwal A, Samaras D, et al (2017) Unsupervised histopathology image synthesis. arXiv:171205021 [cs]Google Scholar
  59. 59.
    Sayres R, Taly A, Rahimy E, Blumer K, Coz D, Hammel N, Krause J, Narayanaswamy A, Rastegar Z, Wu D, Xu S, Barb S, Joseph A, Shumski M, Smith J, Sood AB, Corrado GS, Peng L, Webster DR (2018) Using a deep learning algorithm and integrated gradients explanation to assist grading for diabetic retinopathy. Ophthalmology 0:552–564. Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Preventive Medicine, Graduate School of MedicineThe University of TokyoTokyoJapan

Personalised recommendations