Skip to main content

Advertisement

SpringerLink
Log in

Machine learning approaches for pathologic diagnosis

  • Review and Perspectives
  • Published:
Virchows Archiv Aims and scope Submit manuscript

Abstract

Machine learning techniques, especially deep learning techniques such as convolutional neural networks, have been successfully applied to general image recognitions since their overwhelming performance at the 2012 ImageNet Large Scale Visual Recognition Challenge. Recently, such techniques have also been applied to various medical, including histopathological, images to assist the process of medical diagnosis. In some cases, deep learning–based algorithms have already outperformed experienced pathologists for recognition of histopathological images. However, pathological images differ from general images in some aspects, and thus, machine learning of histopathological images requires specialized learning methods. Moreover, many pathologists are skeptical about the ability of deep learning technology to accurately recognize histopathological images because what the learned neural network recognizes is often indecipherable to humans. In this review, we first introduce various applications incorporating machine learning developed to assist the process of pathologic diagnosis, and then describe machine learning problems related to histopathological image analysis, and review potential ways to solve these problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115:211–252. https://doi.org/10.1007/s11263-015-0816-y

    Article  Google Scholar 

  2. Bejnordi BE, Veta M, van DPJ et al (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318:2199–2210. https://doi.org/10.1001/jama.2017.14585

    Article  Google Scholar 

  3. An augmented reality microscope for cancer detection. In: Google AI Blog. http://ai.googleblog.com/2018/04/an-augmented-reality-microscope.html. Accessed 13 Aug 2018

  4. Ciompi F, Geessink O, Bejnordi BE, et al (2017) The importance of stain normalization in colorectal tissue classification with convolutional networks. arXiv:170205931 [cs]

    Book  Google Scholar 

  5. Bejnordi BE, Litjens G, Timofeeva N et al (2016) Stain specific standardization of whole-slide histopathological images. IEEE Trans Med Imaging 35:404–415. https://doi.org/10.1109/TMI.2015.2476509

    Article  PubMed  Google Scholar 

  6. Khan AM, Rajpoot N, Treanor D, Magee D (2014) A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE Trans Biomed Eng 61:1729–1738. https://doi.org/10.1109/TBME.2014.2303294

    Article  PubMed  Google Scholar 

  7. Li X, Plataniotis KN (2015) A complete color normalization approach to histopathology images using color cues computed from saturation-weighted statistics. IEEE Trans Biomed Eng 62:1862–1873. https://doi.org/10.1109/TBME.2015.2405791

    Article  PubMed  Google Scholar 

  8. Sethi A, Sha L, Vahadane AR, Deaton RJ, Kumar N, Macias V, Gann PH (2016) Empirical comparison of color normalization methods for epithelial-stromal classification in H and E images. J Pathol Inform 7:17. https://doi.org/10.4103/2153-3539.179984

    Article  PubMed  PubMed Central  Google Scholar 

  9. Selvaraju RR, Cogswell M, Das A, et al (2016) Grad-CAM: visual explanations from deep networks via gradient-based localization. arXiv:161002391 [cs]

  10. Koh PW, Liang P (2017) Understanding black-box predictions via influence functions. arXiv:170304730 [cs, stat]

    Google Scholar 

  11. Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M (2014) Striving for simplicity: the all convolutional net. arXiv:14126806 [cs]

    Google Scholar 

  12. Bayramoglu N, Kannala J, Heikkilä J (2016) Deep learning for magnification independent breast cancer histopathology image classification. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp 2440–2445

    Chapter  Google Scholar 

  13. Hou L, Nguyen V, Samaras D, et al (2017) Sparse autoencoder for unsupervised nucleus detection and representation in histopathology images. arXiv:170400406 [cs]

    Google Scholar 

  14. Transfer learning for cell nuclei classification in histopathology images | SpringerLink. https://link.springer.com/chapter/10.1007/978-3-319-49409-8_46. Accessed 22 Nov 2017

  15. Xing F, Yang L (2016) Robust nucleus/cell detection and segmentation in digital pathology and microscopy images: a comprehensive review. IEEE Rev Biomed Eng 9:234–263. https://doi.org/10.1109/RBME.2016.2515127

    Article  PubMed  PubMed Central  Google Scholar 

  16. Roux L, Racoceanu D, Loménie N, Kulikova M, Irshad H, Klossa J, Capron F, Genestie C, Naour GL, Gurcan MN (2013) Mitosis detection in breast cancer histological images an ICPR 2012 contest. J Pathol Inform 4:8. https://doi.org/10.4103/2153-3539.112693

    Article  PubMed  PubMed Central  Google Scholar 

  17. Mitosis detection in breast cancer histological images. http://ludo17.free.fr/mitos_2012/index.html. Accessed 29 Nov 2017

  18. Veta M, van Diest PJ, Willems SM, Wang H, Madabhushi A, Cruz-Roa A, Gonzalez F, Larsen ABL, Vestergaard JS, Dahl AB, Cireşan DC, Schmidhuber J, Giusti A, Gambardella LM, Tek FB, Walter T, Wang CW, Kondo S, Matuszewski BJ, Precioso F, Snell V, Kittler J, de Campos TE, Khan AM, Rajpoot NM, Arkoumani E, Lacle MM, Viergever MA, Pluim JPW (2015) Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal 20:237–248. https://doi.org/10.1016/j.media.2014.11.010

    Article  PubMed  Google Scholar 

  19. Chen H, Qi X, Yu L, Heng PA (2016) DCAN: deep contour-aware networks for accurate gland segmentation. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2487–2496

    Chapter  Google Scholar 

  20. Sirinukunwattana K, Pluim JPW, Chen H, Qi X, Heng PA, Guo YB, Wang LY, Matuszewski BJ, Bruni E, Sanchez U, Böhm A, Ronneberger O, Cheikh BB, Racoceanu D, Kainz P, Pfeiffer M, Urschler M, Snead DRJ, Rajpoot NM (2017) Gland segmentation in colon histology images: the glas challenge contest. Med Image Anal 35:489–502. https://doi.org/10.1016/j.media.2016.08.008

    Article  PubMed  Google Scholar 

  21. Kather JN, Weis C-A (2016) Validation data set for automatic blood vessel segmentation in colorectal cancer histology (IHC)

    Google Scholar 

  22. Gupta V, Bhavsar A (2017) Breast cancer histopathological image classification: is magnification important? In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 769–776

    Chapter  Google Scholar 

  23. Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A (2017) Classification of breast cancer histology images using convolutional neural networks. PLoS One 12:e0177544. https://doi.org/10.1371/journal.pone.0177544

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. CAMELYON17. https://camelyon17.grand-challenge.org/. Accessed 21 Aug 2017

  25. Liu J, Xu B, Zheng C, et al (2018) An end-to-end deep learning histochemical scoring system for breast cancer tissue microarray. arXiv:180106288 [cs]

    Google Scholar 

  26. HALO AI – Indica Labs. http://www.indicalab.com/halo-ai/. Accessed 20 Dec 2018

  27. PAIGE. https://www.paigeai.com/. Accessed 20 Dec 2018

  28. PathAI. https://www.pathai.com/. Accessed 20 Dec 2018

  29. Proscia. https://proscia.com/. Accessed 20 Dec 2018

  30. Contextvision. http://www.contextvision.com/pathology/. Accessed 20 Dec 2018

  31. Luigi: large-scale histopathological image retrieval system using deep texture representations | bioRxiv. https://www.biorxiv.org/content/early/2018/07/19/345785. Accessed 13 Aug 2018

  32. Caicedo JC, González FA, Romero E (2011) Content-based histopathology image retrieval using a kernel-based semantic annotation framework. J Biomed Inform 44:519–528. https://doi.org/10.1016/j.jbi.2011.01.011

    Article  PubMed  Google Scholar 

  33. Mehta N, Raja’S A, Chaudhary V (2009) Content based sub-image retrieval system for high resolution pathology images using salient interest points. In: Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE. IEEE, pp 3719–3722

  34. Qi X, Wang D, Rodero I, Diaz-Montes J, Gensure RH, Xing F, Zhong H, Goodell L, Parashar M, Foran DJ, Yang L (2014) Content-based histopathology image retrieval using CometCloud. BMC Bioinform 15:287

    Article  Google Scholar 

  35. Sridhar A, Doyle S, Madabhushi A (2015) Content-based image retrieval of digitized histopathology in boosted spectrally embedded spaces. J Pathol Inform 6:41. https://doi.org/10.4103/2153-3539.159441

    Article  PubMed  PubMed Central  Google Scholar 

  36. Lafarge MW, Pluim JPW, Eppenhof KAJ, et al (2017) Domain-adversarial neural networks to address the appearance variability of histopathology images. arXiv:170706183 [cs]

  37. ScanNet: a fast and dense scanning framework for metastatic breast cancer detection from whole-slide images - semantic scholar. /paper/ScanNet-A-Fast-and-Dense-Scanning-Framework-for-Me-Lin-Chen/9484287f4d5d52d10b5d362c462d4d6955655f8e. Accessed 22 Nov 2017

  38. Vahadane A, Peng T, Sethi A, Albarqouni S, Wang L, Baust M, Steiger K, Schlitter AM, Esposito I, Navab N (2016) Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans Med Imaging 35:1962–1971. https://doi.org/10.1109/TMI.2016.2529665

    Article  PubMed  Google Scholar 

  39. Goodfellow IJ, Pouget-Abadie J, Mirza M, et al (2014) Generative adversarial networks

  40. Shaban MT, Baur C, Navab N, Albarqouni S (2018) StainGAN: stain style transfer for digital histological images. arXiv:180401601 [cs]

    Google Scholar 

  41. Zanjani FG, Zinger S, Bejnordi BE, et al (2018) Histopathology stain-color normalization using deep generative models

  42. Mariani G, Scheidegger F, Istrate R, et al (2018) BAGAN: data augmentation with balancing GAN

    Google Scholar 

  43. Komura D, Ishikawa S (2018) Machine learning methods for histopathological image analysis. Comput Struct Biotechnol J 16:34–42. https://doi.org/10.1016/j.csbj.2018.01.001

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. grand-challenges - Home. https://grand-challenge.org/. Accessed 17 Dec 2018

  45. Peikari M, Zubovits J, Clarke G, Martel AL (2015) Clustering analysis for semi-supervised learning improves classification performance of digital pathology. In: Machine learning in medical imaging. Springer, Cham, pp 263–270

    Chapter  Google Scholar 

  46. Doyle S, Monaco J, Feldman M, Tomaszewski J, Madabhushi A (2011) An active learning based classification strategy for the minority class problem: application to histopathology annotation. BMC Bioinform 12(424). https://doi.org/10.1186/1471-2105-12-424

  47. Padmanabhan RK, Somasundar VH, Griffith SD, Zhu J, Samoyedny D, Tan KS, Hu J, Liao X, Carin L, Yoon SS, Flaherty KT, DiPaola RS, Heitjan DF, Lal P, Feldman MD, Roysam B, Lee WMF (2014) An active learning approach for rapid characterization of endothelial cells in human tumors. PLoS One 9:e90495. https://doi.org/10.1371/journal.pone.0090495

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. [1805.06983] Terabyte-scale deep multiple instance learning for classification and localization in pathology. https://arxiv.org/abs/1805.06983. Accessed 13 Aug 2018

  49. Li Z, Wang C, Han M, et al (2017) Thoracic disease identification and localization with limited supervision. arXiv:171106373 [cs, stat]

    Google Scholar 

  50. Gal Y, Ghahramani Z (2015) Bayesian convolutional neural networks with Bernoulli approximate variational inference. arXiv:150602158 [cs, stat]

    Google Scholar 

  51. Novak R, Xiao L, Lee J, et al (2018) Bayesian convolutional neural networks with many channels are Gaussian processes. arXiv:181005148 [cs, stat]

  52. Shridhar K, Laumann F, Maurin AL, et al (2018) Bayesian convolutional neural networks with variational inference. arXiv:180605978 [cs, stat]

    Google Scholar 

  53. Zhao G, Liu F, Oler JA, Meyerand ME, Kalin NH, Birn RM (2018) Bayesian convolutional neural network based MRI brain extraction on nonhuman primates. NeuroImage 175:32–44. https://doi.org/10.1016/j.neuroimage.2018.03.065

    Article  PubMed  PubMed Central  Google Scholar 

  54. Buda M, Maki A, Mazurowski MA (2017) A systematic study of the class imbalance problem in convolutional neural networks. https://doi.org/10.1016/j.neunet.2018.07.011

    Book  Google Scholar 

  55. Haixiang G, Yijing L, Shang J, Mingyun G, Yuanyue H, Bing G (2017) Learning from class-imbalanced data: review of methods and applications. Expert Syst Appl 73:220–239. https://doi.org/10.1016/j.eswa.2016.12.035

    Article  Google Scholar 

  56. Mobadersany P, Yousefi S, Amgad M, Gutman DA, Barnholtz-Sloan JS, Velázquez Vega JE, Brat DJ, Cooper LAD (2018) Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci U S A 115:E2970–E2979. https://doi.org/10.1073/pnas.1717139115

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Krause J, Gulshan V, Rahimy E, Karth P, Widner K, Corrado GS, Peng L, Webster DR (2018) Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy. Ophthalmology 125:1264–1272. https://doi.org/10.1016/j.ophtha.2018.01.034

    Article  PubMed  Google Scholar 

  58. Hou L, Agarwal A, Samaras D, et al (2017) Unsupervised histopathology image synthesis. arXiv:171205021 [cs]

    Google Scholar 

  59. Sayres R, Taly A, Rahimy E, Blumer K, Coz D, Hammel N, Krause J, Narayanaswamy A, Rastegar Z, Wu D, Xu S, Barb S, Joseph A, Shumski M, Smith J, Sood AB, Corrado GS, Peng L, Webster DR (2018) Using a deep learning algorithm and integrated gradients explanation to assist grading for diabetic retinopathy. Ophthalmology 0:552–564. https://doi.org/10.1016/j.ophtha.2018.11.016

    Article  Google Scholar 

Download references

Acknowledgments

This study was supported by the Practical Research for Innovative Cancer Control from the Japan Agency for Medical Research and Development (AMED) (S.I.).

Contributions

Ishikawa S and Komura D wrote and reviewed the manuscript.

Funding

This research was supported by AMED under the Practical Research for Innovative Cancer Control, grant number JP19ck0106400 (S.I.).

Author information

Authors and Affiliations

  1. Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan

    Daisuke Komura & Shumpei Ishikawa

Authors
  1. Daisuke Komura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shumpei Ishikawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shumpei Ishikawa.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 34402 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Komura, D., Ishikawa, S. Machine learning approaches for pathologic diagnosis. Virchows Arch 475, 131–138 (2019). https://doi.org/10.1007/s00428-019-02594-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00428-019-02594-w

Keywords

Search

Navigation