Skip to main content
SpringerLink
Log in

Survey of XAI in Digital Pathology

  • Chapter
  • First Online:
Artificial Intelligence and Machine Learning for Digital Pathology

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12090))

Abstract

Artificial intelligence (AI) has shown great promise for diagnostic imaging assessments. However, the application of AI to support medical diagnostics in clinical routine comes with many challenges. The algorithms should have high prediction accuracy but also be transparent, understandable and reliable. Thus, explainable artificial intelligence (XAI) is highly relevant for this domain. We present a survey on XAI within digital pathology, a medical imaging sub-discipline with particular characteristics and needs. The review includes several contributions. Firstly, we give a thorough overview of current XAI techniques of potential relevance for deep learning methods in pathology imaging, and categorise them from three different aspects. In doing so, we incorporate uncertainty estimation methods as an integral part of the XAI landscape. We also connect the technical methods to the specific prerequisites in digital pathology and present findings to guide future research efforts. The survey is intended for both technical researchers and medical professionals, one of the objectives being to establish a common ground for cross-disciplinary discussions.

This work was supported by the Swedish e-Science Research Center.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018)

    Article  Google Scholar 

  2. Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B.: Sanity checks for saliency maps. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS 2018, pp. 9525–9536. Curran Associates Inc., Red Hook (2018). https://doi.org/10.5555/3327546.3327621

  3. Alsallakh, B., Jourabloo, A., Ye, M., Liu, X., Ren, L.: Do convolutional neural networks learn class hierarchy? IEEE Trans. Visual Comput. Graphics 24(1), 152–165 (2018). https://doi.org/10.1109/TVCG.2017.2744683

    Article  Google Scholar 

  4. Alvarez-Melis, D., Jaakkola, T.: A causal framework for explaining the predictions of black-box sequence-to-sequence models. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 412–421. Association for Computational Linguistics, Copenhagen, September 2017. https://doi.org/10.18653/v1/D17-1042

  5. Alzubaidi, L., Resan, R., Abdul Hussain, H.: A robust deep learning approachto detect nuclei in histopathological images. Int. J. Innov. Res. Comput. Commun. Eng. 5, 7–12 (2017)

    Google Scholar 

  6. Arvaniti, E., et al.: Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Sci. Rep. 8(1) (2018). https://doi.org/10.1038/s41598-018-30535-1

  7. Ayhan, M.S., Berens, P.: Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks. In: Medical Imaging with Deep Learning (Midl 2018), pp. 1–9 (2018)

    Google Scholar 

  8. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7) (2015). https://doi.org/10.1371/journal.pone.0130140

  9. Balkenhol, M., et al.: Deep learning assisted mitotic counting for breast cancer. Lab. Invest. 99 (2019). https://doi.org/10.1038/s41374-019-0275-0

  10. Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A.: Network dissection: quantifying interpretability of deep visual representations. In: Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, January 2017, pp. 3319–3327. Institute of Electrical and Electronics Engineers Inc., November 2017. https://doi.org/10.1109/CVPR.2017.354

  11. Bera, K., Schalper, K.A., Rimm, D.L., Velcheti, V., Madabhushi, A.: Artificial intelligence in digital pathology – new tools for diagnosis and precision oncology. Nat. Rev. Clin. Oncol. 16(11), 703–715 (2019). https://doi.org/10.1038/s41571-019-0252-y

    Article  Google Scholar 

  12. Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight uncertainty in neural networks. In: 32nd International Conference on Machine Learning, ICML 2015, vol. 2, pp. 1613–1622, May 2015

    Google Scholar 

  13. Bouchacourt, D., Pawan Kumar, M., Nowozin, S.: DISCO nets: DISsimilarity COefficient Networks. In: Advances in Neural Information Processing Systems, pp. 352–360 (2016)

    Google Scholar 

  14. Bulten, W., et al.: Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21, 233–241 (2020)

    Article  Google Scholar 

  15. Cai, C.J., et al.: Human-centered tools for coping with imperfect algorithms during medical decision-making. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3290605.3300234

  16. Cannon, A.J.: Non-crossing nonlinear regression quantiles by monotone composite quantile regression neural network, with application to rainfall extremes. Stoch. Environ. Res. Risk Assess. 32(11), 3207–3225 (2018). https://doi.org/10.1007/s00477-018-1573-6

    Article  Google Scholar 

  17. Carter, S., Armstrong, Z., Schubert, L., Johnson, I., Olah, C.: Activation atlas. Distill 4(3) (2019). https://doi.org/10.23915/distill.00015

  18. Chen, H., Dou, Q., Wang, X., Qin, J., Heng, P.A.: Mitosis detection in breast cancer histology images via deep cascaded networks. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI 2016, pp. 1160–1166. AAAI Press (2016)

    Google Scholar 

  19. Chen, J., Srinivas, C.: Automatic lymphocyte detection in H&E images with deep neural networks. CoRR abs/1612.03217 (2016)

    Google Scholar 

  20. Dabkowski, P., Gal, Y.: Real time image saliency for black box classifiers. In: Advances in Neural Information Processing Systems, December 2017, pp. 6968–6977 (2017)

    Google Scholar 

  21. Depeweg, S., Hernandez-Lobato, J.M., Doshi-Velez, F., Udluft, S.: Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In: 35th International Conference on Machine Learning, ICML 2018, vol. 3, pp. 1920–1934 (2018)

    Google Scholar 

  22. Der Kiureghian, A., Ditlevsen, O.: Aleatory or epistemic? Does it matter? Struct. Saf. 31, 105–112 (2008). https://doi.org/10.1016/j.strusafe.2008.06.020

    Article  Google Scholar 

  23. Dosovitskiy, A., Brox, T.: Inverting visual representations with convolutional networks. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, December 2017, pp. 4829–4837 (2016). https://doi.org/10.1109/CVPR.2016.522

  24. Došilović, F.K., Brčć, M., Hlupić, N.: Explainable artificial intelligence: a survey. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 0210–0215, May 2018. https://doi.org/10.23919/MIPRO.2018.8400040

  25. Efron, B., Tibshirani, R.: An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability, vol. 57. Chapman & Hall, London (1994)

    Book  Google Scholar 

  26. Ehteshami Bejnordi, B., et al.: The CAMELYON16 consortium: diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017). https://doi.org/10.1001/jama.2017.14585

    Article  Google Scholar 

  27. Eilertsen, G., Jönsson, D., Ropinski, T., Unger, J., Ynnerman, A.: Classifying the classifier: dissecting the weight space of neural networks (2020). arXiv preprint, arXiv: 2002.05688

  28. Erhan, D., Bengio, Y., Courville, A., Vincent, P.: Visualizing higher-layer features of a deep network. Technical report, Univeristé de Montréal, January 2009

    Google Scholar 

  29. Fang, K., Shen, C., Kifer, D.: Evaluating aleatoric and epistemic uncertainties of time series deep learning models for soil moisture predictions (2019). arXiv preprint, arXiv:1906.04595

  30. Fong, R.C., Vedaldi, A.: Interpretable explanations of black boxes by meaningful perturbation. In: Proceedings of the IEEE International Conference on Computer Vision, October 2017, pp. 3449–3457 (2017). https://doi.org/10.1109/ICCV.2017.371

  31. Fraz, M.M., Shaban, M., Graham, S., Khurram, S.A., Rajpoot, N.M.: Uncertainty driven pooling network for microvessel segmentation in routine histology images. In: Stoyanov, D., et al. (eds.) OMIA/COMPAY -2018. LNCS, vol. 11039, pp. 156–164. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00949-6_19

    Chapter  Google Scholar 

  32. Gal, Y., Ghahramani, Z.: Bayesian convolutional neural networks with Bernoulli approximate variational inference (2015). arXiv preprint, arXiv:1506.02158

  33. Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: 33rd International Conference on Machine Learning, ICML 2016, vol. 3, pp. 1651–1660 (2016)

    Google Scholar 

  34. Garcia, E., Hermoza, R., Castanon, C.B., Cano, L., Castillo, M., Castanñeda, C.: Automatic lymphocyte detection on gastric cancer IHC images using deep learning. In: 2017 IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS), pp. 200–204, June 2017. https://doi.org/10.1109/CBMS.2017.94

  35. Garud, H., et al.: High-magnification multi-views based classification of breast fine needle aspiration cytology cell samples using fusion of decisions from deep convolutional networks. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, July 2017, pp. 828–833 (2017). https://doi.org/10.1109/CVPRW.2017.115

  36. Goebel, R., et al.: Explainable AI: the new 42? In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2018. LNCS, vol. 11015, pp. 295–303. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99740-7_21

    Chapter  Google Scholar 

  37. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.org

    MATH  Google Scholar 

  38. Hafner, D., Tran, D., Lillicrap, T., Irpan, A., Davidson, J.: Reliable uncertainty estimates in deep neural networks using noise contrastive priors. In: ICLR, pp. 1–14 (2019). https://doi.org/10.1163/156856192X00700

  39. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, June 2016. https://doi.org/10.1109/CVPR.2016.90

  40. Hernández-Lobato, J.M., Adams, R.P.: Probabilistic backpropagation for scalable learning of Bayesian neural networks. In: 32nd International Conference on Machine Learning, ICML 2015, vol. 3, pp. 1861–1869 (2015)

    Google Scholar 

  41. Höfener, H., Homeyer, A., Weiss, N., Molin, J., Lundström, C.F., Hahn, H.K.: Deep learning nuclei detection: a simple approach can deliver state-of-the-art results. Comput. Med. Imaging Graph. 70, 43–52 (2018). https://doi.org/10.1016/j.compmedimag.2018.08.010

    Article  Google Scholar 

  42. Hohman, F., Kahng, M., Pienta, R., Chau, D.H.: Visual analytics in deep learning: an interrogative survey for the next frontiers. IEEE Trans. Visual Comput. Graphics 25(8), 2674–2693 (2019). https://doi.org/10.1109/TVCG.2018.2843369

    Article  Google Scholar 

  43. Holzinger, A., Carrington, A., Müller, H.: Measuring the quality of explanations: the System Causability Scale (SCS). comparing human and machine explanations. In: Kersting, K. (ed.) Special Issue on Interactive Machine Learning, vol. 34. KI - Künstliche Intelligenz (German Journal of Artificial intelligence) (2020)

    Google Scholar 

  44. Holzinger, A., Langs, G., Denk, H., Zatloukal, K., Müller, H.: Causability and explainabilty of artificial intelligence in medicine. Wiley Interdisc. Rev. Data Min. Knowl. Discov. e1312 (2019). https://doi.org/10.1002/widm.1312

  45. Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2424–2433 (2016). https://doi.org/10.1109/cvpr.2016.266

  46. Huang, Y., Chung, A.C.S.: Evidence localization for pathology images using weakly supervised learning. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 613–621. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_68

    Chapter  Google Scholar 

  47. HTI Inc.: Image search engine for pathology (2020). http://www.hurondigitalpathology.com/image-search/. Accessed 17 Jan 2020

  48. Jung, H., Lodhi, B., Kang, J.: An automatic nuclei segmentation method based on deep convolutional neural networks for histopathology images. BMC Biomed. Eng. 1(1), 24 (2019). https://doi.org/10.1186/s42490-019-0026-8

    Article  Google Scholar 

  49. Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., Wortman Vaughan, J.: Interpreting interpretability: understanding data scientists’ use of interpretability tools for machine learning. In: 2020 CHI Conference on Human Factors in Computing Systems, CHI 2020 (2020)

    Google Scholar 

  50. Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: Advances in Neural Information Processing Systems, December 2017, pp. 5575–5585 (2017)

    Google Scholar 

  51. Kevin, F., et al.: Visualizing histopathologic deep learning classification and anomaly detection using nonlinear feature space dimensionality reduction. BMC Bioinformatics 19, 173 (2018). https://doi.org/10.1186/s12859-018-2184-4

    Article  Google Scholar 

  52. Khan, M.E., Nielsen, D., Tangkaratt, V., Lin, W., Gal, Y., Srivastava, A.: Fast and scalable Bayesian deep learning by weight-perturbation in Adam. In: 35th International Conference on Machine Learning, ICML 2018. vol. 6, pp. 4088–4113 (2018)

    Google Scholar 

  53. Khan, M.E.E., Immer, A., Abedi, E., Korzepa, M.: Approximate inference turns deep networks into gaussian processes. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 3088–3098. Curran Associates, Inc. (2019)

    Google Scholar 

  54. Kim, B., et al.: Interpretability beyond feature attribution: quantitative testing with concept activation vectors (TCAV). In: 35th International Conference on Machine Learning, ICML 2018, vol. 6, pp. 4186–4195 (2018)

    Google Scholar 

  55. Kindermans, P.J., et al.: Learning how to explain neural networks: PatternNet and PatternAttribution. In: 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings (2018)

    Google Scholar 

  56. Koelzer, V., et al.: Digital image analysis improves precision of programmed death ligand 1 (PD-L1) scoring in cutaneous melanoma. Histopathology 73 (2018). https://doi.org/10.1111/his.13528

  57. Koh, P.W., Liang, P.: Understanding black-box predictions via influence functions. In: Proceedings of the 34th International Conference on Machine Learning, ICML 2017, vol. 70. p. 1885–1894. JMLR.org (2017). https://doi.org/10.5555/3305381.3305576

  58. Komura, D., Ishikawa, S.: Machine learning methods for histopathological image analysis. Comput. Struct. Biotechnol. J. 16 (2017). https://doi.org/10.1016/j.csbj.2018.01.001

  59. Kwon, Y., Won, J.H., Kim, B.J., Paik, M.C.: Uncertainty quantification using Bayesian neural networks in classification: application to biomedical image segmentation. Comput. Stat. Data Anal. 142, 106816 (2020). https://doi.org/10.1016/j.csda.2019.106816

    Article  MathSciNet  MATH  Google Scholar 

  60. Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. In: Advances in Neural Information Processing Systems, December 2017, pp. 6403–6414 (2017)

    Google Scholar 

  61. Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., Müller, K.R.: Unmasking Clever Hans predictors and assessing what machines really learn. Nat. Commun. 10(1) (2019). https://doi.org/10.1038/s41467-019-08987-4

  62. Leino, K., Sen, S., Datta, A., Fredrikson, M., Li, L.: Influence-directed explanations for deep convolutional networks. In: Proceedings - International Test Conference, October 2018 (2019). https://doi.org/10.1109/TEST.2018.8624792

  63. Litjens, G.J.S., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)

    Article  Google Scholar 

  64. Liu, Y., et al.: Detecting cancer metastases on gigapixel pathology images. CoRR abs/1703.02442 (2017)

    Google Scholar 

  65. Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 4765–4774. Curran Associates, Inc. (2017)

    Google Scholar 

  66. Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

  67. Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. Technical report (2015). https://doi.org/10.1109/CVPR.2015.7299155

  68. Mahendran, A., Vedaldi, A.: Visualizing deep convolutional neural networks using natural pre-images. Int. J. Comput. Vis. 120(3), 233–255 (2016). https://doi.org/10.1007/s11263-016-0911-8

    Article  MathSciNet  Google Scholar 

  69. Meinshausen, N.: Quantile regression forests. J. Mach. Learn. Res. 7, 983–999 (2006)

    MathSciNet  MATH  Google Scholar 

  70. Mittelstadt, B., Russell, C., Wachter, S.: Explaining explanations in AI. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019, pp. 279–288 (2019). Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3287560.3287574

  71. Molin, J., Bodén, A., Treanor, D., Fjeld, M., Lundström, C.: Scale stain: multi-resolution feature enhancement in pathology visualization (2016). arXiv preprint, arXiv:1610.04141

  72. Molin, J., Woundefinedniak, P.W., Lundström, C., Treanor, D., Fjeld, M.: Understanding design for automated image analysis in digital pathology. In: Proceedings of the 9th Nordic Conference on Human-Computer Interaction, NordiCHI 2016, Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2971485.2971561

  73. Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Müller, K.R.: Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recogn. 65, 211–222 (2017). https://doi.org/10.1016/j.patcog.2016.11.008

    Article  Google Scholar 

  74. Mueller, S.T., Hoffman, R.R., Clancey, W., Emrey, A., Klein, G.: Explanation in human-AI systems: a literature meta-review, synopsis of key ideas and publications, and bibliography for explainable AI (2019). arXiv preprint, arXiv:1902.01876

  75. Nagpal, K., et al.: Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. NPJ Digit. Med. 2(1) (2019). https://doi.org/10.1038/s41746-019-0112-2

  76. Narayanan, P.L., Raza, S.E.A., Dodson, A., Gusterson, B., Dowsett, M., Yuan, Y.: DeepSDCS: dissecting cancer proliferation heterogeneity in Ki67 digital whole slide images (2018). arXiv preprint, arXiv: 1806.10850

  77. Neal, R.M.: Bayesian learning for neural networks. Ph.D. thesis, CAN (1995). aAINN02676

    Google Scholar 

  78. Nguyen, A., Dosovitskiy, A., Yosinski, J., Brox, T., Clune, J.: Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS 2016, pp. 3395–3403. Curran Associates Inc., Red Hook (2016)

    Google Scholar 

  79. Nguyen, A., Yosinski, J., Clune, J.: Multifaceted feature visualization: uncovering the different types of features learned by each neuron in deep neural networks (2016). arXiv preprint, arXiv: 1602.03616

  80. Nie, W., Zhang, Y., Patel, A.: A theoretical explanation for perplexing behaviors of backpropagation-based visualizations. In: ICML (2018)

    Google Scholar 

  81. Olah, C., et al.: The building blocks of interpretability. Distill 3(3) (2018). https://doi.org/10.23915/distill.00010

  82. Osband, I., Blundell, C., Pritzel, A., Van Roy, B.: Deep exploration via bootstrapped DQN. In: Advances in Neural Information Processing Systems, pp. 4033–4041 (2016)

    Google Scholar 

  83. Palatnik de Sousa, I., Maria Bernardes Rebuzzi Vellasco, M., Costa da Silva, E.: Local interpretable model-agnostic explanations for classification of lymph node metastases. Sensors 19(13), 2969 (2019). https://doi.org/10.3390/s19132969

  84. Papadopoulos, H., Vovk, V., Gammerman, A.: Conformal prediction with neural networks. In: Proceedings - International Conference on Tools with Artificial Intelligence, ICTAI, vol. 2, pp. 388–395 (2007). https://doi.org/10.1109/ICTAI.2007.47

  85. Papernot, N., McDaniel, P.: Deep k-nearest neighbors: towards confident, interpretable and robust deep learning (2018). arXiv preprint, arXiv: 1803.04765

  86. Pearce, T., Leibfried, F., Brintrup, A., Zaki, M., Neely, A.: Uncertainty in neural networks: approximately Bayesian ensembling (2018). arXiv preprint, arXiv: 1810.05546

  87. Pearce, T., Zaki, M., Brintrup, A., Neely, A.: High-quality prediction intervals for deep learning: a distribution-free, ensembled approach. In: 35th International Conference on Machine Learning, ICML 2018, vol. 9, pp. 6473–6482 (2018)

    Google Scholar 

  88. Pohn, B., Kargl, M., Reihs, R., Holzinger, A., Zatloukal, K., Muller, H.: Towards a deeper understanding of how a pathologist makes a diagnosis: Visualization of the diagnostic process in histopathology. In: 2019 IEEE Symposium on Computers and Communications (ISCC), 2019 IEEE Symposium on Computers and Communications (ISCC), pp. 1081–1086 (2019)

    Google Scholar 

  89. Postels, J., Ferroni, F., Coskun, H., Navab, N., Tombari, F.: Sampling-free epistemic uncertainty estimation using approximated variance propagation (2019). arXiv preprint, arXiv: 1908.00598

  90. Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should I trust you?” Explaining the predictions of any classifier. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 13–17 August, pp. 1135–1144 (2016). https://doi.org/10.1145/2939672.2939778

  91. Ritter, H., Botev, A., Barber, D.: A scalable Laplace approximation for neural networks. In: ICLR, pp. 1–15 (2018). https://doi.org/10.5121/ijfcst.2014.4504

  92. Saha, M., Chakraborty, C.: Her2Net: a deep framework for semantic segmentation and classification of cell membranes and nuclei in breast cancer evaluation. IEEE Trans. Image Process. 27(5), 2189–2200 (2018). https://doi.org/10.1109/TIP.2018.2795742

    Article  MathSciNet  MATH  Google Scholar 

  93. Samek, W., Binder, A., Montavon, G., Lapuschkin, S., Müller, K.: Evaluating the visualization of what a deep neural network has learned. IEEE Trans. Neural Netw. Learn. Syst. 28(11), 2660–2673 (2017). https://doi.org/10.1109/TNNLS.2016.2599820

    Article  MathSciNet  Google Scholar 

  94. Seah, J., Tang, J., Kitchen, A., Seah, J.: Generative visual rationales (2018). arXiv preprint, arXiv: 1804.04539

  95. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, October 2017, pp. 618–626 (2017). https://doi.org/10.1109/ICCV.2017.74

  96. Serag, A., et al.: Translational AI and deep learning in diagnostic pathology. Front. Med. 6, 185 (2019). https://doi.org/10.3389/fmed.2019.00185

    Article  Google Scholar 

  97. Sharma, H., Zerbe, N., Klempert, I., Hellwich, O., Hufnagl, P.: Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopathology. Comput. Med. Imaging Graph. 61, 2–13 (2017). https://doi.org/10.1016/j.compmedimag.2017.06.001. Selected papers from the 13th European Congress on Digital Pathology

    Article  Google Scholar 

  98. Shimoda, W., Yanai, K.: Distinct class saliency maps for multiple object images. In: Workshop Track - ICLR 2016, vol. 1 (2016)

    Google Scholar 

  99. Shridhar, K., Laumann, F., Liwicki, M.: A comprehensive guide to Bayesian convolutional neural network with variational inference (2019). arXiv preprint, arXiv: 1901.02731

  100. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: Visualising image classification models and saliency maps. In: 2nd International Conference on Learning Representations, ICLR 2014 - Workshop Track Proceedings (2014)

    Google Scholar 

  101. Smilkov, D., Thorat, N., Kim, B., Viégas, F., Wattenberg, M.: SmoothGrad: removing noise by adding noise (2017). arXiv preprint, arXiv:1706.03825

  102. Snoek, J., et al.: Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 13969–13980. Curran Associates, Inc. (2019)

    Google Scholar 

  103. Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: The all convolutional net. In: 3rd International Conference on Learning Representations, ICLR 2015 - Workshop Track Proceedings (2015)

    Google Scholar 

  104. Srinivas, S., Fleuret, F.: Full-gradient representation for neural network visualization. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 4126–4135. Curran Associates, Inc. (2019)

    Google Scholar 

  105. Stacke, K., Eilertsen, G., Unger, J., Lundström, C.: A closer look at domain shift for deep learning in histopathology (2019). arXiv preprint, arXiv: 1909.11575

  106. Ström, P., et al.: Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol. 21, 222–232 (2020)

    Article  Google Scholar 

  107. Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: 34th International Conference on Machine Learning, ICML 2017, vol. 7, pp. 5109–5118 (2017)

    Google Scholar 

  108. Swiderska-Chadaj, Z., et al.: Learning to detect lymphocytes in immunohistochemistry with deep learning. Med. Image Anal. 58, 101547 (2019). https://doi.org/10.1016/j.media.2019.101547

    Article  Google Scholar 

  109. Tagasovska, N., Lopez-Paz, D.: Single-model uncertainties for deep learning. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 6414–6425. Curran Associates, Inc. (2019)

    Google Scholar 

  110. Tang, Z., et al.: Interpretable classification of Alzheimer’s disease pathologies with a convolutional neural network pipeline. Nat. Commun. 10 (2019). https://doi.org/10.1038/s41467-019-10212-1

  111. Tellez, D., et al.: Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Med. Image Anal. 58, 101544 (2019)

    Article  Google Scholar 

  112. Teye, M., Azizpour, H., Smith, K.: Bayesian uncertainty estimation for batch normalized deep networks. In: 35th International Conference on Machine Learning, ICML 2018, vol. 11, pp. 7824–7833 (2018)

    Google Scholar 

  113. Thorstenson, S., Molin, J., Lundström, C.: Implementation of large-scale routine diagnostics using whole slide imaging in Sweden: digital pathology experiences 2006–2013. J. Pathol. Inf. 5 (2014). https://doi.org/10.4103/2153-3539.129452

  114. Tjoa, E., Guan, C.: A survey on explainable artificial intelligence (XAI): towards medical XAI (2019). arXiv preprint, arXiv: abs/1907.07374

  115. Veta, M., et al.: Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge. CoRR abs/1807.08284 (2018)

    Google Scholar 

  116. Wu, T., Song, X.: Towards interpretable object detection by unfolding latent structures. In: The IEEE International Conference on Computer Vision (ICCV), October 2019

    Google Scholar 

  117. Xue, Y., Ray, N., Hugh, J., Bigras, G.: Cell counting by regression using convolutional neural network. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 274–290. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46604-0_20

    Chapter  Google Scholar 

  118. Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization (2015). arXiv preprint, arXiv: 1506.06579

  119. Young, G.A., Smith, R.L.: Essentials of Statistical Inference. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge (2005)

    Book  Google Scholar 

  120. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  121. Zhang, J., Bargal, S.A., Lin, Z., Brandt, J., Shen, X., Sclaroff, S.: Top-down neural attention by excitation backprop. Int. J. Comput. Vis. 126(10), 1084–1102 (2018). https://doi.org/10.1007/s11263-017-1059-x

    Article  Google Scholar 

  122. Zhang, Q.S., Zhu, S.C.: Visual interpretability for deep learning: a survey. Front. Inf. Technol. Electron. Eng. 19(1), 27–39 (2018)

    Article  Google Scholar 

  123. Zhang, Q., Wu, Y.N., Zhu, S.C.: Interpretable convolutional neural networks. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 8827–8836 (2018). https://doi.org/10.1109/CVPR.2018.00920

  124. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Object detectors emerge in deep scene CNNs. In: ICLR 2015 (2014)

    Google Scholar 

  125. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, December 2016, pp. 2921–2929 (2016). https://doi.org/10.1109/CVPR.2016.319

  126. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: The IEEE International Conference on Computer Vision (ICCV), October 2017

    Google Scholar 

  127. Zintgraf, L.M., Cohen, T.S., Adel, T., Welling, M.: Visualizing deep neural network decisions: prediction difference analysis. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Science and Technology, Linköping University, Linköping, Sweden

    Milda Pocevičiūtė, Gabriel Eilertsen & Claes Lundström

  2. Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden

    Milda Pocevičiūtė, Gabriel Eilertsen & Claes Lundström

  3. Sectra AB, Linköping, Sweden

    Claes Lundström

Authors
  1. Milda Pocevičiūtė
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gabriel Eilertsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Claes Lundström
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Milda Pocevičiūtė .

Editor information

Editors and Affiliations

  1. Medical University of Graz, Graz, Austria

    Andreas Holzinger

  2. University of Alberta, Edmonton, AB, Canada

    Randy Goebel

  3. University of Alberta, Edmonton, AB, Canada

    Michael Mengel

  4. Medical University of Graz, Graz, Austria

    Heimo Müller

A Reviewed Methods

A Reviewed Methods

Table 1. List of methods and their categorisations, ordered by the year of publishing.
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Pocevičiūtė, M., Eilertsen, G., Lundström, C. (2020). Survey of XAI in Digital Pathology. In: Holzinger, A., Goebel, R., Mengel, M., Müller, H. (eds) Artificial Intelligence and Machine Learning for Digital Pathology. Lecture Notes in Computer Science(), vol 12090. Springer, Cham. https://doi.org/10.1007/978-3-030-50402-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-50402-1_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-50401-4

  • Online ISBN: 978-3-030-50402-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation