Abstract
Histopathological evaluation of tissue samples is a key practice in patient diagnosis and drug development, especially in oncology. Historically, Hematoxylin and Eosin (H&E) has been used by pathologists as a gold standard staining. However, in many cases, various target specific stains, including immunohistochemistry (IHC), are needed in order to highlight specific structures in the tissue. As tissue is scarce and staining procedures are tedious, it would be beneficial to generate images of stained tissue virtually. Virtual staining could also generate in-silico multiplexing of different stains on the same tissue segment. In this paper, we present a sample application that generates FAP-CK virtual IHC images from Ki67-CD8 real IHC images using an unsupervised deep learning approach based on CycleGAN. We also propose a method to deal with tiling artifacts caused by normalization layers and we validate our approach by comparing the results of tissue analysis algorithms for virtual and real images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bayramoglu, N., Kaakinen, M., Eklund, L., Heikkilä, J.: Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks. In: ICCV Workshops, pp. 64–71 (2017)
Bini, J.M., Spain, J., Nehal, K.S., Rajadhyaksha, M., Hazelwood, V., DiMarzio, C.A.: Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance. J. Biomed. Opt. 16(7), 076008 (2011)
Burlingame, E.A., Margolin, A., Gray, J.W., Chang, Y.H.: Shift: speedy histopathological-to-immunofluorescent translation of whole slide images using conditional generative adversarial networks. In: Medical Imaging 2018: Digital Pathology, vol. 10581, p. 1058105. SPIE (2018)
Can, A., Bello, M.O., Gerdes, M.J., Li, Q.: System and methods for mapping fluorescent images into a bright field color space. US Patent 8,269,827 18 Sept 2012
Chen, J., Pan, X., Monga, R., Bengio, S., Jozefowicz, R.: Revisiting distributed synchronous SGD. arXiv preprint arXiv:1604.00981 (2016)
Christiansen, E.M., et al.: In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173(3), 792–803 (2018)
Gareau, D.S.: Feasibility of digitally stained multimodal confocal mosaics to simulate histopathology. J. Biomed. Opt. 14(3), 034050 (2009)
Giacomelli, M.G., et al.: Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging. PLoS ONE 11(8), e0159337 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE CVPR, pp. 770–778 (2016)
Kelly, T., Huang, Y., Simms, A.E., Mazur, A.: Fibroblast activation protein-\(\alpha \): a key modulator of the microenvironment in multiple pathologies. Int. Rev. Cell Mol. Biol. 297, 83–116 (2012)
Kuse, M., Sharma, T., Gupta, S.: Classification scheme for lymphocyte segmentation in H&E stained histology images. In: Ünay, D., Çataltepe, Z., Aksoy, S. (eds.) ICPR 2010. LNCS, vol. 6388, pp. 235–243. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17711-8_24
Lahiani, A., Gildenblat, J., Klaman, I., Navab, N., Klaiman, E.: Generalising multistain immunohistochemistry tissue segmentation using end-to-end colour deconvolution deep neural networks. IET Image Process. 13(7), 1066–1073 (2019)
Lahiani, A., Klaiman, E., Grimm, O.: Enabling histopathological annotations on immunofluorescent images through virtualization of hematoxylin and eosin. JPI 9, 1 (2018)
Rivenson, Y., Wang, H., Wei, Z., Zhang, Y., Gunaydin, H., Ozcan, A.: Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue. arXiv preprint arXiv:1803.11293 (2018)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: the missing ingredient for fast stylization. arxiv 2016. arXiv preprint arXiv:1607.08022 (2016)
Zhu, J.Y.: Freezing instance normalization during test phase? (2017). https://github.com/junyanz/CycleGAN/issues/32. Accessed 01 Oct 2018
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: ICCV (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Lahiani, A., Gildenblat, J., Klaman, I., Albarqouni, S., Navab, N., Klaiman, E. (2019). Virtualization of Tissue Staining in Digital Pathology Using an Unsupervised Deep Learning Approach. In: Reyes-Aldasoro, C., Janowczyk, A., Veta, M., Bankhead, P., Sirinukunwattana, K. (eds) Digital Pathology. ECDP 2019. Lecture Notes in Computer Science(), vol 11435. Springer, Cham. https://doi.org/10.1007/978-3-030-23937-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-23937-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23936-7
Online ISBN: 978-3-030-23937-4
eBook Packages: Computer ScienceComputer Science (R0)