Abstract
Pathological examination is the gold standard for the diagnosis of cancer. In general, common pathological examinations include hematoxylin-eosin (H&E) staining and immunohistochemistry. H&E staining examination has the advantages of short dyeing duration and low cost, which is the most common one in the clinical practice. However, in some cases, the pathologist is hard to conduct an accurate diagnosis of cancer only according to the H&E staining images. Whereas, the immunohistochemistry examination can further provide enough evidence for the diagnosis process. Hence, the generation of virtual Ki-67 staining sections from H&E staining sections by computer assisted technology will be a good creative solution. Currently, this is still a challenge due to the lack of pixel-level paired data. In this paper, we propose a new method based on Cycle-GAN to generate Ki-67 staining images from the available H&E images, and our method is validated on a neuroendocrine tumor dataset. Massive experiment results show that the addition of skip connection and structural consistency constraint can further improve the performance of Cycle-GAN in unpaired pathological image-to-image transfer tasks. The quantification evaluation demonstrates that our proposed method achieves the state of art and reveals significant potential in clinical virtual staining.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Weinstein, R.S., et al.: Overview of telepathology, virtual microscopy, and whole slide imaging: prospects for the future. Human Pathol. 40(8), 1057–1069 (2009)
Soares, C.T., Frederigue-Junior, U., de Luca, L.A.: Anatomopathological analysis of sentinel and nonsentinel lymph nodes in breast cancer: hematoxylin-eosin versus immunohistochemistry. Int. J. Surg. Pathol. 15(4), 358–368 (2007)
Sheikh, R.A., et al.: Correlation of Ki-67, p53, and Adnab-9 immunohistochemical staining and ploidy with clinical and histopathologic features of severely dysplastic colorectal adenomas. Dig. Dis. Sci. 48(1), 223–229 (2003). https://doi.org/10.1023/A:1021727608133
Wang, Y., Sun, L.L., Jin, Q.: Enhanced diagnosis of pneumothorax with an improved real-time augmentation for imbalanced chest x-rays data based on DCNN. In: IEEE/ACM Transactions on Computational Biology and Bioinformatics (2019)
Tang, Z., et al.: An augmentation strategy for medical image processing based on statistical shape model and 3D thin plate spline for deep learning. IEEE Access 7, 133111–133121 (2019)
Yang, J., et al.: Joint correction of attenuation and scatter in image space using deep convolutional neural networks for dedicated brain 18F-FDG PET. Phys. Med. Biol. 64(7), 075019 (2019)
Tang, Z., Wang, M., Song, Z.: Rotationally resliced 3D prostate segmentation of MR images using Bhattacharyya similarity and active band theory. Physica Med. 54, 56–65 (2018)
Zhang, B., et al.: Cerebrovascular segmentation from TOF-MRA using model-and data-driven method via sparse labels. Neurocomputing 380, 162–179 (2020)
Goodfellow, I., et al.: Generative adversarial nets. Adv. Neural. Inf. Process. Syst. 27, 2672–2680 (2014)
Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
Isola, P., et al.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
Zhu, J.-Y., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision. (2017)
Wang, Z., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Liu, L., et al.: Understanding the Difficulty of Training Transformers. arXiv preprint arXiv:2004.08249 (2020)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: The Thirty-Seventh Asilomar Conference on Signals, Systems and Computers, vol. 2. IEEE (2003)
Weng, L., Preneel, B.: A secure perceptual hash algorithm for image content authentication. In: De Decker, B., Lapon, J., Naessens, V., Uhl, A. (eds.) CMS 2011. LNCS, vol. 7025, pp. 108–121. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24712-5_9
Acknowledgment
This research was made possible with the financial support from National Science Foundation of China (NSFC) (61875102, 81871395, 61675113), Science and Technology Research Program of Shenzhen City (JCYJ20170816161836562, JCYJ20170817111912585, JCYJ20160427183803458, JCYJ20170412171856582, JCY20180508152528735), Oversea cooperation foundation, Graduate School at Shenzhen, Tsinghua University (HW2018007).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Liu, S. et al. (2021). The Generation of Virtual Immunohistochemical Staining Images Based on an Improved Cycle-GAN. In: Guan, M., Na, Z. (eds) Machine Learning and Intelligent Communications. MLICOM 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 342. Springer, Cham. https://doi.org/10.1007/978-3-030-66785-6_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-66785-6_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66784-9
Online ISBN: 978-3-030-66785-6
eBook Packages: Computer ScienceComputer Science (R0)