Generating Virtual Chromoendoscopic Images and Improving Detectability and Classification Performance of Endoscopic Lesions

  • Akihiro FukudaEmail author
  • Tadashi Miyamoto
  • Shunsuke Kamba
  • Kazuki Sumiyama
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11795)


Endoscopy is a standard method for the diagnosis and detection of colorectal lesions. As a method to enhance the detectability of lesions, the effectiveness of pancolonic chromoendoscopy with indigocarmine has been reported. On the other hand, computer-aided diagnosis (CAD) has attracted attention. However, existing CAD systems are mainly for white light imaging (WLI) endoscopy, and the effect of the combination of CAD and indigocarmine dye spraying is not clear. Besides, it is difficult to gather a lot of indigocarmine dye-sprayed (IC) images for training. Here, we propose image-to-image translation from WLI to virtual indigocarmine dye-sprayed (VIC) images based on unpaired cycle-consistent Generative Adversarial Networks. Using this generator as preprocess part, we constructed detection models to evaluate the effectiveness of VIC translation for localization and classification of lesions. We also compared the localization and classification performance with and without image augmentation by using generated VIC images. Our results show that the model trained on IC and VIC images had the highest performance in both localization and classification. Therefore, VIC images are useful for the augmentation of IC images.


  1. 1.
    Adler, A., Pohl, H., Papanikolaou, I.S., et al: A prospective randomized study on narrow-band imaging versus conventional colonoscopy for adenoma detection: does NBI induce a learning effect? Gut 57 (2008)Google Scholar
  2. 2.
    Ahn, S.B., Han, D.S., Bae, J.H., et al.: The miss rate for colorectal adenoma determined by quality-adjusted, back-to-back colonoscopies. Gut Liver 6, 64–70 (2012)CrossRefGoogle Scholar
  3. 3.
    Brooker, J.C., Saunders, B.P., Shah, S.G., et al.: Total colonic dye-spray increases the detection of diminutive adenomas during routine colonoscopy: a randomized controlled trial. Gastrointest. Endosc. 56, 333–338 (2002)CrossRefGoogle Scholar
  4. 4.
    Corley, D., Jensen, C., Marks, A., et al.: Adenoma detection rate and risk of colorectal cancer and death. N. Engl. J. Med. 370, 2541 (2014)CrossRefGoogle Scholar
  5. 5.
    Engelhardt, S., De Simone, R., Full, P.M., et al.: Improving surgical training phantoms by hyperrealism: deep unpaired image-to-image translation from real surgeries. arXiv pre-prints arXiv:1806.03627 (2018)
  6. 6.
    Isola, P., Zhu, J., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004 (2016)
  7. 7.
    Kaminski, M., Regula, J., Kraszewska, E., et al: Quality indicators for colonoscopy and the risk of interval cancer. N. Engl. J. Med. 362 (2010)Google Scholar
  8. 8.
    Ki Min, J., Kwak, M., Myung Cha, J.: Overview of deep learning in gastrointestinal endoscopy. Gut Liver 13(4), 388–393 (2019)CrossRefGoogle Scholar
  9. 9.
    Kumar, S., Thosani, N., Ladabaum, U., et al.: Adenoma miss rates associated with a 3-minute versus 6-minute colonoscopy withdrawal time: a prospective, randomized trial. Gastrointest. Endosc. 85, 1273–1280 (2017)CrossRefGoogle Scholar
  10. 10.
    Mori, Y., Kudo, S.E., Berzin, T.M., et al.: Computer-aided diagnosis for colonoscopy. Endoscopy 49, 813–819 (2017)CrossRefGoogle Scholar
  11. 11.
    Pohl, J., Lotterer, E., Balzer, C., et al.: Computed virtual chromoendoscopy versus standard colonoscopy with targeted indigocarmine chromoscopy: a randomised multicentre trial. Gut 58, 73–78 (2009)CrossRefGoogle Scholar
  12. 12.
    Pohl, J., Schneider, A., Vogell, H., et al.: Pancolonic chromoendoscopy with indigo carmine versus standard colonoscopy for detection of neoplastic lesions: a randomised two-centre trial. Gut 60, 485–490 (2011)CrossRefGoogle Scholar
  13. 13.
    Redmon, J., Farhadi, A.: YoLov3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
  14. 14.
    Repici, A., Wallace, M.B., East, J.E., et al.: Efficacy of per-oral methylene blue formulation for screening colonoscopy\(^{\rm a}\). Gastroenterology S0016–5085(19), 2198–2207 (2019)Google Scholar
  15. 15.
    Rex, D.K., Helbig, C.C.: High yields of small and flat adenomas with high-definition colonoscopes using either white light or narrow band imaging. Gastroenterology 133, 42–47 (2007)CrossRefGoogle Scholar
  16. 16.
    Urban, G., Tripathi, P., Alkayali, T., et al.: Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 155, 1069–1078 (2018)CrossRefGoogle Scholar
  17. 17.
    Wang, P., Xiao, X., Glissen Brown, J.R., et al.: Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nat. Biomed. Eng. 2, 741–748 (2018)CrossRefGoogle Scholar
  18. 18.
    Yi, Z., Zhang, H., Tan, P., et al.: DualGAN: unsupervised dual learning for image-to-image translation. arXiv preprint arXiv:1704.02510 (2017)
  19. 19.
    Zhong, Z., Zheng, L., Kang, G., et al.: Random erasing data augmentation. arXiv preprint arXiv:1708.04896 (2017)
  20. 20.
    Zhu, J., Park, T., Isola, P., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593 (2017)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Akihiro Fukuda
    • 1
    Email author
  • Tadashi Miyamoto
    • 1
  • Shunsuke Kamba
    • 2
  • Kazuki Sumiyama
    • 2
  1. 1.Research and Development DepartmentLPixel Inc.TokyoJapan
  2. 2.Department of EndoscopyThe Jikei University School of MedicineTokyoJapan

Personalised recommendations